backend / cloud aws / application integration / README.md

AWS SQS (Simple Queue Service) - Interview Guide

3 min read index source

AWS SQS (Simple Queue Service) - Interview Guide

Overview

Amazon SQS is a fully managed message queuing service that enables decoupling and scaling of microservices, distributed systems, and serverless applications. SQS offers two queue types: Standard (high throughput, at-least-once delivery) and FIFO (exactly-once processing, ordered delivery).

Key Features

  • Fully managed: No server management
  • Scalable: Handles any volume of messages
  • Standard & FIFO Queues: Choose based on ordering and deduplication needs
  • Dead Letter Queues (DLQ): Handle message failures
  • Visibility Timeout: Prevents multiple consumers from processing the same message
  • Long Polling: Reduces empty responses and cost
  • Server-side Encryption: Protects sensitive data

Architecture

Producer → SQS Queue → Consumer(s)
   |           |           |
  API      Standard/FIFO   Lambda, EC2, ECS, etc.
  • Producer: Sends messages to the queue
  • Queue: Stores messages until processed
  • Consumer: Retrieves and processes messages

Use Cases

  • Decoupling microservices
  • Task scheduling and background jobs
  • Buffering and load leveling
  • Order processing
  • Event-driven serverless workflows

Basic Usage Example

import boto3

sqs = boto3.client('sqs')
queue_url = 'https://sqs.us-east-1.amazonaws.com/123456789012/my-queue'

# Send a message
sqs.send_message(QueueUrl=queue_url, MessageBody='Hello, SQS!')

# Receive messages
response = sqs.receive_message(QueueUrl=queue_url, MaxNumberOfMessages=1)
for msg in response.get('Messages', []):
    print(msg['Body'])
    # Delete after processing
    sqs.delete_message(QueueUrl=queue_url, ReceiptHandle=msg['ReceiptHandle'])

Best Practices

  • Use DLQs for error handling
  • Set appropriate visibility timeout
  • Use batching for high throughput
  • Monitor queue length and age
  • Secure queues with IAM policies
  • Use FIFO queues for strict ordering and deduplication

Security

  • IAM Policies: Restrict access to queues
  • SSE: Enable server-side encryption for sensitive data
  • VPC Endpoints: Private connectivity

Monitoring

  • CloudWatch Metrics: Monitor queue length, age, and failed messages
  • CloudTrail: Audit API calls

Cost Optimization

  • Use long polling to reduce API calls
  • Delete processed messages promptly
  • Clean up unused queues

Common Interview Questions

Q: SQS vs SNS?

  • SQS: Queue, pull-based, one-to-one
  • SNS: Pub/sub, push-based, one-to-many

Q: How does SQS guarantee delivery?

  • Standard: At-least-once, possible duplicates
  • FIFO: Exactly-once, ordered

Q: What is a Dead Letter Queue?

  • A secondary queue for messages that can’t be processed after max attempts

Q: How do you handle message ordering?

  • Use FIFO queues with MessageGroupId

Q: How do you avoid message loss?

  • Use DLQs, monitor metrics, set proper visibility timeout

Summary

AWS SQS is essential for building scalable, decoupled, and reliable distributed systems. Mastery of SQS concepts, patterns, and best practices is crucial for cloud and backend interviews.

Interview angle

  • “What is the visibility timeout and how do you get it wrong?” - the window during which a received message is hidden from other consumers. Set it shorter than your processing time and a second consumer picks up the same message, so the work runs twice. Set it far too long and a crashed consumer’s messages stall. Extend it with a heartbeat for variable-length work.
  • “Standard or FIFO?” - standard gives near-unlimited throughput, at-least-once delivery and best-effort ordering. FIFO gives exactly-once processing within a deduplication window and strict order per message group, at a much lower throughput ceiling. Most systems take standard plus idempotent consumers.
  • “How do you handle a poison message?” - a redrive policy to a dead-letter queue after N receives, then alarm on DLQ depth. Without one, a message that always fails is retried forever and blocks throughput.
  • “Long or short polling?” - long polling (WaitTimeSeconds up to 20). Short polling samples a subset of servers, so it returns empty responses even when messages exist, and it costs more in API calls.
  • “How do you scale consumers?” - on queue depth or age of the oldest message, not CPU. Age of oldest message is the better signal because it reflects user-visible lag.