backend / cloud aws / application integration / README.md

AWS SNS (Simple Notification Service) - Complete Guide

12 min read index source

AWS SNS (Simple Notification Service) - Complete Guide

Overview

Amazon Simple Notification Service (SNS) is a fully managed pub/sub messaging service that enables you to decouple microservices, distributed systems, and serverless applications. SNS provides topics for high-throughput, push-based, many-to-many messaging.

Table of Contents

  1. Core Concepts
  2. Key Features
  3. Architecture
  4. Use Cases
  5. Implementation Examples
  6. Best Practices
  7. Security
  8. Monitoring and Logging
  9. Cost Optimization
  10. Common Interview Questions
  11. Integration Patterns
  12. Troubleshooting

Core Concepts

Topics

  • Publishers: Send messages to topics
  • Subscribers: Receive messages from topics
  • Topics: Logical access points for communication
  • Messages: Data sent between publishers and subscribers

Message Types

  • Standard Topics: Best-effort delivery, at-least-once delivery
  • FIFO Topics: Exactly-once processing, ordered delivery

Subscription Types

  • HTTP/HTTPS: Webhook endpoints
  • Lambda: Serverless functions
  • SQS: Queue integration
  • Email/Email-JSON: Email notifications
  • SMS: Mobile text messages
  • Application: Mobile push notifications

Key Features

1. High Availability

  • 99.9% availability SLA
  • Multi-AZ deployment
  • Automatic failover

2. Scalability

  • No message ordering limits
  • Automatic scaling
  • Global deployment

3. Security

  • IAM integration
  • VPC endpoints
  • Server-side encryption (SSE)
  • Cross-account access

4. Message Filtering

  • JSON path filtering
  • Attribute-based filtering
  • Message deduplication

Architecture

Basic Architecture

Publisher → Topic → Subscribers
    ↓           ↓         ↓
  Lambda    SNS Topic   Lambda
  EC2                  SQS Queue
  API Gateway          HTTP/HTTPS

Advanced Architecture

Application → SNS Topic → Multiple Subscribers
    ↓              ↓              ↓
  Event      Message Filtering   Lambda
  Source     Dead Letter Queue   SQS
  Monitoring CloudWatch         HTTP

Use Cases

1. Event-Driven Architecture

import boto3
import json

# Publisher
sns = boto3.client('sns')

def publish_order_event(order_data):
    topic_arn = 'arn:aws:sns:us-east-1:123456789012:order-events'
    
    message = {
        'order_id': order_data['id'],
        'customer_id': order_data['customer_id'],
        'total': order_data['total'],
        'timestamp': order_data['timestamp']
    }
    
    response = sns.publish(
        TopicArn=topic_arn,
        Message=json.dumps(message),
        MessageAttributes={
            'event_type': {
                'DataType': 'String',
                'StringValue': 'order_created'
            },
            'priority': {
                'DataType': 'String',
                'StringValue': 'high'
            }
        }
    )
    
    return response['MessageId']

2. Microservices Communication

# Order Service
def create_order(order_data):
    # Process order
    order = process_order(order_data)
    
    # Publish event
    publish_order_event({
        'order_id': order.id,
        'status': 'created',
        'items': order.items,
        'total': order.total
    })
    
    return order

# Inventory Service (Subscriber)
def handle_order_created(event, context):
    order_data = json.loads(event['Records'][0]['Sns']['Message'])
    
    # Update inventory
    for item in order_data['items']:
        update_inventory(item['product_id'], item['quantity'])
    
    # Publish inventory updated event
    publish_inventory_event(order_data['order_id'])

3. Fan-out Pattern

# Single publisher, multiple subscribers
def publish_user_registration(user_data):
    topic_arn = 'arn:aws:sns:us-east-1:123456789012:user-events'
    
    message = {
        'user_id': user_data['id'],
        'email': user_data['email'],
        'name': user_data['name'],
        'timestamp': datetime.utcnow().isoformat()
    }
    
    # All subscribers receive this message
    response = sns.publish(
        TopicArn=topic_arn,
        Message=json.dumps(message),
        MessageAttributes={
            'event_type': {
                'DataType': 'String',
                'StringValue': 'user_registered'
            }
        }
    )
    
    return response['MessageId']

# Subscribers:
# - Email Service: Send welcome email
# - Analytics Service: Track user registration
# - Notification Service: Send push notification
# - Database Service: Update user statistics

Implementation Examples

1. Creating Topics and Subscriptions

import boto3
from botocore.exceptions import ClientError

class SNSService:
    def __init__(self):
        self.sns = boto3.client('sns')
    
    def create_topic(self, topic_name, tags=None):
        """Create an SNS topic"""
        try:
            response = self.sns.create_topic(
                Name=topic_name,
                Tags=tags or []
            )
            return response['TopicArn']
        except ClientError as e:
            print(f"Error creating topic: {e}")
            return None
    
    def create_subscription(self, topic_arn, protocol, endpoint, attributes=None):
        """Create a subscription"""
        try:
            response = self.sns.subscribe(
                TopicArn=topic_arn,
                Protocol=protocol,
                Endpoint=endpoint,
                Attributes=attributes or {}
            )
            return response['SubscriptionArn']
        except ClientError as e:
            print(f"Error creating subscription: {e}")
            return None
    
    def publish_message(self, topic_arn, message, subject=None, attributes=None):
        """Publish a message to a topic"""
        try:
            response = self.sns.publish(
                TopicArn=topic_arn,
                Message=message,
                Subject=subject,
                MessageAttributes=attributes or {}
            )
            return response['MessageId']
        except ClientError as e:
            print(f"Error publishing message: {e}")
            return None

# Usage
sns_service = SNSService()

# Create topic
topic_arn = sns_service.create_topic('order-events')

# Create subscriptions
lambda_subscription = sns_service.create_subscription(
    topic_arn, 'lambda', 'arn:aws:lambda:us-east-1:123456789012:function:process-order'
)

sqs_subscription = sns_service.create_subscription(
    topic_arn, 'sqs', 'arn:aws:sqs:us-east-1:123456789012:order-queue'
)

# Publish message
message_id = sns_service.publish_message(
    topic_arn,
    '{"order_id": "123", "status": "created"}',
    'Order Created',
    {
        'event_type': {
            'DataType': 'String',
            'StringValue': 'order_created'
        }
    }
)

2. Message Filtering

# Create subscription with filter policy
def create_filtered_subscription(topic_arn, protocol, endpoint, filter_policy):
    """Create subscription with message filtering"""
    try:
        response = sns.subscribe(
            TopicArn=topic_arn,
            Protocol=protocol,
            Endpoint=endpoint,
            Attributes={
                'FilterPolicy': json.dumps(filter_policy)
            }
        )
        return response['SubscriptionArn']
    except ClientError as e:
        print(f"Error creating filtered subscription: {e}")
        return None

# Filter policy examples
high_priority_filter = {
    "priority": ["high", "critical"]
}

order_filter = {
    "event_type": ["order_created", "order_updated"],
    "amount": [{"numeric": [">", 100]}]
}

# Create filtered subscriptions
high_priority_subscription = create_filtered_subscription(
    topic_arn,
    'lambda',
    'arn:aws:lambda:us-east-1:123456789012:function:high-priority-handler',
    high_priority_filter
)

order_subscription = create_filtered_subscription(
    topic_arn,
    'sqs',
    'arn:aws:sqs:us-east-1:123456789012:order-queue',
    order_filter
)

3. Dead Letter Queue Integration

def create_subscription_with_dlq(topic_arn, protocol, endpoint, dlq_arn):
    """Create subscription with dead letter queue"""
    try:
        response = sns.subscribe(
            TopicArn=topic_arn,
            Protocol=protocol,
            Endpoint=endpoint,
            Attributes={
                'RedrivePolicy': json.dumps({
                    'deadLetterTargetArn': dlq_arn
                })
            }
        )
        return response['SubscriptionArn']
    except ClientError as e:
        print(f"Error creating subscription with DLQ: {e}")
        return None

# Create SQS queue for dead letters
sqs = boto3.client('sqs')
dlq_response = sqs.create_queue(
    QueueName='sns-dlq',
    Attributes={
        'MessageRetentionPeriod': '1209600'  # 14 days
    }
)
dlq_arn = dlq_response['QueueArn']

# Create subscription with DLQ
subscription_arn = create_subscription_with_dlq(
    topic_arn,
    'lambda',
    'arn:aws:lambda:us-east-1:123456789012:function:process-message',
    dlq_arn
)

4. FIFO Topics

def create_fifo_topic(topic_name, content_based_deduplication=True):
    """Create FIFO topic with content-based deduplication"""
    try:
        response = sns.create_topic(
            Name=f"{topic_name}.fifo",
            Attributes={
                'FifoTopic': 'true',
                'ContentBasedDeduplication': str(content_based_deduplication).lower()
            }
        )
        return response['TopicArn']
    except ClientError as e:
        print(f"Error creating FIFO topic: {e}")
        return None

def publish_fifo_message(topic_arn, message, group_id, deduplication_id=None):
    """Publish message to FIFO topic"""
    try:
        publish_kwargs = {
            'TopicArn': topic_arn,
            'Message': message,
            'MessageGroupId': group_id
        }
        
        if deduplication_id:
            publish_kwargs['MessageDeduplicationId'] = deduplication_id
        
        response = sns.publish(**publish_kwargs)
        return response['MessageId']
    except ClientError as e:
        print(f"Error publishing FIFO message: {e}")
        return None

# Usage
fifo_topic_arn = create_fifo_topic('order-events')

# Publish messages with same group ID for ordering
for i in range(5):
    message_id = publish_fifo_message(
        fifo_topic_arn,
        f'{{"order_id": "123", "step": {i}}}',
        'order-123'  # Same group ID ensures ordering
    )

Best Practices

1. Message Design

# Good: Structured message with metadata
def publish_well_structured_message(topic_arn, event_data):
    message = {
        'metadata': {
            'event_id': str(uuid.uuid4()),
            'timestamp': datetime.utcnow().isoformat(),
            'version': '1.0',
            'source': 'order-service'
        },
        'data': event_data,
        'correlation_id': get_correlation_id()
    }
    
    attributes = {
        'event_type': {
            'DataType': 'String',
            'StringValue': event_data['type']
        },
        'priority': {
            'DataType': 'String',
            'StringValue': event_data.get('priority', 'normal')
        }
    }
    
    return sns.publish(
        TopicArn=topic_arn,
        Message=json.dumps(message),
        MessageAttributes=attributes
    )

# Bad: Simple string message
def publish_simple_message(topic_arn, message):
    return sns.publish(
        TopicArn=topic_arn,
        Message=message  # No structure, no metadata
    )

2. Error Handling

def publish_with_retry(topic_arn, message, max_retries=3):
    """Publish message with exponential backoff retry"""
    for attempt in range(max_retries):
        try:
            response = sns.publish(
                TopicArn=topic_arn,
                Message=message
            )
            return response['MessageId']
        except ClientError as e:
            if e.response['Error']['Code'] == 'ThrottlingException':
                if attempt < max_retries - 1:
                    time.sleep(2 ** attempt)  # Exponential backoff
                    continue
            raise e
    return None

3. Monitoring and Alerting

import boto3
from datetime import datetime, timedelta

def monitor_sns_metrics(topic_arn, hours=24):
    """Monitor SNS metrics for a topic"""
    cloudwatch = boto3.client('cloudwatch')
    
    end_time = datetime.utcnow()
    start_time = end_time - timedelta(hours=hours)
    
    # Get delivery success rate
    success_metric = cloudwatch.get_metric_statistics(
        Namespace='AWS/SNS',
        MetricName='NumberOfNotificationsDelivered',
        Dimensions=[
            {
                'Name': 'TopicName',
                'Value': topic_arn.split(':')[-1]
            }
        ],
        StartTime=start_time,
        EndTime=end_time,
        Period=3600,  # 1 hour
        Statistics=['Sum']
    )
    
    # Get failure rate
    failure_metric = cloudwatch.get_metric_statistics(
        Namespace='AWS/SNS',
        MetricName='NumberOfNotificationsFailed',
        Dimensions=[
            {
                'Name': 'TopicName',
                'Value': topic_arn.split(':')[-1]
            }
        ],
        StartTime=start_time,
        EndTime=end_time,
        Period=3600,
        Statistics=['Sum']
    )
    
    return {
        'success_count': sum(point['Sum'] for point in success_metric['Datapoints']),
        'failure_count': sum(point['Sum'] for point in failure_metric['Datapoints'])
    }

Security

1. IAM Policies

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "sns:Publish",
                "sns:GetTopicAttributes"
            ],
            "Resource": "arn:aws:sns:us-east-1:123456789012:order-events"
        },
        {
            "Effect": "Allow",
            "Action": [
                "sns:Subscribe",
                "sns:Receive"
            ],
            "Resource": "arn:aws:sns:us-east-1:123456789012:order-events"
        }
    ]
}

2. VPC Endpoints

import boto3

# Configure SNS client with VPC endpoint
sns = boto3.client(
    'sns',
    endpoint_url='https://sns.us-east-1.amazonaws.com',
    region_name='us-east-1'
)

3. Server-Side Encryption

def create_encrypted_topic(topic_name, kms_key_id):
    """Create topic with server-side encryption"""
    try:
        response = sns.create_topic(
            Name=topic_name,
            Attributes={
                'KmsMasterKeyId': kms_key_id
            }
        )
        return response['TopicArn']
    except ClientError as e:
        print(f"Error creating encrypted topic: {e}")
        return None

Monitoring and Logging

1. CloudWatch Metrics

def get_sns_metrics(topic_arn):
    """Get key SNS metrics"""
    cloudwatch = boto3.client('cloudwatch')
    
    metrics = {
        'delivery_success': 'NumberOfNotificationsDelivered',
        'delivery_failure': 'NumberOfNotificationsFailed',
        'publish_success': 'NumberOfMessagesPublished',
        'publish_failure': 'NumberOfNotificationsFailed'
    }
    
    results = {}
    for metric_name, cloudwatch_metric in metrics.items():
        response = cloudwatch.get_metric_statistics(
            Namespace='AWS/SNS',
            MetricName=cloudwatch_metric,
            Dimensions=[
                {
                    'Name': 'TopicName',
                    'Value': topic_arn.split(':')[-1]
                }
            ],
            StartTime=datetime.utcnow() - timedelta(hours=1),
            EndTime=datetime.utcnow(),
            Period=300,
            Statistics=['Sum']
        )
        results[metric_name] = response['Datapoints']
    
    return results

2. CloudTrail Logging

def enable_cloudtrail_logging(trail_name, s3_bucket):
    """Enable CloudTrail logging for SNS"""
    cloudtrail = boto3.client('cloudtrail')
    
    try:
        response = cloudtrail.create_trail(
            Name=trail_name,
            S3BucketName=s3_bucket,
            IncludeGlobalServiceEvents=True,
            IsMultiRegionTrail=True
        )
        
        # Start logging
        cloudtrail.start_logging(Name=trail_name)
        
        return response['TrailARN']
    except ClientError as e:
        print(f"Error creating CloudTrail: {e}")
        return None

Cost Optimization

1. Message Batching

def batch_publish_messages(topic_arn, messages, batch_size=10):
    """Publish messages in batches to reduce API calls"""
    message_ids = []
    
    for i in range(0, len(messages), batch_size):
        batch = messages[i:i + batch_size]
        
        # Publish batch
        for message in batch:
            response = sns.publish(
                TopicArn=topic_arn,
                Message=json.dumps(message)
            )
            message_ids.append(response['MessageId'])
    
    return message_ids

2. Subscription Management

def cleanup_unused_subscriptions():
    """Remove unused subscriptions to reduce costs"""
    sns = boto3.client('sns')
    
    # List all subscriptions
    paginator = sns.get_paginator('list_subscriptions')
    
    for page in paginator.paginate():
        for subscription in page['Subscriptions']:
            # Check if subscription is confirmed and active
            if (subscription['SubscriptionArn'] != 'PendingConfirmation' and
                subscription['Protocol'] in ['http', 'https']):
                
                # Check if endpoint is responding
                if not is_endpoint_healthy(subscription['Endpoint']):
                    print(f"Removing unhealthy subscription: {subscription['SubscriptionArn']}")
                    sns.unsubscribe(SubscriptionArn=subscription['SubscriptionArn'])

Common Interview Questions

1. Basic Questions

Q: What is the difference between SNS and SQS?

  • SNS: Pub/sub messaging, one-to-many, push-based
  • SQS: Queue messaging, one-to-one, pull-based
  • SNS: No message persistence, immediate delivery
  • SQS: Message persistence, consumer pulls messages

Q: What are the delivery guarantees of SNS?

  • Standard Topics: At-least-once delivery, best-effort ordering
  • FIFO Topics: Exactly-once delivery, strict ordering
  • HTTP/HTTPS: At-least-once delivery with retries
  • Lambda: At-least-once delivery with automatic retries

Q: How does message filtering work in SNS?

# Filter policy example
filter_policy = {
    "event_type": ["order_created", "order_updated"],
    "priority": ["high", "critical"],
    "amount": [{"numeric": [">", 100]}]
}

# Only messages matching ALL conditions are delivered

2. Advanced Questions

Q: How would you implement idempotency with SNS?

def publish_idempotent_message(topic_arn, message, deduplication_id):
    """Publish message with deduplication"""
    try:
        response = sns.publish(
            TopicArn=topic_arn,
            Message=json.dumps(message),
            MessageAttributes={
                'deduplication_id': {
                    'DataType': 'String',
                    'StringValue': deduplication_id
                }
            }
        )
        return response['MessageId']
    except ClientError as e:
        if e.response['Error']['Code'] == 'DuplicateMessage':
            # Message already published
            return None
        raise e

Q: How do you handle message ordering in SNS?

# Use FIFO topics for strict ordering
def create_fifo_topic_with_ordering(topic_name):
    """Create FIFO topic with message ordering"""
    response = sns.create_topic(
        Name=f"{topic_name}.fifo",
        Attributes={
            'FifoTopic': 'true',
            'ContentBasedDeduplication': 'true'
        }
    )
    return response['TopicArn']

def publish_ordered_message(topic_arn, message, group_id):
    """Publish message with ordering"""
    return sns.publish(
        TopicArn=topic_arn,
        Message=json.dumps(message),
        MessageGroupId=group_id  # Messages with same group ID are ordered
    )

Q: How would you implement a dead letter queue for SNS?

def create_subscription_with_dlq(topic_arn, protocol, endpoint):
    """Create subscription with dead letter queue"""
    # Create SQS queue for dead letters
    sqs = boto3.client('sqs')
    dlq_response = sqs.create_queue(QueueName='sns-dlq')
    dlq_arn = dlq_response['QueueArn']
    
    # Create subscription with DLQ
    response = sns.subscribe(
        TopicArn=topic_arn,
        Protocol=protocol,
        Endpoint=endpoint,
        Attributes={
            'RedrivePolicy': json.dumps({
                'deadLetterTargetArn': dlq_arn,
                'maxReceiveCount': '3'
            })
        }
    )
    return response['SubscriptionArn']

3. Architecture Questions

Q: How would you design a notification system using SNS?

class NotificationSystem:
    def __init__(self):
        self.sns = boto3.client('sns')
        self.topics = {
            'user_events': 'arn:aws:sns:us-east-1:123456789012:user-events',
            'order_events': 'arn:aws:sns:us-east-1:123456789012:order-events',
            'system_alerts': 'arn:aws:sns:us-east-1:123456789012:system-alerts'
        }
    
    def send_user_notification(self, user_id, message, notification_type):
        """Send user notification via multiple channels"""
        event_data = {
            'user_id': user_id,
            'message': message,
            'type': notification_type,
            'timestamp': datetime.utcnow().isoformat()
        }
        
        # Publish to user events topic
        # Subscribers: Email service, SMS service, Push notification service
        return self.sns.publish(
            TopicArn=self.topics['user_events'],
            Message=json.dumps(event_data),
            MessageAttributes={
                'notification_type': {
                    'DataType': 'String',
                    'StringValue': notification_type
                }
            }
        )
    
    def send_system_alert(self, alert_level, message, service):
        """Send system alert"""
        alert_data = {
            'level': alert_level,
            'message': message,
            'service': service,
            'timestamp': datetime.utcnow().isoformat()
        }
        
        return self.sns.publish(
            TopicArn=self.topics['system_alerts'],
            Message=json.dumps(alert_data),
            MessageAttributes={
                'alert_level': {
                    'DataType': 'String',
                    'StringValue': alert_level
                }
            }
        )

Integration Patterns

1. SNS + Lambda Pattern

# Lambda function triggered by SNS
def lambda_handler(event, context):
    """Process SNS messages in Lambda"""
    for record in event['Records']:
        sns_message = record['Sns']
        message_body = json.loads(sns_message['Message'])
        
        # Process message based on type
        if message_body.get('type') == 'order_created':
            process_order_created(message_body)
        elif message_body.get('type') == 'user_registered':
            process_user_registration(message_body)
        else:
            print(f"Unknown message type: {message_body.get('type')}")

def process_order_created(order_data):
    """Process order created event"""
    # Update inventory
    # Send confirmation email
    # Update analytics
    pass

def process_user_registration(user_data):
    """Process user registration event"""
    # Send welcome email
    # Create user profile
    # Update metrics
    pass

2. SNS + SQS Pattern

# SNS publishes to SQS queue
def setup_sns_sqs_integration(topic_arn, queue_arn):
    """Set up SNS to SQS integration"""
    # Create subscription
    response = sns.subscribe(
        TopicArn=topic_arn,
        Protocol='sqs',
        Endpoint=queue_arn
    )
    
    # Configure SQS queue policy to allow SNS
    sqs_policy = {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Principal": {
                    "Service": "sns.amazonaws.com"
                },
                "Action": "sqs:SendMessage",
                "Resource": queue_arn,
                "Condition": {
                    "ArnEquals": {
                        "aws:SourceArn": topic_arn
                    }
                }
            }
        ]
    }
    
    sqs = boto3.client('sqs')
    sqs.set_queue_attributes(
        QueueUrl=queue_arn.replace(':sqs:', ':sqs:').replace('arn:aws:sqs:', ''),
        Attributes={
            'Policy': json.dumps(sqs_policy)
        }
    )
    
    return response['SubscriptionArn']

3. Fan-out Pattern

# Single publisher, multiple subscribers
def implement_fan_out_pattern():
    """Implement fan-out pattern with SNS"""
    topic_arn = 'arn:aws:sns:us-east-1:123456789012:user-events'
    
    # Multiple subscribers
    subscribers = [
        {
            'protocol': 'lambda',
            'endpoint': 'arn:aws:lambda:us-east-1:123456789012:function:email-service'
        },
        {
            'protocol': 'lambda',
            'endpoint': 'arn:aws:lambda:us-east-1:123456789012:function:analytics-service'
        },
        {
            'protocol': 'sqs',
            'endpoint': 'arn:aws:sqs:us-east-1:123456789012:notification-queue'
        },
        {
            'protocol': 'http',
            'endpoint': 'https://api.example.com/webhooks/user-events'
        }
    ]
    
    # Create subscriptions
    for subscriber in subscribers:
        sns.subscribe(
            TopicArn=topic_arn,
            Protocol=subscriber['protocol'],
            Endpoint=subscriber['endpoint']
        )
    
    # Publish message (all subscribers receive it)
    return sns.publish(
        TopicArn=topic_arn,
        Message=json.dumps({
            'user_id': '123',
            'event': 'user_registered',
            'timestamp': datetime.utcnow().isoformat()
        })
    )

Troubleshooting

1. Common Issues

def troubleshoot_sns_issues():
    """Common SNS troubleshooting steps"""
    
    # 1. Check topic permissions
    def check_topic_permissions(topic_arn):
        try:
            response = sns.get_topic_attributes(TopicArn=topic_arn)
            return response['Attributes']
        except ClientError as e:
            print(f"Permission error: {e}")
            return None
    
    # 2. Check subscription status
    def check_subscription_status(subscription_arn):
        try:
            response = sns.get_subscription_attributes(
                SubscriptionArn=subscription_arn
            )
            return response['Attributes']
        except ClientError as e:
            print(f"Subscription error: {e}")
            return None
    
    # 3. Test message delivery
    def test_message_delivery(topic_arn):
        try:
            response = sns.publish(
                TopicArn=topic_arn,
                Message='Test message',
                Subject='Test'
            )
            print(f"Test message sent: {response['MessageId']}")
            return response['MessageId']
        except ClientError as e:
            print(f"Publish error: {e}")
            return None
    
    # 4. Check CloudWatch metrics
    def check_delivery_metrics(topic_arn):
        cloudwatch = boto3.client('cloudwatch')
        
        response = cloudwatch.get_metric_statistics(
            Namespace='AWS/SNS',
            MetricName='NumberOfNotificationsFailed',
            Dimensions=[
                {
                    'Name': 'TopicName',
                    'Value': topic_arn.split(':')[-1]
                }
            ],
            StartTime=datetime.utcnow() - timedelta(hours=1),
            EndTime=datetime.utcnow(),
            Period=300,
            Statistics=['Sum']
        )
        
        return response['Datapoints']

2. Debugging Tools

def enable_sns_logging(topic_arn):
    """Enable detailed logging for SNS topic"""
    try:
        response = sns.set_topic_attributes(
            TopicArn=topic_arn,
            AttributeName='DeliveryPolicy',
            AttributeValue=json.dumps({
                'healthyRetryPolicy': {
                    'numRetries': 3,
                    'minDelayTarget': 20,
                    'maxDelayTarget': 20,
                    'numMaxDelayRetries': 0,
                    'numNoDelayRetries': 0,
                    'backoffFunction': 'linear'
                },
                'sicklyRetryPolicy': {
                    'numRetries': 3,
                    'minDelayTarget': 20,
                    'maxDelayTarget': 20,
                    'numMaxDelayRetries': 0,
                    'numNoDelayRetries': 0,
                    'backoffFunction': 'linear'
                },
                'throttlePolicy': {
                    'maxReceivesPerSecond': 10
                }
            })
        )
        return response
    except ClientError as e:
        print(f"Error enabling logging: {e}")
        return None

Summary

AWS SNS is a powerful messaging service that enables:

  • Decoupled Communication: Publishers and subscribers are independent
  • Scalable Architecture: Automatic scaling with no message limits
  • Multiple Protocols: Support for HTTP, Lambda, SQS, Email, SMS
  • Message Filtering: Attribute-based filtering for selective delivery
  • High Availability: 99.9% SLA with multi-AZ deployment
  • Security: IAM integration, VPC endpoints, encryption

Key use cases include:

  • Event-driven architectures
  • Microservices communication
  • Notification systems
  • Fan-out patterns
  • System monitoring and alerting

Best practices include:

  • Proper message structure and metadata
  • Error handling and retry logic
  • Monitoring and alerting
  • Security configuration
  • Cost optimization through batching

Interview angle

  • “SNS or SQS?” - SNS is push, one-to-many, fan-out: every subscriber gets a copy, and there is no retention if nobody is listening. SQS is pull, one consumer per message, with durable retention. The standard answer to “both” is the fan-out pattern: SNS topic with SQS queues subscribed to it, so each consumer gets its own durable buffer.
  • “Why put a queue between SNS and a consumer?” - retries and backpressure. A direct SNS-to-Lambda or SNS-to-HTTP subscription retries on a fixed schedule and then drops; an SQS queue holds the message until the consumer is healthy and gives you a dead-letter queue.
  • “FIFO or standard?” - standard for almost everything: higher throughput, at-least-once delivery, no ordering guarantee. FIFO when strict ordering within a group genuinely matters, at much lower throughput. Both mean you still design consumers to be idempotent.
  • “How do you filter?” - subscription filter policies on message attributes, so a subscriber only receives what it cares about. Filtering at the topic is cheaper than delivering everything and discarding in the consumer.