Question

Difficulty: HardMessage-Based Integration using Amazon SQS and SNS

A developer is designing a message-based integration pattern for an e-commerce platform. Order event messages are published to an Amazon SNS topic, which fans them out to an Amazon SQS queue. A backend worker application polls the SQS queue and processes the events. The developer has two requirements for handling failed messages:

- If the SQS queue is temporarily unreachable or misconfigured (e.g., due to an incorrect IAM policy), any messages that SNS cannot deliver to the SQS queue must be captured for troubleshooting.
- If the backend worker application receives a message from SQS but fails to process it successfully after 33 attempts, the message must be safely set aside.

Which configuration should the developer implement to satisfy both requirements?

  1. Configure an Amazon SQS queue as a dead-letter queue (DLQ) for the Amazon SNS subscription to capture delivery failures, and configure another SQS queue as a DLQ for the main SQS queue with a redrive policy specifying a maxReceiveCount of 33.Answer
  2. B
    Configure a single Amazon SQS queue as a dead-letter queue (DLQ) on the main SQS queue, and set the SQS visibility timeout to at least three times the application processing time to allow SNS to automatically forward delivery failures to the same DLQ.
  3. C
    Configure an Amazon SQS queue as a dead-letter queue (DLQ) at the Amazon SNS topic level, and configure a redrive policy on the SQS queue with a maxReceiveCount of 33 pointing back to the SNS topic to retry delivery.
  4. D
    Configure an Amazon SQS queue as a dead-letter queue (DLQ) for the main SQS queue with a redrive policy specifying a maxReceiveCount of 33, which will automatically capture both worker processing failures and SNS delivery failures to the queue.

Answer

Configure an Amazon SQS queue as a dead-letter queue (DLQ) for the Amazon SNS subscription to capture delivery failures, and configure another SQS queue as a DLQ for the main SQS queue with a redrive policy specifying a maxReceiveCount of 33.
The correct answer is to configure two separate dead-letter queues. The SNS subscription DLQ is designed to capture messages that SNS fails to deliver to the SQS endpoint (due to client or server errors). The SQS queue-level DLQ (configured with a redrive policy and a maxReceiveCount of 33) handles messages that were successfully delivered to the queue but failed to be processed and deleted by the worker application.

Step-by-Step Solution

1
Analyze the requirement for SNS-to-SQS delivery failures.
Identify that if SNS cannot write to SQS (due to permissions, deleted queue, etc.), this represents an SNS delivery failure. SNS handles this by forwarding messages to a DLQ configured on the subscription itself.
To ensure messages that fail to reach the queue are not lost.
2
Analyze the requirement for SQS consumer-side processing failures.
Identify that if the backend worker retrieves a message but fails to process it 33 times, this is a consumer processing failure. This is handled by SQS using a redrive policy with maxReceiveCount set to 33 that points to an SQS DLQ.
To prevent poison-pill messages from blocking the queue processing indefinitely.
3
Synthesize the architecture.
Combine an SNS subscription-level DLQ with an SQS queue-level DLQ to address both distinct failure vectors independently.
To satisfy both requirements using the native capabilities of each service.

Key Concept

Handling message delivery and processing failures in SNS-SQS fan-out integrations using distinct dead-letter queue (DLQ) mechanisms.
Rate this question