Question

Difficulty: MediumMessage-Based Integration using Amazon SQS and SNS

A developer is using an AWS Lambda function to process messages from an Amazon SQS standard queue. The Lambda function processes each message by calling a third-party API, which usually takes 10 seconds10\text{ seconds} but can take up to 50 seconds50\text{ seconds} under high load. The SQS queue's visibility timeout is configured to 30 seconds30\text{ seconds}, and the Lambda function's timeout is set to 60 seconds60\text{ seconds}. The developer notices that when the third-party API is slow, messages are processed multiple times, resulting in duplicate database entries. Which change should the developer make to resolve this duplicate processing issue?

  1. Increase the SQS queue's visibility timeout to 360 seconds360\text{ seconds} or more.Answer
  2. B
    Decrease the SQS queue's visibility timeout to 10 seconds10\text{ seconds} to match the average API response time.
  3. C
    Increase the Lambda function's timeout to 300 seconds300\text{ seconds} while keeping the SQS visibility timeout at 30 seconds30\text{ seconds}.
  4. D
    Initialize the AWS SDK client inside the Lambda handler using hardcoded AWS access keys.

Answer

Increase the SQS queue's visibility timeout to 360 seconds360\text{ seconds} or more.
The correct answer is to increase the SQS queue's visibility timeout to 360 seconds360\text{ seconds} or more. If the queue's visibility timeout is shorter than the Lambda function's timeout, SQS will make the message visible to other consumers while the active execution is still running. AWS recommends configuring the queue's visibility timeout to at least 6 times the Lambda function's timeout to ensure proper handling and retries.

Step-by-Step Solution

1
Analyze the relationship between the SQS visibility timeout and the Lambda function timeout.
The current SQS visibility timeout (30 seconds30\text{ seconds}) is less than the Lambda function's timeout (60 seconds60\text{ seconds}) and the maximum API response time (50 seconds50\text{ seconds}).
When the visibility timeout is shorter than the execution time, SQS assumes the consumer failed and makes the message visible to other consumers before the current Lambda function finishes, causing duplicates.
2
Apply the AWS recommended best practice for SQS-Lambda integrations.
Determine that the SQS visibility timeout should be at least 6 times the Lambda function's timeout.
Setting the visibility timeout to at least 6×Lambda timeout6 \times \text{Lambda timeout} (6×60 seconds=360 seconds6 \times 60\text{ seconds} = 360\text{ seconds}) prevents concurrent processing of the same message and allows adequate time for processing and potential retries.

Key Concept

SQS Visibility Timeout vs. Consumer Processing Time
Rate this question