Examkin

Question

Difficulty: HardMessage-Based Integration using Amazon SQS and SNS

A developer is building a video translation system where upload events are published to an Amazon SNS topic. The events are fanned out to an Amazon SQS standard queue, which is polled by a fleet of containerized workers running on Amazon ECS. Each video transcription job takes between 2 to 5 minutes to complete, but occasionally takes up to 12 minutes for larger files. The default visibility timeout of the SQS queue is configured to 3 minutes. During peak traffic, the developer notices that some videos are transcribed multiple times, resulting in duplicate outputs and wasted compute resources.

Which of the following actions should the developer take to resolve this issue in a secure and efficient manner?

  1. A
    Decrease the default visibility timeout of the SQS queue to 1 minute to ensure that if a worker fails, another worker immediately picks up the message.
  2. B
    Migrate the transcription logic to an AWS Lambda function with the default timeout, relying on the reuse of the execution context to persist and complete the 12-minute transcription across separate invocations.
  3. Modify the ECS worker to call the ChangeMessageVisibility API to dynamically extend the message's visibility timeout during transcription, or increase the default visibility timeout of the SQS queue to 13 minutes.Answer
  4. D
    Implement a helper in the ECS worker to call the ChangeMessageVisibility API, initializing the SQS client by hardcoding an IAM User's access key and secret access key in the source code to ensure proper authentication.

Answer

Modify the ECS worker to call the ChangeMessageVisibility API to dynamically extend the message's visibility timeout during transcription, or increase the default visibility timeout of the SQS queue to 13 minutes.
The correct option addresses the duplicate processing issue by ensuring that the SQS message remains invisible to other consumers while it is actively being processed. By increasing the SQS queue's default visibility timeout to 13 minutes (which is greater than the maximum potential processing time of 12 minutes) or using the ChangeMessageVisibility API to dynamically extend the visibility of a message while work is in progress, the developer prevents other workers from picking up and processing the same message concurrently.

Step-by-Step Solution

1
Analyze the processing times of the ECS worker against the default SQS visibility timeout.
The workers take up to 12 minutes, which is significantly longer than the 3-minute default visibility timeout.
When the processing duration exceeds the visibility timeout, SQS makes the message visible again, allowing other workers to retrieve it and cause duplicate execution.
2
Determine the correct visibility timeout configuration or runtime API calls needed.
The queue's default visibility timeout should be set to at least 13 minutes, or the worker must issue ChangeMessageVisibility calls to heart-beat the message's visibility during execution.
This ensures that no other consumer can pull the message until the current worker either finishes and deletes it, or fails and lets the timeout expire.
3
Evaluate the security and architecture constraints of the proposed solutions.
The SDK client should be configured to run securely using IAM Task Roles instead of hardcoded credentials, and avoiding incorrect assumptions about Lambda default timeouts and execution context reuse.
Using IAM roles for tasks ensures secure access to SQS, and avoiding default Lambda timeouts prevents runtime failures.

Key Concept

Understanding and configuring SQS visibility timeouts to align with application processing times, and dynamically managing message visibility using the AWS SDK.
Rate this question

Topics

Message-Based Integration using Amazon SQS and SNS

More questions from this topic