Question

Difficulty: MediumDatabase and Storage Strategy

A company is designing a high-throughput event processing platform to ingest JSON telemetry logs from 50,00050,000 IoT devices. The platform must support two primary requirements:

1. Low-latency, single-digit millisecond writes to ingest telemetry logs, which must be stored for real-time dashboarding. The telemetry data must automatically expire after 3030 days to minimize storage costs.
2. An audit compliance mandate requires raw logs to be archived for 77 years. These archives must be queryable via standard SQL on-demand, without provisioning permanent database compute resources.

Which two database and storage strategies will satisfy these requirements?

  1. Use Amazon DynamoDB to ingest and store the active telemetry logs, and enable Time to Live (TTL) to automatically delete items older than 3030 days.Answer
  2. Use Amazon S3 to store the raw logs, configure an S3 Lifecycle policy to transition the logs to Amazon S3 Glacier Flexible Retrieval after 3030 days, and query the archived logs on-demand using Amazon Athena.Answer
  3. C
    Use Amazon Aurora Global Database to ingest the telemetry logs, and configure Aurora Auto Scaling to dynamically provision reader instances in the primary Region to handle the write-heavy spike loads.
  4. D
    Use Amazon S3 to store the raw logs, encrypt them using the AWS-managed KMS key (aws/s3), and modify the KMS key policy to delegate read permissions to a cross-account audit role.
  5. E
    Use Amazon ElastiCache for Memcached to ingest the active telemetry logs, and configure multi-AZ replication to ensure data persistence and high availability for the 3030-day window.

Answer

Use Amazon DynamoDB to ingest and store active logs with Time to Live (TTL) enabled, and use Amazon S3 to archive raw logs with an S3 Lifecycle policy transitioning them to Amazon S3 Glacier Flexible Retrieval while querying them via Amazon Athena.
The correct strategy uses Amazon DynamoDB to handle the high-write, low-latency telemetry ingestion, utilizing Time to Live (TTL) to automatically expire data after 3030 days at no extra cost. For long-term archiving, Amazon S3 coupled with a Lifecycle policy to transition logs to S3 Glacier Flexible Retrieval offers the most cost-effective solution, while Amazon Athena allows serverless, on-demand SQL queries without the need for active database compute resources.

Step-by-Step Solution

1
Evaluate the requirements for active telemetry storage.
The active store must handle high-throughput, low-latency writes and automatically delete items after 3030 days.
Amazon DynamoDB is selected because it scales horizontally to support high write throughput with single-digit millisecond latency, and offers built-in TTL to automatically expire data at no additional cost.
2
Evaluate the requirements for long-term archiving and auditing.
Raw logs must be stored cost-effectively for 77 years and remain queryable via SQL on-demand without provisioning server resources.
Amazon S3 combined with S3 Lifecycle policies allows archiving raw logs to a cold storage class like S3 Glacier Flexible Retrieval, reducing cost. Amazon Athena provides a serverless SQL query capability directly over the archived data.
3
Analyze and eliminate incorrect options based on AWS service features.
Identify flaws: Aurora reader scaling does not help write workloads; AWS-managed KMS key policies cannot be modified for cross-account access; ElastiCache for Memcached does not support replication or durability.
Eliminating options that introduce single points of failure, administrative limitations, or incorrect engine behaviors ensures the design is both viable and optimal.

Key Concept

Selecting and integrating optimal database and storage services based on performance, cost-effective archiving, and query requirements.
Rate this question