Question

Difficulty: HardData Store Operations with Amazon DynamoDB

A developer is designing a real-time notification service for a collaborative task management application. The application stores notifications in an Amazon DynamoDB table with the following schema:

- Partition key: `RecipientUserID` (String)
- Sort key: `NotificationTimestamp` (String)
- Attributes: `IsRead` (Boolean), `Message` (String)

The service needs to retrieve only the unread notifications for a specific user, sorted from newest to oldest. The developer implements a `Query` operation on the base table using a `KeyConditionExpression` of RecipientUserID=:userId\text{RecipientUserID} = \text{:userId} and a `FilterExpression` of IsRead=:false\text{IsRead} = \text{:false}.

As the number of read notifications per user grows over time, the application experiences latency spikes and frequently receives `ProvisionedThroughputExceededException` errors, even though the volume of unread notifications remains low. Which approach is the most cost-effective and performant way to optimize this read operation?

  1. Modify the application to write a new attribute `UnreadTimestamp` only for unread notifications, removing it when marked as read. Create a Global Secondary Index (GSI) with `RecipientUserID` as the partition key and `UnreadTimestamp` as the sort key to query unread notifications directly.Answer
  2. B
    Change the operation to a `Scan` on the base table with a `FilterExpression` of RecipientUserID=:userId\text{RecipientUserID} = \text{:userId} and IsRead=:false\text{IsRead} = \text{:false} to bypass partition-key sorting constraints, and scale up the table's provisioned Read Capacity Units (RCUs).
  3. C
    Keep the current base table query configuration and increase the provisioned Read Capacity Units (RCUs) on the table to handle the capacity consumed by the `FilterExpression`, as the error indicates overall table throughput limits have been reached.
  4. D
    Initialize the AWS SDK client inside the application code by hardcoding high-privilege IAM credentials with a dedicated policy that bypasses read capacity throttling for the target DynamoDB table.

Answer

Modify the application to write a new attribute `UnreadTimestamp` only for unread notifications, removing it when marked as read. Create a Global Secondary Index (GSI) with `RecipientUserID` as the partition key and `UnreadTimestamp` as the sort key to query unread notifications directly.
The correct answer utilizes a sparse Global Secondary Index (GSI). By writing the sort key attribute `UnreadTimestamp` only for unread notifications and removing it when they are read, the index only contains unread notifications. Querying this GSI returns only the relevant items, significantly reducing RCU consumption and preventing throughput throttling.

Step-by-Step Solution

1
Analyze base table query behavior and capacity consumption
DynamoDB consumes Read Capacity Units (RCUs) based on the size of all items returned by the key condition expression (RecipientUserID = :userId), before the filter expression (IsRead = :false) is applied. As read notifications grow, this results in high RCU consumption and eventual throttling.
To identify why the current implementation fails to scale and why increasing RCUs on the base table is not the correct solution.
2
Design a sparse index strategy to filter items at the storage layer
Create a schema modification where an attribute like `UnreadTimestamp` only exists when a notification is unread. When the notification is read, the attribute is deleted.
DynamoDB Global Secondary Indexes (GSIs) are sparse by default: they only contain items that possess both the GSI partition key and sort key. This ensures the index size remains minimal and only contains active unread items.
3
Configure the GSI keys and query the index
Define the GSI with `RecipientUserID` as the partition key and `UnreadTimestamp` as the sort key. Query the GSI using `RecipientUserID = :userId` to fetch only unread notifications sorted chronologically.
To retrieve the required dataset directly without wasting RCUs on read notifications, achieving optimal performance and cost efficiency.

Key Concept

Sparse Global Secondary Indexes in Amazon DynamoDB
Rate this question