Question

Difficulty: Very hardHigh-Performing and Scalable Storage Solutions

A financial analytics platform runs a simulation engine on a fleet of 120 Linux EC2 instances. The simulation requires concurrent, shared access to a POSIX-compliant file system. The instances must read and write transient datasets simultaneously. During peak workloads, the simulation engine generates an aggregate write throughput of 6 GB/s6\text{ GB/s} (6,000 MB/s6,000\text{ MB/s}) and requires sub-millisecond metadata latencies. Which storage solution meets these performance requirements with the lowest operational complexity?

  1. A
    Amazon EFS file system configured with Elastic Throughput and mounted on the instances using the EFS mount helper.
  2. B
    Amazon EBS gp3 volume with Multi-Attach enabled, formatted with a clustered file system like GFS2 and mounted on the instances.
  3. Amazon FSx for Lustre scratch file system configured with the required capacity to deliver the target throughput, mounted on the instances using the Lustre client.Answer
  4. D
    Amazon S3 bucket mounted on the instances using Mountpoint for Amazon S3 to support concurrent write workloads.

Answer

Amazon FSx for Lustre scratch file system configured with the required capacity to deliver the target throughput, mounted on the instances using the Lustre client.
The correct answer is the Amazon FSx for Lustre scratch file system. Amazon FSx for Lustre is specifically engineered for high-performance computing (HPC) workloads and transient datasets, offering sub-millisecond latencies and the ability to scale throughput to tens of gigabytes per second. It provides a native POSIX-compliant interface and allows hundreds of Linux instances to access the shared data concurrently.

Step-by-Step Solution

1
Analyze workload storage requirements.
Identify key requirements: concurrent read/write from 120 Linux instances, POSIX compliance, 6 GB/s6\text{ GB/s} write throughput, sub-millisecond latencies, and transient datasets.
Establishing the storage capabilities checklist eliminates non-conforming services.
2
Evaluate EBS Multi-Attach limitations.
EBS gp3 does not support Multi-Attach. EBS io1/io2 volumes support Multi-Attach but are restricted to 16 instances, failing the 120-instance requirement.
This rules out EBS-based block storage solutions.
3
Evaluate EFS throughput capabilities.
Amazon EFS write throughput caps out at 3 GiB/s3\text{ GiB/s} under Elastic Throughput mode, which is below the needed 6 GB/s6\text{ GB/s}.
This rules out Amazon EFS as a high-performance candidate for this write-heavy load.
4
Evaluate Amazon S3 mounting solutions.
Mountpoint for Amazon S3 is not a fully POSIX-compliant interface and lacks support for concurrent writes and file locking.
This rules out S3-based options for concurrent writing simulation nodes.
5
Select the correct FSx deployment.
Amazon FSx for Lustre scratch file systems are optimized for transient, high-performance workloads, supporting tens of GB/s of throughput and sub-millisecond latencies with POSIX compliance.
FSx for Lustre is the only solution that satisfies all throughput, latency, concurrency, and compliance requirements.

Key Concept

Selecting high-performance shared storage solutions based on throughput scale, protocol compliance, and concurrency constraints.
Rate this question