Examkin

Question

Difficulty: HardHigh-Performing and Scalable Storage Solutions

A self-driving vehicle company is running deep learning model training workloads on a cluster of GPU-enabled Amazon EC2 instances. The training jobs process a shared dataset of 60 TB60\text{ TB} consisting of millions of small image files. The storage solution must provide a POSIX-compliant filesystem, support concurrent read and write access from all EC2 instances, deliver aggregate throughput of up to 15 GB/s15\text{ GB/s} with sub-millisecond latencies, and integrate directly with Amazon S3 to load and unload data.

Which storage solution should a solutions architect recommend to meet these requirements?

  1. Configure an Amazon FSx for Lustre SSD-backed file system linked to the Amazon S3 bucket containing the dataset.Answer
  2. B
    Deploy an Amazon EFS file system configured in Elastic Throughput mode and General Purpose performance mode.
  3. C
    Create an Amazon EBS gp3 volume, enable Amazon EBS Multi-Attach, and mount the volume on all EC2 instances.
  4. D
    Store the dataset in Amazon S3 Glacier Instant Retrieval and mount the bucket on the EC2 instances using the AWS File Gateway.

Answer

Configure an Amazon FSx for Lustre SSD-backed file system linked to the Amazon S3 bucket containing the dataset.
The correct option correctly recommends Amazon FSx for Lustre SSD-backed storage because it is optimized for high-performance computing (HPC) and deep learning workloads. It supports POSIX compliance, concurrent access from many GPU instances, sub-millisecond latencies, high throughput scaling, and native bi-directional integration with S3.

Step-by-Step Solution

1
Analyze the workload characteristics and requirements.
The workload requires a POSIX-compliant shared file system, concurrent read/write from multiple EC2 instances, very high aggregate throughput (15 GB/s15\text{ GB/s}), sub-millisecond latencies for processing millions of small files, and S3 integration.
Identifying these key requirements helps filter out block storage and standard general-purpose file storage systems.
2
Evaluate Amazon FSx for Lustre against the requirements.
FSx for Lustre provides a POSIX-compliant file system optimized for parallel computing, machine learning, and high-throughput workloads. SSD-backed deployment delivers sub-millisecond latencies. It integrates natively with S3 to load and write back data automatically.
This confirms that FSx for Lustre is the ideal architectural fit.
3
Evaluate and rule out EFS, EBS, and S3 Glacier options.
EFS does not meet the latency and throughput profile for millions of small files in GPU training. EBS gp3 does not support Multi-Attach. AWS File Gateway with S3 Glacier Instant Retrieval is not built for high-performance local cluster processing.
Ensuring other options are incorrect validates the final recommendation.

Key Concept

Selecting high-performance and scalable shared storage for ML/HPC workloads using Amazon FSx for Lustre.
Rate this question

Topics

High-Performing and Scalable Storage Solutions

More questions from this topic