Question

Difficulty: HardHigh-Performing and Scalable Storage Solutions

A research institution is deploying a distributed high-performance computing (HPC) pipeline on AWS for genomic sequencing. The pipeline is deployed across a fleet of Amazon EC2 instances. The pipeline requires a shared storage solution that supports concurrent read and write access from hundreds of instances, delivers sub-millisecond latency, and scales throughput up to 20 GB/s20\text{ GB/s}. Additionally, the solution must automatically load raw input datasets from a designated Amazon S3 bucket when files are first accessed, and export output results back to the S3 bucket upon pipeline completion.

Which combination of steps should the solutions architect take to meet these performance and integration requirements? (Choose two.)

  1. Create an Amazon FSx for Lustre file system with a data repository association linked to the Amazon S3 bucket.Answer
  2. Mount the FSx for Lustre file system on the EC2 instances using the Lustre client.Answer
  3. C
    Provision an Amazon EBS io2 volume with Multi-Attach enabled, format it with a clustered file system, and mount it to the instances.
  4. D
    Create an Amazon Elastic File System (Amazon EFS) file system with Elastic throughput mode, and link it to the S3 bucket using an AWS Storage Gateway.
  5. E
    Deploy an Amazon FSx for OpenZFS file system, configure a data repository association with the S3 bucket, and mount it using NFSv4.

Answer

Create an Amazon FSx for Lustre file system with a data repository association linked to the Amazon S3 bucket, and mount the file system on the EC2 instances using the Lustre client.
The combination of creating an Amazon FSx for Lustre file system with a data repository association and mounting it using the Lustre client meets all criteria. FSx for Lustre is designed for HPC workloads, supporting parallel client access, throughput scaling to tens of gigabytes per second, and sub-millisecond latency. The data repository association automates the ingestion from and exportation to Amazon S3.

Step-by-Step Solution

1
Select the appropriate storage type for high-performance computing (HPC) with S3 integration.
Amazon FSx for Lustre is selected because it is a parallel file system that supports sub-millisecond latencies, scales to hundreds of gigabytes per second of throughput, and integrates directly with Amazon S3.
Genomic sequencing requires high-throughput parallel access (20 GB/s20\text{ GB/s}) and seamless S3 synchronization, which FSx for Lustre natively provides.
2
Configure the S3 bucket integration using Data Repository Associations (DRA).
A Data Repository Association is configured between the Amazon FSx for Lustre file system and the designated Amazon S3 bucket.
This configuration allows the file system to lazily load raw input datasets from S3 when first requested and to automatically export outputs back to S3.
3
Mount the file system on the EC2 instances using the correct protocol.
The file system is mounted on the EC2 instances using the dedicated Lustre client.
The Lustre client is required to connect to the FSx for Lustre parallel file system and achieve the high-throughput performance characteristics.

Key Concept

Amazon FSx for Lustre provides high-performance parallel file systems that integrate directly with Amazon S3 using Data Repository Associations, enabling high-performance computing (HPC) applications to process large datasets at low latencies.
Estimated Time:2m 30s
Rate this question