Soru

Zorluk: Çok zorHigh-Performing and Scalable Storage Solutions

A biotech company is running a distributed genomic analysis application on a cluster of 3232 Amazon EC2 instances. The application requires concurrent, shared access to a dataset that demands up to 80 GB/s80\text{ GB/s} read throughput and sub-millisecond latencies. The raw input data is stored in an Amazon S3 bucket, and the output files must be stored back in the same S3 bucket after the batch run completes. The storage solution must minimize both cost and operational overhead. Which storage configuration meets these requirements?

  1. Create an Amazon FSx for Lustre file system using a scratch deployment, link it to the Amazon S3 bucket as a data repository, and mount it on the instances.Cevap
  2. B
    Deploy an Amazon EFS file system configured with Elastic Throughput and mount it on the instances.
  3. C
    Provision an Amazon EBS io2 Block Express volume with Multi-Attach enabled, format it with a clustered file system, and attach it to all instances.
  4. D
    Provision an Amazon FSx for NetApp ONTAP file system with multi-pathing (nconnect) enabled and mount it on the instances using NFSv4.

Cevap

Create an Amazon FSx for Lustre file system using a scratch deployment, link it to the Amazon S3 bucket as a data repository, and mount it on the instances.
The correct option is to deploy an Amazon FSx for Lustre scratch file system integrated with the Amazon S3 bucket. FSx for Lustre is optimized for parallel workloads like high-performance computing (HPC) and genomic analysis. It easily scales to tens of gigabytes per second of throughput and millions of IOPS with sub-millisecond latencies. The scratch deployment option is highly cost-effective because it does not replicate data across availability zones, which is appropriate since the source and output data are stored durably in Amazon S3. The integration with S3 allows automated import and export of data with minimal operational effort.

Adım Adım Çözüm

1
Analyze the scaling and instance count requirements.
The workload requires concurrent access from 3232 EC2 instances, which rules out EBS Multi-Attach since it has a maximum limit of 1616 instances.
Eliminate options that cannot scale to the required number of concurrent writers/readers.
2
Evaluate the performance requirements.
The workload requires 80 GB/s80\text{ GB/s} of throughput and sub-millisecond latencies. Amazon EFS does not meet the 80 GB/s80\text{ GB/s} read throughput requirements economically or within standard limits.
Select a storage service designed for massive parallel file system performance.
3
Evaluate cost and operational complexity for the remaining solutions.
Amazon FSx for Lustre scratch deployment is specifically designed for temporary processing of data synced with Amazon S3. It provides high performance at a lower cost than persistent deployments by omitting replication, since the master copy of the data is safe in Amazon S3.
Determine the most cost-effective and operationally simple deployment model.

Anahtar Kavram

Selecting high-performance, cost-effective, and scalable shared storage for parallel processing workloads linked with Amazon S3.
Tahmini Süre:3m 0s
Bu soruyu puanla