A machine learning platform runs distributed training jobs on a cluster of Amazon ECS tasks running on AWS Fargate. During training, the tasks must concurrently read and write millions of small training images and weights (averaging each) from a shared namespace. The workload requires consistent single-digit millisecond latency and must scale to support up to transactions per second (TPS). The data is transient and can be easily regenerated if an Availability Zone failure occurs. Which storage architecture will meet these performance requirements with the lowest operational complexity and latency?
- Configure Amazon S3 Express One Zone directory buckets to store the training data, and access it directly from the ECS tasks using the S3 API.Cevap
- BProvision an Amazon EBS io2 Block Express volume, enable Multi-Attach, and mount the volume across the ECS tasks running on AWS Fargate.
- CCreate an Amazon EFS file system configured with Elastic throughput and General Purpose performance mode, and mount it to the ECS tasks.
- DDeploy an Amazon FSx for Lustre file system using a persistent SSD deployment type, and mount the file system to the ECS tasks.
Cevap
Configure Amazon S3 Express One Zone directory buckets to store the training data, and access it directly from the ECS tasks using the S3 API.
The correct option is to use Amazon S3 Express One Zone directory buckets. It is purpose-built to deliver consistent, single-digit millisecond latency for performance-critical applications and can scale to hundreds of thousands of transactions per second. Since the ECS tasks are running on AWS Fargate, using the standard S3 API avoids the operational complexity of mounting file systems and requires no OS-level client configuration.
Adım Adım Çözüm
Anahtar Kavram
Selecting high-performance and low-latency storage solutions for containerized workloads on AWS Fargate.