An enterprise needs to migrate a 2.4\text{ PB} Hadoop Distributed File System (HDFS) datas… — AWS Certified Solutions Architect - Professional

An enterprise needs to migrate a $2.4\text{ PB}$ Hadoop Distributed File System (HDFS) dataset to an Amazon S3 bucket. The enterprise has a $1\text{ Gbps}$ dedicated network connection, but only a maximum of $400\text{ Mbps}$ can be allocated for migration activities to avoid impacting daily operations. The migration must be completed within $45$ days. During the migration, the source dataset is actively modified on-premises. The final migrated data in Amazon S3 must be securely accessed by an external analytics partner operating in a separate AWS account, and all data must be encrypted at rest. Which architecture strategy will migrate the dataset within the timeline and satisfy the security requirements?

Order multiple AWS Snowball Edge Storage Optimized devices. Copy the bulk data from the HDFS NFS Gateway to the Snowball Edge devices, encrypting the data with an AWS KMS Customer Managed Key (CMK). Ship the devices back to AWS. After the bulk data is imported into Amazon S3, deploy AWS DataSync agents on-premises and configure a DataSync task with an HDFS source location to transfer the incremental changes to the S3 bucket over the network. Configure the S3 bucket policy and the KMS CMK key policy to grant read access to the partner's AWS account.Answer
B
Deploy AWS DataSync agents on-premises, configure the source location to point to the HDFS cluster, and configure the target to point to the Amazon S3 bucket. Encrypt the bucket using an AWS KMS Customer Managed Key (CMK). Execute the DataSync task to transfer the entire $2.4\text{ PB}$ dataset over the dedicated network connection using the allocated $400\text{ Mbps}$ bandwidth, then keep the source and target synchronized. Grant read access to the partner's AWS account using the KMS CMK key policy.
C
Order multiple AWS Snowball Edge Storage Optimized devices. Copy the bulk data from the HDFS NFS Gateway to the Snowball Edge devices, encrypting the data using the default AWS managed key (`aws/s3`). Ship the devices back to AWS. After the data is imported, deploy AWS DataSync agents on-premises to sync incremental changes from HDFS to Amazon S3 over the network. Grant read access to the external partner's AWS account by modifying the S3 bucket policy.
D
Order multiple AWS Snowball Edge Storage Optimized devices. Copy the bulk data from the HDFS NFS Gateway to the Snowball Edge devices, encrypting with an AWS KMS Customer Managed Key (CMK). After the bulk import, configure AWS DataSync to sync changes from on-premises to S3 VPC endpoints in multiple VPCs. Route this DataSync traffic transitively across the VPCs using a Direct Connect Gateway directly connected to the on-premises network, without using AWS Transit Gateway. Configure the KMS CMK key policy to grant access to the partner's account.

Answer

The strategy that uses multiple AWS Snowball Edge Storage Optimized devices for the initial bulk HDFS data copy, utilizes AWS DataSync to sync HDFS incremental changes over the network, and encrypts the data using an AWS KMS Customer Managed Key (CMK) to allow cross-account access to the partner's account.

The correct strategy combines offline transfer via multiple AWS Snowball Edge Storage Optimized devices for the initial bulk data and online replication via AWS DataSync to copy incremental changes from the HDFS cluster over the network. Since

2.4\text{ PB}

cannot be transferred over

400\text{ Mbps}

within

45

days, the bulk must go offline. After import, the DataSync agent reads from the HDFS gateway to capture updates. Furthermore, a Customer Managed Key (CMK) is required because AWS-managed keys cannot be shared with external AWS accounts.

Step-by-Step Solution

Calculate the transfer duration for the

2.4\text{ PB}

dataset over the allocated

400\text{ Mbps}

network bandwidth.

The calculation (

2.4\text{ PB} / 400\text{ Mbps}

) results in a transfer time of over

550

days, which exceeds the

45

-day window, indicating that physical transfer (AWS Snowball Edge) is required for the bulk phase.

To determine whether an online or offline migration strategy is feasible under the given network constraints.

Address the requirement to transfer active updates during the migration period.

AWS DataSync is selected to sync incremental HDFS changes over the network post-bulk import, leveraging its native support for HDFS source locations.

To capture and replicate the on-premises changes to AWS while the bulk data is being shipped and ingested.

Select the correct encryption key configuration for cross-account data sharing.

An AWS KMS Customer Managed Key (CMK) is used, and its key policy is modified to grant decryption rights to the partner's AWS account.

AWS-managed keys (such as `aws/s3`) cannot be used for cross-account decryption because their key policies cannot be customized to grant access to external accounts.

Key Concept

Combining offline Snowball Edge devices for bulk data transfer with online AWS DataSync HDFS replication for active incremental changes, while using KMS Customer Managed Keys to facilitate cross-account resource sharing.

Question

Answer

Step-by-Step Solution

Key Concept

Common Mistakes

Topics

More questions from this topic