A company wants to design a serverless data ingestion pipeline to collect real-time application logs. The pipeline must transform the log format from JSON to Apache Parquet and store the transformed data in Amazon S3 for long-term analytics. The solution must minimize operational overhead. Which TWO actions should the solutions architect take to meet these requirements? (Select TWO.)
- Create an Amazon Data Firehose stream to ingest the logs and enable data format conversion using the AWS Glue Data Catalog.Cevap
- Configure the Amazon Data Firehose stream to deliver the transformed logs directly to the destination Amazon S3 bucket.Cevap
- CIngest the log events into an Amazon SQS standard queue and use an AWS Lambda function to read and order the logs before writing them to Amazon S3.
- DIngest the logs using a single-shard Amazon Kinesis Data Stream and write a custom consumer application to transform and upload the data.
- EDeliver the raw JSON logs directly to Amazon S3 Glacier Flexible Retrieval and use Amazon Athena to perform real-time SQL queries.
Cevap
The correct solution involves creating an Amazon Data Firehose stream to ingest logs and perform format conversion using the AWS Glue Data Catalog, and configuring the stream to deliver the transformed logs directly to the target Amazon S3 bucket.
To design a serverless ingestion pipeline with minimum operational overhead, Amazon Data Firehose is the optimal service because it is fully managed, auto-scales, and provides built-in integration with AWS Glue Data Catalog to convert JSON data to Apache Parquet. Once transformed, it can natively write the files directly to an Amazon S3 bucket without requiring any custom consumer applications or compute resources.
Adım Adım Çözüm
Anahtar Kavram
Serverless real-time data ingestion and inline transformation using Amazon Data Firehose and AWS Glue.
Tahmini Süre:1m 0s