Examkin

Question

Difficulty: HardData Integration and Analytical Storage

A global media streaming platform is designing an analytical pipeline to process and analyze content viewing sessions. The platform requires a solution to store and query historical logs with the following characteristics:

* Daily batch uploads of session log files in Parquet format, totaling 5 TB5\text{ TB} of new data per day.
* A total of 800 TB800\text{ TB} of historical logs stored in a central repository.
* Data scientists must run sporadic, ad-hoc exploratory queries to analyze viewing trends, which can tolerate a query latency of up to 30 seconds30\text{ seconds}.
* The architecture must minimize administrative overhead and reduce costs by avoiding continuously running compute resources.
* The storage solution must remain resilient and survive a datacenter outage in the primary region.

Which two components should you include in the storage and query design?

  1. An Azure Synapse Analytics serverless SQL pool to query the Parquet files directlyAnswer
  2. B
    An Azure Synapse Analytics dedicated SQL pool to store and index the data
  3. An Azure Data Lake Storage Gen2 account configured with Geo-Redundant Storage (GRS)Answer
  4. D
    An Azure Data Lake Storage Gen2 account configured with Locally Redundant Storage (LRS)

Answer

The optimal design includes an Azure Synapse Analytics serverless SQL pool and an Azure Data Lake Storage Gen2 account configured with Geo-Redundant Storage (GRS).
The combination of an Azure Synapse Analytics serverless SQL pool and a Geo-Redundant Storage (GRS) account meets all criteria. The serverless SQL pool allows querying Parquet files in place without maintaining active compute clusters, eliminating idle costs and reducing management overhead. GRS replicates the logs to a secondary region to satisfy the requirement to survive a primary datacenter outage.

Step-by-Step Solution

1
Analyze the query and compute requirements.
The requirements specify sporadic, ad-hoc queries with up to 30 seconds30\text{ seconds} of latency and a constraint to avoid continuously running compute while minimizing administrative overhead.
This points to a serverless query engine rather than a dedicated persistent cluster.
2
Select the query engine.
Azure Synapse Analytics serverless SQL pool is selected because it queries files directly in the data lake, scales dynamically, and bills only for data processed, eliminating idle compute costs.
Dedicated SQL pools require persistent running compute or active management of pause/resume states, which increases overhead and costs.
3
Analyze the storage and resiliency requirements.
The solution must survive a datacenter outage in the primary region.
This rules out Locally Redundant Storage (LRS) and requires a geo-redundant storage option.
4
Select the storage redundancy option.
Azure Data Lake Storage Gen2 with Geo-Redundant Storage (GRS) is selected.
GRS replicates data asynchronously to a secondary region, providing protection against a regional datacenter failure.

Key Concept

Selecting cost-effective analytical query engines and appropriate storage redundancy levels in Azure data architectures.
Rate this question

Topics

Data Integration and Analytical Storage

More questions from this topic