A financial services firm is designing an analytical data platform to query historical audit logs. The platform must meet the following requirements:
- Data volume: of historical records stored in CSV format.
- Ingestion rate: Data is uploaded weekly in a single batch.
- Query latency: Data analysts run exploratory ad-hoc queries sporadically; results are expected within a few minutes.
- Resiliency: The data must survive a primary datacenter outage.
- Security: External auditors require access via Shared Access Signatures (SAS) that can be instantly revoked without rotating primary storage keys.
You need to design a query and storage solution that minimizes operational overhead and compute costs. Which solution should you recommend?
- AAzure Synapse Analytics dedicated SQL pools with the data loaded into dedicated database tables, stored in a Geo-Redundant Storage (GRS) Azure Data Lake Storage Gen2 account, using Shared Access Signatures (SAS) associated with a stored access policy.
- BAzure Synapse Analytics serverless SQL pools querying the data stored in a Locally Redundant Storage (LRS) Azure Data Lake Storage Gen2 account, using Shared Access Signatures (SAS) associated with a stored access policy.
- Azure Synapse Analytics serverless SQL pools querying the data stored in a Geo-Redundant Storage (GRS) Azure Data Lake Storage Gen2 account, using Shared Access Signatures (SAS) associated with a stored access policy.Answer
- DAzure Synapse Analytics serverless SQL pools querying the data stored in a Geo-Redundant Storage (GRS) Azure Data Lake Storage Gen2 account, using ad-hoc Shared Access Signatures (SAS) valid for one year without a stored access policy.
Answer
Azure Synapse Analytics serverless SQL pools querying the data stored in a Geo-Redundant Storage (GRS) Azure Data Lake Storage Gen2 account, using Shared Access Signatures (SAS) associated with a stored access policy.
Using Azure Synapse Analytics serverless SQL pools is the most cost-effective solution for ad-hoc, sporadic querying as you are only billed for the data processed by each query. Storing the data in a Geo-Redundant Storage (GRS) account satisfies the resiliency requirement to survive a primary datacenter outage. Utilizing Shared Access Signatures (SAS) associated with a stored access policy allows the security team to instantaneously revoke access to the external auditors by modifying or deleting the policy, without needing to rotate storage account keys.
Step-by-Step Solution
Key Concept
Data Integration and Analytical Storage