Enterprise Data Platform Architecture Design (2026 Standard)
Design a comprehensive, cloud-native data platform for mid-to-large enterprises supporting real-time analytics, AI/ML pipelines, data mesh, and strong compliance across multi-cloud environments.
Prompt Content
Copy and paste directly into your model or internal evaluation tool.
You are a senior Data Platform Architect with 15+ years of experience in designing scalable data infrastructure, modern data stacks, and real-time analytics systems. Design a comprehensive data platform architecture for a mid-to-large enterprise (500+ employees, multi-cloud environment) that must support:
- Real-time analytics on streaming and batch data
- AI/ML model training and inference pipelines
- Strong data governance, lineage, and quality monitoring
- Multi-domain data mesh with federated ownership
- Cost-efficient storage tiering and compute optimization
- Compliance with data privacy regulations across regions
Deliverables include: Architecture Overview (component diagram in text/markdown, technology stack with justification, cloud deployment strategy), Data Ingestion Layer (batch patterns like CDC/ELT, streaming with Kafka/Pulsar, handling late data and exactly-once semantics), Storage & Lakehouse Design (Delta Lake vs Iceberg vs Hudi, medallion architecture, object storage optimization, hot/warm/cold tiering), Processing & Compute (batch/stream frameworks, SQL engine, autoscaling), AI/ML Integration (feature store, model training/serving, vector DB for RAG), Data Governance & Quality (catalog, lineage, quality checks, access control, PII masking), Data Mesh Implementation (decentralized ownership, self-serve platform, data contracts), Observability & Cost Management (monitoring, cost attribution, dashboards), Migration & Implementation Roadmap (phased migration, team structure, 6-18 month timeline), and Security & Compliance (encryption, network isolation, audit logging, cross-border data transfer).
Justify every technology choice with trade-off analysis, include concrete configuration examples, consider vendor lock-in vs. portability, address technical debt reduction and future extensibility, and include disaster recovery planning. Use Architecture Decision Records (ADRs) format for key decisions and describe diagrams in Mermaid or ASCII art where helpful.
Use Cases
Reference Output
The architecture should include: a lakehouse storage layer using Delta Lake or Apache Iceberg with domain-based Medallion architecture (bronze/silver/gold); event-driven ingestion via Kafka/Pulsar with Schema Registry; stream processing using Flink for stateful computations and exactly-once semantics; batch processing with Spark and Airflow orchestration; SQL query layer using Trino or DuckDB for BI tools; feature store using Feast or custom solution; model training integrated with MLflow for experiment tracking; vector database (e.g., Pinecone or Weaviate) for RAG; data catalog using DataHub or Apache Atlas; governance layer with column-level lineage and automated PII detection; ABAC-based access control; cost management via tagging and spot instance utilization; end-to-end encryption and private endpoints for security; implementation roadmap in three phases: POC (3 months), core domain migration (6 months), full rollout (12 months).
Scoring Rubric
Focus on evaluating executability, factual accuracy, boundary control, and structural completeness.
User Rating
0 ratingsYour rating
Log in to rate
Comments
0Log in to comment
Related Prompts
Product Marketing - Monochrome Avant-Garde Fashion Portrait
A high-fashion, monochrome editorial prompt for a sharp portrait with dramatic lighting and futuristic accessories, mimicking a luxury brand campaign.
Social Media Post - Magical Night Garden Fashion Portrait
A complex, high-quality prompt for a whimsical fantasy fashion editorial featuring glowing lights and a romantic atmosphere.
Social Media Post - Dreamy Woman in Wildflower Field
A cinematic, photorealistic prompt for a serene portrait of a woman in a field of daisies, emphasizing soft natural light and sharp focus on foreground details.
Social Media Post - Mediterranean Riviera Male Menswear
A comprehensive professional photography prompt for a sharp, high-contrast menswear editorial set against sun-drenched stone architecture.