Open source · Data + AI agents
Agentic data platforms on Amazon EKS.
Production-ready blueprints for Spark, Flink, Kafka and Trino, plus the agents, MCP servers and semantic layers that let AI work with your data. Build, deploy, and scale with confidence.
- Ready-to-use Data Stacks
- 30+
- GitHub Stars
- 800+
- Official Project
- AWS
Reference architecture
One platform, end to end.
From streaming ingest to GPU inference, every layer is an open-source blueprint on Amazon EKS, built on an open lakehouse, with AI agents working across it.
Spark troubleshooting agent
Reads Spark event logs through MCP, finds the bottleneck stage in failed or slow jobs and recommends a fix.
Acts on BatchSemantic layer · MCPKubeflow Semantic Operator
Governed semantic models as Kubernetes resources, so agents query business metrics instead of guessing at raw tables.
Acts on Query & serveStarRocks MCP · Amazon BedrockAgentic ad intelligence
Watches live ad metrics in StarRocks, investigates anomalies across every dimension and writes the root-cause report.
Acts on Query & serve01 Ingest
Events, CDC and logs
Real-time topics also feed Pinot, ClickHouse and StarRocks directly.
02 Stream
Parallel choices
↓ Writes to the lakehouse
03 Batch
Spark at scale
↕ Reads and writes the lakehouse
04 Query & serve
SQL, OLAP, search, BI and cache
↑ Reads from the lakehouse
05 AI inference
Batch, on GPUs
Reads Iceberg tables, writes predictions back and embeddings to S3 Vectors.
↕ Reads tables, writes results and embeddings
Open lakehouse
Tables, files and vectors on Amazon S3, governed by one catalog
The platform
What is Data on EKS?
Data on EKS is an open-source project for building agentic data platforms on Amazon EKS. It packages proven open-source data tools and AWS infrastructure as Terraform and ArgoCD blueprints, backed by performance benchmarks and best practices, so teams can run complex workloads across thousands of nodes with confidence. The agentic layer adds AI agents that investigate incidents and diagnose slow jobs, plus MCP servers and semantic layers that give AI structured access to your data.
Powered by Open Source & AWS
Production-ready data stacks on Amazon EKS
Agentic DataNew
Production-ready reference architectures for AI agents built for data platforms. Deploy working agents that investigate metric anomalies, diagnose slow or failing Spark jobs, and trace pipeline breaks, built with open source frameworks like LangGraph and AWS services like Amazon Bedrock AgentCore.
Data Analytics
Transform your data with enterprise-grade analytics solutions. Deploy Apache Spark, Ray, and Jupyter environments with production-ready configurations. Scale from terabytes to petabytes with confidence using battle-tested architectures.
Streaming Data Platforms
Build real-time data pipelines that never sleep. Process millions of events per second with Apache Kafka, Flink, and Kinesis. From IoT sensors to financial transactions, handle any streaming workload at any scale.
Amazon EMR on EKS
Run enterprise-grade Spark workloads on Kubernetes with Amazon EMR on EKS. Get optimized Spark runtime, automatic scaling, simplified job management, and seamless integration with AWS services for faster, more cost-effective big data processing.
Workflow Orchestration
Orchestrate complex data workflows with precision. Deploy Apache Airflow and Argo Workflows to automate ETL pipelines, ML training, and data quality checks. Never miss a dependency again.
Distributed Databases & Query Engines
Query anything, anywhere, anytime. Deploy Trino, Presto, and ClickHouse for lightning-fast analytics across data lakes, warehouses, and real-time streams. Join data across 50+ sources in milliseconds.
Watch
Featured videos
Ship your first data stack today.
Terraform and ArgoCD blueprints for Spark, Flink, Kafka, Trino, Airflow and more, plus AI agents for your data.








