Skip to main content

ClickHouse Infrastructure Deployment

ClickHouse on EKS Stack​

ClickHouse is a high-performance, column-oriented SQL database management system (DBMS) for online analytical processing (OLAP) that is open sourced under the Apache 2.0 license.

OLAP is software technology you can use to analyze business data from different points of view. Organizations collect and store data from multiple data sources, such as websites, applications, smart meters, and internal systems. OLAP helps organizations process and benefit from a growing amount of information by combining and groups this data into categories to provide actionable insights for strategic planning. For example, a retailer stores data about all the products it sells, such as color, size, cost, and location. The retailer also collects customer purchase data, such as the name of the items ordered and total sales value, in a different system. OLAP combines the datasets to answer questions such as which color products are more popular or how product placement impacts sales.

Some key benefits of Clickhouse include:

  • Real-Time Analytics: ClickHouse can handle real-time data ingestion and analysis, making it suitable for use cases such as monitoring, logging, and event data processing.
  • High Performance: ClickHouse is optimized for analytical workloads, providing fast query execution and high throughput.
  • Scalability: ClickHouse is designed to scale horizontally across multiple nodes, allowing users to store and process petabytes of data across a distributed cluster. It supports sharding and replication for high availability and fault tolerance.
  • Column-Oriented Storage: ClickHouse organizes data by columns rather than rows, which allows for efficient compression and faster query processing, especially for queries that involve aggregations and scans of large datasets.
  • SQL Support: ClickHouse supports a subset of SQL, making it familiar and easy to use for developers and analysts who are already familiar with SQL-based databases.
  • Integrated Data Formats: ClickHouse supports various data formats, including CSV, JSON, Apache Avro, and Apache Parquet, making it flexible for ingesting and querying different types of data.

Getting Started​

This stack provisions the platform layer for running ClickHouse on EKS: an EKS cluster with Karpenter for node autoscaling, ArgoCD for GitOps application management, and the ClickHouse Kubernetes operator ready to manage ClickHouseCluster and KeeperCluster custom resources.

./deploy.sh does not deploy an actual ClickHouse cluster — it only provisions the operator. To stand up a sharded, replicated cluster and run analytical queries, follow the Sample Workload — Hits Dataset guide once this infrastructure is up.

Architecture​

EKS Cluster
controllermanaged resource
GitOps
ArgoCD
deploys
Applications
Karpenter
ClickHouse Operator
Other add-ons
provisions
reconciles
Managed Resources
Compute · Karpenter
EC2 nodes (on-demand & spot)
Workload · ClickHouse Operator
StatefulSet
Service
ConfigMap
PersistentVolumeClaim
  • Amazon EKS Cluster: The core Kubernetes environment that hosts everything in the diagram.
  • ArgoCD: Deploys all Applications — including Karpenter, the ClickHouse operator, and other platform add-ons — in a GitOps fashion.
  • Karpenter: Provisions EC2 nodes on demand when pods are scheduled (using nodeSelector constraints).
  • ClickHouse Operator: Watches ClickHouseCluster (and KeeperCluster) custom resources and reconciles them into the underlying Kubernetes resources — StatefulSet, ConfigMap, Service, PersistentVolumeClaim, etc.

Deploy the stack​

Clone the repository, then run the deployment script:

export CLICKHOUSE_DIR=$(git rev-parse --show-toplevel)/data-stacks/clickhouse-on-eks
cd $CLICKHOUSE_DIR
./deploy.sh

Deployment will take ~20 minutes to complete.

Verify deployment​

The deployment script displays ArgoCD credentials at the end. Access the UI:

# Port forward ArgoCD server
export KUBECONFIG=$CLICKHOUSE_DIR/kubeconfig.yaml
kubectl port-forward svc/argocd-server -n argocd 8080:443

Open https://localhost:8080 in your browser:

  • Username: admin
  • Password: Displayed at end of deploy.sh output
info

It may take an additional ~5 minutes for all applications (including the ClickHouse operator) to show Synced and Healthy status.

ArgoCD Applications

Next Steps​

With infrastructure deployed, you can now run any ClickHouse example:

Cleanup​

To remove all resources, use the dedicated cleanup script:

# Navigate to stack directory
cd $CLICKHOUSE_DIR

# Run cleanup script
./cleanup.sh
warning

This command will delete all resources and data. Make sure to backup any important data first.

note

If cleanup fails:

  • Rerun the same command: ./cleanup.sh
  • Keep rerunning until all resources are deleted
  • Some AWS resources may have dependencies that require multiple cleanup attempts