📄️ Trino on EKS Best Practices
Trino deployment on Amazon Elastic Kubernetes Service (EKS) delivers distributed query processing with cloud-native scalability. Organizations can optimize costs by selecting specific compute instances and storage solutions that match their workload requirements while they combine the power of Trino with the scalability and flexibility of EKS using Karpenter.
📄️ EMR on EKS Best Practices
EMR Containers Best Practices Guides
📄️ Spark on EKS Best Practices
This page is a practical guide to running Apache Spark on Amazon Elastic Kubernetes Service (EKS). It focuses on the decisions that most affect cost, performance, and stability as your workloads grow.
📄️ Apache Celeborn Best Practices
Apache Celeborn is a Remote Shuffle Service (RSS). It moves Spark shuffle data off executor disks onto dedicated worker nodes, which lets you use true dynamic allocation and removes the local-disk shuffle bottleneck at scale.
📄️ Preventing OOM Kills in Spark on EKS
Every organization running large scale Spark workloads on Kubernetes has dealt with this: a job runs for hours, processes terabytes of data, completes 80% of its work, and then executors start disappearing. No JVM exception. No heap dump. No warning in Spark UI. Just exit code 137 and hours of compute burned. The standard response is to throw more memory at it, bump memoryOverhead by another 10 GB, and hope for the best. That works until the next data spike.
📄️ In-Place EBS Volume Modification for Stateful Workloads
Almost every stateful data workload on EKS sits on EBS. Celeborn shuffle workers, Valkey cluster nodes, Kafka brokers, Trino spill, Pinot servers, ClickHouse, Starrocks. They all run as StatefulSets where each pod owns a PersistentVolumeClaim backed by an EBS volume. The capacity and IOPS that looked right when the cluster was first provisioned rarely stay right for long. This page is about modifying those volumes in place, while the pods keep serving traffic, with zero restarts and zero cluster impact.
📄️ Faster Spark Image Pulls with SOCI
Spark on Kubernetes has an image pull problem that most teams discover the hard way. Spark images are big. Whether you run the open source apache/spark image, Amazon EMR on EKS Spark, or a custom image built on either, 3 to 5 GB compressed is common once you layer in connectors, application jars, and Python dependencies. Every time Karpenter provisions a fresh node for a burst of executors, the first pod on that node has to download and unpack the full image before a single task runs.