Skip to main content

Data Analytics

📄️ Preventing OOM Kills in Spark on EKS

Every organization running large scale Spark workloads on Kubernetes has dealt with this: a job runs for hours, processes terabytes of data, completes 80% of its work, and then executors start disappearing. No JVM exception. No heap dump. No warning in Spark UI. Just exit code 137 and hours of compute burned. The standard response is to throw more memory at it, bump memoryOverhead by another 10 GB, and hope for the best. That works until the next data spike.

📄️ In-Place EBS Volume Modification for Stateful Workloads

Almost every stateful data workload on EKS sits on EBS. Celeborn shuffle workers, Valkey cluster nodes, Kafka brokers, Trino spill, Pinot servers, ClickHouse, Starrocks. They all run as StatefulSets where each pod owns a PersistentVolumeClaim backed by an EBS volume. The capacity and IOPS that looked right when the cluster was first provisioned rarely stay right for long. This page is about modifying those volumes in place, while the pods keep serving traffic, with zero restarts and zero cluster impact.

📄️ Faster Spark Image Pulls with SOCI

Spark on Kubernetes has an image pull problem that most teams discover the hard way. Spark images are big. Whether you run the open source apache/spark image, Amazon EMR on EKS Spark, or a custom image built on either, 3 to 5 GB compressed is common once you layer in connectors, application jars, and Python dependencies. Every time Karpenter provisions a fresh node for a burst of executors, the first pod on that node has to download and unpack the full image before a single task runs.