PerfectScale
Fine-Grained Pod-Scaling in Kubernetes: Beyond HPA and VPA
This page is also available in Deutsch, Español, Français, Italiano, 日本語, and Português.
About Joshua Fox
Joshua Fox has been a software architect in innovative technology companies for 20 years. Now, he advises tech startups and growth companies on architecture and cost optimization for Google Cloud Platform and Amazon Web Services; also publishing articles, and speaking to cloud engineers.
He has a PhD from Harvard University and a BA in math from Brandeis.
My personal page1. Introduction
Kubernetes autoscaling is designed to maintain robustness while saving money. Autoscalers can adjust the number of nodes with and Cluster Autoscaler, or the number and size of pods with a Vertical or Horizontal Pod Autoscaler.
There are limitations: Standard autoscalers rely on broad metrics like average CPU usage. This is too imprecise and either wasteful resources are retained, or the system risks instability under load. In this article, I will focus on pod autoscalers and describe how fine-grained telemetry from cAdvisor, Kube-State-Metrics, and eBPF can make more precise, workload-aware scaling decisions. The decisions are then applied to live pods using the Kubernetes InPlacePodVerticalScaling API.
2. The Flaws of Existing Autoscaling
Standard autoscaling has three core limitations:
- Insufficient signal granularity: Simple CPU averages do not reflect the full performance profile of complex applications, leading to delayed or incorrect scaling decisions.
- HPA/VPA conflict: HPA scales based on requested resources, while VPA modifies those requests. When both operate independently, they can issue contradictory instructions and cause oscillation.
- Eviction-based resizing: Legacy VPA requires pod evictions to apply resource changes, disrupting stateful or cache-sensitive workloads that cannot tolerate cold-start latency.
3. The Multi-Layered Telemetry Pipeline
We start with node-level metrics as a baseline, then add four additional telemetry layers for finer-grained signal:
- Layer 0 (Baseline): Node-level CPU and memory utilization from the Kubernetes Metrics Server, providing the coarse-grained foundation.
- Layer 1: Container-level telemetry via cAdvisor. This layer captures micro-burst latency — short-lived CPU spikes that inflate averages without reflecting sustained load — and memory working sets, preventing scaling decisions based on benign kernel page cache growth.
- Layer 2: Kube-State-Metrics (KSM) provides workload health context, such as pending replicas and HPA saturation signals. This lets the controller determine whether scaling is necessary or already in progress before issuing further actions.
- Layer 3: eBPF probes the kernel to surface deep application signals, such as JVM heap allocation rates and Garbage Collection (GC) pressure. This enables preemptive resource adjustment before OOM events occur. For example, a
bpftraceprobe using HotSpot USDT markers likeusdt:/path/to/libjvm.so:hotspot:mem__pool__gc__begincan detect GC frequency spikes seconds before they affect latency, giving the controller a head start. - Layer 4: GPUs are now a critical resource for AI/ML cost and performance. GPU-specific metrics via NVIDIA DCGM enable scaling based on GPU bottlenecks, such as SM utilization or memory bandwidth saturation.
4. Building the Unified Scaling Controller
A unified controller harmonizes these signals as a state machine: It maintains explicit states (stable, scaling-horizontal, scaling-vertical, cooldown) and only transitions between them on defined conditions. This is distinct from existing scalers that generally operate as rule engines, evaluating each signal independently, thereby posing a risk of issuing conflicting instructions.
The controller operates in three phases:
Phase 1: Signal Aggregation
Each telemetry layer contributes a weighted score toward a single pressure metric per workload. Application-layer signals like Layer 3 (eBPF/GC pressure) receive a higher weight.
Phase 2: Axis Arbitration
The controller avoids conflict by detecting that horizontal scaling is in progress; in-progress scaling is shown by AbleToScale=True alongside a non-zero difference between currentReplicas and desiredReplicas.
Conversely, if a workload is declared singleton or stateful (via annotation or StatefulSet ownership), the controller routes to vertical scaling, never horizontal.
Phase 3: Reconciliation and Cooldown
After any scaling action, a cooldown period prevents thrashing. The controller re-evaluates only after observing at least two consecutive metric windows above or below the threshold, ensuring transient spikes do not trigger repeated scale events.
5. In-Place Pod Vertical Scaling
The InPlacePodVerticalScaling API, which reached General Availability in December 2025 with Kubernetes v1.35, allows resource mutation without pod eviction. The controller patches the pod specification, and then the kubelet updates the container's cgroups dynamically — no connection drops, no process restart.
A concrete example: if the eBPF layer detects sustained Java GC pressure indicating that heap exhaustion is likely within 30 seconds, the controller issues a PATCH /api/v1/namespaces/{ns}/pods/{name} with updated resources.requests.memory and resources.limits.memory. The kubelet applies the change by adjusting the container's cgroup memory limit in place.
This is particularly valuable for services with significant cold-start times, like JVM applications, database sidecars, and in-memory caches, where eviction-based resizing would introduce latency spikes lasting tens of seconds.
6. Conclusion
Standard HPA and VPA configurations often fail in maintaining both high uptime and cost efficiency. By combining fine-grained observability from eBPF and cAdvisor with coordinated horizontal/vertical scaling through a unified state machine, and applying changes via in-place pod resizing, organizations can build self-healing, right-sized clusters with minimal disruption.
PerfectScale is an implementation inspired by the architectural patterns described here. I work in Forward Deployed Engineering at DoiT, advising customers on AWS and Google Cloud Platform. Feel free to reach out with questions.