GKE Autoscaling: Fix the Signal That's Costing You Money

Most clusters scale on CPU because that's what the default template said. Then the bill grows while the latency graph stays ugly.

On September 23, 2026, Google announced built-in support in GKE for processing Prometheus metrics. That lets the Horizontal Pod Autoscaler (HPA) run PromQL queries against Cloud Monitoring and Google Managed Service for Prometheus (GMP, Google's hosted Prometheus backend) without a third-party adapter, which is a meaningful shift for GKE autoscaling design. The feature is in Preview, with self-hosted Prometheus servers planned after preview as it moves toward general availability (Google Cloud). Check the current status before you plan around it.

What this really removes is an excuse. The reason most teams scale on CPU was never that CPU is the right signal. It's that anything else required an adapter someone had to install, patch, and grant IAM to.

What the HPA is actually doing

Picture a queue consumer at 2am: CPU is idle, and 40,000 messages are waiting. What does the autoscaler see?

It sees one number, and it changes the replica count to keep that number near a target. That's the whole mechanism, and it's the crux of doing GKE autoscaling well. Everything about its cost behavior follows from which number you picked.

CPU is the default choice, and it's a proxy. Think of it as a thermometer in the hallway: it tells you the building is warm, not which room is on fire. A queue consumer can sit at 40% CPU with a backlog thirty minutes deep. An inference service can show a busy GPU while throughput is poor.

The number that belongs in the target is the one sitting upstream of the pain: backlog depth for a queue consumer, a latency percentile or request rate for a web service. Google's own inference guidance recommends queue size to maximize throughput within a target latency threshold, and batch size when you need lower latency than queue-based scaling reaches. It also notes that GPU utilization (DCGM_FI_DEV_GPU_UTIL) measures duty cycle only and doesn't capture how much work is done, which makes it hard to map to a latency or throughput target (GKE docs).

Almost nobody does this

Datadog's State of Containers and Serverless report found that over 64% of Kubernetes organizations have adopted HPA. Among those users, 47% run it in all clusters, and 39% in most (Datadog). Adoption isn't the gap; the signal is. This gap defines the real GKE autoscaling problem: broad adoption, poor signal choice.

In that same dataset, only 20% of HPA-enabled deployments use custom metrics such as queue depth or request rate. Among the rest, 80% scale on CPU or memory alone. Another 14% use custom metrics only, and 6% use both (Datadog).

Set that next to Cast AI's 2026 measurements across tens of thousands of production clusters before optimization. Average CPU utilization sat at 8%, memory at 20%, and GPU at 5%. CPU overprovisioning rose from 40% to 69% year over year (Cast AI). Both vendors sell optimization products, so read the numbers with that in mind. The direction still holds: the common cluster is heavily autoscaled and heavily overprovisioned at the same time.

Why removing the adapter changes the numbers

Google's case against external metrics adapters has three parts: the management overhead of installing and patching third-party components you then have to monitor, the reliability risk of adapter pods sitting inside a critical autoscaling loop, and IAM complexity from Kubernetes-to-Cloud service account mappings (Google Cloud). This adapter-free approach is what makes native GKE autoscaling less brittle in production.

On the native path, GKE says the Kubernetes Default Node Service Agent already holds read permission to Cloud Monitoring and Managed Service for Prometheus in the same project, so no extra service accounts, keys, or federation parameters are required. The controller runs on the GKE control plane, and a system pod only lands on user nodes when a PromQL metric is actively requested. Google states that AutoscalingMetric polls the metrics backend every 15 seconds (Google Cloud). AutoscalingMetric is the GKE resource that declares which PromQL query the HPA should watch. Google also states that its earlier pod-level custom metrics support cut metrics reading latency to 5 seconds by routing straight to the HPA. Those are Google's own figures; we haven't seen an independent benchmark of the native path against an adapter.

There's a real trade-off here. AutoscalingMetric is a Google-specific CRD surface. KEDA's ScaledObject, whatever its YAML bulk, is portable across clouds. If your platform team runs more than GCP, that portability is worth pricing.

Scale-to-zero, and the floor underneath it

Kubernetes v1.37 makes HPA scale-to-zero Beta and enabled by default. minReplicas: 0 requires at least one object or external metric; the API server rejects an HPA containing only resource metrics like CPU or memory, because at zero replicas there are no Pods left to measure (Kubernetes). This is one more reason GKE autoscaling design should start from the right metric, not the default one.

That constraint is the same lesson from a different angle. A backlog metric exists whether or not a worker does, which is exactly why it can bring a workload back from zero.

The upstream post is honest about the edges: the default downscale stabilization window is five minutes, Services don't buffer requests while no Pods are ready, and an HPA can't scale from zero when its metric is unavailable. It reports ScalingActive=False with FailedGetExternalMetric (Kubernetes). Google's answer to the cold-start half is capacity buffers: pooled warm capacity so a workload jumping from 0 to 1 claims resources immediately, rather than waiting 60-90 seconds for a new node (Google Cloud). You still pay for the buffer, so call it a cheaper floor rather than a free one.

Where better signals don't help

Be clear about the limits, because the honest version of this GKE autoscaling argument is narrower than the exciting one.

• Padded requests are a separate problem. Cluster autoscalers treat inflated resource requests as real demand and provision nodes to match (Cast AI). No HPA metric fixes that denominator.
• HPA is reactive by construction. It scales on symptoms, not causes. The delay from spike to available capacity ranges from 30 seconds to over 3 minutes with cold nodes and slow readiness probes (ScaleOps).
• Rightsizing can backfire. Cutting a request from 200m to 100m can turn 75% utilization into 150%, since averageUtilization is computed against resources.requests, triggering aggressive scale-out (ScaleOps).
• Custom metrics add failure modes. Metric pipelines can lag or fail, leaving the HPA without a reliable signal. Add monitoring and a fallback to resource metrics; noisy signals like request rate can also cause pod churn (Datadog).
• CPU is often fine. 46% of unscaled workloads see multiple significant CPU spikes per day, making them straightforward candidates for CPU-based scaling (Datadog). The custom-metric case is for workloads that genuinely aren't CPU-bound.

Make it a review item

The practical move for GKE autoscaling is small. Add one question to your next infrastructure review: what is each HPA actually watching, and does that number sit upstream of the cost?

For most estates the answer will be a short list, queue consumers and inference services mostly, and a longer list where CPU is doing a fine job. Pilot the native PromQL path on one non-critical queue consumer while the feature is in Preview, keep a resource-metric fallback wired up, and set thresholds slightly beneath the observed maximum to dampen oscillation.

Then watch the bill. A badly chosen autoscaling signal rarely announces itself as a latency problem; it shows up as node-hour drift. What a customer needs to see is hourly spend movement at project and SKU level, well before month-end. That's the view Cloud Companion gives our customers, and a scoped pilot on one queue consumer is exactly the kind of engagement our platform team runs.