AI Cost Governance: A Three-Layer Budgeting Model for Agents

Static cloud budgeting was built for provisioned capacity. You bought a VM, you knew roughly what it cost per month, and a monthly threshold alert was enough to catch a mistake before it mattered.

Token-metered agentic workloads break that model on three separate assumptions at once. A single agent run can spend more in an afternoon than a VM does in a month, and the billing system will tell you about it tomorrow. The fix isn't a tighter budget. It's a three-layer control loop: forecasting at the commit layer, hourly anomaly detection at the burn layer, and per-agent attribution at the ownership layer. That's the three-layer model, in practice.

This isn't a fringe concern anymore. The FinOps Foundation's sixth annual State of FinOps reports that 98% of respondents now manage AI spend, up from 31% two years ago, and names "FinOps for AI" the top forward-looking priority, with AI cost management the number one skillset teams say they need to develop.

Why your VM-era budget model can't see an agent

Static budgeting rests on three assumptions. Agentic workloads violate all of them. What you need instead are assumptions built for token-metered, bursty workloads, not static VM math.

• Predictable unit consumption. A VM costs the same per hour whether it's busy or idle. An agent's cost depends on steps per task, context length per step, and which model handled each step, all of which vary run to run.
• Slow burn rate. Provisioned capacity accumulates cost at a rate you can watch monthly. A retry loop accumulates it at machine speed.
• Resource-level ownership. A project or instance maps to a team. A shared API key maps to nobody.

The timing problem is the sharpest of the three. Microsoft engineers Tisha Chawla and Susheem Koul put it plainly in an August 2026 post: a monthly budget cap "can't stop a runaway, for a simple reason: it acts on a total that is only known after the fact." Their argument goes further. Gateways scope budgets per request, key or team, but an agent run is a graph spanning retries, growing context, sub-agents and hand-offs. The unit of accounting and the unit that spends the money are two different units.

Google's own documentation is honest about the lag. Cloud products report usage and cost data at varying intervals, so "typically, your costs are available within a day, but can sometimes take more than 24 hours." Third-party practitioner guidance puts the practical delay for budget notifications higher still. Either way: no billing-derived signal stops a runaway mid-run.

The failure modes, named

Giving these a name is half the battle. It lets a platform team recognize them in their own bill instead of arguing about "AI spend" in the abstract, and it turns governance from a vague goal into a checklist of specific failure modes to control. This grouping is our framework rather than settled industry vocabulary, but each mode is documented somewhere in practitioner writing.

• Retry storms. Each retry replays the accumulated context. One April 2026 engineering analysis reports that uncontrolled agent retry loops can produce 200x the token cost of a single successful execution, with waste tokens spiking above 80% during a storm. Treat that as illustrative, it comes from a practitioner blog, not audited data.
• Runaway loops. An agent that can call itself, with no step ceiling and no dollar-denominated circuit breaker.
• Context-window inflation. Re-searching and re-appending until input tokens dominate the bill. Unblocked's Dennis Pilarinos names broad search, re-search, mis-routing and runaway loops as the four runtime waste mechanisms.
• Model-tier drift. A workflow prototyped on the largest model and never right-sized before it went to production.
• Shared-key attribution blindness. Five teams, one key, no way to split the invoice.

Layer 1, Usage forecasting: budget in tokens and calls

Extrapolating last month's spend forward is how VM budgets were built. It fails here because the drivers move independently of one another.

Forecast from workload drivers instead: expected tasks per period, steps per task. Then layer in input and output tokens per step and the model mix across those steps. Multiply out, then convert to currency last. That structure makes the sensitivity visible: a change in steps per task moves your forecast far more than a change in task volume.

This has a direct commercial consequence. Claude models on Google Cloud are documented as fully managed, serverless APIs billed pay-as-you-go, or via a fixed fee using Provisioned Throughput. Provisioned Throughput is a fixed-cost, fixed-term subscription, you pay for the reserved term regardless of usage. That decision should rest on a token-and-throughput forecast.

The FinOps Foundation's Tokenomics work is worth reading alongside this. Its practitioner survey identified managing the cost and use of tokens in SaaS-model AI as the top challenge facing practitioners today. Developer-led purchasing, opaque billing and the lack of a built-in way to allocate spend are the main reasons why. It also notes that hyperscaler routes are "significantly easier to govern" than direct provider APIs, because billing data lands in existing systems, tagging infrastructure exists, and spend can count against an existing committed-spend agreement.

Layer 2, Hourly anomaly detection: the window that matters

For AI spend, detection latency is the binding constraint. A perfectly tuned monthly threshold that fires three days late hasn't protected anything. Closing that gap is core to the burn layer.

But hourly resolution isn't automatically an improvement, and the research here cuts both ways. ProsperOps argues that detection without automated corrective action surfaces noise without preventing recurring waste. It also argues that the precision-recall trade-off is the core operational problem: low precision burns engineering time on false alarms, and low recall lets real anomalies persist. A March 2026 Holori guide goes further, calling a high-noise program "arguably worse than no program at all," because raw-cost detection can't distinguish healthy growth from inefficiency. That's why mature teams work from unit economics rather than raw spend.

So an hourly loop needs four things:

1. Baselines scoped per workload rather than per account, so seasonality and growth don't fire alarms on a schedule.
2. Unit-economics thresholds, cost per task, per session, per customer, which require Layer 3 to already exist.
3. A named owner for each detection signal. Two-minute detection followed by two-week resolution is still expensive.
4. A containment action. The FinOps for AI Overview recommends combining usage limits with anomaly detection against historical baselines, for example investigating when token consumption doubles without clear justification.

One honest boundary: because billing data lags, in-run enforcement has to live in the agent runtime, step ceilings, per-tool retry budgets, dollar-denominated circuit breakers. The billing layer does reconciliation and pattern detection. Containment happens upstream, in the runtime.

Layer 3, Per-agent attribution: ownership before optimization

Spend that nobody owns is spend that nobody improves. Every proposed change lands on a team that can dispute the number, and it gets argued about.

On Google Cloud the attribution primitive is documented and available now. Labels attached to API calls such as generateContent and rawPredict are forwarded to the billing system, so billed charges can be broken down by label in billing reports and queried in billing data exports, including for partner models such as Claude.

The limits shape your schema. Each call can carry up to 64 labels for Google models and up to 32 for partner models. Each label key supports up to 1000 unique values across all requests over the life of the billing account, and keys may be dropped without notice above that. That's why Google recommends against high-cardinality labels such as timestamps or per-call identifiers.

Read that constraint carefully: naive per-run or per-session labeling will quietly break your attribution. Label the stable dimensions, team, use case, environment, agent name, model tier, and carry run identifiers in your own telemetry instead.

Pair labels with API key governance. The Tokenomics recommendations are unambiguous about sequencing: establish key governance as the foundational control layer, deploy a proxy or observability layer to inject tagging metadata providers don't supply natively, implement unit cost metrics, and treat model right-sizing as the single highest-impact optimization. Attribution is the foundation here, because spend without an owner can't be optimized.

Why caps without attribution backfire

A cap bounds the worst case. Right-sizing lowers the expected bill. Each does a job the other can't. A forced model downgrade applied without attribution just moves the problem: spend falls, and nobody can say what quality was traded away, or on whose behalf.

Wiring the layers into one control loop

The three layers reinforce each other. Attribution sharpens the forecast, because you can see which use case drives which driver. The forecast sets the detection baselines. The detection signals correct the forecast at the end of each period. Three layers, correcting each other continuously, is what this looks like in practice.

Starting from zero, the adoption order is dictated by that dependency chain:

1. Attribution first. Key scoping and a stable label schema on every model call. Nothing downstream works without it.
2. Forecasting second. Driver-based, in tokens and calls, feeding commit and Provisioned Throughput decisions.
3. Detection third. Hourly analysis over labeled billing export, with unit-economics thresholds and a routing rule to a named owner.
4. Runtime guardrails in parallel. Step ceilings and retry budgets in the agent code, from day one, because no billing layer can enforce them.

This is also where the buyer question resolves. State of FinOps 2026 reports that 78% of FinOps teams now sit inside the CTO or CIO organization, with Platform Engineering increasingly participating in shift-left cost conversations. The instrumentation lives in your platform. So the model belongs to your platform team.

What to instrument this quarter

Each item below supports one of the three layers described above. A short, honest checklist:

• Inventory every API key that can reach a model endpoint, and scope each one to a single team and use case.
• Define a label schema with fewer than 1000 values per key, and enforce it in your agent framework rather than trusting each caller.
• Turn on BigQuery billing export and confirm labeled model spend is landing and splitting correctly.
• Build one driver-based forecast for your highest-volume agent before any commit or Provisioned Throughput renewal.
• Add a step ceiling and a per-tool retry budget to every production agent loop.
• Stand up one hourly baseline on one workload, with a named owner and a documented containment action, before scaling detection across the estate.

Granular monitoring of AI spend, tokens, LLM requests and GPU utilization, is the top item in State of FinOps 2026's top-3 tooling expectations, and most teams are building it themselves over billing export today.

If you'd rather not build the detection layer from scratch, Cloud Companion provides hourly spend analytics and recommendations over your Google Cloud billing data, which shortens the detection window at the burn layer. The runtime guardrails still belong in your agent code: no billing-derived signal can stop a run that's already in flight. Get the attribution schema right first, and both layers become worth having.