Agentic AI Infrastructure: Why Modernise Before You Build

Google Cloud's own research finds that the single biggest agentic AI infrastructure gap named by IT leaders is difficulty integrating with legacy APIs and data sources. Not a model problem. A foundation problem, and most teams are trying to solve it in the wrong order.

The Google Cloud 2026 State of AI Infrastructure report, drawn from more than 1,400 senior IT leaders, puts a precise number on something many engineering teams already feel: 43% cite difficulty integrating with legacy APIs and data sources as their biggest agentic AI infrastructure gap. A further 83% say they need infrastructure upgrades before they can run production-grade agents at all. Only 17% have full confidence their current stack can support mission-critical agentic work.

Sit with that last figure. Seventeen percent. The conversation in most boardrooms is about which model to use, which vendor to partner with, and how fast to move. The data says the conversation should be about plumbing.

The Bottleneck Is Integration Debt, Not Model Maturity

Here is the uncomfortable truth the industry keeps talking around: the models are good enough. Anthropic's 2026 State of AI Agents report, drawn from over 500 technical leaders, found that model capability did not appear in the top barriers to scaling agentic AI. What did? System integration, cited by 46% of organizations; data access and quality, cited by 42%; change management needs, cited by 39%. The bottleneck isn't the intelligence layer. It's the data plumbing and integration debt that agentic workflows expose the moment you try to run them at production scale.

This distinction changes the order of operations entirely. Teams that buy the model first and modernize the data estate later aren't moving fast, they're accumulating expensive technical debt that will stop their agents cold at exactly the wrong moment.

What Agentic AI Actually Demands From Your Infrastructure

Standard generative AI, a chatbot or a single-inference summarization task, makes modest demands on your infrastructure. An autonomous agent makes a categorically different one. Industry estimates put agentic AI token consumption an order of magnitude or more above standard generative AI, requiring proportionally more compute. The infrastructure that handles a thousand daily chatbot queries will not scale to an agent that operates continuously, triggers workflows, reads from multiple data sources, and hands off context across systems.

McKinsey's April 2026 analysis projects IT infrastructure costs will grow two to three times by 2030, driven by compute and storage demand from agentic AI, even as budgets stay roughly flat. That pressure doesn't arrive gradually. It arrives the day your first production agent goes live and starts hitting every fragile API and ungoverned data source in your estate.

McKinsey's framework for agent-ready infrastructure names four foundational capabilities an enterprise needs before deploying agents at scale:

• Repeatable and executable actions through secure, modern APIs
• Reliable, clean operational data that agents can trust
• Embedded controls and agent governance from day one
• Agent lifecycle management with interoperability, memory, and handoff continuity

Most legacy estates satisfy none of these. APIs are absent, poorly documented, or so tightly coupled to monolithic systems that exposing them safely requires a dedicated re-architecture project. Data is siloed, inconsistently labeled, and lacks semantic ownership, meaning agents can't determine which source to trust when two systems disagree. And governance for autonomous, non-human workers is a new discipline that most IT organizations haven't yet built.

The Hidden Cost of Getting the Order Wrong

Google Cloud's infrastructure report found that 81% of leaders identified operational complexity and engineering overhead as top unforeseen expenses when scaling AI, much of it traced to the time engineers spend manually patching agents across disconnected systems. Separately, 62% are seeing high inference costs driven by data egress, storage bloat, and idle specialized hardware inside legacy architectures. Google Cloud VP Nirav Mehta put it directly: traditional architectures are cracking under this pressure.

The downstream consequence is predictable. Gartner predicts that over 40% of agentic AI projects will be canceled by end of 2027, not because the models failed, but because of escalating costs, unclear business value, and inadequate risk controls. McKinsey reports that less than 10% of agentic programs reach meaningful scale, and that in any given business function no more than 10% of organizations say they are scaling AI agents. That's not a failure of ambition. It's a failure of sequencing.

There's also a subtler problem Gartner flags: agent washing. Thousands of vendors are rebranding existing chatbots and RPA workflows as agentic AI. Gartner considers only around 130 of them genuine. Teams that can't distinguish between true agentic systems and rebadged automation are particularly exposed, they may believe they've solved the problem while the real integration debt quietly grows.

MCP Solves the Envelope, Not the Contents

One source of false confidence deserves specific attention. Anthropic's Model Context Protocol, released in November 2024 and donated to the Linux Foundation in December 2025, has become the de facto standard for wiring agents into enterprise systems. By early 2026, over 10,000 active public MCP servers existed, with more than 97 million monthly SDK downloads. MCP is genuinely useful, it standardises how agents connect to systems.

What it doesn't do is fix what the data means or who owns it. Data contracts, semantic clarity, and source-of-truth designation remain manual problems that no protocol automatically resolves. An MCP connector wired into a legacy CRM that holds three conflicting customer records gives your agent three conflicting answers, faster. The connector is the easy part. Data quality and ownership are the hard part, and they require modernization work that precedes the agentic build, not follows it.

The winners in 2026 and beyond won't necessarily be the ones with the smartest agents. They'll be the ones who can feed those agents the right knowledge, securely, cost-effectively, and at scale.

The Right Order: Modernize First, Then Build

The data earns a clear position: infrastructure and application modernization must precede agent deployment, not trail it. Teams that skip this step aren't accelerating, they're building on sand. The payoff, once the foundation is in place, is substantial. McKinsey's experience indicates that agentic AI can automate 60 to 80% of routine infrastructure work over time, translating to a 20 to 40% run-rate cost reduction in initial deployments, but only after automation, environment simplification, and active cost governance are already running.

Practically, the right order looks like this. Before the first agent goes to production, your team should be able to answer yes to three questions:

• Do your critical systems expose clean, secure, well-documented APIs that an agent can call without human intervention?
• Is your operational data governed, semantically clear, and sourced from a single point of truth per domain?
• Do you have an agent governance model, covering access controls, audit logging, failure handling, and rollback, that your security and compliance teams have signed off on?

If the answer to any of these is no, the agent deployment isn't ready. The infrastructure project is. Gartner makes this point with characteristic directness: in many cases, rethinking workflows with agentic AI from the ground up is the ideal path, not layering agents onto existing systems that were never designed to hand off context, trigger downstream actions, or expose clean interfaces to an autonomous non-human worker.

Where to Start If You Are Behind

The 83% of organizations that know they need infrastructure upgrades but haven't completed them aren't uniformly behind, they're at different points on the same path. Some have clean cloud-native applications but ungoverned data estates. Some have well-structured data but legacy monoliths with no API surface an agent can safely call. Some have both problems and need to triage.

The diagnostic questions are always the same: Where does your data actually live? Which systems are agents expected to read from and write to? What APIs exist today, and which would need to be built? What governance is in place for non-human access? The answers tell you whether you're weeks away from a meaningful agentic build or quarters away from one.

IT leaders who also cite a lack of specialized, high-throughput vector databases as a key gap are pointing at the same underlying issue from a different angle: agents need efficient retrieval infrastructure to carry business context across multi-step reasoning chains. That's not a model configuration problem. It's a data infrastructure investment.

The Argument, Plainly Stated

Agentic AI works. When the foundation holds, it works well. The problem is that the foundation is the variable most teams are under-investing in, and the research across Google Cloud, McKinsey, Anthropic, and Gartner all points to the same diagnosis: most organizations are trying to solve an infrastructure problem with a model selection decision.

Sound agentic AI infrastructure isn't a follow-on project. It's the prerequisite. The organizations running production-grade agentic systems at scale in 2027 won't necessarily be the ones with the largest AI budgets today, they'll be the ones that treated modernization as a prerequisite and started that work before the agent project brief landed on anyone's desk.

If you're trying to work out whether your data estate and application architecture are ready to support production agents, or where to begin modernizing if they're not, WALT Labs' Workshop and Assessment engagement is designed for exactly that starting point: an independent architecture review, risk assessment, and a prioritised roadmap before any larger commitment.