Agents can't reason on a
broken foundation 

Most enterprises have now lived through the first wave of AI. Chatbots answer customer questions, copilots draft emails and code, and generative tools sit inside workflows that already existed. The results have been real, but bounded. They made existing work faster; they rarely changed what the work was.

The next wave is different in kind, not degree. Agentic AI is not a more powerful chatbot. It is a system that reasons, plans, and recovers from failure, often with no user-facing interface at all. Gartner expects 33% of enterprise software applications to include agentic AI by 2028, up from fewer than 1% in 2024. That trajectory makes architectural readiness one of the most consequential technology decisions of the decade, and most organizations are not ready. They are trying to deploy next-generation agents on first-generation foundations. That mismatch, more than model capability, explains the gap between agentic AI's theoretical potential and its enterprise reality.

The data layer is the decisive, unglamorous work

Agents need more than data. They need data that is explainable, actionable, and organized around outcomes rather than records. MIT Sloan Management Review reports that 78% of executives cite data quality and explainability as the primary barriers to scaling AI. For agentic systems the finding bites harder, because an agent's decisions are only as good as the information it can reason over.

Basic semantic virtualization, a friendly layer of business names over scattered tables, is not enough. An agent asked to resolve a supply disruption or rebalance a credit portfolio needs to understand relationships, context, and intent:

That is the job of a rich ontology. It is also the difference between an agent that reasons and one that guesses confidently.

Move the logic out of the model

The dominant architecture places too much logic inside the LLM. It is the path of least resistance, and the source of maximum long-term cost.

The first cost is opacity. If the reasoning lives inside the model, it cannot be audited, explained, or trusted in regulated or high-stakes settings. Forrester Research finds that organizations investing in explainable, auditable AI application architecture are 2.3× more likely to achieve regulatory approval for AI deployments in high-stakes industries. The second cost is tokens. Consumption at enterprise scale is a significant and chronically underestimated constraint, and it grows with every agent, step, and retry.

The answer is to place application logic grounded in industry value chains outside the LLM, in explainable, testable code that the model calls rather than improvises. The model handles language and judgement; the application layer handles rules, sequencing, and domain truth. BCG Henderson Institute estimates that companies redesigning around outcome-focused data models and externally grounded logic achieve 40% lower total cost of AI ownership over three years than those relying on LLM-centric architectures. Organizations that move logic out are making a bet that compounds.

The emerging direction of zero data transfers to LLMs takes this further. It is not a cost-cutting idea. It is an architectural philosophy: agents should reason with references to data, not copies of it. Sensitive records stay where they are governed; the model works with pointers, schemas, and results. That single principle reshapes security, cost, and system design at the foundation. IDC forecasts enterprise spending on AI infrastructure, including the data and application layers agentic systems require, to reach $200 billion globally by 2027.

What agents do when the foundation is right

Get both layers right and something qualitatively new becomes possible: non-linear reasoning. An agent makes a plan, hits a dead end, reassesses, and tries a different route. That is the line between agentic AI and automation. Automation replicates a workflow. An agent transforms it, and can find a path the workflow's designer never anticipated. McKinsey Global Institute estimates that agents designed to adapt and plan dynamically can unlock 3–5× greater productivity value than rule-based automation applied to the same workflows. But that only happens if the data layer is rich enough to reason over, and the application logic is structured enough to guide the search.

So before asking what your agents will do, ask what they will build from. The data layer and the application layer are not infrastructure decisions; they are strategic ones. The organizations investing in this foundation now are not merely preparing for the next generation of tools. They are building the conditions for a different category of value: workflows that adapt, decisions that improve, and systems that navigate complexity rather than just process it. The goal is not to automate today's workflow faster. It is to build agents that find a better path, one that may look nothing like the current process. Those that skip the foundation will find their agents fast, cheap, and deeply limited.