Big Tech spent trillions on AI but the ROI is still a ghost

Seven trillion dollars is a staggering amount of capital to deploy into a single technology bet, yet we are seeing a widening gap between the infrastructure spend and the actual revenue generated by LLMs. The current trajectory suggests that the hyperscalers are trapped in a cycle of "competitive spending"—buying H100s not because they have a clear product-market fit for every cluster, but because they can't afford to let their rivals have more compute.

The Infrastructure Trap

The primary issue is the decoupling of Capex and Opex. We are seeing massive deployment of data centers and energy grids, but the applications sitting on top of this hardware are still mostly wrappers or incremental productivity tools. For a company to justify a trillion-dollar investment, you need "killer apps" that create entirely new economies, not just a faster way to summarize an email or generate a mediocre image.

The risk here is a massive write-down. If the scaling laws hit a plateau—where adding ten times the compute only yields a 2% increase in reasoning capability—the economic model for these massive clusters collapses. We've seen this movie before with the fiber optic bubble of the late 90s. The infrastructure was necessary for the internet to exist, but the companies that built it went bust before the actual utility of the web caught up.

LLM Agents as the Only Exit

The only way out of this trap is a shift from "Chatbots" to "LLM agents" that can actually execute complex, multi-step workflows without human hand-holding. A tool that helps me write a report is a feature; a tool that manages my entire supply chain, negotiates with vendors, and handles logistics autonomously is a business model.

To get there, we need a massive leap in prompt engineering and agentic frameworks. Most current "agents" are just loops with a system prompt that frequently hallucinate or get stuck in infinite cycles. For Big Tech to see a real return, these agents need to be reliable enough to handle real-world financial transactions and critical infrastructure.

The Compute Paradox

We are currently in a state where compute is the new oil, but we don't have enough "engines" (applications) to burn it. The industry is pushing for larger and larger models, but the real-world demand is shifting toward smaller, distilled models that can run on the edge. If the market pivots toward efficient, specialized SLMs (Small Language Models), the trillion-dollar bets on monolithic, general-purpose giants might actually be a strategic error.

The technical debt isn't just in the code; it's in the physical hardware. You can't just "pivot" a 100,000-GPU cluster if the world decides it only needs 7B parameter models tuned for specific tasks. We are essentially gambling that intelligence scales linearly with power and data, and so far, the evidence is anecdotal rather than systemic.