What happens to the price of intelligence when the machines that make it are sold out for two years?
**
Two numbers about artificial intelligence are moving in opposite directions. The price of a given level of machine intelligence is falling faster than the price of almost anything economists have tracked. The capacity that produces it is so scarce that the world’s largest cloud provider says it has little left to sell before 2028.
Both are true, and the space between them is where much of the next decade’s AI economics will be settled. Intelligence is getting cheaper by the unit. Whether it gets cheaper by the job depends on how much of it each job wastes. Those who secure the most compute will have real advantages. A second advantage, open to many more players, will go to those who need the least of it for each piece of useful work.
The argument of this essay is simple. AI now has two scarcity problems. The first is physical: chips, memory, power and the capacity commitments that bundle them. The second is economic: useful intelligence can be too expensive at the level of a workflow, because agents burn tokens re-reading, re-reasoning and retrying. Both point to the same destination, which is avoidable compute spending. Do not repeatedly rent what your system already knows. In marketing I have called this principle Never Pay Twice. Applied to AI, it suggests that India’s best move is not to bid harder in the capacity auction, but to change the unit of competition.
1
Sold out until 2028
On 23 October, Amazon’s chief executive, Andy Jassy, told an Andreessen Horowitz podcast that AWS capacity for 2026 is sold out, 2027 is largely sold out, and a good chunk of 2028 is gone too. AWS keeps some capacity back for on-demand customers and for its EC2 and Bedrock services, but the big blocks are committed. This from a company that expects capital expenditure of about $220 billion in 2026, roughly $600 million a day, and whose chief executive said it could invest a lot more without it being a bad use of capital. He also said AWS would not squeeze its existing customers to make room for AI labs willing to pay almost anything.
The statement needs reading carefully. “Sold out” here describes forward commitments. It does not mean a developer cannot make an API call this afternoon. It means the large blocks of capacity, the ones that decide who can train and serve at scale, have been bought years ahead by those willing to commit early and in size. When capacity is sold years in advance, the price that matters is not the list price per token. It is access.
That changes the question a company or a country should ask. For three years the question was which model to use. The next question is how much capacity a given amount of useful work requires, because capacity has become the constraint and a forward market has formed around it. That is a question about engineering and design as much as about procurement.
Two scarcities. Physical compute is scarce because chips, memory, power and grid connections cannot be built as fast as demand grows. Useful intelligence is scarce because, at the level of a completed workflow, it can still cost too much to deploy widely. The first is largely beyond any one company’s control. The second is largely within it.
2
The two prices of AI
The other number comes from Epoch AI. In The plunging price of thought, published on 22 September, Epoch estimates that the cost of reaching a given level of performance on five benchmarks covering mathematics, science and games of skill has fallen about 47% a quarter since 2023, or 13 times a year. On one science benchmark, the cost of a given score fell from about $0.30 to $0.0004 a question in under 18 months: a 725-fold fall. Epoch puts the pace ahead of the historic declines in the prices of electricity, compute, batteries and DNA sequencing.
The detail matters as much as the headline. The decline is steepest just after a level of performance first becomes state of the art, at about 66% a quarter, and slows to about 32% a quarter two years later. Epoch measures the cheapest model able to reach a fixed score, not the newest model, and its authors note that this assumes a user who keeps switching to the cheapest capable model, which real users rarely do. On SWE-bench Verified, the benchmark in Epoch’s set closest to agentic software work, the fall has averaged about 28% a quarter: steep, but roughly half the headline.
The price that is plunging is the price of yesterday’s capability. The price of today’s capability, consumed at today’s volumes, is a different number. A buyer who wants the newest model pays the premium that comes with it. A workflow that takes more steps consumes more tokens. And the inputs are getting dearer: Epoch itself notes that the AI boom is now powerful enough to raise the prices of the chips, power and even the electricians it needs.
So AI has two prices. Fixed capability is getting cheaper at a rate without precedent. Compute, the physical capacity that delivers any capability at all, is getting dearer and harder to secure. In When AI Gets Cheaper, What Gets More Valuable? I asked which side of the first price a business sits on. This essay is about the second, and about what sits between them: the cost of a completed piece of work.
Three propositions need to be kept apart. The cost of a fixed capability is falling: that is measured. The cost of increasingly ambitious agentic workflows may rise: that is a forecast. The share of that spending which could be avoided without losing quality is unknown: that is the opportunity this essay proposes to test. The third does not depend on the second proving right.
| Price | Direction | Status |
| Fixed capability (cost of a given benchmark score) | Falling about 47% a quarter since 2023 | Measured (Epoch AI, September 2026) |
| Capacity (chips, memory, power, data-centre space) | Scarce; large blocks sold forward into 2028 | Reported (AWS, October 2026; IEA, 2026) |
| An increasingly ambitious agentic workflow | Forecast to rise more than fivefold through 2028 | Forecast (Gartner, August 2026) |
| The avoidable share of that spending | Unknown | Hypothesis to test (this essay) |
Figure 1. Two prices, and the two questions between them
3
The compute strait
In Every Business Has a Strait, I argued that profits pool at choke points: narrow passages in valuable flows, where the alternatives cost enough that the passage acquires power out of all proportion to its size. Physical compute now has one.
The International Energy Agency’s updated outlook sees electricity consumption by data centres roughly doubling from 485 TWh in 2025 to about 950 TWh in 2030, around 3% of global demand, with consumption by AI-focused data centres tripling. The same report says a shortage of high-bandwidth memory, the specialised memory packaged with AI chips, developed over the preceding six months and is expected to persist through at least the end of 2027. The IEA’s 2025 report had already estimated that around 20% of planned data-centre projects could be delayed by strain on grids. And the money has outgrown balance sheets. The IEA notes that the largest technology companies spent more than $400 billion on capital expenditure in 2025, expects a further 75% rise in 2026, and observes that the build-out now depends on capital markets and investor sentiment.
A strait does not have to close to set the price. It only has to be narrow. Models improve every quarter. Grids, transformers, turbines, memory plants and financing move on a timescale of years. Even the IEA, which expects the build-out to continue, says bottlenecks across the value chain are making the most aggressive near-term scenarios less likely.
The strait sits in the physical world, and no single company or country can widen it quickly. But the same report makes a second observation that matters more for what follows. The energy used per AI task has been falling by at least an order of magnitude a year. Yet newer uses, including reasoning and agentic tasks, can consume hundreds or thousands of times more energy per query than simple text generation. The strait is in the hardware. Much of the waste is in the software.
4
The intelligence waste problem
In August, Gartner forecast that inference costs per agentic workflow will rise more than fivefold through 2028, even as the price of foundation-model tokens keeps falling. Gartner calls this the inference paradox. Its analysts say that sending a task to an agentic reasoning model raises inference costs at least fivefold compared with a basic chatbot exchange, and often far more as the task gets harder. This is a forecast from an econometric model, not an observation. And a rising cost per workflow is not, by itself, waste. A workflow that costs five times as much because it does ten times as much work is good economics. The forecast cannot tell us how much of the rise is ambition and how much is waste. Gartner’s own prescription points at the second: it warns that defaulting to generic autonomous intelligence produces unbounded costs, and argues for tiering inference instead.
Measured evidence of waste exists, though only in one domain. A Stanford-led study of coding agents found that input tokens, the context sent to the model again and again, rather than output tokens drive their cost. Runs on the same task varied by up to 30 times in total tokens, and spending more tokens did not buy more accuracy: accuracy often peaked at intermediate cost. The mechanism behind those numbers is visible in any agent log.
An agent re-reads: at every step it re-sends the same context, the same policy document, the same customer history, the same tool descriptions. It re-reasons, working out from scratch what the system worked out yesterday for a near-identical case. It calls a frontier model to check a date or add up a column, work that a line of code would do exactly and almost free. And when a step fails, it retries the chain.
Marketing has a name for this pattern. In The Profit You Already Own I described the reacquisition tax: brands paying advertising platforms to reach customers they already have, because nothing in their own systems kept those customers’ attention. Agents pay the same tax on intelligence: they rent reasoning again for what the system already knows. The cause is the same in both cases. Without memory, every interaction starts from zero, and starting from zero is billed by the unit. Never Pay Twice, applied to AI, means a system should pay for an inference once and keep the result for as long as it remains valid.
How much agentic spending is avoidable? At the level of a real workflow, nobody yet knows. The best measured evidence comes from routing. RouteLLM, built by researchers at Berkeley and LMSYS, trained routers that send each query to either a strong or a weak model. Compared with sending everything to GPT-4, the routers cut costs by over 85% on a conversational benchmark, 45% on a knowledge test and 35% on a mathematics test, while keeping 95% of GPT-4’s performance. Those are real measurements, but of single questions on benchmarks, with one routing decision and no memory, caching or code. They are not measurements of multi-step agents doing live work.
The avoidable share of agentic spending is a hypothesis, not a finding. It can be tested cheaply. Take a month of logged traces for one workflow. Replay them through a system that checks code and memory first, uses a small model by default and escalates to a frontier model only when needed. Compare the result with the original on a held-out sample, on cost per completed task, error rate and human intervention. If the hybrid cannot cut cost per completed task by at least a third without raising errors, the thesis is weak for that workflow. If it cuts it by three-quarters, as the routing benchmarks suggest is possible, the waste is large and the case for removing it is larger.
Cost per completed task. Total spend on a task, including inference, compute, human review and rework, divided by the number of tasks that reached a verified, correct end state. In From Rented to Owned, I argued that the comparison that matters is total cost per completed outcome; for AI work, the unit is the task. A cheap call that leaves the job unfinished is not cheap.
5
The frugal intelligence architecture
The alternative to an agent that burns tokens is not an agent starved of them. It is a system built in layers, in which each request travels only as far as it must.
- Code first. Dates, sums, rules, eligibility and state are deterministic. Software handles them exactly, instantly and for almost nothing, so a model should never be asked to do work it can hand to a function.
- Memory next. What the system already knows, such as yesterday’s answer, the decision reached for a similar case, the customer’s history or a document already summarised, should be looked up rather than re-derived. The cheapest token is the one never generated.
- Small models for routine language. Classifying, extracting, routing and drafting standard replies are within the reach of small, cheap models that run on modest hardware.
- Frontier models for hard cases. Ambiguity, novel reasoning and high-stakes judgement deserve the best model available. Used only for the cases that need it, the frontier stops being a cost problem.
Figure 2. The frugal intelligence ladder: each request stops at the cheapest layer that can complete it correctly
*Relative costs are illustrative assumptions reflecting typical price gaps between model tiers, not measurements.
Every request should stop at the cheapest layer that can complete it correctly. The order is the point. Memory sits above the model because a lookup is cheaper and more consistent than an inference. Code sits above both because exactness costs nothing.
Memory carries its own risk. A customer’s eligibility, a policy clause, a payment status or a student’s misconception can change. A cached answer may be cheaper than a fresh inference and more expensive than both if it triggers a wrong action. Reuse what remains valid; recompute what has changed. That requires memory with provenance, so every entry traces to a source; versions, so a changed rule replaces the old one rather than sitting beside it; permissions, so one person’s record is not everyone’s; and expiry, so a stale fact retires itself. In From Rented to Owned, I argued that memory earns its value only by improving the next decision. Here it also has to know when it no longer applies. Routing needs the same discipline: evidence that the cheaper path completed the work correctly, not only that it was cheaper.
| Frontier-only | Small-only | Hybrid | |
| Cost per task (assumed index) | 100 | 2–5 | 5–15 |
| Completion | Highest on hard cases | Falls on ambiguous or novel cases | Close to frontier, if routing is accurate |
| Errors | Few reasoning errors; can still slip on arithmetic and state that code gets exact | Confident mistakes on hard cases | Fewest, when code owns the deterministic steps and hard cases escalate |
| Latency | Slowest, especially with long reasoning | Fastest | Fast for most requests; slow for the few escalated |
| Human intervention | Low | High and unpredictable | Low, concentrated on cases the router flags |
Figure 3. Three ways to build the same workflow
The cost index is an assumption (frontier-only = 100), not a measurement. Section 7 works through two cases.
The hybrid has one weak point: the router. A router that sends hard cases to the small model produces the worst of both, a cheap call and a failed task. That is why a hybrid must be judged by cost per completed task, not cost per call. Small-only systems look best on an invoice and worst on outcomes. Frontier-only systems look safest and cost the most. A hybrid wins only if routing is measured, and the measurement is part of the architecture, not an afterthought.
There is also a scale effect. Memory, caching, evaluation and routing are expensive to build and cheap to share. A large enterprise can build them for itself; a small firm, a school or a district hospital cannot. In The Dormant Goldmine, I argued that work no single participant can afford to operate becomes viable when many share one factory. Frugal intelligence is the same kind of problem. Its components are infrastructure, and infrastructure is cheapest when pooled.
6
India’s different race
A country can respond to scarce AI in three ways.
| Strategy | Aim | Measured in | Constraint | Best fit |
| Compute sovereignty | Own enough capacity at home | GPUs, megawatts | Capital, power, imported chips | Countries that can finance capacity at scale |
| Model sovereignty | Train national foundation models | Capability, language coverage | Talent, data, training compute | Where language, culture or security require it |
| Intelligence affordability | Deliver useful AI at the lowest cost per task | Cost per completed task | Engineering discipline, shared infrastructure | Vast, price-sensitive, multilingual demand |
Figure 4. Three national strategies for scarce AI
India is pursuing all three. The IndiaAI Mission’s shared compute pool passed 45,000 GPUs by June 2026. The government told the Lok Sabha in August that it had sanctioned 93.18 lakh GPU-hours across 237 projects; sanctioned is the operative word, since an allocation is not a measure of use. Access is subsidised, at an average of about Rs 65 per GPU-hour and Rs 92 for an H100. Twenty indigenous model proposals, twelve large and eight small, have been selected for support. These are serious commitments. They are also small against the global build-out: India hosts an estimated 3–4% of the world’s data-centre capacity, a share the electronics ministry describes as indicative rather than precise.
None of the frugal argument is new to Indian policy. The Economic Survey 2025-26, tabled in January, argued for a bottom-up approach to AI built on application-specific small models and frugal innovation, rather than replicating the capital-intensive frontier race. On strategy, India was already there.
What is new is the price signal. In January, frugality read as prudence: a sensible stance for a country short of capital, power and chips. Ten months later, the largest cloud provider has sold its capacity years forward, memory is short into 2027, and the cost of ambitious agentic workflows is forecast to multiply. In that world, frugality stops being a constraint to manage and becomes an advantage to press. For a country with a small share of the world’s compute, halving the compute needed per task does the work of doubling capacity, and it can be done in months rather than years.
That is what changing the unit of competition means. The compute race is measured in GPUs and megawatts; the model race in parameters and benchmark scores. The race India is best placed to win is measured in rupees per completed task, across a billion people and many languages. The 93.18 lakh GPU-hours already sanctioned would go several times further if the work they serve were built frugally from the start.
Why should India be better at this than anyone else? Caching, routing and small models are not Indian inventions, and every large AI provider is investing in them. India’s possible advantage lies in the combination: price sensitivity that makes cost per task the binding constraint rather than a line item; multilingual demand that rewards small, specialised models; applications at population scale, where a fraction of a rupee saved per task becomes a large total; and a record of building shared digital public infrastructure. No other large market has all four pressing on it at once.
That points to one concrete proposal. IndiaAI should publish comparative cost-per-completed-task benchmarks for Indian public-service workflows, reporting quality, error rates, latency and human intervention alongside cost. Progress today is reported in GPUs onboarded, GPU-hours sanctioned and model benchmark scores. Those measure inputs. A benchmark for a grievance resolved, a form checked or a lesson completed to a stated standard would measure what the inputs buy, and would turn the Economic Survey’s frugal-AI philosophy into a measurable national programme. What a country measures, its builders optimise.
The supporting pieces are less glamorous than a supercomputer: shared memory and caching services, open evaluation sets for Indian workflows and languages, and small-model serving at population scale. Each is infrastructure that no single start-up can justify and every start-up needs. Subsidising access to compute is necessary. Building the infrastructure, benchmarks and incentives that make every unit of compute accomplish more is the larger prize.
7
The price of a completed job
Two cases show the arithmetic. Every price below is an assumption, chosen to reflect the typical gap between frontier and small models rather than any vendor’s list: $5 per million input tokens and $25 per million output tokens for a frontier model, and $0.10 and $0.40 for a small one.
A tutoring hour. An AI tutor works through one hour of practice with a student: thirty exchanges. The frontier-only tutor re-sends the lesson and the full history at every turn (assumed at 20,000 input and 1,500 output tokens a turn). The small-only tutor does the same, with the same token counts, on a small model. The hybrid lets code mark three answers outright and keeps a compact student record in memory, so that a small model handles twenty-four turns with 4,000 input and 500 output tokens each. It sends three hard turns, the ones where a misconception must be diagnosed, to the frontier, each with 6,000 input tokens (the student record plus the exchange in question) and 1,500 output tokens. When the AI cannot complete, a human teacher steps in, at an assumed $2 an intervention. Safeguarding concerns go to a person in every architecture.
| Frontier-only | Small-only | Hybrid | |
| AI cost per hour | $4.13 | $0.08 | $0.22 |
| Hours completed (assumed) | 85% | 60% | 83% |
| Teacher interventions (assumed) | 5% | 25% | 7% |
| Cost per completed hour | $4.97 | $0.96 | $0.43 |
Figure 5. One tutoring hour, three architectures (all inputs assumed)
Hybrid AI cost = 24 × (4,000 × $0.10 + 500 × $0.40) ÷ 1 million + 3 × (6,000 × $5 + 1,500 × $25) ÷ 1 million = $0.22. Cost per completed hour = (AI cost + interventions × $2) ÷ completion rate. Completion and intervention rates are assumptions, not results; a real comparison would measure learning, not only completed hours.
The cheapest call is not the cheapest job. The small-only tutor costs about a third as much per hour as the hybrid and more than twice as much per completed hour. The frontier-only tutor costs about eleven times as much as the hybrid for a similar completion rate. At a crore tutoring hours a month, the gap between the first and last columns is about $45 million a month.
Living Emails at billions of opens. A living email is composed when it is opened, not when it is sent, so the price, stock level, score or status it shows is current. The naive way to build one at scale is to call a model at every open. Assume 3,000 input and 300 output tokens per open. On the small model that is about $420 per million opens; on the frontier, about $22,500. At a billion opens a month, that is $420,000 or $22.5 million, before the second problem: a model call adds seconds to an experience that must load in a fraction of one.
The frugal design moves inference away from the open. Models write content in advance, when something changes: say 10,000 new items a day on a frontier model and ten variants of each on a small one. Code decides overnight, from customer data, who should see what. At open, the email reads the latest values from a database and renders them. The open becomes a lookup. Where an email client cannot render live content, a static fallback carries the same data, and consent and security rules apply to every action.
| A model at every open | Live in data | |
| Inference runs when | Someone opens | Something changes |
| Monthly inference cost only | $0.42 million (small) to $22.5 million (frontier) | About $8,000 |
| Cost grows with | Audience | Content |
| Latency at open | Seconds | Milliseconds |
Figure 6. A billion opens a month, two designs (all inputs assumed)
Both designs are assumed to deliver the same freshness and function. The comparison covers inference only; database reads, rendering, caching, evaluation and fallbacks for clients that cannot show live content are additional costs in both.
Living emails should live in data, not in inference. The email is alive because the data underneath it is live, not because a model rethinks it every time someone looks. Inference runs when something new needs writing; reads run when someone opens. That one decision turns a cost that grows with the audience into a cost that grows with the content. It is the difference between a feature that works in a pilot and one that works across billions of opens.
8
How the thesis could fail
The argument rests on claims of different strength, and they should not be bundled together.
| Claim | Likelihood | Why | What would lower it |
| Bottlenecks in memory, power and grids persist through 2027 | High | Memory shortage expected to last to at least end-2027; AWS capacity committed into 2028 | Memory supply recovers early; large cloud providers cut capital spending |
| The largest buyers get priority access; others face queues or premiums | Plausible | Big blocks are committed years ahead; on-demand is what remains | On-demand capacity stays ample and its prices fall |
| Everyday cost per task rises for ordinary users | Uncertain | Forecast for agentic workflows; token prices still falling | Measured cost per workflow falls through 2027 |
| A general shortage rations AI for everyone | Speculative | No sign yet of API access being rationed for ordinary developers | Only widespread rationing of inference would raise it |
Figure 7. The likelihood ladder
The thesis does not need a general shortage. It needs the first two rungs to hold, and a reason to care about the third. Four failure modes would undo it.
| Failure | What it would mean | What would show it |
| Efficiency outruns demand | Cost per task falls faster than agents multiply tasks, and waste stops mattering | Measured cost per agentic workflow falls through 2027; forecasts like Gartner’s are revised down |
| Overbuild turns to glut | Capacity catches up and overshoots; compute becomes cheap and plentiful | Rental prices per GPU-hour fall sharply; utilisation of new data centres drops |
| Financing breaks | Capital markets stop funding the build-out; scarcity tightens but demand weakens with it | Capital-spending guidance is cut; data-centre debt becomes hard to raise |
| Small models fail | Hybrids cannot match frontier completion on real work; routing errors cost more than they save | Replay tests show no gain in cost per completed task over frontier-only |
Figure 8. Four ways the thesis fails
Two of these deserve particular weight. A glut would remove the scarcity premium and turn the argument from strategy into housekeeping: waste would still cost money, but it would no longer cost access. And if small models fail on real work, the frugal architecture collapses into frontier-only with extra steps.
Four tests, each answerable by 2028, will decide it:
- Has commissioned AI capacity lagged announced plans because of power, memory or grid connections, rather than weak demand?
- Matched workflow cost. On the same workflows, not more ambitious ones, has the fully loaded cost per completed task risen or fallen since 2026, including retries, quality checks and human handling?
- Hybrid gain. In controlled deployments with a held-out comparison, do hybrid systems complete tasks for a third or less of the frontier-only cost per completed task, at equal completion, error and safety rates?
- India’s unit. Is India publishing GPU-hours consumed, not only sanctioned, and cost-per-completed-task benchmarks for at least one public-service workflow?
If a frontier-only system reliably completes more work at equal or lower total cost, route to it. If small models need so much supervision that their savings vanish, stop using them for that task. If hybrids win, measure the gap and keep widening it. Frugality is not an ideology. It is an experiment with a P&L and a control group.
If capacity loosens, matched workflow costs fall on their own and hybrids show no gain, the compute strait was a passing squeeze and frugality was only good manners.
**
The compute race will be decided by those who can build and finance the most capacity. A second race runs alongside it, and it will be decided by those who need the least capacity for each piece of useful work. The first rewards scale. The second rewards discipline: code before models, memory before inference, small before frontier, and paying once for what the system already knows.
The countries that own the largest compute clusters may dominate the supply of intelligence. The countries that learn to use intelligence most efficiently may capture more of its economic value.
