Advisory research · Compute · Inference economics · 2 August 2026

The inference spark spread

On 30 July a frontier lab cut the published price of one model tier by 80% in three weeks. Over the past year the neocloud GPU rental rate that tier is served on roughly doubled off its trough. Those two facts are in direct tension, and the tension has an exact form: it is a spark spread — the structure a power trader uses to price a generator's margin as the power price less the gas price times the heat rate. Written that way, the inference version reduces to a single unobservable term, tokens per GPU-hour, and the arithmetic inverts to a threshold: the serving efficiency a rate card requires in order to clear.

The heat rate is the whole difficulty. In power it is published per unit and regulated; in inference nobody publishes it. This piece computes it directly — 1,024 measured vLLM serving runs on Llama-3.1-70B across four H100s, joined sample-by-sample to the NVML power traces recorded alongside them, released under CC-BY by a national laboratory. That gives a measured denominator to put underneath the July rate cards. This is a required-efficiency bound computed from published prices. It is not an accounting of any company's margins, and nothing here is a claim about any firm's financial condition.

01The divergence, stated with verified numbers

Both legs move, in opposite directions, and both are tiered. The tiering is the first finding.

On 30 July 2026 OpenAI repriced the GPT-5.6 family, three weeks after its 9 July launch. The cut was not uniform. Luna fell 80%, from $1.00/$6.00 to $0.20/$1.20 per million input/output tokens. Terra fell 20%, from $2.50/$15.00 to $2.00/$12.00. Sol, the flagship, was left unchanged at $5.00/$30.00. Batch and Flex processing are reported at half standard price, putting Luna's batch tier at $0.10/$0.60.

The stated driver is competitive. A CNBC investigation found Chinese-origin models reaching a weekly peak of 46% of US enterprise token volume on OpenRouter, and at least 30% of it every week since 8 February 2026 — against 11% averaged over the prior twelve months and 4.5% in the first half of 2025. OpenRouter's own attribution is price: open-weight Chinese models running 60–90% cheaper than the US frontier — and at the extreme, well beyond that band: as of June 2026 DeepSeek V4 Flash listed at $0.14/M input against the then-current GPT-5.5 flagship at $5.00, a 97% gap. OpenAI framed the cut as passing through efficiency gains in its serving stack.

The demand side is moving the other way, which is what makes the divergence interesting rather than merely deflationary. Silicon Data's LLM Token Expenditure Index shows aggregate LLM spending has doubled since 2025 even as cost per token fell more than 90% over three years, and the Federal Reserve Bank of Atlanta expects firms to raise AI spend roughly 50% in 2026, to $280bn. Falling unit price with rising total spend is the P×Q problem the inference hedging piece works in its §04: every hedgeable instrument lives on price, while quantity outran it. A spark spread is a per-unit margin — it says nothing about volume, and volume is where the money has actually gone.

The compute leg is tiered too — and "GPU prices are rising" is only half true

The convenient framing is that GPU rentals rose while token prices fell. The verified picture is more specific, and the specificity matters because a lab's input cost depends on which tier it buys. Silicon Data's Q3 2026 regional cut shows hyperscaler H100 rates falling 26–54% by region since Q1 2025, while non-hyperscaler (neocloud) rates bottomed in mid-2025 and recovered strongly — US East from roughly $1.35 to about $2.90, the Nordics from about $1.75 to roughly $3.30, now above where the series started. Silicon Data computes the average premium across those seven regions as falling from roughly 250% to about 120% — a per-region average of quarterly medians, which is why the range midpoints in the table below do not reproduce it directly.

So the divergence is real but it is a neocloud divergence. A lab renting marginal capacity from the specialist tier has watched its input price roughly double off the trough while its output price on one tier fell 80%. A lab buying hyperscaler on-demand has seen its input price fall — but no lab at scale buys marginal inference capacity there. The relevant leg for this analysis is the neocloud tier, and it is the leg that rose.

---

02The spark spread, constructed

The analogy is exact in form and breaks in one specific place. Both halves matter.

A gas generator's margin is the spark spread: the price it sells power at, less the price it buys gas at, scaled by how much gas it burns per unit of power. The scaling term is the heat rate, in MMBtu per MWh, and it is the whole reason the spread is tradeable — it is published per unit, it is an engineering property of the turbine, and it changes slowly.

The inference version has the identical shape. A model lab sells tokens at a published $/M-token rate and buys GPU-hours at a rental rate. The scaling term is the number of tokens the stack produces per GPU-hour — which enters as a divisor rather than a multiplier, because it is expressed as output per input rather than input per output.

This is the cost equation from the inference hedging piece — cost = R·T·p_tok + (1−R)·T·(g/E)·k — restricted to the self-hosted channel and rearranged as a margin rather than a cost. E is the heat rate of inference. 1/E is tokens' "gas burn": GPU-hours consumed per token delivered.

Where the analogy breaks

---

03The heat rate of inference, measured

1,024 vLLM serving runs, joined to 715,989 NVML power samples over a 19.9-hour window. Same hardware, same model, same weights — only the offered load changes.

The NLR generative-AI power dataset (Vercellino et al., arXiv:2604.07345, CC-BY-4.0) was published to support data-centre load modelling, and the practice used it previously for duty factors and capacity-factor derates. It contains something the power analysis did not need: the raw vLLM benchmark logs that ran alongside the wattmeter traces. Those logs carry total_token_throughput, total_input_tokens and total_output_tokens per run. Dividing throughput by the tensor-parallel width gives measured tokens per GPU-hour directly, with no modelling step.

What this measurement is and is not

• It is a 70B-class dense reference, not a universal constant. Tokens per GPU-hour varies by an order of magnitude with parameter count, sparsity, quantization, sequence length and hardware generation. An 8B model at INT4 on the same hardware would be multiples faster; a 400B mixture-of-experts would be slower per GPU. Use the number as one anchor point on a curve, never as the curve.
• It is bfloat16 on Hopper. FP8 or INT4 weight-only quantization on the same H100 would raise it substantially, and Blackwell raises it again. The measurement is therefore a floor on what a 2026-current stack achieves on a 70B-class model — the house convention from the flat-load work: present the computed result as a bound with its assumptions attached.
• The mix is 1.1:1, not 3:1. Prefill is far more compute-efficient per token than decode, so a heavier input mix would produce a higher total-token throughput than measured here. Comparing this E against a 3:1-blended rate card is conservative in the same direction — it understates achievable efficiency.
• It is one framework, one version, one node. A frontier lab's serving stack is private and is very likely better than public vLLM defaults. Treat the measured value as a public lower bound on what a well-run operator achieves.

---

04Break-even efficiency: what the rate card requires

Set the spread to zero and solve for E. The result is the number of tokens per GPU-hour a stack must produce for a given rate card to cover a given rental rate.

This is the inference analogue of the market-implied heat rate a power trader computes as the power price divided by the gas price: the efficiency the market is implicitly demanding at prevailing prices. Cut the numerator's price and the required efficiency rises proportionally. The 80% Luna cut therefore raises the required efficiency by exactly 5×, mechanically, holding the compute leg fixed.

Required efficiency by tier and GPU price

Luna's new card, across overhead and GPU price

Sensitivity to the input:output blend

Input and output tokens carry different prices and different costs, and the blend assumption does real work. Luna's 6:1 price ratio between output and input means an output-heavy workload is better for the lab per token; an input-heavy one is worse on price but cheaper to serve.

---

05What the answer means, in three cases

The arithmetic supports a conclusion about model class, not about margins. That is a narrower claim than the headline invites, and it is the one the data will carry.

The spread, valued

Running the spread rather than the threshold, at the measured 70B-class median and a 3:1 blend:

---

06The hedge, and the answer to Pirrong

Work the sign convention and the two sides of the market fall out — including the one Pirrong argued does not exist.

Who is long the spread and who is short it

Pirrong's objection, and what survives it

Craig Pirrong's May 2026 critique — catalogued in the index methodology work — makes three arguments against compute futures: compute prices trend (tech-cycle driven) rather than mean-revert; the value chain is concentrated and vertically integrated, so there is weak natural two-sided hedging demand; and chip-specific contracts risk orphaning on 18–24 month silicon cycles, as DRAM futures did.

One objection of three is answered. That is worth stating plainly rather than claiming the critique has been disposed of — Pirrong's remaining two are the harder ones, and the efficiency basis this piece measures is an argument for his orphaning concern, not against it.

---

07What would have to exist

A tradeable inference spark spread needs three published inputs. Two exist in some form. The third exists nowhere.

---

08What this analysis is not

Repeated deliberately, because the arithmetic is more inviting than it is authoritative.

• Rate cards are not realised prices. Enterprise discounts, committed-use agreements and bundling mean published per-token rates overstate what large buyers actually pay. This is the identical critique the methodology catalogue makes of quote-based GPU indices — and the symmetry is worth naming: both legs of this spread are built from quotes, and both legs inherit the same weakness. A realised-price spread would differ, in an unknown direction.
• A lab's marginal cost is not the rental rate. Frontier labs own capacity, or reserve it at negotiated prices, or hold it through equity-linked compute arrangements. The neocloud spot rate is an upper bound on their input cost — which makes every threshold in §04 conservative in their favour, and every spread in §05 a lower bound.
• Tokens per GPU-hour varies by an order of magnitude. Model size, sparsity, quantization, batching, sequence length and hardware generation each move it multiplicatively. A single number is meaningless without its conditions. Everything here is stated with its conditions attached, and the calculator is built to be moved rather than read at its defaults.
• The input:output blend does unresolved work. Heavier input mixes lower the blended price and raise the required efficiency, but also raise achievable efficiency. This dataset has a fixed 1.1:1 prompt mix and cannot separate the two effects. That is the largest single uncertainty in the calculation.
• This is a required-efficiency bound, not an accounting of margins. It uses published rate cards and published GPU rental indices, and states what arithmetic they jointly imply about a threshold. It makes no claim about any named company's financial condition, solvency, unit economics or realised margins, and it is not a security recommendation or a short thesis. It is market-structure analysis.
• One reported input remains unverified — the H100 forward contango. See the note in §01. The measured convenience yield from the corpus is used in its place.