By end of 2027, at least two top-tier AI labs will offer a generally available high-throughput serving tier of ~500+ tokens/second on non-GPU silicon

The prediction

In mid-2026 the fastest frontier-model serving moved from open-weight demos to a proprietary flagship: OpenAI began serving GPT-5.6 Sol on Cerebras wafer-scale silicon at roughly 750 tokens per second — about fifteen times a typical GPU stack — but only for a hand-picked set of select customers. Cerebras separately serves trillion-parameter models such as Kimi K2.6 near 981 tokens per second. The capability exists; what is rationed today is access. Extrapolating the axis shift from capability to latency:

By December 31, 2027, at least two of the top-tier AI labs (measured among OpenAI, Anthropic, Google, xAI, Meta, and DeepSeek) will offer a generally available — self-serve, any-developer, publicly listed — high-throughput serving tier that delivers at least roughly 500 output tokens per second on a frontier-class model, running on wafer-scale or other non-GPU inference silicon.

"Generally available" means a documented, purchasable option any developer can select through the standard API or platform — the way context length and price tiers are selectable today — not a private arrangement limited to select customers or a single flagship enterprise. It must be a frontier-class model, not a small distilled one, and the throughput must be a published or independently measured single-stream figure at or above roughly 500 tokens per second.

Why 62 percent confidence

The direction is well-supported. Capability at the frontier has converged, which forces competition onto cost and speed; wafer-scale and alternative silicon already demonstrate the required throughput on frontier-scale models; and one lab has already put a proprietary flagship on that substrate. Turning a select-customer deployment into a listed, self-serve tier is a productization and capacity step, not a research one — exactly the kind of move labs make when an axis becomes strategically important, and the incentive to differentiate on latency is only strengthening as agentic workloads dominate.

Confidence is held at 62 rather than higher because the binding constraint is supply and business model, not technology. Wafer-scale capacity is scarce and expensive to build, so labs may keep the fast tier as a rationed premium for large accounts well past 2027 rather than opening it to every developer. "At least two labs" is also a demanding bar within eighteen months — one lab crossing is plausible, two crossing and publicly listing the tier is a stronger claim. And "generally available" could be met in spirit but fail on the letter if the fast path stays gated behind enterprise sales.

What would falsify it

If, at December 31, 2027, fewer than two of the named top-tier labs offer a self-serve, publicly listed high-throughput tier at roughly 500 tokens per second or more on a frontier-class model — because the fast path remains confined to select-customer deals, single flagship demos, or private enterprise arrangements — the prediction is wrong. A capability that exists but stays rationed does not satisfy it; the claim is specifically about latency graduating from a deployment footnote to a purchasable, generally available axis of the product, the way context length and price already did.