---
title: "Mainframes became personal. So will your data center."
description: "Local models now answer 89% of everyday chat \u0026 reasoning queries as well as frontier models, \u0026 their efficiency per watt has improved 5.3x in two years."
categories: ["AI"]
date: 2026-08-21
lastmod: 2026-08-21
canonical_url: https://www.tomtunguz.com/intelligence-per-watt/
author: "Tomasz Tunguz"
---


Local AI models can already answer 89% of everyday chat & reasoning queries as well as a frontier cloud model, a result that holds broadly across more than a million real queries & 20+ local models tested.[^1]

That means we're generating more intelligence per watt of electricity : more work from the same number of electrons.[^1]

Tracking computing efficiency over time is not new. Koomey's law found that computing power per watt doubled roughly every 1.5 years for decades, a trend that shrank the power of a mainframe into a laptop's chassis.[^2][^8]

> Just as performance-per-watt guided the mainframe-to-PC transition, intelligence-per-watt will guide AI's transition to the edge.
>
> — Jon Saad-Falcon, Avanika Narayan, et al., "Intelligence per Watt"

GPUs follow a more languid curve today, doubling efficiency roughly every 2.7 years over the last 15 years, not every 1.5 years.[^3]

The impact is no less impressive : the best local model's win/tie rate against a frontier model, rose from 23.2% in 2023 to 71.3% in 2025, adding roughly 20 percentage points a year. In 2026, a local model selected for the task by another local AI bumps that number to nearly 90%.[^6]

Efficiency improved alongside the trend line : intelligence-per-watt rose 5.3x over the same period, split into a 3.1x gain from better models & a 1.7x gain from better chips. Computers are faster, models are smarter. And the end user benefits.

{{< email_image src="dae9kgsrexypykneyfwi" alt="Local models match cloud models on 71.3% of queries in 2025, up from 23.2% in 2023; a 2026 bar shows the ensemble-routed ceiling of 89%, a different metric than the single-model trend" width="540" height="285" >}}[^6]

Cloud remains essential for long multi-step reasoning, the hardest technical domains, & workloads where scale & parallelization matter. Cloud hardware still holds an edge over local hardware in many use cases. Cloud inference delivers a 40% energy efficiency gain relative to local models.[^5] Datacenters batch queries, a trick local hardware serving one user at a time cannot use, yet.

{{< email_image src="iu5udlxcbdirt2tqm8et" alt="Local AI efficiency improved 18x in 16 months, split between gains from better accelerators and better models" width="540" height="334" >}}

But for much of everyday knowledge work, there is no reason to send the query to the data center at all. The combination of local models plus a router is sufficient for the supermajority of work.[^4]

It also cuts energy 80%, compute 77%, & cost 74% against an all-cloud baseline.

Mainframes became personal. So will your data center.

[^1]: Jon Saad-Falcon, Avanika Narayan, et al., "Intelligence per Watt: Measuring Intelligence Efficiency of Local AI," Stanford University & Together AI, November 2025. [arXiv:2511.07885](https://arxiv.org/abs/2511.07885). See also the [Stanford Hazy Research overview](https://hazyresearch.stanford.edu/blog/2025-11-11-ipw).

[^2]: [Koomey's law](https://en.wikipedia.org/wiki/Koomey%27s_law), Wikipedia.

[^3]: Anson Ho, Ege Erdil & Tamay Besiroglu, "Limits to the Energy Efficiency of CMOS Microprocessors," 2023. [arXiv:2312.08595](https://arxiv.org/abs/2312.08595).

[^4]: [Most AI Work Can Wait](https://tomtunguz.com/ai-execution-routing/), tomtunguz.com.

[^5]: Jon Saad-Falcon, Avanika Narayan, et al., "Intelligence per Watt: Measuring Intelligence Efficiency of Local AI," Stanford University & Together AI, November 2025. Cloud accelerators deliver at least 1.4x higher intelligence-per-watt than local chips running the same models, roughly a 40% efficiency premium. [arXiv:2511.07885](https://arxiv.org/abs/2511.07885).

[^6]: 71.3% & 89% are two different measurements from the same 2025 data, not the same number at different times. 71.3% is the best single local model's win/tie rate against a frontier model, the trend line this chart plots (23.2% in 2023, 48.7% in 2024, 71.3% in 2025). 89% is a separate, higher ceiling: routing each query to whichever of the 20+ local models tested handles it best beats any single model by 16.3 to 28.8 percentage points. That gain is a selection effect: the paper notes local routing draws from 20+ diverse models versus three frontier cloud models, so on some benchmarks the best-of-local ensemble even surpasses best-of-cloud. More candidates to choose from, not just smarter routing, is what raises accuracy. Both figures come from the same study & neither supersedes the other.

[^8]: Watts measure power, the rate energy is drawn, & joules measure the energy itself; Koomey's original metric was computations per joule, but the underlying trend is the same one intelligence per watt now tracks for AI models.