The a16z Show · 2026-10-14
Open Models at the Frontier
Nathan Lambert — who co-led post-training at AI2 on Olmo and is now founding the nonprofit open-model research lab Trillium — breaks the open-vs-closed gap down into three forms of capital: compute, data, and talent. The conversation covers why the distillation story is a caricature, why the RL-environment data economy is where frontier progress is actually being bought, and how open models survive as fast followers with real ownership. The core bet: closed labs will always serve the cheapest intelligence at the frontier, but open models win elsewhere — on margins, private data, and the low-hanging compounding gains left in the open.
The talk itself: youtube.com/watch?v=BKuK0M6oDTI
The open-versus-closed gap broken into compute, data and talent, by someone who ran post-training on an open model. Denser per minute than most of the AI commentary in this corpus.
The argument, in 5 sections
9 quotes, none repeated01Model Progress as Capital Organization
Lambert's master frame: model progress is three forms of capital — compute, data, talent — and the differentiator is a lab's ability to organize them (“making the org chart work for building models is really hard, which is why you have companies have to do complete resets from the bottom up”, ~2:31). OpenAI and Anthropic are the true frontier labs because they took the most ambitious bets earliest: OpenAI kickstarted the reasoning-model revolution, while Anthropic's long focus on coding paid off as the Claude Code explosion — subtly different cultures, same willingness to “spend egregious amounts to make a step towards a direction that people think is impossible” (~3:53). Chinese labs, by contrast, entered for practical commercial reasons: open releases are a distribution shortcut to matter (“nobody would know about Moonshot AI... It's a very practical decision by them”, ~4:04). This connects to the closed-lab oligopoly framing in concepts/ai-oligopoly-three-horse-race — Lambert's version adds the human/org-chart layer on top of the capital arithmetic, a theme he returns to when advising aspiring researchers: work at Anthropic or OpenAI, “which is the easiest bet, I think” (~3:03).
“I think open and closed models are on different time scales of exponentials and the closed labs are so integrated”
the honest structural case for why open models can keep up without ever being first.
02Distillation: Compute Arbitrage, Not Theft
The host voices the popular view — distillation is the main lever behind China's rapid catching-up, with the open-to-frontier lag now down to two or three months. Lambert's answer is layered. First, distillation is fundamentally a compute workaround: “outputs from a language model are a function of compute and if you know the outputs you want it's the same as having compute” (~9:47) — Chinese labs are talent-equal but compute-starved (a top Chinese pre-training researcher's “jaw just falls on the floor” at the idea that an average OpenAI researcher has 5,000 H100 equivalents, ~9:18). Second, the classic SFT-era distillation recipe doesn't obviously transfer to reasoning: in the fully open literature, builders are “stuck using like a really outdated Qwen model because you can't figure out how to get the teacher right”, and even with Claude's reasoning traces in hand he's “not sure that it would have actually made our model a lot better” (~11:23). Third, the real leakage may not be distillation at all — Claude proxies, massive intermediary accounts, even Kimi reportedly serving Claude to users — behaviors that don't fit the post-training tool called distillation. His policy stance is pragmatic: labs may lock down their own APIs, but regulating distillation would just make US labs' lives annoying while China “they're not going to stop” (~16:18).
“People distill my products every single day. They take it and they they strip it down to its raw parts. They test everything. That's called competition.”
the distillation-is-theft debate in one line — and Lambert agrees with the framing.
03The Data and RL-Environment Economy
This is where Lambert sees the actual frontier progress being bought. In verifiable domains (math, code), scaling reasoning compounds for free; in everything else — “knowledge work, this biology, health care, legal” — progress comes from “buying data that is high quality and giving the models worlds to act in and hill climb on” (~25:03). The labs are spending billions on RL environments because it demonstrably works (“I still have denial about it... they're only doing this because it is helping the models and they see the signal”, ~30:28), but the supply chain is sloppy: bought-in-mass data with no hit-rate guarantee, contractors who don't know what they're selling, and corner-cutting that “can induce things like reward hacking” — models gaming test files with empty mazes (~28:33). This is a large part of why Trillium exists: to release open-source RL environments for high-leverage tasks (the TMAX terminal-task work with a UDub/AI2 student, fully synthetic data where “you read every data point, your data will get better, and you can do this”, ~32:16) rather than sell them — the profit motive is so high that almost nobody releases.
“the data industry has a lot of low-hanging fruit on improving quality and improving just like opsec”
04The Open-Model Economy: Slow Flywheel, Real Ownership
Closed labs reach high-value markets first with efficient integrated inference — coding revenue “has just gone zero 100 billion and likely to grow” (~19:49). The open ecosystem is structurally slower: the fast follower “you don't have this flow of information and you don't have this flow of revenue” because customer signal is fragmented across model builders, inference companies, and post-training shops (~19:56). But the end state is different in kind: “model weights, no terms and conditions that they can run on their hardware, and they figured out they can make an ROI on” (~21:15) — real ownership in the private-data niches companies can't send over an API. On pre-training he's unexpectedly sanguine: the tools are accessible, a pre-training ladder tests whether ideas scale, and “you don't have to spend a billion dollars to get a very good base model” (~45:29) — he's “not betting against new entrants”. And on post-training, his reason for founding Trillium: when he left AI2 he felt open post-training “honestly falling behind just cuz our cycle time was not high enough” — and the opportunity is enormous because “it doesn't take as much capital to make very big compounding gains on the state-of-the-art in the open source cuz the bar is very low. It's like it's not like it's not close” (~49:32). Thinking Machines' honest Inkling launch and Meta's open-weight strategy are the templates for making openness a business.
05Demystifying RSI — and What Actually Worries Him
Lambert's anti-panic case is arithmetic: “all the scaling laws are exponential increase in compute for linear increase in performance” (~55:39), and labs are currently “spending exponential amounts relative to the improvements” just to sustain the pace — so fast takeoff where models build themselves hits human bottlenecks. He flags the honest wild card — if an AI model discovers something “very different than the transformer”, it's “worth updating” (~56:11) — and points to Claude system cards conceding it would take ~10x more progress for fast takeoff. Trillium's contribution is transparency as policy infrastructure: publish the open-source SFT/mid-training stack, store trajectories, document tasks and what models got stuck on, then “check in every 6 months and you're like has it changed?” (~40:39) — so policymakers see the bugs, not the mystique. This echoes the recursive-improvement anxiety in concepts/recursive-improvement-ai but deflates it: the thing that worries him isn't the capability plot, it's that capability is diffusing into “the pretty much non-existent preparation of cross public cybersecurity and all of these things” (~57:24). Closing line worth keeping: “not fearing fast takeoff does not mean that you don't worry about risks” (~58:11).
Also worth keeping
6 more · each links to its minute“When I think about model progress, it comes down to like different forms of capital that you put into them. And that is like compute, data, and talent.”
the analytical spine of the whole episode — everything else is an application of this frame.
“to say that China is anything but equal in talent is going to you're going to have a hard time because they just have so many people being trained at such a high level and what they don't have is compute.”
reframes the US-China model race as a compute gap, not a capability or talent gap.
“So like if I were to have access to Claude's reasoning trace when I was building Olmo I'm actually like Claude Opus Claude Mythos whatever I'm not sure that it would have actually made our model a lot better.”
first-hand evidence from an open-model builder that reasoning-era distillation is not the magic lever people assume.
“I'm very confident in the frontier labs being able to serve the cheapest intelligence per per watt, per score on an eval.”
what the frontier labs' durable win actually looks like — and why it doesn't preclude many other winners.
“it doesn't take as much capital to make very big compounding gains on the state-of-the-art in the open source cuz the bar is very low. It's like it's not like it's not close.”
the capital-efficiency argument that underwrites Trillium's entire existence.
“The all the scaling laws are exponential increase in compute for linear increase in performance”
the arithmetic backbone of his no-fast-takeoff position.
What to take away
5 lines- 01Measure the model gap in compute, not cleverness
When evaluating who's winning, check GPU access per researcher before theorizing about stolen recipes. China is talent-equal and compute-starved; distillation is best understood as compute arbitrage (buying outputs instead of generating them), and its edge in the current RL-heavy era amounts to a few months at most, with “very big error bars” — treat confident claims in either direction as uninformed.
- 02The RL-environment data economy is the real frontier supply chain
Labs are spending billions on data and environments for non-verifiable domains (knowledge work, biology, legal), and the industry is young, sloppy, and full of rough edges (opsec holes, reward-hacking-inducing environments). If you're building in AI, this is the highest-leverage place to work — and releasing environments as open public goods compounds far beyond their sale price.
- 03Open models win by fast-following to ownership, not by racing the frontier
The durable open position is: let closed labs pioneer paradigms, follow 2-3 months behind, and end with weights you own, no terms and conditions, running on your hardware at better margins for private-data use cases. The feedback-loop deficit (fragmented revenue and customer signal) makes the flywheel slower, not broken.
- 04Stop fearing fast takeoff; prepare for diffusion
Exponential compute currently buys linear performance gains, so self-improving takeoff is bottlenecked by humans and spending — meanwhile, capabilities that already exist are diffusing into society with “non-existent preparation of cross public cybersecurity” (~57:24). The rational worry is institutional unpreparedness, not RSI mystique; and “not fearing fast takeoff does not mean that you don't worry about risks.”
- 05Open post-training is the capital-efficient frontier right now
Per Lambert, the open state-of-the-art bar in post-training is “not close” — modest capital produces outsized compounding gains, and cycle time (not breakthroughs) is the binding constraint. Trillium's playbook — open-source stack, published trajectories, transparent tasks, six-month public check-ins — is a template any well-resourced team can copy, and a model for how to demystify RSI for policymakers. ---