The Pulse
Trillium Labs Puts Frontier AI Training Research in Public
Trillium Labs, a nonprofit founded by Nathan Lambert and Tom Zick, plans to publish open post-training recipes, data, code, evaluations and model checkpoints. Its launch puts the case for wider scrutiny in tension with the view that restric
AI.info Team · Written with AI ·
Trillium Labs is betting that publishing research on risky AI capabilities can improve safety, while some AI companies argue that limiting access to powerful systems helps contain their risks. The nonprofit announced its launch on October 1, saying it will release detailed post-training methods and materials for outside researchers to inspect. Its founders, Nathan Lambert and Tom Zick, are challenging the idea that keeping consequential work inside a small number of labs is the safer course.
Trillium’s open recipe for post-training
The lab plans to publish post-training recipes, including data, code, evaluations and intermediate checkpoints. Post-training refers to work done after a model has been built; Trillium says making those materials available would let researchers study how training decisions shape model behavior. Outside teams could then test interventions, investigate failures and adapt the methods to questions the original researchers did not pursue.
The founders argue that researchers increasingly see the outputs of frontier systems without access to enough information to reproduce how those systems were trained. They point to earlier work, including the transformer paper “Attention Is All You Need” and OpenAI’s InstructGPT research, as examples of public methods that let researchers beyond the originating teams study and adapt influential techniques.
“Our measure of success is how much independent research those resources make possible,” the founders write in their launch announcement. They say the nonprofit will document failed runs and publish work even when results complicate its assumptions, rather than holding research back to protect intellectual property.
Transparency meets the case for restricted access
The disagreement is not simply about publishing papers. Trillium wants to share materials that could let others reproduce training experiments, while supporters of restricted access argue that keeping powerful systems in the hands of a limited group can reduce the chance of harm. WIRED’s report on the launch describes that divide and notes that frontier models can automate vulnerability discovery and probe computer systems.
Lambert told WIRED that secrecy limits outsiders’ ability to scrutinize ideas and contribute alternatives. “The current closed trajectory of frontier AI development is taking us a step backwards,” he said. Tim Fist, director of emerging technology policy at the Institute for Progress, also told WIRED he supports more transparency in research and development.
Compute and funding set the limits
Publishing a recipe is not enough to make it independently reproducible. Trillium says serious training research requires substantial compute and engineering, resources that academic researchers often lack. Its plan is to build a shared base of experiments and documentation so researchers can start from more than a model’s visible outputs.
The launch announcement names Halcyon Futures and Schmidt Sciences as early supporters and says the lab is fundraising. WIRED reports that the amount raised has not been disclosed; the founders told the publication they aim to raise $40 million to $100 million in total and plan to spend $30 million on training over the next 18 months. The company announcement also says Trillium is hiring and seeking compute support.
The test is whether others can use the work
Trillium says it will build on open research from the Allen Institute for AI, EleutherAI, OpenAthena, Nvidia and Hugging Face, while arguing that existing efforts do not provide enough capacity for the questions it wants to study. Its planned work includes how training affects model behavior and how reinforcement learning shapes capabilities and conduct. WIRED reports that the founders also want to investigate recursive self-improvement, in which AI systems contribute to developing new models.
The nonprofit’s stated measure is not how many models it releases, but how much independent research its materials enable. The first concrete commitments are a public collection of post-training data, code, evaluations and checkpoints—and an attempt to fund the compute needed to produce them.