Ai2 參與 COLM 2026：從模型到科學研究智能體的開放研究

This week Ai2 was at COLM 2026 in San Francisco. The work we shared reflects our wider mission: building AI research artifacts anyone can study and extend, and developing AI for science tools in collaboration with domain experts.

Our evening gathering gave colleagues from across the COLM and wider open-source community another setting to connect.

Those exchanges are central to how we approach research. As a fully nonprofit lab, we pursue fundamental scientific questions while sharing the models, data, code, and methods that let others investigate them too. COLM brings us together with the people who use our resources we release, challenge our assumptions, and contribute their own ideas.

Sharing the work in open collaboration with others isn’t an afterthought for us—it’s a fundamental part of how we approach research.

From Olmo Hybrid to the next Olmo

Our paper “Olmo Hybrid: From Theory to Practice and Back”, presented at the conference, examines what happens when a language model combines transformer attention with linear recurrent layers. Attention can retrieve specific details from earlier text; recurrent layers maintain a compact state that updates as the model processes a sequence of text tokens.

The research connects theory about what these architectures can represent with controlled training experiments. Olmo Hybrid provides evidence that combining attention and recurrence can improve training efficiency: in the study, it reached the same accuracy as Olmo 3 7B on MMLU, a benchmark covering knowledge across many subjects, using 49% fewer training tokens.

That research is helping inform the next Olmo model, now in pre-training with a hybrid mixture-of-experts architecture. The hybrid design combines attention and recurrence. Mixture-of-experts offers another way to improve efficiency, routing each token through a subset of the model’s components.

We are also opening up the infrastructure behind this direction. Released ahead of COLM,

Olmo-core 3

adds a redesigned system for training large mixture-of-experts models. Researchers can use the code to develop their own models and investigate training decisions, alongside the checkpoints and technical reports available through the broader Olmo ecosystem. And Google

recently announced

that they recreated our Olmo 3 7B training run in MaxText on Google Cloud TPUs, highlighting the reproducibility that our fully open approach enables.

In our conversation from the conference, Noah A. Smith, Ai2’s senior director of NLP research, joined our comms lead Kyle Wiggers to discuss the next iteration of Olmo and our work on agentic models:

Developing AI for science with scientists

Scientific tools need to fit the tasks scientists actually do. That means creating them with researchers and studying both their capabilities and their limits—which we are doing in our development of Asta, our agentic platform for scientific work.

The recently released

AstaBrief

is an open-weights model that generates cited reports from research questions and retrieved literature. Scientists can use it in Asta’s Fast mode or download the model to run on their own infrastructure.

We are also working directly with scientific communities to understand where general-purpose models fall short and what future open models should support.

Ai2 senior research scientist Bodhisattwa Prasad Majumder was at COLM to further these conversations, presenting a number of relevant papers:

Bolmo in Nature

During the conference, our paper “Retrofitting language models to operate over bytes”, the research behind Bolmo, was published in Nature. Most language models first divide text into chunks from a fixed vocabulary. Bolmo works with the underlying bytes that represent characters, allowing it to handle details that subword tokenization can obscure. Our paper describes how to convert an existing language model to this approach with a relatively short additional training run.

Alongside the publication, we released additional checkpoints so researchers can experiment with the architecture themselves. Publishing the findings and opening the materials behind them gives others a way to test the approach and take it further.

Thank you to everyone who connected with us at COLM, joined our evening event, and shared their work with us. The papers and tools above offer ways to continue the discussion—read the research, try the models, and tell us what you find.

If you would like to help build this work, explore our open roles.