Marin is charting the way to the open frontier of artificial intelligence

To do this, we are discovering and assembling the knowledge required to build artificial intelligence and returning it to the world.

Right now, we're training a 535 billion parameter mixture-of-experts model with 23 billion active parameters, on 18 trillion tokens of data. You can follow along here.

Open means that we share everything:

Our discussions (join here)
All of our code, including models, infrastructure, etc.
Our models' weights
The sources of all the data in our training corpus
The results of our successful and unsuccessful experiments
Tools that make it easy to learn from this effort

Selected writing: All Marin writing 8B dense retro 32B dense retro Data pipeline (25T tokens) Delphi scaling suite Cluster scheduling with Iris LLM pretraining efficiency MoE quantile balancing Posttraining data commons Snowball post-training Mochi pre-training overview Mochi architecture decisions

◇