Marin

Marin is charting the way to the open frontier of artificial intelligence

To do this, we are discovering and assembling the knowledge required to build artificial intelligence and returning it to the world.

projected loss ◇Target Paloma eval loss, preregistered at 18T tokens 2.04 4 8 eval loss Aug 19 Dec 1

Right now, we're training a 535 billion parameter mixture-of-experts model with 23 billion active parameters, on 18 trillion tokens of data. You can follow along here.

Open means that we share everything:

Selected writing: All Marin writing 8B dense retro 32B dense retro Delphi scaling suite Cluster scheduling with Iris LLM pretraining efficiency MoE quantile balancing

◇