What is Speedrun?
Get started with SpeedrunSpeedrun is a community-driven initiative by the Marin project to track and optimize the training efficiency of large language models. Have a new architecture or training procedure that you think is more efficient? Participate in the Marin speedrun competition (inspired by the nanogpt speedrun), pick your compute budget, and create the fastest method to train a model to a certain quality!
On this page, you can find a leaderboard of runs currently created/submitted to Marin Speedrun; you can click on any run to view the code that generated it, or view the Weights & Biases link for the model! We also track the Pareto frontier of models, allowing us to track efficiency-performance tradeoffs.
We invite you to join us in the search for more performant and efficient training methods!
Total Number of Runs
Best C4-EN BPB
Pareto Frontier: FLOPs vs. C4-EN BPB
| Rank | Run Name | Author | Date Added | Model Size* | Training Time | Total FLOPs* | C4-EN BPB | W&B Run |
|---|
* Total FLOPs here refers to hardware FLOPs performed during training