What is Speedrun?
Get started with SpeedrunSpeedrun is a community-driven initiative by the Marin project to track and optimize the training efficiency of large language models. Have a new architecture or training procedure that you think is more efficient? Participate in the Marin speedrun competition (inspired by the nanogpt speedrun), pick your compute budget, and create the fastest method to train a model to a certain quality!
On this page, you can find leaderboards for different speedrun tracks, each targeting a specific loss threshold. You can click on any run to view the code that generated it, or view the Weights & Biases link for the model! We also track the overall Pareto frontier of models, allowing us to track efficiency-performance tradeoffs across all tracks.
We invite you to join us in the search for more performant and efficient training methods!
Speedrun Tracks
Select a track to view leaderboard results for a specific model run. Each track now corresponds to a particular Llama model size and its best run's BPB.
Select a track to see its description and leaderboard.
Total Runs in Track
Best FLOPs in Track
Best C4-EN BPB
Overall Pareto Frontier: FLOPs vs. C4-EN BPB
Track Leaderboard
Select a track to view its leaderboard
| Rank | Run Name | Author | Date Added | Model Size* | Training Time | Total FLOPs* | C4-EN BPB | W&B Run |
|---|
* Total FLOPs here refers to hardware FLOPs performed during training