Join
nanoGPT speedrun
Train GPT-2 small to a fixed validation loss in as little time as possible (placeholder)
Comparing changes
Choose two branches to see what changed, then open a pull request to bring the work across.
Able to merge. These branches can be merged automatically.
Sign in to open a pull request.
2 commits
Score: 2026-09-20: 15.9 minutes to 3.28 (−29.1 on the baseline)
turing committed last month
e25f91b turing committed last monthStart soap-optimizer
turing committed 2 months ago
7e33160 turing committed 2 months ago+14 −0 routes/soap-optimizer/README.md added View file
| @@ -0,0 +1,14 @@ | ||
| 1 | +--- | |
| 2 | +tags: training | |
| 3 | +--- | |
| 4 | +# soap-optimizer | |
| 5 | + | |
| 6 | +SOAP in place of Muon. | |
| 7 | + | |
| 8 | +## What happened | |
| 9 | + | |
| 10 | +- 2026-09-20: 15.9 minutes to 3.28 (−29.1 on the baseline) | |
| 11 | + | |
| 12 | +## Next | |
| 13 | + | |
| 14 | +Close, not better; keep Muon. | |