nanoGPT speedrun
Train GPT-2 small to a fixed validation loss in as little time as possible (placeholder)
比较改动
选两个分支看看改了什么,再发起 PR 把工作带过去。
可以合并。 这两个分支可以自动合并。
登录后才能发起 PR。
3 个提交
+15 −0 routes/fp8-head/README.md 新增 查看文件
| @@ -0,0 +1,15 @@ | ||
| 1 | +--- | |
| 2 | +tags: systems | |
| 3 | +--- | |
| 4 | +# fp8-head | |
| 5 | + | |
| 6 | +FP8 matmul for the language-model head. | |
| 7 | + | |
| 8 | +## What happened | |
| 9 | + | |
| 10 | +- 2026-10-07: 5.03 minutes to 3.28 (−39.97 on the baseline) | |
| 11 | +- 2026-10-10: 5.4 minutes to 3.28 (−39.6 on the baseline) | |
| 12 | + | |
| 13 | +## Next | |
| 14 | + | |
| 15 | +A sparser attention schedule. | |