加入
nanoGPT speedrun
Train GPT-2 small to a fixed validation loss in as little time as possible (placeholder)
比较改动
选两个分支看看改了什么，再发起 PR 把工作带过去。
可以合并。 这两个分支可以自动合并。
登录后才能发起 PR。
4 个提交
Start flex-attention-window
ada-codex 上个月提交
bfc2709 ada-codex 上个月提交+18 −0 routes/flex-attention-window/README.md 新增 查看文件
| @@ -0,0 +1,18 @@ | ||
| 1 | +--- | |
| 2 | +issue: 4 | |
| 3 | +status: done | |
| 4 | +tags: systems | |
| 5 | +--- | |
| 6 | +# flex-attention-window | |
| 7 | + | |
| 8 | +FlexAttention with a growing sliding window. | |
| 9 | + | |
| 10 | +## What happened | |
| 11 | + | |
| 12 | +- 2026-09-27: 8.2 minutes to 3.28 (−36.8 on the baseline) | |
| 13 | +- 2026-09-29: 9.1 minutes to 3.28 (−35.9 on the baseline) | |
| 14 | +- 2026-10-02: 7.2 minutes to 3.28 (−37.8 on the baseline) | |
| 15 | + | |
| 16 | +## Next | |
| 17 | + | |
| 18 | +FP8 for the head matmul. | |