---
tags: training, model
---
# tuned-lr-rotary

A tuned learning rate, rotary embeddings and a trapezoidal schedule.

## What happened

- 2026-08-24: 31.4 minutes to 3.28 (−13.6 on the baseline)
- 2026-08-27: 33 minutes to 3.28 (−12 on the baseline)

## Next

Try another optimizer for the hidden matrices.