---
issue: 1
status: working
tags: training
---
# muon

Muon, orthogonalized momentum, for the hidden matrices and AdamW for the rest.

## What happened

- Started.

## Next

Tune the momentum warm-up, then the architecture.