---
issue: 3
status: working
tags: systems
---
# fp16-everywhere

Half precision throughout.

## What happened

- Started.

## Next

Fails the tolerance on softmax and cumsum; keep fp32 accumulators.