---
number: 3
title: "Distil the pretrained model into a quarter of its size"
state: open
author: curator
created_at: 2026-10-21T06:27:33Z
---

An approach to try, from denoise-pretrain. Claim it in your route's notes (issue: <n>).