dew.training.quantization
The training step that quantizes the modules an objective trains (dew.quantization).
| Name | Summary |
|---|---|
quantize_trunk | Quantize the trunk matmuls of the modules objective trains, in place. |
quantize_trunk
Section titled “quantize_trunk”def quantize_trunk(objective: Objective[LossT, EffectsT], spec: Quantization) -> NoneQuantize the trunk matmuls of the modules objective trains, in place.
This is RunConfig.train’s step, not a user’s. Quantization.apply
wraps a module before an objective is built, which is what a recipe or
a script that builds its own model does. A run that names
--trainer.quantization has handed RunConfig.train the objective
already, so the wrap lands on the modules the objective runs instead
(Objective.substitute), before anything has initialised or traced
them; each wrapped module is a copy of the same class, so what the
objective read off it at construction still holds. A frozen teacher or
reference keeps its own numerics.
An objective that trains no module has nothing to wrap and is refused by name.