dew.nn.protocols
What a model can be asked for, as structural protocols.
An objective, a task or a server reads a model through these capabilities
instead of asking which class it is. A model has a capability by defining
the method; there is no Dew base class to inherit and no wrapper around a
model. A caller checks one with isinstance(model, Logits) and calls it
through Flax’s own apply, with rngs and mutable passed to apply:
logits = model.apply(variables, tokens, method='logits')states = model.apply(variables, tokens, train=True, method='hidden_states', rngs={'dropout': key})table = model.apply(variables, method='output_table')Logits and HiddenStates are token models’ full-sequence reads, with no
cache: a call writes no cache collection, and TokenModel says whether
those states attend causally and names the id masked diffusion corrupts
tokens to. DenoisingModel is the call every image and video
denoiser already has, the raw network that dew.diffusion.process.Denoiser
wraps into a prediction a solver steps with. AffineHead and
LogitsFromHidden let a loss score the vocabulary without a second trunk
pass: output_table gives the matrix a tiled loss contracts in place of the
logits, and logits_from_hidden is the exact head for a head no matrix alone
gives.
DenoisingModel is an annotation only: every Flax module has a __call__, so
an isinstance check on it would hold for any model. RequiresText,
IntervalModel and TimeScaled are what a denoiser declares about the
text and time it reads.
The serving and training hooks are read off the model itself, not through
apply, from the layers it declares; a wrapper answers for its decoder.
The keywords a model takes past its tokens are ModelKwarg values:
ModelInputs.kwargs() gives a request’s token fields and conditioning, and
callers add flags and absent values.
| Name | Summary |
|---|---|
AffineHead | A model whose logits may be one matrix product of its final states. |
BlockDenoiser | A diffusion language model that generates a canvas at a time, as DiffusionGemma does. |
BlockDrafting | A decoder that may carry a block drafter: DeepSeek’s DSpark (V4.1, V4-Flash-0731), which drafts a whole block per pass from the context its target layers record instead of chaining prediction depths. |
Bounded | A model that declares its vocabulary and cache capacity, which the host checks read to refuse an id outside the vocabulary or a request that would overflow the cache; a model that declares neither is checked for neither. |
CacheCapacity | A decoder at another decode-cache capacity: the same variables, and the same draws for a request that fits. |
Capability | What a consumer can need of a model past its protocols’ presence: its logits (Logits, or states through a head), causal or bidirectional attention (TokenModel.causal), packed rows apart, a token by token decode cache (Decoding), an interval’s duration (IntervalModel), a guidance input, block outputs a discriminator reads, or a canvas against a committed cache (BlockDenoiser). |
Counted | Analytical forward FLOPs summed over real row lengths, on the host. |
CountedCondition | A condition encoder owns its work and the geometry its denoiser reads. |
CountedConditionLayout | A denoiser selects the valid keys or physical positions its forward reads. |
CountedConditionedSpatial | A spatial model also consumes the encoder’s pooled and time-id geometry. |
CountedDecisions | A decision head counts the layout’s real questions and options with its rows. |
CountedImages | A patch-token model’s forward work with channel/conditioning geometry. |
CountedLogits | A counted token backbone with separate vocabulary-head work. |
CountedLogitsTraining | An adapted LM head’s training work, separately from its backbone. |
CountedMatrixUsage | A model’s declared call geometry for an adapted native matrix path. |
CountedMultimodal | A joint-stream model whose input maps infer their widths at init. |
CountedParameters | Forward work of learned matrix maps, separately from state/attention MACs. |
CountedPredictionParameters | Prediction depths declare their learned maps and adapted scopes separately. |
CountedPredictionTraining | An adapted prediction stack’s frozen-base and trainable-factor training work. |
CountedPredictions | The extra decoder depths and heads trained beside a language model’s trunk. |
CountedSequences | A joint/cross-attention backbone with separate image and text geometry. |
CountedSpatial | A convolutional denoiser with a resolution-specific stage walk. |
CountedTraining | An adapted backbone’s training work, including its factors, not 3*forward. |
CountedVideo | A factorized video model with independent frame and spatial axes. |
DecoderTraining | What a decoder trains beside its next token: its multi-token prediction depths (arXiv 2412.19437, section 2.2; apply’s method='mtp_hidden_states' and 'mtp_logits', Predicting), which with mtp_hyper_connections set read the residual streams a forward sows under prediction_inputs/states, and the MLA mixers it declares with DeepSeek’s lightning indexer. |
Decoding | A decoder that continues its rows token by token: init_cache opens the cache its decode calls (decode=True) write, a position a call. |
DenoisingModel | A diffusion network’s output for sample at time, given its conditions: arrays, encoded text (TextContext) or a process’s DenoisingCondition. |
ExpertRouted | A model that may route tokens to experts: the routers among its own whose expert weights are fixed whatever they score (dew.nn.backbones.decoder_block.Mixture.fixed_weights), each by name and whether a parameter, by its path in the model’s own tree, is its, which no term of a plain loss reaches (dew.objectives.routing.refuse_fixed_router). |
Exposing | A decoder that hands back its hidden states beside its logits, which a drafting strategy seeds from; without them a model decodes but cannot draft. |
Generating | A decoder that says, after examining the layers it builds, why a generation loop cannot continue its rows token by token, or None; a sampler refuses with that reason without knowing where a family keeps the layers that cause it. |
HiddenStates | A token model’s [B, S, features] final states, the head’s input. |
IntervalModel | A denoising model that, with its interval field set (model.clone(interval=True)), embeds the duration an interval process hands it beside its time, a missing one as zero; unset, it refuses one. |
Lacks | A consumer’s refusal of a model for a capability it lacks, raised where the consumer first reads it: as it admits the model, or in the step that reads it. |
LayerScalars | A model with per-layer output scalars that a fine-tune trains, as DiffusionGemma’s published SFT does: with_trainable_layer_scalars is the model that trains them, and trainable_variables the source’s tree as that model reads it. |
LayerStates | A model whose forward keeps its layers’ residual states, each layer’s output before the final norm, for a reader of a middle layer’s features: applied with capture_intermediates=model.capture_layers and model.layer_collections mutable, the collections it returns hold them, and model.layer_state(collections, index) reads layer index’s (0-based; -1 is the last). |
Layered | A decoder that declares its decode cache’s layout, as CausalTransformer does, and gives the same model over another (with_kv_cache): a wrapper answers for the decoder that holds the cache. |
Logits | A token model’s [B, S, vocab] fp32 logits over the whole sequence. |
LogitsFromHidden | The exact head from final states, for scoring states a loss already holds. |
Looping | A model whose layer stack can run a block of it several times over its own output, and the same model at steps passes of its block (None keeps its count), or of the block layers, [first, end), which gives a model with no loop one; None where they cannot. |
ModelKwarg | Documented in dew.nn.inputs. |
OutputTable | A vocabulary head as a matrix a loss contracts: matrix as the tree stores it, [vocab, features] when vocab_major and [features, vocab] otherwise, an optional fp32 [vocab] scale multiplying the product (nGPT’s per-token logit gain) and bias added after it, the softcap applied last, and the precision the product runs at. |
Predicting | A decoder that declares how many multi-token prediction depths it carries. |
ProjectionGroup | Projections under path in params that read one input: members, widths wide, concatenate on the output axis into packed, which their module reads in their place, one product for a decode step. |
ProjectsParameters | A model whose training keeps a constraint on the parameters it stores. |
QuantizesInputs | A model computing under a checkpoint’s input quantization (dew.quantization.InputQuantization): the quantizer a bound module’s input takes, which a head read as a table (OutputTable) applies as the module’s own call does. |
ReadsClips | An encoder that says whether it reads a clip’s frames, (B, T, H, W, C), or an image: a run over clips takes one that does, and a run over images one that doesn’t. |
ReadsTrain | A module whose call takes train, for a stochastic layer of its own (a token mixer’s or a feed-forward’s dropout, a vision tower’s drop path): its owner passes train where reads_train says the call reads it, whatever the module’s family. |
Recomputing | A model one remat rung up (None at the top), its rung as a checkpoint records it, and the model at a record’s rung where that is above its own: a resumed run never compiles a lighter rung than it trained on. |
RematChoosing | A decoder whose layers start at a chosen RematPolicy, or None for no remat. |
RequiresText | A denoising model that cannot run without text, which reaches it under the keyword text_keyword on every call. |
Selective | A decoder that scores one position per row instead of them all, so its prefill runs the head only on the slot the first draw reads. |
Serving | What a server reads off a decoder. |
StagedStacks | A model whose stacks of like layers a pipeline over the stage axis runs: where in store, the variables a stage-local state is made from, each stack’s run lies and how the pipeline over stages stages stacks it (DecoderBank), so a trainer keeps that run stacked by stage on the stage’s own devices (dew.training.stages). |
TimeScaled | A denoising model whose time_scale field is its time’s unit: at scale s on time t (and duration d) it computes what it does at s / k on k t (and k d). |
TokenModel | Whether a token model’s hidden_states attend causally (each position to itself and the ones before it) or to the whole sequence, and the vocabulary id masked diffusion corrupts tokens to, None without one. |
TritonGemm | Whether a mixer of a model keeps XLA’s Triton GEMM fusions (MixerBase.keeps_triton_gemm). |
VocabularyRows | A model whose head scores its final states against one row per vocabulary token: each token’s [vocab, features] row as OutputTable.rows gives it, read from the bound variables, where no matrix alone is the head too (an AffineHead returning None): past a nonlinear head transform the matrix’s rows, with an adapter on the head its rows as the adapter updates them. |
label_logits | model’s own logits [B, N, K, V] at each of the [K, V] label ids of each answer state of answers [B, N, features], a class read as its label’s tokens’ logits (a verbalizer): its output table at the labels’ columns alone, computed in fp32 (OutputTable.columns), where one matrix is its head (head_table); else its exact head over the answer states, a few rows (LogitsFromHidden), the labels’ columns taken. |
AffineHead
Section titled “AffineHead”class AffineHead(Protocol)A model whose logits may be one matrix product of its final states.
It reads the variables apply binds, so an adapter’s frozen base weights
are there too, and apply copies nothing: under a gradient the matrix is
the caller’s parameter, which a loss keeps for its backward pass.
AffineHead.output_table
Section titled “AffineHead.output_table”def output_table() -> OutputTable | NoneBlockDenoiser
Section titled “BlockDenoiser”class BlockDenoiser(Protocol)A diffusion language model that generates a canvas at a time, as
DiffusionGemma does. init_cache allocates batch_size rows of a
prefix cache, and encode appends clean tokens to it: a prompt, with its
ModelInputs fields, or a committed canvas. Calling the model refines a
canvas_length canvas against that frozen cache and returns its [B, canvas_length, vocab_size] logits, self-conditioned on the previous step’s
(self_conditioning_logits, a zero signal in rows whose
self_conditioning_mask is false); max_seq_len bounds the prompt and
its canvases. A canvas is refined at the logical positions that continue
its prompt’s, None where the prompt gives none and the cache’s cursor
places it, and committed at them laid out as its coordinates, the one
layout a method past __call__ reads (ledger 1122). A causal language
model scores tokens too, but runs none of these, so it is not one.
BlockDenoiser.init_cache
Section titled “BlockDenoiser.init_cache”def init_cache(batch_size: int) -> NoneBlockDenoiser.encode
Section titled “BlockDenoiser.encode”def encode(tokens: jax.Array, *, coordinates: Coordinates | None = None) -> jax.ArrayBlockDrafting
Section titled “BlockDrafting”class BlockDrafting(Protocol)A decoder that may carry a block drafter: DeepSeek’s DSpark (V4.1, V4-Flash-0731), which drafts a whole block per pass from the context its target layers record instead of chaining prediction depths.
Bounded
Section titled “Bounded”class Bounded(Protocol)A model that declares its vocabulary and cache capacity, which the host checks read to refuse an id outside the vocabulary or a request that would overflow the cache; a model that declares neither is checked for neither.
CacheCapacity
Section titled “CacheCapacity”class CacheCapacity(Protocol)A decoder at another decode-cache capacity: the same variables, and the same draws for a request that fits.
CacheCapacity.with_cache_capacity
Section titled “CacheCapacity.with_cache_capacity”def with_cache_capacity(capacity: int) -> SelfCapability
Section titled “Capability”What a consumer can need of a model past its protocols’ presence: its logits (Logits, or states
through a head), causal or bidirectional attention (TokenModel.causal), packed rows apart, a token
by token decode cache (Decoding), an interval’s duration (IntervalModel), a guidance input, block
outputs a discriminator reads, or a canvas against a committed cache (BlockDenoiser).
Counted
Section titled “Counted”class Counted(Protocol)Analytical forward FLOPs summed over real row lengths, on the host.
A token backbone counts its hidden states, excluding the vocabulary
head, which an LM objective adds through CountedLogits; a decision
objective instead adds its decision head’s work. Lookup embeddings,
rematerialization and padded tokens are excluded. Geometry beyond one
sequence is supplied by the objective through a family’s named count.
A family that cannot count it raises MissingFlopsCount, never HLO.
Counted.model_flops
Section titled “Counted.model_flops”def model_flops(lengths: np.ndarray) -> floatCountedCondition
Section titled “CountedCondition”class CountedCondition(Protocol)A condition encoder owns its work and the geometry its denoiser reads.
CountedCondition.conditioning_flops
Section titled “CountedCondition.conditioning_flops”def conditioning_flops(tokens: Mapping[str, np.ndarray]) -> ConditionWorkCountedConditionLayout
Section titled “CountedConditionLayout”class CountedConditionLayout(Protocol)A denoiser selects the valid keys or physical positions its forward reads.
CountedConditionLayout.condition_lengths
Section titled “CountedConditionLayout.condition_lengths”def condition_lengths(lengths: np.ndarray, spans: np.ndarray) -> np.ndarrayCountedConditionedSpatial
Section titled “CountedConditionedSpatial”class CountedConditionedSpatial(Protocol)A spatial model also consumes the encoder’s pooled and time-id geometry.
CountedConditionedSpatial.conditioned_spatial_flops
Section titled “CountedConditionedSpatial.conditioned_spatial_flops”def conditioned_spatial_flops( sample_shape: tuple[int, ...], rows: int, condition: ConditionWork,) -> floatCountedDecisions
Section titled “CountedDecisions”class CountedDecisions(Protocol)A decision head counts the layout’s real questions and options with its rows.
CountedDecisions.decision_flops
Section titled “CountedDecisions.decision_flops”def decision_flops( lengths: np.ndarray, questions: np.ndarray, options: np.ndarray, *, pooled: np.ndarray | None = None,) -> floatThe head’s work over rows of lengths tokens, their real questions and options, its spans
pooled over pooled tokens a row where they are not the row’s (a context’s).
CountedImages
Section titled “CountedImages”class CountedImages(Protocol)A patch-token model’s forward work with channel/conditioning geometry.
Models whose patch projection infers its channels at init cannot derive
that map’s input width from token lengths alone. The objective supplies
input_channels and the pooled text width (None for no text).
CountedImages.image_flops
Section titled “CountedImages.image_flops”def image_flops( lengths: np.ndarray, input_channels: int, text_features: int | None = None,) -> floatCountedImages.image_lengths
Section titled “CountedImages.image_lengths”def image_lengths(sample_shape: tuple[int, ...], rows: int) -> np.ndarrayCountedLogits
Section titled “CountedLogits”class CountedLogits(Counted, Protocol)A counted token backbone with separate vocabulary-head work.
lengths counts predicted real tokens. A plain head contributes
2widthvocabulary per token, whether its matrix is tied or tiled.
CountedLogits.logits_flops
Section titled “CountedLogits.logits_flops”def logits_flops(lengths: np.ndarray) -> floatCountedLogitsTraining
Section titled “CountedLogitsTraining”class CountedLogitsTraining(CountedTraining, Protocol)An adapted LM head’s training work, separately from its backbone.
CountedLogitsTraining.logits_training_flops
Section titled “CountedLogitsTraining.logits_training_flops”def logits_training_flops(lengths: np.ndarray) -> floatCountedMatrixUsage
Section titled “CountedMatrixUsage”class CountedMatrixUsage(Protocol)A model’s declared call geometry for an adapted native matrix path.
CountedMatrixUsage.matrix_usage
Section titled “CountedMatrixUsage.matrix_usage”def matrix_usage( path: tuple[str, ...], lengths: np.ndarray, *, logits: bool = False,) -> np.ndarrayCountedMultimodal
Section titled “CountedMultimodal”class CountedMultimodal(Protocol)A joint-stream model whose input maps infer their widths at init.
CountedMultimodal.multimodal_flops
Section titled “CountedMultimodal.multimodal_flops”def multimodal_flops( lengths: np.ndarray, context_lengths: np.ndarray, input_channels: int, text_features: int,) -> floatCountedMultimodal.image_lengths
Section titled “CountedMultimodal.image_lengths”def image_lengths(sample_shape: tuple[int, ...], rows: int) -> np.ndarrayCountedParameters
Section titled “CountedParameters”class CountedParameters(Protocol)Forward work of learned matrix maps, separately from state/attention MACs.
An adapter freezes these base matrices: their training work is twice forward work, while its trainable factors cost three times their own forward work. Nonparameter attention/state contractions remain separate.
CountedParameters.parameter_flops
Section titled “CountedParameters.parameter_flops”def parameter_flops(lengths: np.ndarray) -> floatCountedPredictionParameters
Section titled “CountedPredictionParameters”class CountedPredictionParameters(CountedPredictions, Protocol)Prediction depths declare their learned maps and adapted scopes separately.
CountedPredictionParameters.prediction_parameter_flops
Section titled “CountedPredictionParameters.prediction_parameter_flops”def prediction_parameter_flops( lengths: np.ndarray, *, prediction_lengths: Sequence[tuple[np.ndarray, np.ndarray]] | None = None,) -> floatCountedPredictionParameters.prediction_matrix_usage
Section titled “CountedPredictionParameters.prediction_matrix_usage”def prediction_matrix_usage( path: tuple[str, ...], lengths: np.ndarray, *, prediction_lengths: Sequence[tuple[np.ndarray, np.ndarray]] | None = None,) -> np.ndarrayCountedPredictionTraining
Section titled “CountedPredictionTraining”class CountedPredictionTraining(CountedPredictions, Protocol)An adapted prediction stack’s frozen-base and trainable-factor training work.
CountedPredictionTraining.prediction_training_flops
Section titled “CountedPredictionTraining.prediction_training_flops”def prediction_training_flops( lengths: np.ndarray, *, prediction_lengths: Sequence[tuple[np.ndarray, np.ndarray]] | None = None,) -> floatCountedPredictions
Section titled “CountedPredictions”class CountedPredictions(Protocol)The extra decoder depths and heads trained beside a language model’s trunk.
CountedPredictions.prediction_flops
Section titled “CountedPredictions.prediction_flops”def prediction_flops( lengths: np.ndarray, *, prediction_lengths: Sequence[tuple[np.ndarray, np.ndarray]] | None = None,) -> floatCountedSequences
Section titled “CountedSequences”class CountedSequences(Protocol)A joint/cross-attention backbone with separate image and text geometry.
The objective supplies independent lengths and the actual image, context and pooled widths, because Dense input axes are inferred from the arrays.
CountedSequences.multisequence_flops
Section titled “CountedSequences.multisequence_flops”def multisequence_flops( lengths: np.ndarray, context_lengths: np.ndarray, *, input_channels: int, text_features: int, pooled_features: int | None = None,) -> floatCountedSequences.image_lengths
Section titled “CountedSequences.image_lengths”def image_lengths(sample_shape: tuple[int, ...], rows: int) -> np.ndarrayCountedSpatial
Section titled “CountedSpatial”class CountedSpatial(Protocol)A convolutional denoiser with a resolution-specific stage walk.
CountedSpatial.spatial_flops
Section titled “CountedSpatial.spatial_flops”def spatial_flops( sample_shape: tuple[int, ...], rows: int, context_lengths: np.ndarray | None = None, text_features: int | None = None,) -> floatCountedTraining
Section titled “CountedTraining”class CountedTraining(Protocol)An adapted backbone’s training work, including its factors, not 3*forward.
CountedTraining.training_flops
Section titled “CountedTraining.training_flops”def training_flops(lengths: np.ndarray) -> floatCountedVideo
Section titled “CountedVideo”class CountedVideo(Protocol)A factorized video model with independent frame and spatial axes.
CountedVideo.video_lengths
Section titled “CountedVideo.video_lengths”def video_lengths( sample_shape: tuple[int, ...], rows: int,) -> tuple[np.ndarray, np.ndarray]CountedVideo.spatiotemporal_flops
Section titled “CountedVideo.spatiotemporal_flops”def spatiotemporal_flops( frames: np.ndarray, patches: np.ndarray, input_channels: int, text_features: int | None = None,) -> floatDecoderTraining
Section titled “DecoderTraining”class DecoderTraining(Predicting, Protocol)What a decoder trains beside its next token: its multi-token prediction
depths (arXiv 2412.19437, section 2.2; apply’s method='mtp_hidden_states'
and 'mtp_logits', Predicting), which with mtp_hyper_connections set
read the residual streams a forward sows under prediction_inputs/states,
and the MLA mixers it declares with DeepSeek’s lightning indexer.
Decoding
Section titled “Decoding”class Decoding(Protocol)A decoder that continues its rows token by token: init_cache opens the
cache its decode calls (decode=True) write, a position a call. A caller
that appends with those calls itself, past generate’s and the server’s
host checks, reads dew.nn.kv_cache.overflowed after them.
Decoding.init_cache
Section titled “Decoding.init_cache”def init_cache(batch_size: int) -> NoneDenoisingModel
Section titled “DenoisingModel”class DenoisingModel(Protocol)A diffusion network’s output for sample at time, given its conditions:
arrays, encoded text (TextContext) or a process’s DenoisingCondition.
ExpertRouted
Section titled “ExpertRouted”class ExpertRouted(Protocol)A model that may route tokens to experts: the routers among its own whose expert weights are fixed
whatever they score (dew.nn.backbones.decoder_block.Mixture.fixed_weights), each by name and whether
a parameter, by its path in the model’s own tree, is its, which no term of a plain loss reaches
(dew.objectives.routing.refuse_fixed_router).
ExpertRouted.fixed_routers
Section titled “ExpertRouted.fixed_routers”def fixed_routers() -> Mapping[str, Callable[[tuple[str, ...]], bool]]Exposing
Section titled “Exposing”class Exposing(Protocol)A decoder that hands back its hidden states beside its logits, which a drafting strategy seeds from; without them a model decodes but cannot draft.
Exposing.states_and_logits
Section titled “Exposing.states_and_logits”def states_and_logits( tokens: jax.Array, **kwargs: ModelKwarg = {},) -> tuple[jax.Array, jax.Array]Generating
Section titled “Generating”class Generating(Protocol)A decoder that says, after examining the layers it builds, why a generation loop cannot continue its rows token by token, or None; a sampler refuses with that reason without knowing where a family keeps the layers that cause it.
Generating.generation_refusal
Section titled “Generating.generation_refusal”def generation_refusal() -> str | NoneHiddenStates
Section titled “HiddenStates”class HiddenStates(Protocol)A token model’s [B, S, features] final states, the head’s input.
HiddenStates.hidden_states
Section titled “HiddenStates.hidden_states”def hidden_states( tokens: jax.Array, *, train: bool = False, **fields: ModelKwarg = {},) -> jax.ArrayIntervalModel
Section titled “IntervalModel”class IntervalModel(Protocol)A denoising model that, with its interval field set
(model.clone(interval=True)), embeds the duration an interval process
hands it beside its time, a missing one as zero; unset, it refuses one.
class Lacks(capability: Capability, message: str)A consumer’s refusal of a model for a capability it lacks, raised where the consumer first
reads it: as it admits the model, or in the step that reads it. capability names what it
lacks, so a caller tells the refusal from any other error by that and not by the message. A
TypeError, as a model of the wrong kind, and a ValueError, as one configured otherwise, as numpy’s
AxisError is both of its errors.
LayerScalars
Section titled “LayerScalars”class LayerScalars(Protocol)A model with per-layer output scalars that a fine-tune trains, as
DiffusionGemma’s published SFT does: with_trainable_layer_scalars is
the model that trains them, and trainable_variables the source’s tree
as that model reads it.
LayerScalars.with_trainable_layer_scalars
Section titled “LayerScalars.with_trainable_layer_scalars”def with_trainable_layer_scalars() -> SelfLayerScalars.trainable_variables
Section titled “LayerScalars.trainable_variables”def trainable_variables(variables: VariableDict) -> VariableDictLayerStates
Section titled “LayerStates”class LayerStates(Protocol)A model whose forward keeps its layers’ residual states, each layer’s output before the
final norm, for a reader of a middle layer’s features: applied with
capture_intermediates=model.capture_layers and model.layer_collections mutable, the
collections it returns hold them, and model.layer_state(collections, index) reads layer
index’s (0-based; -1 is the last). A layer a loop runs more than once is read at its last
pass, the state the layers after it read.
LayerStates.capture_layers
Section titled “LayerStates.capture_layers”def capture_layers(module: nn.Module, method: str) -> boolLayerStates.layer_state
Section titled “LayerStates.layer_state”def layer_state(collections: Mapping[str, Mapping], index: int) -> jax.ArrayLayered
Section titled “Layered”class Layered(Protocol)A decoder that declares its decode cache’s layout, as CausalTransformer does, and gives
the same model over another (with_kv_cache): a wrapper answers for the decoder that holds
the cache.
Layered.with_kv_cache
Section titled “Layered.with_kv_cache”def with_kv_cache(layout: KVCache) -> SelfLogits
Section titled “Logits”class Logits(Protocol)A token model’s [B, S, vocab] fp32 logits over the whole sequence.
Logits.logits
Section titled “Logits.logits”def logits( tokens: jax.Array, *, train: bool = False, **fields: ModelKwarg = {},) -> jax.ArrayLogitsFromHidden
Section titled “LogitsFromHidden”class LogitsFromHidden(Protocol)The exact head from final states, for scoring states a loss already holds.
LogitsFromHidden.logits_from_hidden
Section titled “LogitsFromHidden.logits_from_hidden”def logits_from_hidden(hidden: jax.Array) -> jax.ArrayLooping
Section titled “Looping”class Looping(Protocol)A model whose layer stack can run a block of it several times over its own output,
and the same model at steps passes of its block (None keeps its count), or of the block
layers, [first, end), which gives a model with no loop one; None where they cannot.
backprop_steps trains through the final passes and remat checkpoints each pass;
None keeps an existing loop’s value, or a fresh loop’s default.
Looping.with_passes
Section titled “Looping.with_passes”def with_passes( steps: int | None = None, layers: tuple[int, int] | None = None, *, backprop_steps: int | None = None, remat: bool | None = None,) -> Self | NoneOutputTable
Section titled “OutputTable”class OutputTable( vocab_major: bool = struct.field(pytree_node=False), bias: jax.Array | None = None, softcap: float | None = struct.field(pytree_node=False, default=None), precision: PrecisionLike = struct.field(pytree_node=False, default=None), compute: DTypeLike | None = struct.field(pytree_node=False, default=None), input_quantizer: InputQuantizer | None = None, scale: jax.Array | None = None, projection: Projection | None = None,)A vocabulary head as a matrix a loss contracts: matrix as the tree
stores it, [vocab, features] when vocab_major and [features, vocab]
otherwise, an optional fp32 [vocab] scale multiplying the product
(nGPT’s per-token logit gain) and bias added after it, the softcap
applied last, and the precision the product runs at. compute, the
model’s compute dtype, decides the product’s operands
(dew.nn.precision.head_product; the states’ dtype where None), and an
input_quantizer rounds the states first as a checkpoint’s head input
quantization does. Where the matrix is an embedding’s own width, the states
reach it through a projection first (Projection), so every reader of
the states the product takes (states) reads them at the matrix’s width.
Contracting the model’s final states with it (logits) reproduces the
model’s own logits, scale, bias, softcap and any adapter on the head
included. Where no matrix does, output_table returns None, and the
logits come from logits_from_hidden or logits instead.
OutputTable.states
Section titled “OutputTable.states”def states(hidden: jax.Array) -> jax.Arrayhidden as the product reads it: through the projection where the head has one, then the
input quantizer’s QDQ.
OutputTable.logits
Section titled “OutputTable.logits”def logits(hidden: jax.Array, *, temperature: float = 1.0) -> jax.Arrayhidden’s [..., vocab] logits, at least fp32: the states through
the input quantizer, the product (head_product), the scale and the
bias (scaled_logits), the softcap, then temperature, which a
sampler applies to the head’s logits.
OutputTable.columns
Section titled “OutputTable.columns”def columns( hidden: jax.Array, ids: Sequence[int] | jax.Array, *, temperature: float = 1.0, exact: bool = False,) -> jax.Arrayhidden’s logits at the vocabulary columns ids alone, [..., len(ids)]: logits
restricted to them by construction, the same states (the input quantizer’s QDQ), product
(head_product) at the table’s precision and compute, scale and bias at ids
(scaled_logits) and softcap, so a fixed set of a few columns costs no full-vocabulary
product. exact computes the product in fp32 or wider at fp32 precision, whatever the
table’s compute, as a readout that reads a label’s logit exactly does. (dew.objectives.lm .chunked’s support log probabilities restrict each position to its own ids, another
contraction.) Static ids outside the vocabulary are refused (IndexError); traced ids are
the caller’s to bound.
OutputTable.rows
Section titled “OutputTable.rows”def rows() -> jax.ArrayEach vocabulary token’s [vocab, features] row: the matrix’s, times
the token’s scale, whose dot product with a state as the product reads
it (states) is the token’s logit before its bias and softcap, in the
wider of the two dtypes. A reader
of tokens’ vectors (a decision head’s lexical prior) takes these.
Predicting
Section titled “Predicting”class Predicting(Protocol)A decoder that declares how many multi-token prediction depths it carries.
ProjectionGroup
Section titled “ProjectionGroup”class ProjectionGroup( path: tuple[str, ...], packed: str, members: tuple[str, ...], widths: tuple[int, ...],)Projections under path in params that read one input: members,
widths wide, concatenate on the output axis into packed, which their
module reads in their place, one product for a decode step.
ProjectionGroup.of
Section titled “ProjectionGroup.of”def of( path: tuple[str, ...], packed: str, members: tuple[str, ...], projections: Mapping[str, tuple[int, int]],) -> SelfThe group of members, their widths read from the module’s own
projections(), {name: (inputs, outputs)}, the geometry its setup builds.
ProjectionGroup.split
Section titled “ProjectionGroup.split”def split(out: jax.Array) -> tuple[jax.Array, ...]The members’ outputs from the packed projection’s out, in member order.
ProjectionGroup.held
Section titled “ProjectionGroup.held”def held(params: Mapping) -> boolWhether params, the parameters at path, hold the group packed or every member.
ProjectsParameters
Section titled “ProjectsParameters”class ProjectsParameters(Protocol)A model whose training keeps a constraint on the parameters it stores.
projected returns the stored parameters a training step reads and
updates. The trainer calls it at each optimizer commit: first with
updated=False on the parameters the step’s update applies to, which
the optimizer sees (Gemma 3n’s AltUp clamps its prediction coefficients
there, as the reference clamps the stored weight in place before its
forward); then with updated=True on the updated parameters, which the
step stores (a normalized transformer puts its matrices back on the unit
sphere there). The tree a run starts from is projected once with
updated=True, drawn or held, so a projection leaves a tree that already
satisfies the constraint as it is. params is the trainable tree,
which may hold a part of the model’s parameters (an adapter’s frozen
base is absent): a model projects the leaves present and keeps the rest.
ProjectsParameters.projected
Section titled “ProjectsParameters.projected”def projected(params: VariableDict, *, updated: bool) -> VariableDictQuantizesInputs
Section titled “QuantizesInputs”class QuantizesInputs(Protocol)A model computing under a checkpoint’s input quantization
(dew.quantization.InputQuantization): the quantizer a bound
module’s input takes, which a head read as a table (OutputTable) applies
as the module’s own call does.
QuantizesInputs.input_quantizer
Section titled “QuantizesInputs.input_quantizer”def input_quantizer(module: nn.Module) -> InputQuantizer | NoneReadsClips
Section titled “ReadsClips”class ReadsClips(Protocol)An encoder that says whether it reads a clip’s frames, (B, T, H, W, C), or an image: a run over clips takes one that does, and a run over images one that doesn’t.
ReadsTrain
Section titled “ReadsTrain”class ReadsTrain(Protocol)A module whose call takes train, for a stochastic layer of its own (a token mixer’s or a
feed-forward’s dropout, a vision tower’s drop path): its owner passes train where reads_train
says the call reads it, whatever the module’s family.
Recomputing
Section titled “Recomputing”class Recomputing(Protocol)A model one remat rung up (None at the top), its rung as a checkpoint
records it, and the model at a record’s rung where that is above its
own: a resumed run never compiles a lighter rung than it trained on.
recompute_half is the model one rung up where that rung recomputes only
the few values it names, a half rung the planner tries at each split
(dew.training.planning.FitSearch), else None.
Recomputing.recompute_record
Section titled “Recomputing.recompute_record”def recompute_record() -> JSONRecomputing.recompute_more
Section titled “Recomputing.recompute_more”def recompute_more() -> Self | NoneRecomputing.recompute_half
Section titled “Recomputing.recompute_half”def recompute_half() -> Self | NoneRecomputing.restore_recompute
Section titled “Recomputing.restore_recompute”def restore_recompute(record: JSON) -> SelfRematChoosing
Section titled “RematChoosing”class RematChoosing(Protocol)A decoder whose layers start at a chosen RematPolicy, or None for no remat.
CausalTransformer and the multimodal and DiffusionGemma models that wrap
it answer this capability. Image diffusion models choose a RematChoice
instead of named decoder residuals, so they do not. A policy outside
DECODER_REMAT is valid, but Recomputing.recompute_more cannot climb
from it.
RematChoosing.with_remat
Section titled “RematChoosing.with_remat”def with_remat(policy: RematPolicy | None) -> SelfRequiresText
Section titled “RequiresText”class RequiresText(Protocol)A denoising model that cannot run without text, which reaches it under
the keyword text_keyword on every call. Any other runs unconditionally.
Selective
Section titled “Selective”class Selective(Protocol)A decoder that scores one position per row instead of them all, so its prefill runs the head only on the slot the first draw reads.
Selective.states_and_logits_at
Section titled “Selective.states_and_logits_at”def states_and_logits_at( tokens: jax.Array, slots: jax.Array, **kwargs: ModelKwarg = {},) -> tuple[jax.Array, jax.Array]Serving
Section titled “Serving”class Serving(Protocol)What a server reads off a decoder. inference_projection_groups names
the packed groups variables hold, packed or as every member (the packer
checks the members concatenate; a model that does not decode
autoregressively names none). It reads which parameters the tree holds and
their shapes, never their values or placement, and the model is an unbound,
hashable module, so its answer is kept per model and tree shape
(dew.inference.projections) and is called over jax.ShapeDtypeStruct
leaves. cache_rebuild_position is where cached
keys go stale and the prefix is recomputed (LongRoPE’s long factors,
Phi-3), or None. mixed_admission_refusal says why its layers would not
run a mixed call (dew.nn.inputs.Admitted) as separate decode and
prefill calls would, or None. split_prefill_refusal says why it
prefills each prompt whole, a prompt taken in pieces or after a prefix
another request cached computing something its whole prefill does not,
and what serves those modes instead, or None.
Serving.inference_projection_groups
Section titled “Serving.inference_projection_groups”def inference_projection_groups( variables: Mapping[str, Mapping],) -> tuple[ProjectionGroup, ...]Serving.mixed_admission_refusal
Section titled “Serving.mixed_admission_refusal”def mixed_admission_refusal() -> str | NoneServing.split_prefill_refusal
Section titled “Serving.split_prefill_refusal”def split_prefill_refusal() -> str | NoneStagedStacks
Section titled “StagedStacks”class StagedStacks(Protocol)A model whose stacks of like layers a pipeline over the stage axis runs: where in store, the
variables a stage-local state is made from, each stack’s run lies and how the pipeline over
stages stages stacks it (DecoderBank), so a trainer keeps that run stacked by stage on the
stage’s own devices (dew.training.stages). A stack that runs whole on every stage has none.
StagedStacks.stage_sites
Section titled “StagedStacks.stage_sites”def stage_sites(store: VariableDict, stages: int) -> tuple[DecoderBank, ...]TimeScaled
Section titled “TimeScaled”class TimeScaled(Protocol)A denoising model whose time_scale field is its time’s unit: at scale
s on time t (and duration d) it computes what it does at s / k on k t (and
k d). A loss that differentiates it in time needs the scale small
(dew.objectives.diffusion.few_step.SMOOTH_TIME_SCALE). A table of time
features in the variables, as FourierEmbedding’s, is drawn at init’s
scale and loaded variables keep theirs.
TokenModel
Section titled “TokenModel”class TokenModel(Protocol)Whether a token model’s hidden_states attend causally (each position
to itself and the ones before it) or to the whole sequence, and the
vocabulary id masked diffusion corrupts tokens to, None without one.
TritonGemm
Section titled “TritonGemm”class TritonGemm(Protocol)Whether a mixer of a model keeps XLA’s Triton GEMM fusions (MixerBase.keeps_triton_gemm).
VocabularyRows
Section titled “VocabularyRows”class VocabularyRows(Protocol)A model whose head scores its final states against one row per vocabulary token: each
token’s [vocab, features] row as OutputTable.rows gives it, read from the bound variables,
where no matrix alone is the head too (an AffineHead returning None): past a nonlinear head
transform the matrix’s rows, with an adapter on the head its rows as the adapter updates them.
A reader of tokens’ vectors, a decision head’s lexical prior, takes these.
VocabularyRows.vocabulary_rows
Section titled “VocabularyRows.vocabulary_rows”def vocabulary_rows() -> jax.Arraylabel_logits
Section titled “label_logits”def label_logits( model: nn.Module, variables, answers: jax.Array, ids: Sequence[Sequence[int]],) -> jax.Arraymodel’s own logits [B, N, K, V] at each of the [K, V] label ids of each answer state of
answers [B, N, features], a class read as its label’s tokens’ logits (a verbalizer): its output
table at the labels’ columns alone, computed in fp32 (OutputTable.columns), where one matrix is
its head (head_table); else its exact head over the answer states, a few rows
(LogitsFromHidden), the labels’ columns taken. No product is rebuilt from the vocabulary’s rows,
which carry no bias and no softcap.