Skip to content

dew.decision.objective

DecisionObjective: training a decision model on labelled questions.

NameSummary
tokenizer_ofThe tokenizer a loaded model’s text processor encodes with: a run’s own, or a checkpoint’s Hugging Face tokenizer, alone or inside a multimodal processor, read from the directory it was loaded from, at the revision the model was.
DecisionObjectiveTrains a decision model on questions with known answers.

function source

def tokenizer_of(processor: TaskProcessor | None) -> Tokenizer | None

The tokenizer a loaded model’s text processor encodes with: a run’s own, or a checkpoint’s Hugging Face tokenizer, alone or inside a multimodal processor, read from the directory it was loaded from, at the revision the model was. A processor without a tokenizer gives none, and tokenizer= names one.

class source

class DecisionObjective(
backbone: nn.Module | Source | Adapter | Decide,
*,
loss: ScoringRule | None = None,
tokenizer: Tokenizer | None = None,
specials: Specials | None = None,
layout: Layout | None = None,
head: Head | None = None,
variables: Variables | None | Omitted = OMITTED,
label_smoothing: float = 0.0,
shuffle_options: bool = True,
none_of_the_above: float = 0.0,
temperatures: Temperatures | None = None,
segments: int | None = 16,
context_encoder: ContextEncoder | None = None,
consistency: float = 0.0,
context_window: int | None = None,
none_of_the_above_options: int = 0,
)

Trains a decision model on questions with known answers.

The model is any backbone with final states (dew.nn.protocols.HiddenStates), read by a head, Laya’s DecisionHead unless head says otherwise. backbone is a model built from scratch, a loaded model (Pretrained.load, adapted with LoRA or partly frozen through freeze, as LMObjective accepts one), or a whole Decide task, such as Laya’s released checkpoint, which brings its head and layout with it. Unless layout says otherwise, a head that reads several questions a row reads them in one row (JointLayout), and one that reads one question a row reads Laya’s MarkerLayout over a bidirectional backbone and the StateFirstLayout over a causal one (default_layout). tokenizer and specials default to the loaded model’s.

The loss is loss, a proper scoring rule or a sum of them, computed against each row’s right option, with label_smoothing spread evenly over its options. shuffle_options and none_of_the_above augment the rows as Encoding describes, a choice of fewer than none_of_the_above_options options drawing no “none of the above” (a yes or a no), and data supplies their pipeline steps. The training stream and the validation pass pack rows first-fit into windows of the layout’s max_len tokens (packing), as many rows, questions and options as its bounds let a window hold, so a step pays for its tokens rather than max_len per row, and every batch has one shape. A window holds at most segments rows, which the backbone’s sequence kernel then works on a window (Coordinates.max_segments) whether or not they come: 16 by default, a few more than a window of mixed tasks’ rows was measured to hold, where a window full of the shortest rows would declare tens. A mix of rows shorter than max_len / segments tokens fills its windows less under it and takes a larger one; None lets a window hold as many as the layout’s shortest row fits. A backbone that reads one document a row (it lacks packing, dew.nn.protocols.Lacks) trains and answers at segments=1, one row a window. The loss is the mean over the batch’s scored questions. With consistency w > 0 it adds w times the mean over the batch’s consistency groups (Example.groups) of the generalized Jensen-Shannon divergence of their members’ answers over the group’s canonical options (categorical.group_divergence), reported as the consistency metric; a group’s members stand in one row, so a layout of a question a row refuses an example with groups under it. At 0 the groups are not read, and the loss is the label loss alone.

Evaluation returns the model’s Decisions on the validation rows, which Accuracy, ECE, AURC and the scoring rules read, its probabilities divided by temperatures (none by default: a Decide’s calibration fits its logits before training moves them, so it is not inherited). The saved run loads as a Decide task.

context_length: int | None

The most context tokens a row carries (DecisionModel.context_length).

spans_in_context: bool

Whether a row’s questions are its context (Layout.spans_in_context): a joint row’s schema, or a question row’s question.

bounds: Bounds

The most rows, questions and options a window packs (Layout.bounds), at most segments rows; rows that carry a context, an example’s or their schema, pack as any do, each reading its own (DecisionInputs.context_segments).

def real_lengths(batch: Batch) -> np.ndarray

Each packed row’s own length, not its window’s.

def model_flops(model, batch: Batch) -> float

One backbone pass, its cross-attention over each row’s context where rows carry one (CrossAttending.cross_flops), plus the head’s real question/option geometry. Forming the vocabulary rows a head reads (an adapter’s update merged into them) is execution, as recompute is.

def training_flops(model, batch: Batch) -> float

The backbone’s frozen and trainable work, then three passes of the trained cross-attention blocks and decision head: an adapted backbone’s own count (CountedTraining); a backbone whose held parameters train nothing but its cross-attention blocks, as with_cross_attention holds a source, its forward, input-gradient and no weight-gradient work (frozen_training_flops), so too a block’s reads through its own matrices (CrossBlocks.held_flops); else, trained whole, three passes of everything.

def program_key() -> tuple[ProgramModule, ...]

The backbone, then the head, both trained.

def substitute(modules: Sequence[nn.Module]) -> None
def fresh_variables(key: jax.Array, held: Variables | None) -> Variables

Nothing beyond the held parts: complete_variables draws the part the tree lacks.

def fresh_modules() -> tuple[str, ...]

What the backbone’s source drew fresh (Pretrained.fresh), and the head where it is drawn beside a held backbone (complete_variables).

def complete_variables(key: jax.Array, tree: Variables) -> Variables

Draw whichever of the backbone and the head tree does not hold.

def loss(variables: Variables, batch: Batch, step: Step)
def evaluate(params: Variables, batch: Batch, step: Step) -> Decisions
def data(tokenizer: Tokenizer | None = None) -> ObjectiveData

Fit rows before mixing, augment training layouts, and pack training and evaluation rows alike, by the objective’s own tokenizer, which its bounds count in.

def bind_widths(widths: Mapping[str, object]) -> None

Keep the most questions a row and options a question of the run’s data hold, as its layout steps bound them (widths’ slots, ConfigureLayout), which its saved task pads each pass to (DecideSettings.questions); data that binds none keeps those of the task it fine-tunes.

def task_settings() -> DecideSettings

What a saved run rebuilds its task from beside the backbone.

That is the head, layout, tokenizer, special tokens and the image processor’s source, independently of the text tokenizer. Interop reloads that source’s processor files and derives its media geometry from the source config; the decision record owns neither format.

def served_groups(variables: Variables) -> tuple[ProjectionGroup, ...]

The backbone’s projection groups a served task packs (Decide._served), their paths from the params root: what a checkpoint holds packed where its run trained a loaded task’s packed tree (mains 947ac9262 up to ledger 2299’s fix), which a resume splits back (dew.checkpoints). Carried to 1.0, when it is deleted (ledger 1552).

def recorded_model() -> nn.Module

The backbone; the head is a task setting of its own.

def pipeline(
state: TrainState,
*,
ema: bool | None = None,
processor: TaskProcessor | None | Omitted = OMITTED,
) -> Decide

Return the trained model as a Decide task over the state’s weights.

A Decide encodes with the objective’s own tokenizer and layout, so it takes no processor; it reads images with the backbone’s own processor, which a task the objective fine-tunes brings. Its passes pad to the run’s widths (bind_widths), as the task loaded from its run does.