Skip to content

dew.decision.config

The decision run, as one typed record: a decision model fine-tuned on a table of labelled text, as laya-train --data tickets.csv does.

By default the run starts from Laya’s released English checkpoint, encoder and head. pretrained names another: a Laya-style repository (one with an rl_agent_config.json), or any Hugging Face model, whose head is then drawn fresh, with lora to train an adapter in place of the whole model. A fresh head is Laya’s, reading the layout the backbone’s order asks for, or with head the joint schema head, reading every question of an example in one row (JointLayout). Its data is a chain through decisions over a table, or mixture’s over a mixture manifest whose sets contribute their recorded shares, and its validation the rows a holdout keeps or a store’s own split; validation scores them, and after training they fit the temperatures the task divides its logits by, saved into the run, so Decide.from_run and dew.pipeline answer calibrated. A question type’s temperature needs ten held-out answers and a bucket’s 2000, Laya’s floors.

NameSummary
DecisionRunConfigA run, plus the decision objective’s own knobs.
JointShapeThe sizes of the joint schema head (JointSchemaHead), its own defaults.
LettersThe letter readout (LabelReadout): each option scored by the backbone’s own logit at its label’s token, read at the row’s answer cue, as a slot release reads it; its rows lettered as labels letter them, each label one token of the run’s tokenizer: where labels names none, the first count of LABEL_NAMES the tokenizer spells as one token (Labels.spelled).

dataclass source

class DecisionRunConfig(
model: ModelConfig = (lambda: ModelConfig('causal_transformer', {}))(),
data: Annotated[Dataset | None, tyro.conf.Suppress] = None,
validation: Annotated[Dataset | Mapping[str, Dataset] | None, tyro.conf.Suppress] = None,
data_seed: int = 0,
data_plan_version: int | None = None,
source: SourceOptions = SourceOptions(),
optim: OptimConfig = (lambda: OptimConfig(learning_rate=2.5e-05))(),
trainer: TrainerConfig = TrainerConfig(),
objective: ObjectiveConfig = (lambda: ObjectiveConfig('decision'))(),
lora: Annotated[LoRA, tyro.conf.subcommand(lora)] | None = None,
val_metrics: Annotated[tuple[MetricConfig, ...], record_argument(tuple[MetricConfig, ...])] | None = None,
val_every: Annotated[Mapping[str, int | str], record_argument(Mapping[str, int | str])] = dict(),
val_summaries: Annotated[tuple[Summary, ...], record_argument(tuple[Summary, ...])] = (),
loop_steps: int | None = None,
loop_layers: tuple[int, int] | None = None,
loop_backprop_steps: int | None = None,
loop_remat: bool = False,
pretrained: str = LAYA,
subfolder: str | None = None,
revision: str | None = None,
remat: str | None = None,
head: JointShape | Letters | None = None,
calibrate: bool = True,
max_len: int | None = None,
head_max_len: int | None = None,
option_tokens: int | None = None,
max_state_tokens: int | None = None,
option_text: Literal['named', 'described'] | None = None,
segments: Annotated[tuple[RowSegment, ...], record_argument(tuple[RowSegment, ...])] | None = None,
reads: Annotated[Reads, record_argument(Reads)] | None = None,
parallel: bool | None = None,
shared_state: int | None = None,
siblings: bool | None = None,
unit: Unit | None = None,
order: RowOrder | None = None,
split: Literal['part', 'question', 'whole'] | None = None,
framing: Framing | None = None,
context_layers: tuple[int, ...] | None = None,
context_frozen: bool = True,
context_length: int = 256,
context_encoder: ContextEncoder = 'table',
context_form: Literal['idefics', 'shared'] = 'idefics',
)

A run, plus the decision objective’s own knobs.

objective: ObjectiveConfig

The decision objective and its arguments: its scoring rule (loss), label smoothing, option shuffling and none-of-the-above rate.

pretrained: str

A Laya-style repository, read whole, or a Hugging Face model, read as the backbone.

subfolder: str | None

The checkpoint’s folder inside a repository that bundles several, such as multilingual.

remat: str | None

A REMAT_POLICIES name for the loaded backbone’s layers, applied before looping and LoRA. The trainer’s fitting ladder starts here; a policy outside DECODER_REMAT cannot climb. None keeps the checkpoint’s policy.

head: JointShape | Letters | None

The head over a fresh backbone: the joint schema head, which then reads the joint layout, or the letter readout of the backbone’s own logits (Letters), over lettered rows; None draws Laya’s head.

calibrate: bool

Fit the task’s temperatures on the held-out rows after training.

max_len: int | None

The row’s length in tokens; None keeps the checkpoint’s (Laya’s 512) or the layout’s own.

head_max_len: int | None

The tokens the question and its options may share, a declared cut; None keeps the checkpoint’s (Laya’s release declares 192) or a fresh head’s, which keeps every question whole. Each row it cuts is counted in the run’s data counters, and a row whose cut leaves two options alike is refused.

option_tokens: int | None

The most tokens one option keeps, in a layout of one row per question, a declared cut counted as head_max_len’s is; None keeps the checkpoint’s (Laya’s 48) or a fresh head’s, each option whole.

max_state_tokens: int | None

The most tokens a joint row’s state keeps, whatever room max_len leaves it: a row is its state, cut to this, and its schema, so a large schema (a hundred options, or a document’s many hypotheses) fits under a max_len set for it without lengthening every other row’s state. None lets the state fill the row.

option_text: Literal['named', 'described'] | None

How a question row writes a choice’s option (QuestionLayout.option_text): its key and description (named, as Laya’s release reads its rows), or its description alone (described). None keeps the layout’s own, a release’s named and a fresh head’s described, which its record holds as the build resolved it (layout_writing); a run that leaves it None and resumes takes the saved run’s (BUILT, RunConfig.resumed). A run saved before this field reads named, as it ran, and one whose record holds None the writing its checkpoint’s layout states (dew.config.earlier_option_text, earlier_option_writing). A joint row names each option by its key beside its description in its schema, the named writing, and has no other.

segments: Annotated[tuple[RowSegment, ...], record_argument(tuple[RowSegment, ...])] | None

The row’s parts in order, as its layout reads them. A joint row’s user turn (JointLayout.segments): the schema first, or read as the row’s context through context_layers (Schema(stream="context")), its context_length holding the whole schema. A question row’s text (QuestionLayout.segments): its instructions, option lines and answer cue beside the state, the question first or the state first, its blocks following that order (QuestionLayout.blocks). None keeps the layout’s own, the release’s turn.

reads: Annotated[Reads, record_argument(Reads)] | None

Which of the row’s streams each reads (Reads), where its segments name a stream of the row’s own (Schema(stream=...), Instructions(stream=...)): two streams in one pass, the state reading the schema and the schema itself alone, are its table. None reads the row in order.

parallel: bool | None

Whether a question row reads its options in parallel, each at positions of its own after the question (QuestionLayout.parallel), which a head that reads option slots needs. None keeps the layout’s own.

shared_state: int | None

The most questions a row asks over one copy of their request’s state, each read as its own row reads it (QuestionLayout.shared_state). None keeps a row a question, each with its state.

siblings: bool | None

Whether a question of a row over one state also reads the questions before it in the row (QuestionLayout.siblings). None keeps the layout’s own, each reading the state and itself.

unit: Unit | None

How a question row’s segments encode their text, piece by piece or whole (QuestionLayout.unit). None keeps the layout’s own, piece by piece.

order: RowOrder | None

The order a question row’s own text writes its parts in (QuestionLayout.order): the state first, the question first, or the state, the question and the state again (re_read), each part as the built-in row writes it. None keeps the layout’s own, a causal backbone’s state first.

split: Literal['part', 'question', 'whole'] | None

Where a question row splits into its blocks, where its backbone reads blocks (a block_bidirectional --source.attention, QuestionLayout.split): at its second part, a question-first row’s state; at its question; or nowhere, the whole row one block its backbone reads both ways (whole). None keeps the layout’s own law.

framing: Framing | None

How a joint row frames itself as a conversation (Framing): its system message, the assistant’s prefill, the reasoning switch, a Jinja chat template in place of the tokenizer’s own, and the template’s other variables; recorded with the run’s layout, so the run reloads it. None keeps the layout’s own, the tokenizer’s template with thinking off.

context_layers: tuple[int, ...] | None

The backbone layers that read each example’s context (Example.context) through gated cross-attention, the context encoded as context_encoder says; the blocks train from zero gates beside the head, the loaded backbone frozen (Pretrained.with_cross_attention) unless context_frozen is off, with lora its adapter beside them. None reads no context.

context_frozen: bool

Whether the context layers freeze the backbone beside their blocks, which then train alone with the head; off, they freeze nothing more, so the backbone trains as the run trains it: whole, a full fine-tune, or through its lora adapter over the frozen base (--no-context-frozen). With context_layers alone.

context_length: int

The most tokens of an example’s context the cross-attention reads; an example with a longer context refuses the run where its data reads it, before training for data read into memory, as a request with one is refused. With context_layers alone.

context_encoder: ContextEncoder

How a context becomes the states its cross-attention reads (DecisionModel.context_encoder): the backbone’s token table, or the backbone’s own pass over it. With context_layers alone.

context_form: Literal['idefics', 'shared']

The block each context layer runs: IDEFICS’s gated block, an attention and a feed-forward of its own drawn beside the layer (CrossAttention), or the layer’s own attention reading the context behind a gate it adds, the only weights the form draws (SharedCrossAttention). With context_layers alone.

def pinned() -> DecisionRunConfig

This run with its Hub sources at their commits (RunConfig.pinned), its backbone’s revision among them (SourceReference.pinned), so its record names the commit it read.

def build_run() -> Prepared

The objective over the backbone pretrained names, on the table’s or the mixture’s rows, scored on the held-out ones, which calibrate the run after it trains. The checkpoint decides the backbone; the model flags choose only its compute dtype and attention kernel; trainer.precision.dtypes.params chooses the stored dtype of a checkpoint it trains (RunConfig.source_keywords). remat chooses its layer recomputation before looping or attaching LoRA.

def default_metrics(*, validates: bool) -> tuple[MetricConfig, ...]

The task’s own held-out suite (default_metrics), where the data holds held-out rows.

dataclass source

class JointShape(
width: int = JointSchemaHead.width,
routing_layers: int = JointSchemaHead.routing_layers,
layers: int = JointSchemaHead.layers,
heads: int = JointSchemaHead.heads,
feedforward: int = JointSchemaHead.feedforward,
)

The sizes of the joint schema head (JointSchemaHead), its own defaults.

Its input width is the backbone’s, which the run reads from the model.

dataclass source

class Letters(labels: Labels = Labels(), count: int | None = None, untied: bool = False)

The letter readout (LabelReadout): each option scored by the backbone’s own logit at its label’s token, read at the row’s answer cue, as a slot release reads it; its rows lettered as labels letter them, each label one token of the run’s tokenizer: where labels names none, the first count of LABEL_NAMES the tokenizer spells as one token (Labels.spelled). The head holds count labels, and its rows letter no more: a question of more options, one the objective’s none-of-the-above adds among them, is refused where its row is laid out. untied gives each label a trainable output row of its own, drawn from the backbone’s.

count: int | None

The labels the head holds; None resolves them from the run: the most options a question of its data asks (its slots facts, training’s and validation’s), one more where its objective adds “none of the above”, or where the data states none, every label its tokenizer spells as one token.