dew.decision.rows
The decision objective’s training rows: examples fitted to the layout’s rows and grouped examples’ units
(Fit), encoded with their augmentations (Encoding), packed into windows (PackedRows), and the
fields a row carries (TRAINING_FIELDS); DecisionObjective.data assembles them.
| Name | Summary |
|---|---|
NONE_OF_THE_ABOVE | |
Encoding | How a labelled example becomes the row a step packs and trains on. |
model_row | The fields of a laid-out row a model reads, as a window packs them (packing): its tokens, its questions in width option slots each (question_fields), its tokens’ positions and option slots in the parallel layout, their blocks where it reads them in blocks, and with a context_length its context’s tokens. |
class_id | The class an option of a question is, as two int32 lanes of a 64-bit hash of the question’s name and the option’s key: the same class in every row and split that asks it, whatever slot it fills. |
GroupOf | The group a row’s question stands in, by its name: the group’s index in its unit and the question’s options mapped onto the group’s canonical keys, or None. |
draws_none | Whether a training read may draw a “none of the above” on question (Encoding): a choice of at least fewest options. |
FittedRow | |
drawn_refusal | Why a training read declines the “none of the above” it drew for row: the layout refuses the row (Layout.refusal), or the option’s line cuts more of the state than the row was admitted with (state_cut), which its data report counted (FittedRow.cuts); None where the row stands as drawn. |
Fit | |
FittedUnit | Rows that train together in one window (Fit._unit): each the index of its example in grouped and its fitted row. |
QuestionRows | |
LayoutRows | |
EvaluateRows | |
ConfigureLayout | |
option_width | The option slots a question of a run’s rows takes: the most options a question of its data asks (slots), and one more where a training read draws “none of the above” beside them, which it does only on a choice of at least fewest options (draws_none). |
TRAINING_FIELDS | The fields along a training row’s questions beside the model’s (QUESTION_FIELDS): each question’s gold labels, whether it is scored, its soft targets over its option slots, each option slot’s class (class_id, its question’s name and its option’s key, which a per-class metric reads), and, where its questions stand in consistency groups (Example.groups, GroupedExamples), each one’s group and each option slot’s canonical option, -1 for none, and its row in its unit of rows (question_row, _unit). |
packing | Rows of layout packed first-fit into windows of its max_len, each window’s rows, questions and options within bounds (Layout.bounds), and with a context_window each row’s context along an axis of its own of that many tokens, end to end in the order their rows stand (PackedRows’ context_segments): a window holds rows while both its streams do. |
PackedRows | A window of packed rows (packing) as DecisionInputs reads it: each question’s row as the packer placed it (question_rows), each row’s spans moved to where it starts in the window, along its tokens, or along the window’s contexts where its spans are its context’s (spans_in_context), and with option_slots the window’s real options compacted to that many slots (compacted). |
NONE_OF_THE_ABOVE
Section titled “NONE_OF_THE_ABOVE”NONE_OF_THE_ABOVE = 'none of the above'Encoding
Section titled “Encoding”class Encoding( layout: Layout, tokenizer: Tokenizer, specials: Specials, width: int, shuffle: bool = True, none_of_the_above: float = 0.0, context_length: int | None = None, grouped: bool = False, none_of_the_above_options: int = 0,)How a labelled example becomes the row a step packs and trains on.
A row is its own tokens, at most max_len, one slot per question it lays
out and width option slots each. A layout of one row per question lays
out one answered question; a joint layout lays out every question of the
example, scoring the answered ones. With shuffle, a choice’s options are
laid out in a fresh random order every time the row is read (Laya’s
--shuffle-options), while a score’s levels and a noul’s two answers keep
their order, and a joint row asks its questions in a fresh order too.
With none_of_the_above p, a choice with a single right answer gains a
“none of the above” option with probability p, and half of those also
lose their right option, which makes “none of the above” the right
answer. That teaches the model to say
that no option fits instead of picking the nearest wrong one. A choice
of fewer than none_of_the_above_options options draws none: where a
choice is a yes or a no, “none of the above” changes what it asks. Each scored
slot’s target is its gold distribution (Example.distribution) in the
order the row shows the options. A reader passes the state’s tokens it
fitted the example with (Fit), which the row lays out again.
With a context_length, a row also carries its example’s context as
context, the tokens of its text (render), a context longer than
context_length refused (Fit), and none for an example without one.
grouped: bool-
Whether each row carries its questions’ consistency groups (
_grouped), an objective with a consistency term’s rows; every row then carries them, an example of none with none in each. none_of_the_above_options: int-
The fewest options a choice holds to draw a “none of the above” (
draws_none); 0, every choice.
model_row
Section titled “model_row”def model_row(encoded: Encoded, width: int, context_length: int | None = None) -> BatchThe fields of a laid-out row a model reads, as a window packs them (packing): its tokens, its
questions in width option slots each (question_fields), its tokens’ positions and option slots
in the parallel layout, their blocks where it reads them in blocks, and with a context_length its
context’s tokens. A training row (Encoding) is this and its gold; a scored row is this alone
(dew.decision.task.Decide).
class_id
Section titled “class_id”def class_id(question: str, option: str) -> tuple[int, int]The class an option of a question is, as two int32 lanes of a 64-bit hash of the question’s name and the option’s key: the same class in every row and split that asks it, whatever slot it fills.
GroupOf
Section titled “GroupOf”The group a row’s question stands in, by its name: the group’s index in its unit and the question’s options mapped onto the group’s canonical keys, or None.
draws_none
Section titled “draws_none”def draws_none(question: Question, fewest: int) -> TypeGuard[Choice]Whether a training read may draw a “none of the above” on question (Encoding): a choice of at
least fewest options.
FittedRow
Section titled “FittedRow”class FittedRow( example: Example, names: tuple[str, ...], length: int, truncated: bool, tokens: np.ndarray, cuts: Mapping[str, int] = dict(), declined: str | None = None, context: int = 0,)cuts: Mapping[str, int]-
The tokens each cut removed from the row, by what it cut (
Encoded.cuts). declined: str | None-
Why a training read declines the row’s “none of the above” beside each of its answered choices (
drawn_refusal: the layout refuses it, or it would cut the state further), so a read that draws the option for it lays it out as it was admitted (Encoding); None where the row takes it or no read draws one (Fit._declined). context: int-
The context tokens the row reads beside its own: its example’s, or its schema where the schema is its context (
Encoded.spans_in_context). tokens: np.ndarray-
The state’s tokens, which the encoding lays out again, as int32: a read held in memory keeps them for every example.
drawn_refusal
Section titled “drawn_refusal”def drawn_refusal( layout: Layout, row: Encoded, context_length: int | None, admitted: Mapping[str, int] | None,) -> str | NoneWhy a training read declines the “none of the above” it drew for row: the layout refuses the row
(Layout.refusal), or the option’s line cuts more of the state than the row was admitted with
(state_cut), which its data report counted (FittedRow.cuts); None where the row stands as drawn.
The option’s own line under a declared question budget is the budget’s (options_alike where it
leaves two alike).
class Fit( layout: Layout, tokenizer: Tokenizer, specials: Specials, head: Head, context_length: int | None = None, grouped: bool = False, bounds: Bounds | None = None, none_of_the_above: float = 0.0, context_window: int | None = None, scored: bool = False, none_of_the_above_options: int = 0,)context_length: int | None-
The most context tokens the backbone’s cross-attention reads, or None where it reads none: an example whose context is longer refuses the run (ValueError) where the data reads it, which the data plan’s compile meets before training for data read into memory and a streamed or indexed read meets at that record, so no example is silently cut or dropped.
grouped: bool-
Whether the objective reads consistency groups, whose members may stand in rows of their own: an example’s questions under a layout of a question a row, or grouped examples’ (
GroupedExamples), whose rows then train as one unit (_unit). bounds: Bounds | None-
The most rows and questions a packed window holds, which a unit of rows fits in or is rejected.
none_of_the_above: float-
The rate the training encoding adds a “none of the above” at (
Encoding), for which a unit’s rows are budgeted at their longest. context_window: int | None-
The most context tokens a packed window holds (
DecisionObjective.context_window), which a unit’s rows’ contexts fit in or the unit trains apart. scored: bool-
Whether the rows are scored, as a validation pass or a server reads them, not trained on: a row whose declared budgets cut its options alike is laid out as its release answers it, counted (
options_alikein its cut record), where training leaves it out (Layout.refusal). none_of_the_above_options: int-
The fewest options a choice holds to draw a “none of the above” (
Encoding,draws_none).
Fit.map
Section titled “Fit.map”def map(element: Batch) -> Batch | RejectedFittedUnit
Section titled “FittedUnit”class FittedUnit(grouped: GroupedExamples, rows: tuple[tuple[int, FittedRow], ...])Rows that train together in one window (Fit._unit): each the index of its example in grouped
and its fitted row.
QuestionRows
Section titled “QuestionRows”class QuestionRows()QuestionRows.flat_map
Section titled “QuestionRows.flat_map”def flat_map(element: Batch) -> Sequence[Batch]LayoutRows
Section titled “LayoutRows”class LayoutRows(encoding: Encoding)LayoutRows.random_map
Section titled “LayoutRows.random_map”def random_map(element: Batch, rng: np.random.Generator) -> BatchEvaluateRows
Section titled “EvaluateRows”class EvaluateRows(encoding: Encoding)EvaluateRows.map
Section titled “EvaluateRows.map”def map(element: Batch) -> BatchConfigureLayout
Section titled “ConfigureLayout”class ConfigureLayout( layout: Layout, tokenizer: Tokenizer, specials: Specials, shuffle: bool, none_of_the_above: float, train: bool, context_length: int | None = None, grouped: bool = False, none_of_the_above_options: int = 0,)none_of_the_above_options: int-
The fewest options a choice holds to draw a “none of the above” (
Encoding).
option_width
Section titled “option_width”def option_width( slots: Mapping[str, object], none_of_the_above: float, fewest: int = 0,) -> intThe option slots a question of a run’s rows takes: the most options a question of its data asks
(slots), and one more where a training read draws “none of the above” beside them, which it does
only on a choice of at least fewest options (draws_none).
TRAINING_FIELDS
Section titled “TRAINING_FIELDS”TRAINING_FIELDSThe fields along a training row’s questions beside the model’s (QUESTION_FIELDS): each question’s
gold labels, whether it is scored, its soft targets over its option slots, each option slot’s class
(class_id, its question’s name and its option’s key, which a per-class metric reads), and, where its
questions stand in consistency groups (Example.groups, GroupedExamples), each one’s group and each
option slot’s canonical option, -1 for none, and its row in its unit of rows (question_row, _unit).
packing
Section titled “packing”def packing( layout: Layout, bounds: Bounds, context_window: int | None = None, grouped: bool = False, trained: bool = True,) -> PackRows of layout packed first-fit into windows of its max_len, each
window’s rows, questions and options within bounds (Layout.bounds),
and with a context_window each row’s context along an axis of its own of
that many tokens, end to end in the order their rows stand (PackedRows’
context_segments): a window holds rows while both its streams do. A
trained row carries its gold along its questions; a scored one
(model_row) carries the model’s fields alone.
PackedRows
Section titled “PackedRows”class PackedRows( max_length: int, option_slots: int | None = None, spans_in_context: bool = False,)A window of packed rows (packing) as DecisionInputs reads it: each
question’s row as the packer placed it (question_rows), each row’s spans
moved to where it starts in the window, along its tokens, or along the
window’s contexts where its spans are its context’s (spans_in_context),
and with option_slots the window’s real options compacted to that many
slots (compacted). Every window is max_length tokens, which a token
budget divides into rows.
PackedRows.map
Section titled “PackedRows.map”def map(element: Batch) -> Batch