Skip to content

dew.decision.data

Labelled requests: what a decision model trains and calibrates on.

NameSummary
GroupQuestions of one example that ask the same thing (negated, paraphrased, its options re-ordered), named name: members maps each one’s every option key onto a canonical key the members share, so a consistency term (DecisionObjective.consistency) can hold their answers to one distribution over the canonical keys.
ExampleOne state, its questions, and the right answer to some of them.
GroupedExamplesExamples whose rows train together, so a consistency term (DecisionObjective.consistency) holds questions asked across them to one answer: the same question negated, paraphrased or its options re-ordered in rows of their own, under any layout.
groups_ofA row’s groups, as Example.groups holds them: a mapping of each group’s name to its members, each member’s option keys mapped to canonical keys, or None for none.
ParseExampleA row of an example’s fields as its Example, a row of grouped examples’ fields (examples) as its GroupedExamples, or a row that holds either already as it is.
ParseTableA flat table’s row as a one-question Example.
decisionsdataset’s rows as decisions, each an Example under example, for a chain to batch: a flat table’s text and label columns read as one question over options (its ClassLabel names, or read whole in memory, where none are given), its context column, where named, each example’s context, or rows that hold their examples already.
STREAMED_WINDOWThe rows a corpus read as a stream draws its order from (mixture): decision examples are rows of text, so this many hold tens of megabytes.
SCHEDULE_MIXTUREHow a decision run reads a mixture saved before its corpus order was seeded, as the refusal of its resume names it (dew.data.plan.SCHEDULE’s spelling for a decision mixture, rows 3870, 3976).
mixtureThe decision corpora of the mixture directory root_url (its mixture.json): every corpus of a positive weight, the manifest’s weights over weights, mixed by them for training, and their held rows together, or None where none has any.
corpusCorpus name’s split of the mixture directory root_url as decision loading reads it, from the shards and index its manifest names (or the corpus’s one JSONL file), each row an Example under example; None when the corpus has no such split.
made_byThe manifest a decision mixture’s corpora state they were made by (mixture): read again from the root and descriptor they name, refused where its bytes are not the ones the corpora were read by (row 3969); None where they name none, as a chain recorded before they named it does (row 3968).

dataclass source

class Group(name: str, members: Mapping[str, Mapping[str, str]])

Questions of one example that ask the same thing (negated, paraphrased, its options re-ordered), named name: members maps each one’s every option key onto a canonical key the members share, so a consistency term (DecisionObjective.consistency) can hold their answers to one distribution over the canonical keys.

dataclass source

class Example(
state: JSON,
questions: Mapping[str, Question],
answers: Mapping[str, str | int] = dict(),
targets: Mapping[str, tuple[float, ...]] = dict(),
context: JSON | None = None,
groups: tuple[Group, ...] = (),
)

One state, its questions, and the right answer to some of them.

An answer names an option by its key or its index (Question.index), so a dataset’s integer label column can be used as it is. Where the gold is a distribution, as a teacher’s or a panel’s is, targets gives it instead: one probability per option, in the question’s own order. A question with neither is asked but not scored.

context is a sequence the decision reads beside its state, through the backbone’s cross-attention layers (DecisionObjective’s context), rendered as text; None, or a backbone without those layers, reads none.

groups name questions that ask the same thing (Group), each question in one at most; an objective without a consistency term reads none.

def of(row: Example | Mapping[str, object]) -> Example

Return an example as it is, or build one from a row with state, questions and answers.

The questions may be Questions or in Jev’s wire format.

def group_of(name: str) -> tuple[int, Group] | None

The index and group question name stands in, or None.

def labels() -> dict[str, int]

Return each answered question’s right option, as an index: a target’s most likely one.

def distribution(name: str) -> np.ndarray

Return question name’s gold as a distribution over its options.

That is its target, or else all of the mass on its answer.

dataclass source

class GroupedExamples(examples: tuple[Example, ...], groups: tuple[Group, ...] = ())

Examples whose rows train together, so a consistency term (DecisionObjective.consistency) holds questions asked across them to one answer: the same question negated, paraphrased or its options re-ordered in rows of their own, under any layout.

groups names each set of such questions (Group), its members keyed "<example index>.<question>" by the example’s index in examples and the question’s name there, each mapping its question’s every option key onto canonical keys the members share, as an example’s own groups do. An example’s own groups hold too, numbered after these. The examples’ rows train in one packed window, as a unit the packer does not split (DecisionObjective.data): a unit too long for a window is refused.

def group_of(index: int, name: str) -> tuple[int, Mapping[str, str]] | None

The unit-wide index of the group example index’s question name stands in, counting each example’s own groups after these, and the member’s map onto its canonical keys, or None.

def of(row: GroupedExamples | Mapping[str, object]) -> GroupedExamples

Return grouped examples as they are, or build them from a row with examples (each as Example.of reads one) and groups (groups_of), its members named "<example index>.<question>".

function source

def groups_of(value: JSON) -> tuple[Group, ...]

A row’s groups, as Example.groups holds them: a mapping of each group’s name to its members, each member’s option keys mapped to canonical keys, or None for none.

dataclass source

class ParseExample()

A row of an example’s fields as its Example, a row of grouped examples’ fields (examples) as its GroupedExamples, or a row that holds either already as it is. A Grain transform, so a chain that parses its rows by it is recorded and read back (dew.data.records).

def map(element: Batch) -> Batch

dataclass source

class ParseTable(
options: tuple[str, ...],
text: str = 'text',
label: str = 'label',
question: str = 'label',
instructions: str = 'Classify the input into the correct category.',
by_value: bool = False,
context: str | None = None,
)

A flat table’s row as a one-question Example. A label names its option by key, or by index where it is an int; where the options are the label column’s own values (by_value), every label names its option by value, an int one included. A context column’s value is the example’s context (Example.context), none where the column holds None. A row’s groups field reaches the example as Example.of reads it, so a group over the row’s one question is refused as any group of one member is.

function source

def decisions(
dataset: Dataset,
*,
options: Sequence[str] = (),
text: str = 'text',
label: str = 'label',
question: str = 'label',
instructions: str = 'Classify the input into the correct category.',
context: str | None = None,
) -> Pipeline

dataset’s rows as decisions, each an Example under example, for a chain to batch: a flat table’s text and label columns read as one question over options (its ClassLabel names, or read whole in memory, where none are given), its context column, where named, each example’s context, or rows that hold their examples already. Its stages before this one, a holdout’s selection among them, stand; a decision chain is a store’s rows, and a mixture of them mixes decision chains (mixture). Raises ValueError for a mixture or phases under it.

attribute source

STREAMED_WINDOW = 2 ** 14

The rows a corpus read as a stream draws its order from (mixture): decision examples are rows of text, so this many hold tens of megabytes.

attribute source

SCHEDULE_MIXTURE = 'read its mixture in the schedule it saved (`mixture(..., order="schedule")`)'

How a decision run reads a mixture saved before its corpus order was seeded, as the refusal of its resume names it (dew.data.plan.SCHEDULE’s spelling for a decision mixture, rows 3870, 3976).

function source

def mixture(
root_url: str,
*,
weights: Mapping[str, float] | None = None,
descriptor: str | None = None,
seed: int | None = 0,
order: Literal['seeded', 'schedule'] = 'seeded',
) -> tuple[Pipeline, Pipeline | None]

The decision corpora of the mixture directory root_url (its mixture.json): every corpus of a positive weight, the manifest’s weights over weights, mixed by them for training, and their held rows together, or None where none has any. descriptor pins a replaced manifest by its URL.

Each corpus trains on its rows in an order drawn from seed and its name (Pipeline.shuffle): all of them permuted where the run reads by index, within STREAMED_WINDOW rows where it streams. The mix takes each corpus’s next row in that order, choosing the corpus each position reads by a permutation seed draws for each window of the mix (mix(seed=)), so every window holds each corpus at its share in no fixed order, and a corpus whose file groups its rows by task gives no window the next rows of its file. order="schedule" chooses the corpora in Grain’s schedule, the same every window in their manifest’s order, as a mixture read before its corpus order was seeded (row 3870); seed=None reads each corpus in its file’s order, in that schedule. Stages of the chain, the order resumes and continues into a later phase as the mix read it (dew.data.compile).

function source

def corpus(
root_url: str,
name: str,
split: Literal['train', 'held'] = 'held',
descriptor: str | None = None,
) -> Pipeline | None

Corpus name’s split of the mixture directory root_url as decision loading reads it, from the shards and index its manifest names (or the corpus’s one JSONL file), each row an Example under example; None when the corpus has no such split. A benchmark reads one corpus’s held rows through this, whatever its weight.

function source

def made_by(facts: Mapping[str, object]) -> Mapping[str, object] | None

The manifest a decision mixture’s corpora state they were made by (mixture): read again from the root and descriptor they name, refused where its bytes are not the ones the corpora were read by (row 3969); None where they name none, as a chain recorded before they named it does (row 3968).