dew.data.preferences
Preference pairs for DPO: chosen and rejected rows with their masks.
A PreferencePair step reads records of
chosen and rejected token ids, lists or one-dimensional arrays, with
chosen_mask and rejected_mask marking the completion tokens. Each batch stacks them in
TRL’s layout: chosen rows over rejected rows, padded to the longest row,
with completion_mask alongside the ids and real_mask marking each row’s
ids apart from its padding. A mask must run as long as its
ids; anything else fails naming the row.
| Name | Summary |
|---|---|
IDS_KEY | Batch key holding the [B, 2, S] pair ids, chosen at index 0. |
MASK_KEY | Batch key holding the [B, 2, S] completion marks, 1 on completion tokens. |
REAL_KEY | Batch key holding the [B, 2, S] marks of each half’s ids, 1 on its tokens and 0 on its padding. |
FIELDS | |
PreferencePair | A chosen and a rejected completion’s ids and loss masks laid out as two seq_len rows. |
IDS_KEY
Section titled “IDS_KEY”IDS_KEY = 'input_ids'Batch key holding the [B, 2, S] pair ids, chosen at index 0.
MASK_KEY
Section titled “MASK_KEY”MASK_KEY = 'completion_mask'Batch key holding the [B, 2, S] completion marks, 1 on completion tokens.
REAL_KEY
Section titled “REAL_KEY”REAL_KEY = 'real_mask'Batch key holding the [B, 2, S] marks of each half’s ids, 1 on its tokens and 0 on its padding.
FIELDS
Section titled “FIELDS”FIELDS = ('chosen', 'rejected', 'chosen_mask', 'rejected_mask')PreferencePair
Section titled “PreferencePair”class PreferencePair(seq_len: int = 256, *, pad_id: int = 0)A chosen and a rejected completion’s ids and loss masks laid out as two
seq_len rows.
PreferencePair.declared
Section titled “PreferencePair.declared”def declared(facts: Mapping[str, object]) -> Mapping[str, object]A chosen and a rejected row of seq_len ids and their completion mask, in place of the
completions.
PreferencePair.map
Section titled “PreferencePair.map”def map(element: Batch) -> Batch