dew.config.env
The environment run as one typed record: the environment an agent acts in, the agent’s networks and objective, the rollout that collects its batches there, and a teacher that labels or demonstrates its steps (docs/design/post-training.md §14).
The environment is its class and arguments (EnvConfig) and the rollout its class and settings
(RolloutConfig), each held as its import path, the arguments the run states and the class’s other
defaults as they were recorded, as a run holds its model and its objective. What the environment states,
the run derives: the agent’s action width, the width a policy reads of each observation and the image a
world model sees. The rollout replaces every batch whole, so the run reads no data, and the trainer is
handed a loader of empty rows (idle).
| Name | Summary |
|---|---|
EnvConfig | An environment’s class and the arguments the run gives it, and how the run wraps it. |
EnvRunConfig | A run that trains an agent in an environment: the shared run fields, the environment (env), the agent’s networks (model: a WorldModel, its dynamics a nested TSSM record where it has one, or an ActorCritic, each by its import path) and objective (objective: a DreamerV3Objective or a ControlPPOObjective, by its import path, its imitation among its arguments), the rollout’s settings (rollout), an optional teacher (teacher), the saved run the agent starts from (pretrained) and the episodes each validation flies (episodes). |
EpisodesConfig | A validation split scored by flying the agent (dew.training.Episodes over dew.rl.evaluation.episodes): its greedy policy, or with drawn the policy’s draws as it acts in training (act), flown steps steps from fresh episodes of env, the run’s own where None, each environment’s first episode scored, logged as <split>/return and <split>/steps, and for each observation view in reached its mean at each first episode’s last step (a race’s gates passed), <split>/<view>. |
RolloutConfig | The rollout that collects the run’s batches (dew.training.rollout.Rollout): name an alias of dew.registry.rollouts (dreamer, control) or an import path, held as the path, and fields the settings the run gives it, as records: a DreamerRollout’s batch, length, train_ratio, prefill, capacity and checkpointed_replay, a ControlRollout’s unroll_length, unrolls, epochs and minibatches. |
TeacherConfig | A privileged teacher: the saved environment run in the directory run, a ControlPPOObjective’s, read at its checkpoint step (a step, latest, best or best:<metric>; None for its latest), which the run pins as it starts, so its record names the step it read (EnvRunConfig.pinned). |
idle | The loader of a run whose rollout makes every batch whole (§14): a batch of no records, one value for the step, which any mesh places whole, the same each time it opens (replayed), so a resume reads on; with no record count, so no cadence is an epoch of it. |
EnvConfig
Section titled “EnvConfig”class EnvConfig( name: str, fields: JsonDict = dict(), defaults: JsonDict = dict(), count: int | None = None, episode_length: int | None = None, action_repeat: int = 1,)An environment’s class and the arguments the run gives it, and how the run wraps it.
name is an alias of dew.registry.envs or an import path, held as the path: a JAX Env class, a host
Simulator class, or a function or classmethod that returns one (Playground.load). fields are its
arguments as given, a nested value as its record (a race’s Racing, a GateRace’s renderer as
{"class": ..., "fields": ...}), set one at a time by --set env.<argument>=...; defaults holds
every other argument the class defaults, as a model’s are held. An Env of one environment is wrapped
as dew.rl.playground.playground wraps one: each episode cut after episode_length steps of
action_repeat repeats where that is set (TimeLimit), then count of them stepped together
(Batched). A Simulator is a batch of its own, and takes neither.
EnvConfig.build
Section titled “EnvConfig.build”def build() -> EnvironmentThe environment the record names, wrapped as it says (dew.objectives.rl.run.built).
EnvRunConfig
Section titled “EnvRunConfig”class EnvRunConfig( model: ModelConfig = (lambda: ModelConfig('dew.objectives.rl.dreamer:WorldModel'))(), data: Annotated[Dataset | None, tyro.conf.Suppress] = None, validation: Annotated[Dataset | Mapping[str, Dataset] | None, tyro.conf.Suppress] = None, data_seed: int = 0, data_plan_version: int | None = None, source: SourceOptions = SourceOptions(), optim: OptimConfig = OptimConfig(), trainer: TrainerConfig = (lambda: TrainerConfig(eval_every=None, checkpoint_every=None))(), objective: ObjectiveConfig = (lambda: ObjectiveConfig('dew.objectives.rl.dreamer:DreamerV3Objective'))(), lora: Annotated[LoRA, tyro.conf.subcommand(lora)] | None = None, val_metrics: Annotated[tuple[MetricConfig, ...], record_argument(tuple[MetricConfig, ...])] | None = None, val_every: Annotated[Mapping[str, int | str], record_argument(Mapping[str, int | str])] = dict(), val_summaries: Annotated[tuple[Summary, ...], record_argument(tuple[Summary, ...])] = (), env: Annotated[EnvConfig, tyro.conf.subcommand('env')] | None = None, rollout: RolloutConfig = RolloutConfig(), teacher: Annotated[TeacherConfig, tyro.conf.subcommand('teacher')] | None = None, pretrained: str | None = None, episodes: Annotated[Mapping[str, EpisodesConfig], record_argument(Mapping[str, EpisodesConfig])] = dict(),)A run that trains an agent in an environment: the shared run fields, the environment (env), the
agent’s networks (model: a WorldModel, its dynamics a nested TSSM record where it has one, or an
ActorCritic, each by its import path) and objective (objective: a DreamerV3Objective or a
ControlPPOObjective, by its import path, its
imitation among its arguments), the rollout’s settings (rollout), an optional teacher (teacher), the
saved run the agent starts from (pretrained) and the episodes each validation flies (episodes).
The environment is what the run reads, as data is a run’s, and the rollout how it collects and replays
it, so a run resumed under its name may change them (RESUMABLE): a curriculum resumes the run at a
new difficulty, a later phase of training at longer windows (SkyDreamer’s), the rollout owing its
updates from there at its own pace. The run has no epochs, so its
trainer validates and checkpoints at the cadences it names, none by default, and at its end.
model: ModelConfig-
The agent’s networks. The run derives from its environment each width the class declares: its
action_size, and a world model’sobservation_sizeandimagefrom the view itreads; a record that states one is refused. A run that starts from a saved run trains that run’s model, and names none. objective: ObjectiveConfig-
The agent’s objective and its arguments (
--objective.imitation 1 --objective.imitates label); aControlPPOObjective’sobservation_sizeis derived, the width of the views itreads. env: Annotated[EnvConfig, tyro.conf.subcommand('env')] | None-
The environment the agent acts in (
EnvConfig):--env.name flight --env.count 4096 --env.fields '{"racing": {"hz": 60}}', an argument at a time--set env.difficulty=0.5. rollout: RolloutConfig-
The rollout and its settings (
RolloutConfig):--rollout.name control --rollout.fields '{"unroll_length": 64}', a setting at a time--set rollout.train_ratio=128. teacher: Annotated[TeacherConfig, tyro.conf.subcommand('teacher')] | None-
A privileged teacher’s saved run (
TeacherConfig), turned on by its flags:--teacher.run runs/teacher --teacher.demonstrates. pretrained: str | None-
A saved environment run’s directory the agent starts from, its model and its weights at the step
source.stepselects (the latest unless it names one), which the run pins as it starts; its live weights, or its average withsource.ema. A stage of a chain starts from the stage before it. episodes: Annotated[Mapping[str, EpisodesConfig], record_argument(Mapping[str, EpisodesConfig])]-
The episodes each validation flies, a split of each (
EpisodesConfig) by its name, one JSON value on the command line:--episodes '{"episodes": {"steps": 1000}}'; attrainer.eval_every, or a split’s own cadence inval_every.
EnvRunConfig.build_run
Section titled “EnvRunConfig.build_run”def build_run() -> PreparedThe agent the record names, acting in its environment through its rollout, on the loader the
rollout replaces (idle), validated by its episodes where it names them.
EnvRunConfig.trained_model
Section titled “EnvRunConfig.trained_model”def trained_model() -> ModelConfigThe model the run trains: its own, or where it starts from a saved run that run’s.
EnvRunConfig.agent
Section titled “EnvRunConfig.agent”def agent(env: Environment, variables: Variables | None = None) -> AgentThe run’s objective over its model (trained_model) acting in env, from variables, a fresh
draw where None: each width env states derived for both, and refused where the record states it.
Variables that do not fit the agent are refused.
EnvRunConfig.started
Section titled “EnvRunConfig.started”def started() -> Variables | NoneThe weights the agent starts from: those of the run pretrained names, at the step source.step
selects, live unless source.ema; None where it names none, its source then refused
(built_fresh). A model the run names beside it, or a source option other than the step and the
average, is refused.
EnvRunConfig.taught
Section titled “EnvRunConfig.taught”def taught(objective: Agent, env: Environment) -> TaughtThe rollout’s teacher and demonstrator (TeacherConfig), each the teacher run’s policy acting in
env: its greedy action labelling each step where objective imitates labels, and its draws flying
the prefill where it demonstrates. An objective imitating labels with no teacher, a teacher that
neither labels nor demonstrates, and a teacher run of another objective than PPO’s are refused.
EnvRunConfig.pinned
Section titled “EnvRunConfig.pinned”def pinned() -> SelfThis run with its sources pinned (RunConfig.pinned), and its teacher’s step and the step of the
run it starts from named as each resolves now (Checkpoints.pinned: the latest, or the best), so
its record names the steps it read and a resume reads them again: a resumed run that leaves its
teacher’s step to this takes the saved run’s first (BUILT, RunConfig.resumed, row 3931).
EpisodesConfig
Section titled “EpisodesConfig”class EpisodesConfig( steps: int, env: EnvConfig | None = None, reached: tuple[str, ...] = (), drawn: bool = False,)A validation split scored by flying the agent (dew.training.Episodes over
dew.rl.evaluation.episodes): its greedy policy, or with drawn the policy’s draws as it acts in
training (act), flown steps steps from fresh episodes of env, the run’s own where None, each
environment’s first episode scored, logged as <split>/return and <split>/steps, and for each
observation view in reached its mean at each first episode’s last step (a race’s gates passed),
<split>/<view>. A JAX Env is flown in one program; a Simulator from the host, and only one the
record names for the split: the run’s own simulator holds the rollout’s environments mid episode,
which a flight would move.
EpisodesConfig.split
Section titled “EpisodesConfig.split”def split(objective: Agent, env: Environment) -> EpisodesThe split of objective’s flights, in env where the record names no environment of its own.
RolloutConfig
Section titled “RolloutConfig”class RolloutConfig( name: str = 'dreamer', fields: JsonDict = dict(), defaults: JsonDict = dict(),)The rollout that collects the run’s batches (dew.training.rollout.Rollout): name an alias of
dew.registry.rollouts (dreamer, control) or an import path, held as the path, and fields the
settings the run gives it, as records: a DreamerRollout’s batch, length, train_ratio,
prefill, capacity and checkpointed_replay, a ControlRollout’s unroll_length, unrolls,
epochs and minibatches. defaults holds the class’s others as recorded. The run builds the rest,
the objective, the environment and its teacher (_BUILT); a train_ratio of inf trains on the
prefill alone.
RolloutConfig.build
Section titled “RolloutConfig.build”def build(objective: Agent, env: Environment, taught: Taught) -> RolloutThe rollout of objective in env, with the teacher and demonstrator taught names. Refuses a
setting the run builds, an objective of another class than the rollout’s, and a teacher it takes
none of.
TeacherConfig
Section titled “TeacherConfig”class TeacherConfig(run: str, step: int | str | None = None, demonstrates: bool = False)A privileged teacher: the saved environment run in the directory run, a ControlPPOObjective’s,
read at its checkpoint step (a step, latest, best or best:<metric>; None for its latest), which
the run pins as it starts, so its record names the step it read (EnvRunConfig.pinned). Its policy acts
in the run’s own environment and reads there what it read in its own. It labels every step the rollout
takes with its greedy action where the run’s objective imitates labels (imitates="label", DAgger),
and where it demonstrates, it flies the rollout’s prefill with its policy’s draws (DART), the replay’s
first steps. A teacher that does neither is refused.
def idle() -> LoaderThe loader of a run whose rollout makes every batch whole (§14): a batch of no records, one value for
the step, which any mesh places whole, the same each time it opens (replayed), so a resume reads on;
with no record count, so no cadence is an epoch of it. A run composed in Python passes it to
Trainer.fit as the run kind does.