dew.rl.evaluation
A policy scored by the episodes it flies rather than by records it reads: episodes(policy, env, steps), the scorer dew.training.evaluation.Episodes runs on a run’s validation cadence, beside its
data splits. Each score is a mean over the environments’ first episodes from fresh starts: the
undiscounted return, the steps the episode lasted, and the mean over those steps of each named
observation view’s last step (a race’s gates passed, passed), so a score is one a drone’s episode
earns once, whatever the steps the rollout runs past it.
Every such flight is flights’: a JAX Env flown in one program, a host-stepped batch (a Simulator, or a
DrivenSimulator a world-model learner acts in) from the host, each answering the environment’s own steps.
A judge that reads each environment’s episode whole reads them with first_episode_rows.
| Name | Summary |
|---|---|
episodes | A scorer of policy in fresh environments of env, flown steps steps without restarting (flights): called with the policy’s variables and a key, it returns the first episodes’ mean return and steps, and for each observation view in reached the mean of its value at each first episode’s last step. |
first_episode_rows | Each environment’s first episode from the steps that followed its actions [T, envs]: its undiscounted return, its steps, whether it was terminated (not cut), the reward of its last step (last_reward), and each array of reached ([T, envs, ...], an observation view) at its last step. |
first_episodes | Where each step [T, envs] belongs to its environment’s first episode: the step that ended it included, the steps after it not. |
flights | A flier of policy in fresh environments of env, flown steps steps without restarting: called with the policy’s variables and a key, it answers the steps that followed each action [steps, envs], their observations the environment’s own (a driver’s, Driven or DrivenSimulator, taken off). |
episodes
Section titled “episodes”def episodes( policy: Policy, env: Env | Simulator | DrivenSimulator, steps: int, *, memory: Callable[[int], Memory] | None = None, reached: Sequence[str] = (), models: tuple[Held, ...] = (),) -> Callable[[Params, jax.Array], dict[str, float]]A scorer of policy in fresh environments of env, flown steps steps without restarting
(flights): called with the policy’s variables and a key, it returns the first episodes’ mean return
and steps, and for each observation view in reached the mean of its value at each first episode’s
last step. memory(count) is a recurrent policy’s starting state (DreamerV3Objective.memory), and
models the networks whose compiler options its programs take.
first_episode_rows
Section titled “first_episode_rows”def first_episode_rows( following: TimeStep, reached: Mapping[str, jax.Array] | None = None,) -> dict[str, jax.Array]Each environment’s first episode from the steps that followed its actions [T, envs]: its
undiscounted return, its steps, whether it was terminated (not cut), the reward of its last step
(last_reward), and each array of reached ([T, envs, ...], an observation view) at its last step.
The one reading of first episodes, which episodes averages and a tool’s judge reads whole.
first_episodes
Section titled “first_episodes”def first_episodes(following: TimeStep) -> jax.ArrayWhere each step [T, envs] belongs to its environment’s first episode: the step that ended it
included, the steps after it not.
flights
Section titled “flights”def flights( policy: Policy, env: Env | Simulator | DrivenSimulator, steps: int, *, memory: Callable[[int], Memory] | None = None, models: tuple[Held, ...] = (),) -> Callable[[Params, jax.Array], TimeStep]A flier of policy in fresh environments of env, flown steps steps without restarting: called
with the policy’s variables and a key, it answers the steps that followed each action [steps, envs],
their observations the environment’s own (a driver’s, Driven or DrivenSimulator, taken off). A JAX
Env is flown in one program (dew.rl.env.env_rollout), one environment as a batch of one; a
host-stepped batch, a Simulator or a DrivenSimulator, from the host (simulator_rollout), its
ended episodes left ended or, driven, restarted, which a first episode’s reading leaves out.
memory(count) is a recurrent policy’s starting state over the count environments
(DreamerV3Objective.memory), and models the networks the policy runs, whose compiler options its
programs take (dew.nn.kernels.options.owned_program; an objective’s towers()).