Skip to content

dew.rl.evaluation

A policy scored by the episodes it flies rather than by records it reads: episodes(policy, env, steps), the scorer dew.training.evaluation.Episodes runs on a run’s validation cadence, beside its data splits. Each score is a mean over the environments’ first episodes from fresh starts: the undiscounted return, the steps the episode lasted, and the mean over those steps of each named observation view’s last step (a race’s gates passed, passed), so a score is one a drone’s episode earns once, whatever the steps the rollout runs past it.

Every such flight is flights’: a JAX Env flown in one program, a host-stepped batch (a Simulator, or a DrivenSimulator a world-model learner acts in) from the host, each answering the environment’s own steps. A judge that reads each environment’s episode whole reads them with first_episode_rows.

NameSummary
episodesA scorer of policy in fresh environments of env, flown steps steps without restarting (flights): called with the policy’s variables and a key, it returns the first episodes’ mean return and steps, and for each observation view in reached the mean of its value at each first episode’s last step.
first_episode_rowsEach environment’s first episode from the steps that followed its actions [T, envs]: its undiscounted return, its steps, whether it was terminated (not cut), the reward of its last step (last_reward), and each array of reached ([T, envs, ...], an observation view) at its last step.
first_episodesWhere each step [T, envs] belongs to its environment’s first episode: the step that ended it included, the steps after it not.
flightsA flier of policy in fresh environments of env, flown steps steps without restarting: called with the policy’s variables and a key, it answers the steps that followed each action [steps, envs], their observations the environment’s own (a driver’s, Driven or DrivenSimulator, taken off).

function source

def episodes(
policy: Policy,
env: Env | Simulator | DrivenSimulator,
steps: int,
*,
memory: Callable[[int], Memory] | None = None,
reached: Sequence[str] = (),
models: tuple[Held, ...] = (),
) -> Callable[[Params, jax.Array], dict[str, float]]

A scorer of policy in fresh environments of env, flown steps steps without restarting (flights): called with the policy’s variables and a key, it returns the first episodes’ mean return and steps, and for each observation view in reached the mean of its value at each first episode’s last step. memory(count) is a recurrent policy’s starting state (DreamerV3Objective.memory), and models the networks whose compiler options its programs take.

function source

def first_episode_rows(
following: TimeStep,
reached: Mapping[str, jax.Array] | None = None,
) -> dict[str, jax.Array]

Each environment’s first episode from the steps that followed its actions [T, envs]: its undiscounted return, its steps, whether it was terminated (not cut), the reward of its last step (last_reward), and each array of reached ([T, envs, ...], an observation view) at its last step. The one reading of first episodes, which episodes averages and a tool’s judge reads whole.

function source

def first_episodes(following: TimeStep) -> jax.Array

Where each step [T, envs] belongs to its environment’s first episode: the step that ended it included, the steps after it not.

function source

def flights(
policy: Policy,
env: Env | Simulator | DrivenSimulator,
steps: int,
*,
memory: Callable[[int], Memory] | None = None,
models: tuple[Held, ...] = (),
) -> Callable[[Params, jax.Array], TimeStep]

A flier of policy in fresh environments of env, flown steps steps without restarting: called with the policy’s variables and a key, it answers the steps that followed each action [steps, envs], their observations the environment’s own (a driver’s, Driven or DrivenSimulator, taken off). A JAX Env is flown in one program (dew.rl.env.env_rollout), one environment as a batch of one; a host-stepped batch, a Simulator or a DrivenSimulator, from the host (simulator_rollout), its ended episodes left ended or, driven, restarted, which a first episode’s reading leaves out. memory(count) is a recurrent policy’s starting state over the count environments (DreamerV3Objective.memory), and models the networks the policy runs, whose compiler options its programs take (dew.nn.kernels.options.owned_program; an objective’s towers()).