Skip to content

dew.interop.openenv

OpenEnv environments as episode environments: one OpenEnv session per episode, Dew drawing each turn.

OpenEnv (huggingface/OpenEnv, the openenv distribution, 0.9) serves an environment over a session a client opens, EnvClient or its synchronous .sync(): reset(**options) and step(action) each answer a StepResult holding the observation, the reward of the step and whether the episode is done. OpenEnv draws no model turn; a trainer does. OpenEnv adapts one session to the Environment an EnvironmentSource runs: the rollout server draws each turn from the exact context ids, and the session steps on the action the drawn ids stand for.

Three adapters are the caller’s, since only it knows the model’s chat format and the environment’s action type:

  • prompt(observation) gives the first context’s ids, and the chat messages they hold (Said).
  • reply(observation) gives the ids the environment’s answer appends after the model’s turn, and the messages they hold. The turn’s drawn ids, EOS included, stay as drawn: the next context is the turn’s context, its drawn ids, then these.
  • action(ids) gives the environment’s action for a turn’s drawn ids. Decoding them is the caller’s.

Each step’s reward becomes that turn’s step reward (Observation.reward), so a verifier that returns 0 trains on the environment’s rewards alone, and a done result completes the episode, a reset’s too, before any turn. The messages each observation holds become the episode’s transcript (Observation.messages).

Dew imports nothing of OpenEnv. session(task, identity) opens the caller’s client, a context manager whose exit closes the session, and the protocols below are what Dew reads of it. OpenEnv’s client speaks to its server over a WebSocket, and a connection that drops while a reset or a step waits raises websockets.exceptions.ConnectionClosed: that call raises it as an InfrastructureError, an infrastructure failure the source retries.

NameSummary
OpenEnvRuns each episode in its own OpenEnv session (an environment factory for EnvironmentSource).
SaidWhat an observation adds to the model’s context: its ids, and the chat messages they hold.
SessionOne open OpenEnv session, as EnvClient.sync() serves it.
StepResultWhat an OpenEnv session answers a reset or a step with (openenv.core.StepResult).

dataclass source

class OpenEnv(
session: Callable[[Task, EpisodeId], contextlib.AbstractContextManager[Session[ObservationT, ActionT]]],
prompt: Callable[[ObservationT], Said],
reply: Callable[[ObservationT], Said],
action: Callable[[tuple[int, ...]], ActionT],
options: Callable[[Task, EpisodeId], Mapping[str, JSON]] = lambda task, identity: {},
)

Runs each episode in its own OpenEnv session (an environment factory for EnvironmentSource).

session(task, identity) opens it, options(task, identity) are its reset’s keyword arguments, and prompt, reply and action are the adapters the module docstring describes.

dataclass source

class Said(ids: tuple[int, ...], messages: tuple[Mapping[str, object], ...] = ())

What an observation adds to the model’s context: its ids, and the chat messages they hold.

class source

class Session(Protocol)

One open OpenEnv session, as EnvClient.sync() serves it.

def reset(**options: JSON = {}) -> StepResult[ObservationT]
def step(action: ActionT) -> StepResult[ObservationT]

class source

class StepResult(Protocol)

What an OpenEnv session answers a reset or a step with (openenv.core.StepResult).