dew.rl.sandbox
Run untrusted programs and tool sessions in bounded Linux subprocesses.
SubprocessEnvironment is an episode environment whose worker runs under
the caller’s OS permissions with bounded CPU, memory, time and IO. It has no
filesystem or network isolation, so hostile code needs an outer boundary.
Dew never selects or executes it by default.
A Program holds the files to write into a fresh temporary directory, an
argv to run there without a shell, and the text to feed to its stdin. A
runner executes one program under SandboxLimits and reports an
Outcome: how it ended, its exit code, its capped output and the seconds
it took. SandboxFleet runs programs on workers threads at once, each
program in its own process, and a verifiable reward uses it to score
completions.
Dew has two runners. ProcessRunner starts the program through the same
launcher as SubprocessEnvironment. The program gets RLIMIT_CPU and
RLIMIT_AS, no core dumps, its own session, SIGKILL on parent death and a
minimal environment, and its process group is killed at the wall deadline.
It keeps the caller’s user, filesystem and network, so a program can read
and reach whatever you can. ContainerRunner runs the program in a fresh
Docker or Podman container with no network, a read-only root, the job
directory mounted read-only, no capabilities, an unprivileged user, and
limits on memory, CPU, process count and CPU time. Use it for hostile code.
A program that fails returns an Outcome, and the reward decides what a
timeout or a crash is worth. A runner raises only when it cannot start a
program at all: an OSError when the process cannot start, and an
InfrastructureError when a container runtime starts no container or
stalls removing one. A session source retries either as an infrastructure
failure.
MathReward is a verifiable reward that runs no program. It reads the
completion’s final answer and compares it with the record’s reference as an
exact rational number.
| Name | Summary |
|---|---|
ContainerRunner | Runs each program in a fresh container of image with no network. |
MathReward | Scores 1.0 when the final answer equals the reference as a rational number, and 0.0 otherwise. |
Outcome | |
ProcessRunner | Runs each program as a resource-limited Linux process in a temporary directory. |
Program | The files to write, the argv to run beside them, and the text for its stdin. |
Runner | Runs one program to its end under limits, and raises only when it cannot start it. |
SandboxFleet | Runs programs on workers sandboxed workers at once. |
SandboxLimits | Limits for one worker: RLIMIT_CPU and RLIMIT_AS, plus session and IO limits the parent enforces. |
SubprocessEnvironment | Runs an episode environment as a JSON-lines worker process that you choose. |
Verdict | How a program ended. |
outputs_match | Return whether the program completed and its stdout equals expected. |
ContainerRunner
Section titled “ContainerRunner”class ContainerRunner( image: str, runtime: str = 'docker', cpus: float = 1.0, pids: int = 64, user: str = '65534:65534', python: tuple[str, ...] = ('python', '-I', '-S'),)Runs each program in a fresh container of image with no network.
runtime is the Docker-compatible CLI (docker or podman). The job
directory is mounted read-only at /work, which is the working
directory, and /tmp is a small writable tmpfs. The container caps
memory at memory_bytes with no swap, CPU time at cpu_seconds
(SIGXCPU, then SIGKILL a second later), the processor share at cpus
and the process count at pids, and it runs as user. At the wall
deadline the runner kills the client process, then force-removes the
container by name. A runtime that exits without creating the container
(no daemon, no permission, no image), or that stalls removing it,
raises InfrastructureError.
python: tuple[str, ...]-
The image’s own interpreter, since the host’s path does not exist in the container.
ContainerRunner.command
Section titled “ContainerRunner.command”def command( program: Program, limits: SandboxLimits, directory: str, name: str, cidfile: str,) -> list[str]Return the runtime argv that runs program from directory in a container called name.
The runtime writes the container’s id to cidfile once it creates one.
MathReward
Section titled “MathReward”class MathReward(require_boxed: bool = True)Scores 1.0 when the final answer equals the reference as a rational number, and 0.0 otherwise.
The answer is the last \boxed{} in the completion. With
require_boxed=False, a completion without one uses its last number
instead. Integers, decimals, a/b and \frac{a}{b} compare exactly, so
0.5, 1/2 and \frac{1}{2} agree. The reference is the record’s
ground_truth; one that is not a number is compared as trimmed text.
Outcome
Section titled “Outcome”class Outcome( verdict: Verdict, exit_code: int | None, stdout: str, stderr: str, seconds: float,)ProcessRunner
Section titled “ProcessRunner”class ProcessRunner(python: tuple[str, ...] = (sys.executable, '-I', '-S'))Runs each program as a resource-limited Linux process in a temporary directory.
The runner kills the whole process group at the wall deadline, on
excess output, and after a normal exit, so no child the program forked
outlives it unless the child left the group on purpose. python is this
interpreter, isolated from the environment, site-packages and user site.
Program
Section titled “Program”class Program(files: Mapping[str, str], command: tuple[str, ...], stdin: str = '')The files to write, the argv to run beside them, and the text for its stdin.
File names must be relative paths inside the job directory, and
command must be a nonempty argv tuple; anything else raises
ValueError.
Runner
Section titled “Runner”class Runner(Protocol)Runs one program to its end under limits, and raises only when it cannot start it.
python is the argv that runs a Python file in the place this runner
runs programs.
SandboxFleet
Section titled “SandboxFleet”class SandboxFleet( runner: Runner = _DEFAULT_RUNNER, *, limits: SandboxLimits = _DEFAULT_LIMITS, workers: int = os.cpu_count() or 1,)Runs programs on workers sandboxed workers at once.
runner defaults to a ProcessRunner and workers to the CPU count.
Use it as a context manager, or call close, which waits for the
running programs and cancels the queued ones. submit returns a future
at once, and programs wait in a queue while every worker is busy.
SandboxFleet.submit
Section titled “SandboxFleet.submit”def submit(program: Program) -> Future[Outcome]SandboxFleet.run
Section titled “SandboxFleet.run”def run(programs: Iterable[Program]) -> list[Outcome]Run every program concurrently and return their outcomes in input order.
SandboxFleet.close
Section titled “SandboxFleet.close”def close() -> NoneSandboxLimits
Section titled “SandboxLimits”class SandboxLimits( wall_seconds: float = 30.0, cpu_seconds: int = 10, memory_bytes: int = 256 * 1024 ** 2, message_bytes: int = 1024 ** 2,)Limits for one worker: RLIMIT_CPU and RLIMIT_AS, plus session and IO limits the parent enforces.
cpu_seconds and memory_bytes set RLIMIT_CPU and RLIMIT_AS.
wall_seconds is the wall-clock deadline. message_bytes caps one JSON
message to or from a SubprocessEnvironment worker, and a program’s
stdout and stderr together.
Forked children inherit the resource limits, which apply to each process separately; they are not a cgroup limit on the group’s total. Killing the process group cleans up the descendants that stay in that group.
SubprocessEnvironment
Section titled “SubprocessEnvironment”class SubprocessEnvironment( command: tuple[str, ...], limits: SandboxLimits = SandboxLimits(),)Runs an episode environment as a JSON-lines worker process that you choose.
The worker implements reset and step. command is its argv tuple, run
without a shell in a temporary working directory. Each request has an
operation and either an episode identity or an action record. Each
reply holds context (integer ids), status (running, completed,
truncated, cancelled or error), an optional string detail, an
optional number reward, the step’s reward (Observation.reward), and
an optional list messages of the chat messages it added
(Observation.messages). It runs on Linux only.
Leaving the session kills the process group, whether the episode succeeded, failed or was cancelled. The worker itself also gets SIGKILL if its parent dies. Neither replaces filesystem and network isolation, and neither stops descendants that deliberately leave the group.
Verdict
Section titled “Verdict”class Verdict(Enum)How a program ended.
COMPLETED-
The program exited with status zero.
FAILED-
The program exited with a nonzero status, including after an uncaught exception or a MemoryError.
TIMEOUT-
The program ran past the wall deadline or its CPU-time limit and was killed.
CRASHED-
The program was killed by a signal it did not ask for.
OUTPUT_LIMIT-
The program wrote more than
message_bytesto stdout and stderr together and was killed.
outputs_match
Section titled “outputs_match”def outputs_match(outcome: Outcome, expected: str) -> boolReturn whether the program completed and its stdout equals expected.
Whitespace around the whole output and at the end of each line is ignored.