Skip to content

dew.rl.flight

First-person drone gate racing in JAX: one drone’s race as a dew.rl.env.Env (FlightEnv), on device and jittable whole, flown by dew.rl.quadrotor’s physics; and a batch of them seen through a renderer’s camera as a dew.rl.simulator.Simulator (GateRace).

The reward is the one drone-racing RL has converged on: each step, the progress made toward the next gate’s centre (Kaufmann et al., Nature 2023, Swift, Eq 8; Song et al., Sci. Robot. 2023, Eq 5; SkyDreamer, arXiv 2510.14783, Sec II-C), less a cost on the body rates (Song 2023, b = 0.01 a step at 100 Hz, here 1 a second) and on the action’s rate of change (Swift’s lambda5, here 0.005 a step at 60 Hz as a rate a second); a bonus on passing a gate, the more for passing it centred (SkyDreamer’s r_gate); a reward for the camera’s optical axis pointing at the next gate’s centre (Swift’s perception term, Eq 8: lambda2 exp(lambda3 delta_cam^4), the angle in radians), without which a drone flies its yaw where it likes: a teacher of the privileged views flew tail first, the next gate out of its camera’s view on 95% of its steps (on a G4, 10-31), a flight no policy of the camera alone can learn. Its weights are those of Geles et al.’s policy from pixels (RSS 2024, arXiv 2406.12505, Sec III-A4: 0.025 exp(-delta^4) a step, here 1.5 exp(-delta^4) a second, 0.025 a step at 60 Hz), whose term pays four fifths of its most at a gate 40 degrees off the axis, where Swift’s sharper one (Extended Data Table 1a: 0.02 exp(-10 delta^4)) pays a tenth: a teacher on Swift’s turned to the gate on 59% of its steps at the curriculum’s level 0.8; and a crash’s penalty, which ends the episode (Song 2023’s -10). A gate counts only when its plane is crossed through its opening in its direction; crossing the next gate’s plane beside it misses it, a crash (SkyDreamer; Ferede et al., arXiv 2504.21586). The ground, the gates’ frames and the obstacles are solid, the drone’s path through each physics step tested against them (struck), so a fast drone cannot pass through one between two steps. An open track ends, a success, on passing its last gate.

Each episode is flown on a course of its own (drawn_course), of a family:

  • layout: a published racing layout (uzh-rpg/agile_flight, MIT: Split-S and its real-world variant, AlphaPilot; and FlyDreamer’s loop), each gate moved by up to jitter (Song et al., IROS 2021, Sec IV-C);
  • loop: a closed loop of gates about an ellipse;
  • chain: Song et al. 2021’s gate primitive chained into an open track, G_{j+1} = f(G_j, p, R) (Eq 6): each gate a relative turn, climb and distance from the last, with split-S (a gate below the last, facing back) and dive primitives; the paper gives no ranges, so these match its tracks’ 110 to 150 m and climbs up to 17 m (Fig 7), and its curriculum from near-straight (Sec III-C.3);
  • figure8: a closed figure-eight whose lobes cross at different heights. The difficulty, from 0 to 1, scales the jitter, turns, climbs and primitives, and the obstacles’ number, a curriculum (Song 2021 raised it on the crash rate). Obstacles are vertical pillars beside or on the racing line. An episode starts anywhere on its course: hovering before a random gate (Swift; SkyDreamer’s Table III), or already flying toward it from along the segment before it.

The policy sees the camera’s image alone, the next gate lit (highlight: its colours and pattern drawn per episode, as AI Grand Prix 2026 lights its gates; FlyDreamer highlights its target): which gate is next is in the image. The state, the next gates’ corners, the obstacles and the drone’s drawn parameters are privileged, for a world model’s auxiliary heads (SkyDreamer’s privileged decoding) and a teacher (Swift’s and Song 2023’s observation with the parameters). An action keeps FlyDreamer’s semantics (env.py _action_to_command): its first entry is the throttle, zero the drone’s hover thrust, -1 none and 1 the nominal drone’s full thrust; the rest the body rates over the drone’s rate limit. A command reaches the flight controller after the drone’s latency (Swift measured 40 ms; SkyDreamer’s 11 ms action delay).

The world frame is x forward, y left, z up (FLU), the body frame forward, left, up.

NameSummary
ASPECTSA pillar’s height over its radius, one of these: a renderer draws each as its shape scaled.
FAMILIESThe course families (drawn_course).
GATESThe most gates a course holds; a shorter one’s last are inactive.
INTERFACEThe width of the teacher’s observation in its first race: its first entries keep their meaning.
LAYOUTSRacing layouts’ gate centres in metres, FLU: uzh-rpg/agile_flight at 03dc1179c011f414aba1264bb6db92679be66b4f (envsim/parameters/tracks/SplitS.yaml, real_track.yaml, AlphaPilot.yaml, MIT) and FlyDreamer’s loop (AshishKumar4/FlyDreamer dreamerv3/configs.yaml static_gates, turned from NED).
NEARESTThe obstacles a teacher is shown, the nearest first.
OBSTACLESThe most obstacles a course holds; an inactive one has no radius.
TEACHERThe width of the teacher’s observation (privileged).
VIEWSWhat a gate race can show: the camera’s image, and the privileged views (privileged).
CameraA drone’s camera for an episode: its uptilt and horizontal field of view (radians), its mount’s roll and yaw, and its image’s exposure, white balance [3], gamma, read noise and shutter (seconds); and the looks its renderer draws (look [4]: the gates’, the lit gate’s, the ground’s and the sky’s, each an index the renderer takes modulo its variants).
CommandedA drone a flight controller commands (commanded): its drawn body, its rate_limit, its latency in whole control steps (delay) and the commands pending (delayed), a dataclass whose pending a command replaces.
CourseA course: its gates’ centres [G, 3] and facings [G] (the yaw of their passing direction), how many it holds, whether it is closed (a loop) or open (a track ending at its last gate), its family’s index in FAMILIES, and its obstacles [O, 4], each a vertical pillar’s centre’s x and y, radius and height (an inactive one’s radius 0).
FlightEnvOne drone racing racing’s courses as a dew.rl.env.Env: actions [4] in [-1, 1] (commanded); observations a dict of the privileged views asked for (privileged).
GateRaceA batch of renderer.count drones racing racing’s courses (FlightEnv, stepped on device) and seen through renderer’s cameras (Seen): actions [B, 4] in [-1, 1]; observations a dict of the views asked for (VIEWS), each [B, ...].
RaceOne drone’s race: its course, the gate it heads for and its distance from it after the last step, the steps taken and gates passed; the last action and the commands awaiting their latency [L, 4] (collective thrust in N and body rates in rad/s), its latency in control steps and its rate limit; its drawn body and its flight; its camera; the key its physics draws from; and the curriculum it was drawn at (difficulty, ramp).
RacingThe race.
RendererA renderer of a batch of count drones’ cameras: render(scene, cameras, opened) answers each drone’s image [B, H, W, 3] uint8 of its Scene from its camera at the shutter’s close [B, 12] (cameras) and at its open (for motion blur).
SceneWhat a renderer draws for each drone: its course’s gates [B, G, 3], facings [B, G] and count [B], the gate lit (lit [B], -1 for none), its obstacles [B, O, 4], and its camera’s looks [B, 4] (Camera.look).
SeenA batch of drones of a JAX environment env (stepped on device), seen through renderer’s cameras, as a dew.rl.simulator.Simulator: actions [B, A]; observations a dict of the views asked for, some of those offered by what, each [B, ...], image the renderer’s and the rest env’s, which shows the privileged views asked for (privileged).
ViewedA drone’s state a camera sees from (cameras): its camera and its flight.
airframeThe drone’s parameters a world model learns to read off its latent and a teacher is shown [6]: its thrust-to-weight at its start’s charge over the nominal drone’s, its rate limit over nominal, its motors’ response a control step (one less the decay of their lag over it), its latency in control steps, and its camera’s uptilt and field of view.
barsA gate’s four frame bars [4, 3, 2], each a box’s extent along the gate’s normal, its left and up from its centre, grown by the drone’s radius on every side: where the drone’s centre meets the frame.
camerasA drone’s camera [12] as HyperLux takes it (set_cameras: the position, the ray to the image’s first pixel’s corner and the rays across a pixel row and down a column), its mount tilted up by the camera’s uptilt and turned by its mount’s roll and yaw, its field of view the camera’s: HyperDrone’s camera_base_one (hyperdrone/_native/dynamics/impl.cpp) of the mounted camera; before seconds earlier, the drone moved back along its velocity and turned back by its body rates (a shutter’s open).
commandedA drone’s action [4] in [-1, 1] as the flight controller takes it after the drone’s latency: the collective thrust (N), the throttle’s share of the hover below zero and of the span from the hover to the nominal quadrotor’s full thrust above; the body rates (rad/s), its share of the drone’s rate limit.
cornersThe corners of the openings of the gate headed for and of the ahead - 1 after it (on an open track, its last gate in place of any beyond it), in the body frame [12 * ahead] (Song 2023’s track observation).
delayedA command sent after delay control steps: the commands pending [D + 1, ...], newest first, with command pushed, and the one delay steps old, which the flight controller takes now.
developedA rendered image [H, W, 3] uint8 as a drone’s camera develops it, in the camera’s dtype: its exposure, white balance and gamma, and read noise of the camera’s std (full scale).
drawn_cameraA drone’s camera for an episode (Camera), each quantity uniform in its range, the image’s spreads about neutral scaled by ramp; each of the scene’s looks drawn for a ramp share of episodes, else the first, so a curriculum’s first raise varies a tenth of its scenes rather than every one.
drawn_courseAn episode’s course (Course): its family drawn at racing.families’ weights, then its gates (_layout, _loop, _chain, _figure8), facings and obstacles.
facingsEach gate’s facing [G]: the yaw of the level line from the gate before it to the gate after it, on a closed course of count gates wrapping round, on an open one the first gate’s from itself and the last’s to itself.
gate_axesA gate’s normal (its passing direction) and its left [..., 3], level, from its facing.
lookingThe angle in radians between the drone’s camera’s optical axis (its mount’s forward, mounted) and the direction from the drone to point (Swift’s delta_cam).
mountedA camera’s mount [3, 3]: its columns the camera’s forward, left and up in the body frame, the body’s rolled by the mount’s roll, tilted up by the camera’s uptilt and turned by the mount’s yaw.
nearestThe NEAREST active pillars nearest the drone, each its axis’s closest point’s offset in the body frame (at the drone’s height, held within the pillar) and its radius [4 * NEAREST], an absent one zeros.
physicsThe drone’s physics a teacher is shown and a world model learns to read off [20], each over its nominal: its mass, inertia [3], thrust and drag coefficients and top speed; its rotor, quadratic [2] and vertical drag; its ground effect’s strength; its battery’s voltage now over full and its charge drawn over the pack’s; and its constant force over its mass times the disturbance’s limit and its moment over the nominal inertia times its limit [3] each, within [-1, 1].
privilegedThe privileged views: the state [13] (position, velocity, orientation, body rates), the next two gates’ corners in the body frame [24], the drone’s airframe [6] and physics [20], the obstacles nearest [12], the gates passed so far, whether the gate headed for is an open track’s last (finishing), and the teacher’s observation [80]: the velocity and the direction of gravity in the body frame, the body rates, the corners, the last action and the airframe (the first 43, its interface since FlyDreamer’s first race), then the physics, the obstacles and the finishing flag, each about unit size.
quaternionUnit quaternions (w, x, y, z) of a yaw, then a pitch, then a roll.
racedA step’s race, reward, and whether its episode terminated or was truncated, from the action taken and the drone’s path through the step to its flight after it.
shownRefuses views but where they are some of the views what offered.
startedA drone’s episode (Race) at a curriculum’s difficulty and ramp: its course, its start on it, its drawn body (dew.rl.quadrotor.drawn_body) and camera, its latency and rate limit; its rotors at the speed that holds its weight, its commands awaiting their latency its hover; no pillar within 1.5 m of its start.
struckWhether a drone’s path through a step [S + 1, 3] (its positions before it and after each physics step, each pair joined by a straight segment) crashed, passed the gate it heads for, and how far across from its centre it passed.

attribute source

ASPECTS = (6.0, 12.0, 24.0)

A pillar’s height over its radius, one of these: a renderer draws each as its shape scaled.

attribute source

FAMILIES = ('layout', 'loop', 'chain', 'figure8')

The course families (drawn_course).

attribute source

GATES = 16

The most gates a course holds; a shorter one’s last are inactive.

attribute source

INTERFACE = 43

The width of the teacher’s observation in its first race: its first entries keep their meaning.

attribute source

LAYOUTS

Racing layouts’ gate centres in metres, FLU: uzh-rpg/agile_flight at 03dc1179c011f414aba1264bb6db92679be66b4f (envsim/parameters/tracks/SplitS.yaml, real_track.yaml, AlphaPilot.yaml, MIT) and FlyDreamer’s loop (AshishKumar4/FlyDreamer dreamerv3/configs.yaml static_gates, turned from NED). Each gate faces along the course (facings): the layouts’ own gate rotations follow their simulator’s model’s axes, which the files do not state.

attribute source

NEAREST = 3

The obstacles a teacher is shown, the nearest first.

attribute source

OBSTACLES = 8

The most obstacles a course holds; an inactive one has no radius.

attribute source

TEACHER = 43 + 20 + 4 * NEAREST + 1

The width of the teacher’s observation (privileged).

attribute source

VIEWS = ('image', 'state', 'gates', 'airframe', 'physics', 'obstacles', 'passed', 'finishing', 'teacher')

What a gate race can show: the camera’s image, and the privileged views (privileged).

dataclass source

class Camera()

A drone’s camera for an episode: its uptilt and horizontal field of view (radians), its mount’s roll and yaw, and its image’s exposure, white balance [3], gamma, read noise and shutter (seconds); and the looks its renderer draws (look [4]: the gates’, the lit gate’s, the ground’s and the sky’s, each an index the renderer takes modulo its variants).

class source

class Commanded(Protocol)

A drone a flight controller commands (commanded): its drawn body, its rate_limit, its latency in whole control steps (delay) and the commands pending (delayed), a dataclass whose pending a command replaces.

dataclass source

class Course()

A course: its gates’ centres [G, 3] and facings [G] (the yaw of their passing direction), how many it holds, whether it is closed (a loop) or open (a track ending at its last gate), its family’s index in FAMILIES, and its obstacles [O, 4], each a vertical pillar’s centre’s x and y, radius and height (an inactive one’s radius 0).

dataclass source

class FlightEnv(
racing: Racing = struct.field(pytree_node=False),
views: tuple[str, ...] = struct.field(pytree_node=False, default=('teacher',)),
difficulty: float = struct.field(pytree_node=False, default=1.0),
ramp: float = struct.field(pytree_node=False, default=1.0),
)

One drone racing racing’s courses as a dew.rl.env.Env: actions [4] in [-1, 1] (commanded); observations a dict of the privileged views asked for (privileged). An episode is drawn at its curriculum, difficulty and ramp (started); a curriculum changes the env, its programs compiled again (a few times a run).

def observed(race: Race) -> dict[str, jax.Array]

The privileged views of race asked for (privileged).

def scene(race: Race) -> Scene

What a renderer draws of race (_scene).

def reset(key: jax.Array) -> tuple[Race, TimeStep]
def flown(race: Race, action: jax.Array) -> tuple[Race, jax.Array]

The race after the drone flies a control step under action, and its path through it.

def step(race: Race, action: jax.Array) -> tuple[Race, TimeStep]

class source

class GateRace(
renderer: Renderer | None,
racing: Racing | None = None,
views: tuple[str, ...] = VIEWS,
count: int | None = None,
difficulty: float = 1.0,
ramp: float = 1.0,
)

A batch of renderer.count drones racing racing’s courses (FlightEnv, stepped on device) and seen through renderer’s cameras (Seen): actions [B, 4] in [-1, 1]; observations a dict of the views asked for (VIEWS), each [B, ...]. An episode restarts on a course drawn afresh at the race’s curriculum, difficulty and ramp, which a race starts at and which may change between steps.

dataclass source

class Race()

One drone’s race: its course, the gate it heads for and its distance from it after the last step, the steps taken and gates passed; the last action and the commands awaiting their latency [L, 4] (collective thrust in N and body rates in rad/s), its latency in control steps and its rate limit; its drawn body and its flight; its camera; the key its physics draws from; and the curriculum it was drawn at (difficulty, ramp).

dataclass source

class Racing(
families: tuple[tuple[str, float], ...] = (('loop', 0.4), ('chain', 0.6)),
layouts: tuple[str, ...] = tuple(LAYOUTS),
jitter: tuple[float, float] = (1.5, 0.75),
obstacles: int = OBSTACLES,
on_line: float = 0.3,
hz: float = 60.0,
episode_seconds: float = 25.0,
opening: float = 1.5,
frame: float = 0.25,
depth: float = 0.1,
radius: float = 0.2,
progress_scale: float = 1.0,
rate_cost: float = 1.0,
smoothness_cost: float = 0.005 / 60,
gate_bonus: float = 6.0,
perception_scale: float = 1.5,
perception_sharpness: float = 1.0,
crash_penalty: float = 10.0,
miss_radius: float = 3.0,
ceiling: float = 25.0,
rate_limit: float = 10.0,
rate_spread: float = 0.2,
latency: tuple[float, float] = (0.01, 0.06),
uptilt: tuple[float, float] = (30.0, 30.0),
fov: float = 100.0,
fov_spread: float = 0.0,
mount_spread: float = 0.0,
exposure: tuple[float, float] = (0.5, 1.5),
white_balance: float = 0.1,
gamma: tuple[float, float] = (0.8, 1.25),
noise: tuple[float, float] = (0.0, 0.03),
shutter: tuple[float, float] = (0.002, 0.008),
highlight: bool = True,
flying_share: float = 0.5,
start_speed: float = 8.0,
quadrotor: Quadrotor = Quadrotor(),
)

The race. families weighs the course families drawn (FAMILIES), layouts names the layouts a layout course is drawn from, jitter the most a layout’s gate moves across and up at full difficulty; obstacles the most obstacles a course holds at full difficulty and on_line the share placed on the racing line. The reward’s weights are raced’s; hz the control rate, 60 Hz (a teacher at 60 Hz passed 27% more gates than at 30 in the same time; Swift’s and Song’s run at 90 to 100 Hz); an episode is cut after episode_seconds (Swift’s 1500 steps). Gates’ openings are opening square (1.5 m, SkyDreamer’s and Ferede’s) in a frame frame wide and depth deep; the drone collides as a cube of half-width radius aligned with the gate it meets and as a sphere of radius with the ground and obstacles (struck). rate_limit is the nominal body-rate command’s limit and rate_spread its randomization, latency the command latency’s range in seconds; the camera’s uptilt above the body’s forward axis in degrees, its horizontal field of view fov and fov_spread, its mount’s mount_spread, by default one fixed geometry, 30 degrees up and 100 across, as a real FPV quadrotor’s camera is mounted; the image’s exposure, white_balance, gamma, noise (read noise’s std, of full scale) and shutter (seconds) ranges; highlight whether the next gate is lit; flying_share the share of starts already flying and start_speed their fastest; and the quadrotor flown.

max_steps: int

The steps an episode is cut after.

delays: tuple[int, int]

The latency’s range in whole control steps, each rounded; none where latency is (0, 0).

class source

class Renderer(Protocol)

A renderer of a batch of count drones’ cameras: render(scene, cameras, opened) answers each drone’s image [B, H, W, 3] uint8 of its Scene from its camera at the shutter’s close [B, 12] (cameras) and at its open (for motion blur). Its aspect is its images’ width over height, and gate the opening, frame and depth of the gates it draws, which a race’s must be. Arrays cross by DLPack (dew.rl.simulator.Handed).

def render(scene: Scene, cameras: jax.Array, opened: jax.Array) -> Handed

dataclass source

class Scene()

What a renderer draws for each drone: its course’s gates [B, G, 3], facings [B, G] and count [B], the gate lit (lit [B], -1 for none), its obstacles [B, O, 4], and its camera’s looks [B, 4] (Camera.look).

class source

class Seen(
renderer: Renderer | None,
env,
views: tuple[str, ...],
offered: tuple[str, ...],
what: str,
count: int | None = None,
)

A batch of drones of a JAX environment env (stepped on device), seen through renderer’s cameras, as a dew.rl.simulator.Simulator: actions [B, A]; observations a dict of the views asked for, some of those offered by what, each [B, ...], image the renderer’s and the rest env’s, which shows the privileged views asked for (privileged). env steps one drone (reset, step) and shows its views (observed, its states Viewed) and its scene (scene). The batch is the renderer’s, or count drones where there is none, and then no image.

def privileged(views: tuple[str, ...]) -> tuple[str, ...]

The views of views an environment shows, all but the renderer’s image (the gates passed where there are none).

def reset(seed: int) -> dict[str, jax.Array]
def restart(mask: jax.Array) -> dict[str, jax.Array]
def step(action: jax.Array) -> TimeStep
def branch() -> Iterator[None]

A cooldown branch’s steps (dew.rl.simulator.branched), after which every drone’s state and the key its images’ noise is drawn from stand as the run left them: no step donates them, so keeping them keeps them.

class source

class Viewed(Protocol)

A drone’s state a camera sees from (cameras): its camera and its flight.

function source

def airframe(racing: Racing, race: Race) -> jax.Array

The drone’s parameters a world model learns to read off its latent and a teacher is shown [6]: its thrust-to-weight at its start’s charge over the nominal drone’s, its rate limit over nominal, its motors’ response a control step (one less the decay of their lag over it), its latency in control steps, and its camera’s uptilt and field of view.

function source

def bars(racing: Racing) -> np.ndarray

A gate’s four frame bars [4, 3, 2], each a box’s extent along the gate’s normal, its left and up from its centre, grown by the drone’s radius on every side: where the drone’s centre meets the frame.

function source

def cameras(race: Viewed, aspect: float, before: float = 0.0) -> jax.Array

A drone’s camera [12] as HyperLux takes it (set_cameras: the position, the ray to the image’s first pixel’s corner and the rays across a pixel row and down a column), its mount tilted up by the camera’s uptilt and turned by its mount’s roll and yaw, its field of view the camera’s: HyperDrone’s camera_base_one (hyperdrone/_native/dynamics/impl.cpp) of the mounted camera; before seconds earlier, the drone moved back along its velocity and turned back by its body rates (a shutter’s open).

function source

def commanded(
quadrotor: Quadrotor,
drone: D,
action: jax.Array,
) -> tuple[D, jax.Array, jax.Array]

A drone’s action [4] in [-1, 1] as the flight controller takes it after the drone’s latency: the collective thrust (N), the throttle’s share of the hover below zero and of the span from the hover to the nominal quadrotor’s full thrust above; the body rates (rad/s), its share of the drone’s rate limit. The drone, a gate race’s (Race) or any other with its fields, comes back with the command pushed.

function source

def corners(racing: Racing, race: Race, ahead: int = 2) -> jax.Array

The corners of the openings of the gate headed for and of the ahead - 1 after it (on an open track, its last gate in place of any beyond it), in the body frame [12 * ahead] (Song 2023’s track observation).

function source

def delayed(
pending: jax.Array,
delay: jax.Array,
command: jax.Array,
) -> tuple[jax.Array, jax.Array]

A command sent after delay control steps: the commands pending [D + 1, ...], newest first, with command pushed, and the one delay steps old, which the flight controller takes now.

function source

def developed(image: jax.Array, camera: Camera, key: jax.Array) -> jax.Array

A rendered image [H, W, 3] uint8 as a drone’s camera develops it, in the camera’s dtype: its exposure, white balance and gamma, and read noise of the camera’s std (full scale).

function source

def drawn_camera(racing: Racing, key: jax.Array, ramp: jax.Array) -> Camera

A drone’s camera for an episode (Camera), each quantity uniform in its range, the image’s spreads about neutral scaled by ramp; each of the scene’s looks drawn for a ramp share of episodes, else the first, so a curriculum’s first raise varies a tenth of its scenes rather than every one.

function source

def drawn_course(racing: Racing, key: jax.Array, difficulty: jax.Array) -> Course

An episode’s course (Course): its family drawn at racing.families’ weights, then its gates (_layout, _loop, _chain, _figure8), facings and obstacles.

function source

def facings(center: jax.Array, count: jax.Array, closed: jax.Array) -> jax.Array

Each gate’s facing [G]: the yaw of the level line from the gate before it to the gate after it, on a closed course of count gates wrapping round, on an open one the first gate’s from itself and the last’s to itself.

function source

def gate_axes(facing: jax.Array) -> tuple[jax.Array, jax.Array]

A gate’s normal (its passing direction) and its left [..., 3], level, from its facing.

function source

def looking(race: Race, point: jax.Array) -> jax.Array

The angle in radians between the drone’s camera’s optical axis (its mount’s forward, mounted) and the direction from the drone to point (Swift’s delta_cam).

function source

def mounted(camera: Camera) -> jax.Array

A camera’s mount [3, 3]: its columns the camera’s forward, left and up in the body frame, the body’s rolled by the mount’s roll, tilted up by the camera’s uptilt and turned by the mount’s yaw.

function source

def nearest(race: Race) -> jax.Array

The NEAREST active pillars nearest the drone, each its axis’s closest point’s offset in the body frame (at the drone’s height, held within the pillar) and its radius [4 * NEAREST], an absent one zeros.

function source

def physics(racing: Racing, race: Race) -> jax.Array

The drone’s physics a teacher is shown and a world model learns to read off [20], each over its nominal: its mass, inertia [3], thrust and drag coefficients and top speed; its rotor, quadratic [2] and vertical drag; its ground effect’s strength; its battery’s voltage now over full and its charge drawn over the pack’s; and its constant force over its mass times the disturbance’s limit and its moment over the nominal inertia times its limit [3] each, within [-1, 1]. A term a quadrotor turns off (its nominal or limit zero, so every drone’s zero too) reads as the nominal’s, 1, and an absent disturbance as 0.

function source

def privileged(racing: Racing, race: Race) -> dict[str, jax.Array]

The privileged views: the state [13] (position, velocity, orientation, body rates), the next two gates’ corners in the body frame [24], the drone’s airframe [6] and physics [20], the obstacles nearest [12], the gates passed so far, whether the gate headed for is an open track’s last (finishing), and the teacher’s observation [80]: the velocity and the direction of gravity in the body frame, the body rates, the corners, the last action and the airframe (the first 43, its interface since FlyDreamer’s first race), then the physics, the obstacles and the finishing flag, each about unit size.

function source

def quaternion(roll: jax.Array, pitch: jax.Array, yaw: jax.Array) -> jax.Array

Unit quaternions (w, x, y, z) of a yaw, then a pitch, then a roll.

function source

def raced(
racing: Racing,
race: Race,
action: jax.Array,
path: jax.Array,
) -> tuple[Race, jax.Array, jax.Array, jax.Array]

A step’s race, reward, and whether its episode terminated or was truncated, from the action taken and the drone’s path through the step to its flight after it. The reward is the progress toward the gate headed for, its distance before the step less its distance after (times progress_scale), less rate_cost times the body rates’ norm over the step, less smoothness_cost times the action’s squared rate of change over the step (its squared change over the step’s seconds, a rate a second); plus perception_scale times exp(-perception_sharpness delta^4) over the step (a rate a second, times its seconds), delta the angle in radians between the camera’s optical axis and the centre of the gate headed for after the step (looking); on passing that gate, gate_bonus times one less its offset’s share of half the opening, the drone heading for the next; on a crash (struck), less crash_penalty, which ends the episode. Passing an open track’s last gate ends it, a success. An episode is cut after max_steps steps.

function source

def shown(views: tuple[str, ...], offered: tuple[str, ...], what: str) -> None

Refuses views but where they are some of the views what offered.

function source

def started(
racing: Racing,
key: jax.Array,
difficulty: jax.Array,
ramp: jax.Array,
) -> Race

A drone’s episode (Race) at a curriculum’s difficulty and ramp: its course, its start on it, its drawn body (dew.rl.quadrotor.drawn_body) and camera, its latency and rate limit; its rotors at the speed that holds its weight, its commands awaiting their latency its hover; no pillar within 1.5 m of its start.

function source

def struck(
racing: Racing,
race: Race,
path: jax.Array,
) -> tuple[jax.Array, jax.Array, jax.Array]

Whether a drone’s path through a step [S + 1, 3] (its positions before it and after each physics step, each pair joined by a straight segment) crashed, passed the gate it heads for, and how far across from its centre it passed. It crashes on coming within radius of the ground or climbing past ceiling, on a segment entering a gate’s frame bar grown by radius (bars), on coming within radius of a pillar, or on crossing the next gate’s plane in its direction outside the frame within miss_radius (a missed gate). It passes that gate on crossing its plane in its direction with its centre within radius of the opening’s inside, the larger of its offsets across and up from the centre no more than half the opening less radius.