Skip to content

dew.rl.skydreamer

SkyDreamer’s race (Verraest et al., arXiv 2510.14783) flown from the camera’s RGB image by collective thrust and body rates: its task, reward, termination and starts, with Dew’s quadrotor and renderer.

The task (Sec II-C, Table III): a track’s gates passed in order, each through its pre-gate, itself and its post-gate, planes depth / 2 before and after it. The reward is 5 progress - rate + 30 gate. Progress is the step’s approach to the next pre-gate, zero from the pre-gate through the post-gate: a step that crosses the pre-gate approaches it up to where it crosses. The rate penalty is (exp(min(|rates|_1, 17)) - 1) over 2 f_c 1e5. The gate reward at each of the three planes is 1 - max(|y|, |z|) / size where the drone crosses it. An episode ends, its last reward zero, when the drone crosses one of those planes beyond size of the gate’s centre, where the race has frames strikes the frame of a gate it is not passing (the race’s opening and frame, Racing), or is within 0.5 m of the ground while descending faster than 1 m/s or rolled or pitched past 60 degrees; it is cut after steps steps (2000, 22 s: the length of SkyDreamer’s flights in Fig 5; the paper states no other). An episode starts on one of the race’s tracks, drawn alike, 2 to 4 m before a gate (any gate in anywhere of starts, else the first), within 1 m to its side and up to 1.3 m below its centre (0.7 to 1.3 m before the first), at rest, rolled, pitched and turned from the gate’s heading up to 20 degrees, its body rates up to 0.1 rad/s and its rotors at a quarter to half of their top speed.

A track’s gates are seen or not: SkyDreamer adds gates no one sees to its tracks in training “to enforce the correct flight maneuvers” (Sec III-A), a gate above one to fly over it. Its two tracks, given only in figures (Figs 4 and 6, its 8 x 8 m arena), are reconstructed as INVERTED_LOOP and LADDER; CIRCLE stands four gates on a circle in the same arena.

The drone (QUADCOPTER) is Dew’s quadrotor (dew.rl.quadrotor, its flight controller tracking collective thrust and body rates) with SkyDreamer’s Table II where Dew’s model has the term: thrust k_w per unit mass and the rotors’ speeds from 341.75 to 3100 rad/s (a thrust-to-weight ratio of 6.1), their 30 ms time constant, and the drag along the body’s x and y, linear in the rotors’ speed and quadratic in the velocity; no vertical drag, ground effect or battery sag. An action is FlyDreamer’s and Dream to Fly’s command (Romero et al., arXiv 2501.14377, Sec III-B): its first entry the throttle, zero the drone’s hover thrust, -1 none and 1 its full thrust; the rest the body rates over the drone’s rate limit; it reaches the flight controller latency seconds later (delays; SkyDreamer’s image and action delays together are 33 and 11 ms).

The policy reads the camera’s image (image, the renderer’s, SkyDreamerRace) and, as SkyDreamer’s does, the body rates, the rotors’ speeds (each a share of the top speed, so a decoder’s squared error over them is of the others’ size) and the flight plan (Sec II-H, Eq 12: for the next three gates, the change in position and yaw from the gate before, then their positions and yaws). The plan moves on as the drone crosses a gate’s plane: SkyDreamer moves it at a random point from the pre-gate to the post-gate in training, and where its world model’s decoded position passes 0.15 m before the gate in flight; here it moves at the gate. The privileged entries a world model decodes (state, Sec II-B) are the position and velocity in the world and in the gate headed for, and the orientation, the roll, pitch and yaw in the world and the yaw from the gate’s heading; the camera’s extrinsics are fixed and the dynamics’ parameters are left out. The camera is tilted up 30 degrees, fixed as a real FPV quadrotor’s camera is mounted (Racing’s; SkyDreamer’s is 45 to 55), its horizontal field of view fov.

Randomization is ramp’s, a race’s curriculum from 0 (none) to 1, as dew.rl.flight’s, one level for every track or each track’s own, so a curriculum may raise each track’s as its own flights allow: the drone’s body and its rotors’ noise (dew.rl.quadrotor.drawn_body, flown), its rate limit, the camera’s image and the scene’s looks (dew.rl.flight.drawn_camera), and the latency’s spread, half its nominal either way at 1. The world frame is x forward, y left, z up (dew.rl.flight’s).

NameSummary
CIRCLEFour gates 1.5 m up on a circle of 3 m about the arena’s centre, facing along the circle.
INVERTED_LOOPSkyDreamer’s inverted loop (Fig 6, its frame North-East-Down turned to this one’s): through the first gate, over the second, through it the other way after a split-S, and back over it; the gates’ centres 1.35 m up (its 2.7 m gates standing), the unseen gates 4 m up above the second.
LADDERSkyDreamer’s ladder inverted loop (Fig 4): through the first gate and, after a full turn, over it the same way, then the inverted loop’s second gate.
PLANESThe planes a gate is passed through, in its depth’s halves along its normal: pre-gate, gate, post-gate.
QUADCOPTERDew’s quadrotor at its nominal mass with SkyDreamer’s Table II thrust, rotor speeds, motor time constant and drag (each per unit mass there, so times the mass here), its yaw torque’s ratio to thrust Dew’s.
TRACKSThe tracks a race is drawn over, by name.
VIEWSWhat the race shows: the camera’s image, the body rates and rotor speeds, the flight plan, and privileged, the state and the gates passed.
GateA track’s gate: its centre, its heading (degrees, the way it is passed) and whether it is seen.
LapOne drone’s episode: its drawn body, flight and camera, its rate limit and command latency (whole steps) with the commands pending (dew.rl.flight.delayed), the track it races (into Sky.tracks), the gate it heads for (index) and the plane of it it crosses next (stage, into PLANES), its distance to that gate’s pre-gate after the last step, the gates passed, the steps taken, and the key its rotors’ noise is drawn from.
SkySkyDreamer’s race (the module docstring): its control rate hz, the tracks an episode is drawn over (TRACKS’ names), its gate rewards’ size and the pre- and post-gates’ span depth (Table III), the share of starts before any gate (anywhere), the camera’s horizontal field of view fov (degrees), the command’s nominal latency (seconds), the quadrotor, the episode’s steps, and whether the frames of gates the drone is not passing are struck (frames).
SkyDreamerRaceA batch of renderer.count drones racing sky’s tracks (SkyRace, stepped on device), seen through renderer’s cameras (dew.rl.flight.Seen), as a dew.rl.simulator.Simulator: observations a dict of the views asked for (VIEWS), each [B, ...].
SkyRaceOne drone racing sky’s tracks as a dew.rl.env.Env at randomization ramp (the module docstring), one level for every track or each of sky.tracks’ own in turn, an episode flown at its track’s: actions [4] in [-1, 1]; observations a dict of the views asked for, each but the image (VIEWS, which SkyDreamerRace adds).
gate_ofGate index of track track: its centre and heading.
heading_ofAn orientation’s yaw (its yaw, pitch, roll Euler angles’).
in_gatepoint in gate index’s frame: along its normal, to its left and up from its centre.
planThe flight plan [24] at gate index (Eq 12): for it and the two after it, the change in position and heading from the gate before (a half turn either way), then their positions and headings.
pre_gateGate index’s pre-gate’s centre.
roll_pitchAn orientation’s roll and pitch (its yaw, pitch, roll Euler angles’).
struckWhether the step from before to after crosses the frame of a seen gate of track other than those passing [G] (the gates it heads for, whose planes decide alone): crosses its plane between its opening and its frame’s outer edge (Racing’s opening and frame).

attribute source

CIRCLE: tuple[Gate, ...] = (((3.0, 0.0, 1.5), 90.0, True), ((0.0, 3.0, 1.5), 180.0, True), ((-3.0, 0.0, 1.5), 270.0, True), ((0.0, -3.0, 1.5), 0.0, True))

Four gates 1.5 m up on a circle of 3 m about the arena’s centre, facing along the circle.

attribute source

INVERTED_LOOP: tuple[Gate, ...] = (((3.0, 0.0, 1.35), -90.0, True), ((-2.1, 0.0, 4.0), 90.0, False), ((-2.0, 0.0, 1.35), -90.0, True), ((-2.1, 0.0, 4.0), 90.0, False))

SkyDreamer’s inverted loop (Fig 6, its frame North-East-Down turned to this one’s): through the first gate, over the second, through it the other way after a split-S, and back over it; the gates’ centres 1.35 m up (its 2.7 m gates standing), the unseen gates 4 m up above the second.

attribute source

LADDER: tuple[Gate, ...] = (((3.0, 0.0, 1.35), -90.0, True), ((3.0, 0.0, 4.0), -90.0, False), ((-2.1, 0.0, 4.0), 90.0, False), ((-2.0, 0.0, 1.35), -90.0, True), ((-2.1, 0.0, 4.0), 90.0, False))

SkyDreamer’s ladder inverted loop (Fig 4): through the first gate and, after a full turn, over it the same way, then the inverted loop’s second gate.

attribute source

PLANES = (-1, 0, 1)

The planes a gate is passed through, in its depth’s halves along its normal: pre-gate, gate, post-gate.

attribute source

QUADCOPTER

Dew’s quadrotor at its nominal mass with SkyDreamer’s Table II thrust, rotor speeds, motor time constant and drag (each per unit mass there, so times the mass here), its yaw torque’s ratio to thrust Dew’s.

attribute source

TRACKS = {'circle': CIRCLE, 'inverted_loop': INVERTED_LOOP, 'ladder': LADDER}

The tracks a race is drawn over, by name.

attribute source

VIEWS = ('image', 'rates', 'motors', 'plan', 'state', 'passed')

What the race shows: the camera’s image, the body rates and rotor speeds, the flight plan, and privileged, the state and the gates passed.

attribute source

A track’s gate: its centre, its heading (degrees, the way it is passed) and whether it is seen.

dataclass source

class Lap()

One drone’s episode: its drawn body, flight and camera, its rate limit and command latency (whole steps) with the commands pending (dew.rl.flight.delayed), the track it races (into Sky.tracks), the gate it heads for (index) and the plane of it it crosses next (stage, into PLANES), its distance to that gate’s pre-gate after the last step, the gates passed, the steps taken, and the key its rotors’ noise is drawn from.

dataclass source

class Sky(
hz: float = 90.0,
tracks: tuple[str, ...] = ('circle',),
size: float = 0.8,
depth: float = 0.8,
anywhere: float = 0.7,
fov: float = 2 * math.degrees(math.atan(32 / 25)),
latency: float = 0.0,
quadrotor: Quadrotor = QUADCOPTER,
steps: int = 2000,
frames: bool = False,
)

SkyDreamer’s race (the module docstring): its control rate hz, the tracks an episode is drawn over (TRACKS’ names), its gate rewards’ size and the pre- and post-gates’ span depth (Table III), the share of starts before any gate (anywhere), the camera’s horizontal field of view fov (degrees), the command’s nominal latency (seconds), the quadrotor, the episode’s steps, and whether the frames of gates the drone is not passing are struck (frames).

racing: Racing

The gate race’s settings this race shares (dew.rl.flight.Racing): the camera’s (its fixed tilt and this field of view, its image’s randomization), the rate limit and its spread, the gates’ opening and frame.

layout: tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray]

Each track’s gates [T, G, ...], a shorter track’s padded by repeating its first: their centres [T, G, 3], headings [T, G] (radians) and whether each is seen [T, G], and each track’s gate count [T].

def delays(ramp: float) -> tuple[int, int, int]

The latency’s range in whole control steps at ramp, its nominal spread half its size either way at a ramp of 1, and the longest at any ramp, which the commands pending hold.

class source

class SkyDreamerRace(
renderer: Renderer | None,
sky: Sky | None = None,
views: tuple[str, ...] = VIEWS,
count: int | None = None,
ramp: float | tuple[float, ...] = 0.0,
)

A batch of renderer.count drones racing sky’s tracks (SkyRace, stepped on device), seen through renderer’s cameras (dew.rl.flight.Seen), as a dew.rl.simulator.Simulator: observations a dict of the views asked for (VIEWS), each [B, ...]. An episode is flown at its track’s level of the race’s ramp (SkyRace), which a race starts at and which may change between steps. The renderer’s gates must be the race’s, and a renderer’s fisheye lens (dew.rl.hyperlux.HyperLuxRenderer) the cameras’ field of view.

dataclass source

class SkyRace(
sky: Sky = struct.field(pytree_node=False, default_factory=Sky),
views: tuple[str, ...] = struct.field(pytree_node=False, default=VIEWS[1:]),
ramp: float | tuple[float, ...] = struct.field(pytree_node=False, default=0.0),
)

One drone racing sky’s tracks as a dew.rl.env.Env at randomization ramp (the module docstring), one level for every track or each of sky.tracks’ own in turn, an episode flown at its track’s: actions [4] in [-1, 1]; observations a dict of the views asked for, each but the image (VIEWS, which SkyDreamerRace adds). It cuts its own episodes (Sky.steps); a curriculum changes its ramp, its programs compiled again.

levels: np.ndarray

Each track’s randomization, [len(sky.tracks)], in sky.tracks’ order.

def observed(lap: Lap) -> dict[str, jax.Array]

The views of lap asked for.

def scene(lap: Lap) -> Scene

What a renderer draws of lap: its track’s seen gates, none lit, no pillars, in its camera’s looks.

def reset(key: jax.Array) -> tuple[Lap, TimeStep]

A start of Table III’s (the module docstring) at its track’s level of the race’s ramp.

def step(lap: Lap, action: jax.Array) -> tuple[Lap, TimeStep]

The drone flown a control step, then the planes it crossed in order, the frames it struck, the reward and whether the episode ended (the module docstring).

function source

def gate_of(sky: Sky, track: jax.Array, index: jax.Array) -> tuple[jax.Array, jax.Array]

Gate index of track track: its centre and heading.

function source

def heading_of(orientation: jax.Array) -> jax.Array

An orientation’s yaw (its yaw, pitch, roll Euler angles’).

function source

def in_gate(sky: Sky, track: jax.Array, index: jax.Array, point: jax.Array) -> jax.Array

point in gate index’s frame: along its normal, to its left and up from its centre.

function source

def plan(sky: Sky, track: jax.Array, index: jax.Array) -> jax.Array

The flight plan [24] at gate index (Eq 12): for it and the two after it, the change in position and heading from the gate before (a half turn either way), then their positions and headings.

function source

def pre_gate(sky: Sky, track: jax.Array, index: jax.Array) -> jax.Array

Gate index’s pre-gate’s centre.

function source

def roll_pitch(orientation: jax.Array) -> tuple[jax.Array, jax.Array]

An orientation’s roll and pitch (its yaw, pitch, roll Euler angles’).

function source

def struck(
sky: Sky,
track: jax.Array,
before: jax.Array,
after: jax.Array,
passing: jax.Array,
) -> jax.Array

Whether the step from before to after crosses the frame of a seen gate of track other than those passing [G] (the gates it heads for, whose planes decide alone): crosses its plane between its opening and its frame’s outer edge (Racing’s opening and frame).