dew.rl.skydreamer
SkyDreamer’s race (Verraest et al., arXiv 2510.14783) flown from the camera’s RGB image by collective thrust and body rates: its task, reward, termination and starts, with Dew’s quadrotor and renderer.
The task (Sec II-C, Table III): a track’s gates passed in order, each through its pre-gate, itself and its
post-gate, planes depth / 2 before and after it. The reward is 5 progress - rate + 30 gate. Progress is the
step’s approach to the next pre-gate, zero from the pre-gate through the post-gate: a step that crosses the
pre-gate approaches it up to where it crosses. The rate penalty is (exp(min(|rates|_1, 17)) - 1) over 2 f_c
1e5. The gate reward at each of the three planes is 1 - max(|y|, |z|) / size where the drone crosses it. An
episode ends, its last reward zero, when the drone crosses one of those planes beyond size of the gate’s
centre, where the race has frames strikes the frame of a gate it is not passing (the race’s opening and
frame, Racing), or is within 0.5 m of the ground while descending faster than 1 m/s or rolled or pitched
past 60 degrees; it is cut after steps steps (2000, 22 s: the length of SkyDreamer’s flights in Fig 5; the
paper states no other). An episode starts on one of the race’s tracks, drawn alike, 2 to 4 m before a gate
(any gate in anywhere of starts, else the first), within 1 m to its side and up to 1.3 m below its centre
(0.7 to 1.3 m before the first), at rest, rolled, pitched and turned from the gate’s heading up to 20 degrees,
its body rates up to 0.1 rad/s and its rotors at a quarter to half of their top speed.
A track’s gates are seen or not: SkyDreamer adds gates no one sees to its tracks in training “to enforce the
correct flight maneuvers” (Sec III-A), a gate above one to fly over it. Its two tracks, given only in figures
(Figs 4 and 6, its 8 x 8 m arena), are reconstructed as INVERTED_LOOP and LADDER; CIRCLE stands four
gates on a circle in the same arena.
The drone (QUADCOPTER) is Dew’s quadrotor (dew.rl.quadrotor, its flight controller tracking collective
thrust and body rates) with SkyDreamer’s Table II where Dew’s model has the term: thrust k_w per unit mass
and the rotors’ speeds from 341.75 to 3100 rad/s (a thrust-to-weight ratio of 6.1), their 30 ms time
constant, and the drag along the body’s x and y, linear in the rotors’ speed and quadratic in the velocity;
no vertical drag, ground effect or battery sag. An action is FlyDreamer’s and Dream to Fly’s command (Romero
et al., arXiv 2501.14377, Sec III-B): its first entry the throttle, zero the drone’s hover thrust, -1 none and
1 its full thrust; the rest the body rates over the drone’s rate limit; it reaches the flight controller
latency seconds later (delays; SkyDreamer’s image and action delays together are 33 and 11 ms).
The policy reads the camera’s image (image, the renderer’s, SkyDreamerRace) and, as SkyDreamer’s does, the
body rates, the rotors’ speeds (each a share of the top speed, so a decoder’s squared error over them is of
the others’ size) and the flight plan (Sec II-H, Eq 12: for the next three gates, the change in position and
yaw from the gate before, then their positions and yaws). The plan moves on as the drone crosses a gate’s
plane: SkyDreamer moves it at a random point from the pre-gate to the post-gate in training, and where its
world model’s decoded position passes 0.15 m before the gate in flight; here it moves at the gate. The
privileged entries a world model decodes (state, Sec II-B) are the position and velocity in the world and in
the gate headed for, and the orientation, the roll, pitch and yaw in the world and the yaw from the gate’s
heading; the camera’s extrinsics are fixed and the dynamics’ parameters are left out. The camera is tilted up
30 degrees, fixed as a real FPV quadrotor’s camera is mounted (Racing’s; SkyDreamer’s is 45 to 55), its
horizontal field of view fov.
Randomization is ramp’s, a race’s curriculum from 0 (none) to 1, as dew.rl.flight’s, one level for
every track or each track’s own, so a curriculum may raise each track’s as its own flights allow: the
drone’s body and its rotors’ noise (dew.rl.quadrotor.drawn_body, flown), its rate limit, the camera’s
image and the scene’s looks (dew.rl.flight.drawn_camera), and the latency’s spread, half its nominal
either way at 1.
The world frame is x forward, y left, z up (dew.rl.flight’s).
| Name | Summary |
|---|---|
CIRCLE | Four gates 1.5 m up on a circle of 3 m about the arena’s centre, facing along the circle. |
INVERTED_LOOP | SkyDreamer’s inverted loop (Fig 6, its frame North-East-Down turned to this one’s): through the first gate, over the second, through it the other way after a split-S, and back over it; the gates’ centres 1.35 m up (its 2.7 m gates standing), the unseen gates 4 m up above the second. |
LADDER | SkyDreamer’s ladder inverted loop (Fig 4): through the first gate and, after a full turn, over it the same way, then the inverted loop’s second gate. |
PLANES | The planes a gate is passed through, in its depth’s halves along its normal: pre-gate, gate, post-gate. |
QUADCOPTER | Dew’s quadrotor at its nominal mass with SkyDreamer’s Table II thrust, rotor speeds, motor time constant and drag (each per unit mass there, so times the mass here), its yaw torque’s ratio to thrust Dew’s. |
TRACKS | The tracks a race is drawn over, by name. |
VIEWS | What the race shows: the camera’s image, the body rates and rotor speeds, the flight plan, and privileged, the state and the gates passed. |
Gate | A track’s gate: its centre, its heading (degrees, the way it is passed) and whether it is seen. |
Lap | One drone’s episode: its drawn body, flight and camera, its rate limit and command latency (whole steps) with the commands pending (dew.rl.flight.delayed), the track it races (into Sky.tracks), the gate it heads for (index) and the plane of it it crosses next (stage, into PLANES), its distance to that gate’s pre-gate after the last step, the gates passed, the steps taken, and the key its rotors’ noise is drawn from. |
Sky | SkyDreamer’s race (the module docstring): its control rate hz, the tracks an episode is drawn over (TRACKS’ names), its gate rewards’ size and the pre- and post-gates’ span depth (Table III), the share of starts before any gate (anywhere), the camera’s horizontal field of view fov (degrees), the command’s nominal latency (seconds), the quadrotor, the episode’s steps, and whether the frames of gates the drone is not passing are struck (frames). |
SkyDreamerRace | A batch of renderer.count drones racing sky’s tracks (SkyRace, stepped on device), seen through renderer’s cameras (dew.rl.flight.Seen), as a dew.rl.simulator.Simulator: observations a dict of the views asked for (VIEWS), each [B, ...]. |
SkyRace | One drone racing sky’s tracks as a dew.rl.env.Env at randomization ramp (the module docstring), one level for every track or each of sky.tracks’ own in turn, an episode flown at its track’s: actions [4] in [-1, 1]; observations a dict of the views asked for, each but the image (VIEWS, which SkyDreamerRace adds). |
gate_of | Gate index of track track: its centre and heading. |
heading_of | An orientation’s yaw (its yaw, pitch, roll Euler angles’). |
in_gate | point in gate index’s frame: along its normal, to its left and up from its centre. |
plan | The flight plan [24] at gate index (Eq 12): for it and the two after it, the change in position and heading from the gate before (a half turn either way), then their positions and headings. |
pre_gate | Gate index’s pre-gate’s centre. |
roll_pitch | An orientation’s roll and pitch (its yaw, pitch, roll Euler angles’). |
struck | Whether the step from before to after crosses the frame of a seen gate of track other than those passing [G] (the gates it heads for, whose planes decide alone): crosses its plane between its opening and its frame’s outer edge (Racing’s opening and frame). |
CIRCLE
Section titled “CIRCLE”CIRCLE: tuple[Gate, ...] = (((3.0, 0.0, 1.5), 90.0, True), ((0.0, 3.0, 1.5), 180.0, True), ((-3.0, 0.0, 1.5), 270.0, True), ((0.0, -3.0, 1.5), 0.0, True))Four gates 1.5 m up on a circle of 3 m about the arena’s centre, facing along the circle.
INVERTED_LOOP
Section titled “INVERTED_LOOP”INVERTED_LOOP: tuple[Gate, ...] = (((3.0, 0.0, 1.35), -90.0, True), ((-2.1, 0.0, 4.0), 90.0, False), ((-2.0, 0.0, 1.35), -90.0, True), ((-2.1, 0.0, 4.0), 90.0, False))SkyDreamer’s inverted loop (Fig 6, its frame North-East-Down turned to this one’s): through the first gate, over the second, through it the other way after a split-S, and back over it; the gates’ centres 1.35 m up (its 2.7 m gates standing), the unseen gates 4 m up above the second.
LADDER
Section titled “LADDER”LADDER: tuple[Gate, ...] = (((3.0, 0.0, 1.35), -90.0, True), ((3.0, 0.0, 4.0), -90.0, False), ((-2.1, 0.0, 4.0), 90.0, False), ((-2.0, 0.0, 1.35), -90.0, True), ((-2.1, 0.0, 4.0), 90.0, False))SkyDreamer’s ladder inverted loop (Fig 4): through the first gate and, after a full turn, over it the same way, then the inverted loop’s second gate.
PLANES
Section titled “PLANES”PLANES = (-1, 0, 1)The planes a gate is passed through, in its depth’s halves along its normal: pre-gate, gate, post-gate.
QUADCOPTER
Section titled “QUADCOPTER”QUADCOPTERDew’s quadrotor at its nominal mass with SkyDreamer’s Table II thrust, rotor speeds, motor time constant and drag (each per unit mass there, so times the mass here), its yaw torque’s ratio to thrust Dew’s.
TRACKS
Section titled “TRACKS”TRACKS = {'circle': CIRCLE, 'inverted_loop': INVERTED_LOOP, 'ladder': LADDER}The tracks a race is drawn over, by name.
VIEWS = ('image', 'rates', 'motors', 'plan', 'state', 'passed')What the race shows: the camera’s image, the body rates and rotor speeds, the flight plan, and privileged,
the state and the gates passed.
A track’s gate: its centre, its heading (degrees, the way it is passed) and whether it is seen.
class Lap()One drone’s episode: its drawn body, flight and camera, its rate limit and command latency (whole
steps) with the commands pending (dew.rl.flight.delayed), the track it races (into Sky.tracks), the
gate it heads for (index) and the plane of it it crosses next (stage, into PLANES), its distance to
that gate’s pre-gate after the last step, the gates passed, the steps taken, and the key its rotors’
noise is drawn from.
class Sky( hz: float = 90.0, tracks: tuple[str, ...] = ('circle',), size: float = 0.8, depth: float = 0.8, anywhere: float = 0.7, fov: float = 2 * math.degrees(math.atan(32 / 25)), latency: float = 0.0, quadrotor: Quadrotor = QUADCOPTER, steps: int = 2000, frames: bool = False,)SkyDreamer’s race (the module docstring): its control rate hz, the tracks an episode is drawn
over (TRACKS’ names), its gate rewards’ size and the pre- and post-gates’ span depth (Table III),
the share of starts before any gate (anywhere), the camera’s horizontal field of view fov (degrees),
the command’s nominal latency (seconds), the quadrotor, the episode’s steps, and whether the frames
of gates the drone is not passing are struck (frames).
racing: Racing-
The gate race’s settings this race shares (
dew.rl.flight.Racing): the camera’s (its fixed tilt and this field of view, its image’s randomization), the rate limit and its spread, the gates’ opening and frame. layout: tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray]-
Each track’s gates
[T, G, ...], a shorter track’s padded by repeating its first: their centres[T, G, 3], headings[T, G](radians) and whether each is seen[T, G], and each track’s gate count[T].
Sky.delays
Section titled “Sky.delays”def delays(ramp: float) -> tuple[int, int, int]The latency’s range in whole control steps at ramp, its nominal spread half its size either way
at a ramp of 1, and the longest at any ramp, which the commands pending hold.
SkyDreamerRace
Section titled “SkyDreamerRace”class SkyDreamerRace( renderer: Renderer | None, sky: Sky | None = None, views: tuple[str, ...] = VIEWS, count: int | None = None, ramp: float | tuple[float, ...] = 0.0,)A batch of renderer.count drones racing sky’s tracks (SkyRace, stepped on device), seen through
renderer’s cameras (dew.rl.flight.Seen), as a dew.rl.simulator.Simulator: observations a dict of
the views asked for (VIEWS), each [B, ...]. An episode is flown at its track’s level of the race’s
ramp (SkyRace), which a race starts at and which may change between steps. The renderer’s gates
must be the race’s, and a renderer’s fisheye lens (dew.rl.hyperlux.HyperLuxRenderer) the cameras’
field of view.
SkyRace
Section titled “SkyRace”class SkyRace( sky: Sky = struct.field(pytree_node=False, default_factory=Sky), views: tuple[str, ...] = struct.field(pytree_node=False, default=VIEWS[1:]), ramp: float | tuple[float, ...] = struct.field(pytree_node=False, default=0.0),)One drone racing sky’s tracks as a dew.rl.env.Env at randomization ramp (the module docstring),
one level for every track or each of sky.tracks’ own in turn, an episode flown at its track’s:
actions [4] in [-1, 1]; observations a dict of the views asked for, each but the image (VIEWS,
which SkyDreamerRace adds). It cuts its own episodes (Sky.steps); a curriculum changes its ramp,
its programs compiled again.
levels: np.ndarray-
Each track’s randomization,
[len(sky.tracks)], insky.tracks’ order.
SkyRace.observed
Section titled “SkyRace.observed”def observed(lap: Lap) -> dict[str, jax.Array]The views of lap asked for.
SkyRace.scene
Section titled “SkyRace.scene”def scene(lap: Lap) -> SceneWhat a renderer draws of lap: its track’s seen gates, none lit, no pillars, in its camera’s
looks.
SkyRace.reset
Section titled “SkyRace.reset”def reset(key: jax.Array) -> tuple[Lap, TimeStep]A start of Table III’s (the module docstring) at its track’s level of the race’s ramp.
SkyRace.step
Section titled “SkyRace.step”def step(lap: Lap, action: jax.Array) -> tuple[Lap, TimeStep]The drone flown a control step, then the planes it crossed in order, the frames it struck, the reward and whether the episode ended (the module docstring).
gate_of
Section titled “gate_of”def gate_of(sky: Sky, track: jax.Array, index: jax.Array) -> tuple[jax.Array, jax.Array]Gate index of track track: its centre and heading.
heading_of
Section titled “heading_of”def heading_of(orientation: jax.Array) -> jax.ArrayAn orientation’s yaw (its yaw, pitch, roll Euler angles’).
in_gate
Section titled “in_gate”def in_gate(sky: Sky, track: jax.Array, index: jax.Array, point: jax.Array) -> jax.Arraypoint in gate index’s frame: along its normal, to its left and up from its centre.
def plan(sky: Sky, track: jax.Array, index: jax.Array) -> jax.ArrayThe flight plan [24] at gate index (Eq 12): for it and the two after it, the change in position
and heading from the gate before (a half turn either way), then their positions and headings.
pre_gate
Section titled “pre_gate”def pre_gate(sky: Sky, track: jax.Array, index: jax.Array) -> jax.ArrayGate index’s pre-gate’s centre.
roll_pitch
Section titled “roll_pitch”def roll_pitch(orientation: jax.Array) -> tuple[jax.Array, jax.Array]An orientation’s roll and pitch (its yaw, pitch, roll Euler angles’).
struck
Section titled “struck”def struck( sky: Sky, track: jax.Array, before: jax.Array, after: jax.Array, passing: jax.Array,) -> jax.ArrayWhether the step from before to after crosses the frame of a seen gate of track other than those
passing [G] (the gates it heads for, whose planes decide alone): crosses its plane between its opening
and its frame’s outer edge (Racing’s opening and frame).