Skip to content

dew.objectives.diffusion.few_step

Few-step generators trained from scratch: MeanFlow and shortcut models.

MeanFlow (Geng et al. 2025, “Mean Flows for One-step Generative Modeling”) trains a model of the average velocity u(z_t, r, t) over [r, t] through the MeanFlow identity u = v - (t - r) du/dt, where one JVP takes the total derivative along the flow. One step of the average velocity then crosses the whole interval. The official code is Gsunshine/meanflow’s MeanFlow, in JAX, which tools/meanflow_reference.py runs.

Shortcut models (Frans et al. 2025, “One Step Diffusion via Shortcut Models”) train a velocity conditioned on its step size d. They train flow matching at the smallest step, and on a fraction of the batch they train self-consistency: one step of 2d equals two steps of d. The official code is kvfrans/shortcut-models’ get_targets, in JAX, which tools/shortcut_reference.py runs.

NameSummary
MeanFlowObjectiveTrains MeanFlow on an interval process (presets.MeanFlow).
ShortcutObjectiveTrains a shortcut model on an interval process (presets.Shortcut).
adaptive_lossReturn MeanFlow’s adaptively weighted loss for each row.
guided_velocityReturn MeanFlow’s training-time guided target velocity (the reference’s guidance_fn).
intervalsReturn MeanFlow’s (t, r) interval draw from two independent times.
mean_flow_targetReturn the model’s average velocity at (z, t, r) and its regression target v - (t - r) du/dt.
shortcut_levelsReturn the step levels of the self-consistency rows, where level l is a step of 2^-l.
shortcut_targetReturn the self-consistency target for one step of size step from x at sigma.

class source

class MeanFlowObjective(
model: nn.Module,
process: Process | Preset,
inputs: InputSpec,
*,
instantaneous: float = 0.75,
omega: float = 1.0,
kappa: float = 0.0,
guidance_interval: tuple[float, float] = (0.0, 1.0),
norm_p: float = 1.0,
norm_eps: float = 0.01,
solver: Solver = _EULER,
steps: int = 2,
**kwargs={},
)

Trains MeanFlow on an interval process (presets.MeanFlow).

instantaneous is the fraction of rows trained at r = t (the reference’s data_proportion, 0.75). omega and kappa set the training-time guidance, applied where t lies in guidance_interval; omega 1 and kappa 0 train without guidance. norm_p and norm_eps are the adaptive weighting’s power and epsilon.

The condition is dropped on unconditional_prob of the rows, and the target of a dropped row is the unguided v. As in the reference’s cond_drop, the dropped rows are the first ones, as many as a draw at that rate counts. They therefore fall on the instantaneous rows, which is where the training-time guidance reads the unconditional velocity. Sampling takes steps - 1 Euler steps of the average velocity, one by default. It uses no guidance, because the guidance is trained into the model.

The loss differentiates the model with respect to time, so the model’s time embedding must be smooth in time. A run config sets simple_dit(time_scale=0.002) for this, in place of the default 16.

def loss(variables, batch, step: Step)

class source

class ShortcutObjective(
model: nn.Module,
process: Process | Preset,
inputs: InputSpec,
*,
sections: int = 128,
bootstrap_every: int = 8,
solver: Solver = _EULER,
steps: int = 2,
**kwargs={},
)

Trains a shortcut model on an interval process (presets.Shortcut).

sections is the finest grid, the reference’s denoise_timesteps (128), a power of two. Flow-matching rows train at one step of 1 / sections, on times of that grid. One row in bootstrap_every (8) trains self-consistency at a level from shortcut_levels, on times of that level’s grid. Its target is two half steps of the EMA weights when the run keeps them. The condition is dropped on unconditional_prob of the flow-matching rows.

The path is the reference’s, whose noise never quite vanishes. The state at t is (1 - (1 - 1e-5) t) noise + t data, where Dew’s sigma is 1 - t, and the velocity toward the noise is (1 - 1e-5) noise - data. Sampling takes steps - 1 equal Euler steps with no guidance. Step counts that are powers of two up to sections are the ones the model trained at.

def loss(variables, batch, step: Step)

function source

def adaptive_loss(u, target, power: float, epsilon: float) -> jax.Array

Return MeanFlow’s adaptively weighted loss for each row.

Each row’s squared error is summed over its entries and divided by the stopped (error + epsilon)^power. Power 0 gives the plain error, and power 1 gives a loss near one per row.

function source

def guided_velocity(v, unconditional, conditional, omega, kappa)

Return MeanFlow’s training-time guided target velocity (the reference’s guidance_fn).

It mixes the sample’s v with the model’s own unconditional and conditional velocities: omega v + (1 - omega - kappa) v_u + kappa v_c. With kappa 0 this is v_u + omega (v - v_u).

function source

def intervals(
t: jax.Array,
r: jax.Array,
instantaneous: float,
places: jax.Array | None = None,
total: int | None = None,
) -> tuple[jax.Array, jax.Array]

Return MeanFlow’s (t, r) interval draw from two independent times.

The later time is t and the earlier is r. The first instantaneous fraction of the update’s total examples takes r = t, the plain flow-matching case (the reference’s sample_tr), by each row’s place among them (Step.places; by default the rows, all of the update).

function source

def mean_flow_target(
velocity: Velocity,
z,
t,
r,
v,
trained: Velocity | None = None,
) -> tuple[jax.Array, jax.Array]

Return the model’s average velocity at (z, t, r) and its regression target v - (t - r) du/dt.

One JVP, traced for forward mode (forward_mode_attention), takes the derivative along (dz/dt, dt/dt, dr/dt) = (v, 1, 0); the velocity the loss trains comes from the ordinary forward, which a fused attention kernel differentiates in reverse, as sCM’s student does, trained’s where given (one that records what its call sows). No gradient flows through the target.

function source

def shortcut_levels(rows: int, sections: int) -> jax.Array

Return the step levels of the self-consistency rows, where level l is a step of 2^-l.

Each level gets rows // log2(sections) rows, starting with the finest level, log2(sections) - 1, and ending with level 0. The remaining rows are also at level 0, one step across the whole path (the reference’s get_targets).

function source

def shortcut_target(velocity: Velocity, x, sigma, step) -> jax.Array

Return the self-consistency target for one step of size step from x at sigma.

The target is the mean velocity of two steps of half that size, with each state clipped to [-4, 4] as the reference clips it. velocity(x, sigma, sigma - step) is the model’s velocity over the interval to sigma - step.