dew.sampling.solvers
Diffusion solvers, each taking one reverse step from t to t_next with the model’s prediction at t.
A solver is a value, and a pytree: its settings are static, so they shape
the compiled walk, and only a number a walk varies by request (DPMSolverSDE’s
seed) is a leaf. Whatever it needs between steps is kept in its state,
which init builds and step carries through sample’s scan. The signal and
noise rates come from process. A solver that needs another model evaluation
(Heun’s corrector, RK4’s stages, KDPM2’s midpoint) calls denoise. A solver
that integrates dx / dsigma = eps refuses a schedule whose alpha is not one.
The solvers named after Diffusers 0.34.0 schedulers reproduce their
arithmetic. Before the compiled scan, init checks each algorithm’s endpoint
domain on the concrete grid; finite endpoint limits are tested, and endpoints
where the update is undefined raise an error instead of substituting another
update. A multistep update, which divides by the gaps between its history’s
levels, refuses a grid with two equal levels before its last interval.
| Name | Summary |
|---|---|
DDIM | The DDIM update, where eta sets the stochasticity: 0 is deterministic, 1 is DDPM-like. |
DDPM | Exact ancestral sampler for the reverse diffusion SDE. |
DEIS | DEIS (Zhang and Chen 2023, arXiv 2204.13902) in its log-rho multistep form, Diffusers 0.34.0’s DEISMultistepScheduler: the exponential integrator of eps with the polynomial-in-log(rho) interpolation of the last outputs, rho = sigma / alpha, integrated in closed form over the step. |
KDPM2 | k-diffusion’s DPM-Solver-2 (sample_dpm_2), the update of Diffusers 0.34.0’s KDPM2DiscreteScheduler, and with ancestral its sample_dpm_2_ancestral and KDPM2AncestralDiscreteScheduler. |
LMS | Linear multistep over dx/dsigma = (x - x_0) / sigma, k-diffusion’s sample_lms and Diffusers 0.34.0’s LMSDiscreteScheduler: the last order derivatives interpolated by the Lagrange polynomial through their sigmas and integrated over the step, in closed form where Diffusers quadratures; the order grows with the history. |
PNDM | PNDM as Diffusers 0.34.0’s PNDMScheduler: Adams-Bashforth over eps with DDIM as the transfer. |
RK4 | Classical fourth-order Runge-Kutta over dx/dsigma = eps on a variance-exploding schedule. |
TCD | Trajectory consistency sampling, as Diffusers 0.34.0’s TCDScheduler. |
Consistency | Multistep consistency sampling, the update of Diffusers 0.34.0’s LCMScheduler. |
DPMSolverMultistep | DPM-Solver and DPM-Solver++ as multistep integrators in lambda = log(alpha) - log(sigma). |
DPMSolverSDE | Diffusers 0.34.0’s DPMSolverSDEScheduler, k-diffusion’s sample_dpmpp_sde midpoint solver over a Brownian tree. |
DPMSolverSinglestep | Diffusers 0.34.0’s grouped DPM-Solver updates from each group’s anchor. |
Euler | The DDIM update written as an Euler step of the probability flow ODE. |
EulerAncestral | Euler with k-diffusion’s ancestral noise injection (get_ancestral_step, eta 1). |
Heun | Heun’s second-order method: an Euler step, the derivative re-evaluated at its end, and their average. |
MultiStepDPM | A third order multistep integrator of dx/dsigma = eps on a variance exploding schedule, from finite differences of the last three eps: FlaxDiff’s MultiStepDPM, which despite the name is not DPM-Solver. |
RowKeys | One typed PRNG key per row of a batch, keys [B], for a solver whose draws are row by row (RowSolver): a row’s draws are its own key’s, whatever rows sample beside it, as they are sampled alone, while the model still runs once over the batch. |
RowSolver | A solver that draws each row by its own key (RowKeys): sample hands step_rows the step’s row keys where it hands step one key. |
Solver | The interface for one diffusion solver step and the state it keeps between steps. |
UniPC | UniPC, a unified predictor and corrector in lambda, as Diffusers 0.34.0’s UniPCMultistepScheduler. |
class DDIM(eta: float = struct.field(pytree_node=False, default=0.0))The DDIM update, where eta sets the stochasticity: 0 is deterministic, 1 is DDPM-like.
DDIM is from Song et al. (2021). Diffusers 0.34.0’s DDIMScheduler limits the clean prediction under
clip_sample or thresholding and keeps the model’s own output as its
epsilon, so the direction term uses the unlimited one. That pairing comes
from SourceLimitedPrediction, in the process’s conversion.
DDIM.init
Section titled “DDIM.init”def init(x, times, process, *, key)DDIM.step
Section titled “DDIM.step”def step(x, t, t_next, denoised, eps, state, key, process, denoise)class DDPM( variance: Literal['small', 'large'] = struct.field(pytree_node=False, default='small'),)Exact ancestral sampler for the reverse diffusion SDE.
One step draws from the forward posterior q(x_s | x_t, x_0) for x_t = alpha_t x_0 + sigma_t eps, written in signal and noise rates so it holds for any schedule and any step stride. The posterior mean is alpha_s x_0 + alpha_t sigma_s^2 / (alpha_s sigma_t) eps and its variance is sigma_s^2 (1 - alpha_t^2 sigma_s^2 / (alpha_s^2 sigma_t^2)).
variance selects which of Diffusers 0.34.0’s fixed DDPMScheduler
posterior variances the draw uses. "small" is the posterior’s own
variance (posterior_std). "large" is the forward step’s own noise
(transition_variance), the wider choice from Glide, which Diffusers
writes as the beta 1 - alpha_t^2 / alpha_s^2 of a variance-preserving
table. Both are written in rates, so they are defined on any schedule.
Neither choice adds noise on the step whose own time is the schedule’s
zero, where x_t is already the least noisy state the schedule holds; the
source skips its draw at that time the same way. Elsewhere, when the
target’s alpha is one on a variance-preserving schedule, the wide noise
has standard deviation sigma_t, the square root of the source’s own
current_beta_t.
DDPM.init
Section titled “DDPM.init”def init(x, times, process, *, key)DDPM.step
Section titled “DDPM.step”def step(x, t, t_next, denoised, eps, state, key, process, denoise)class DEIS( order: int = struct.field(pytree_node=False, default=2), lower_order_final: bool = struct.field(pytree_node=False, default=True),)DEIS (Zhang and Chen 2023, arXiv 2204.13902) in its log-rho multistep
form, Diffusers 0.34.0’s DEISMultistepScheduler: the exponential
integrator of eps with the polynomial-in-log(rho) interpolation of the
last outputs, rho = sigma / alpha, integrated in closed form over the
step. The first order is DPM-Solver’s. Orders grow with the history;
lower_order_final is the same short-walk taper as
DPMSolverMultistep’s. At sigma=0 the integrated log-rho basis retains
its history terms. An alpha=0 source contributes a node at infinite rho;
that node’s weight vanishes in subsequent finite-interval integrals.
DEIS.init
Section titled “DEIS.init”def init(x, times, process, *, key)DEIS.step
Section titled “DEIS.step”def step(x, t, t_next, denoised, eps, state, key, process, denoise)class KDPM2(ancestral: bool = struct.field(pytree_node=False, default=False))k-diffusion’s DPM-Solver-2 (sample_dpm_2), the update of Diffusers
0.34.0’s KDPM2DiscreteScheduler, and with ancestral its
sample_dpm_2_ancestral and KDPM2AncestralDiscreteScheduler.
It takes an Euler step to the geometric midpoint of sigma_t and the target
level, evaluates the model there, and steps from x with that midpoint
derivative. The target is sigma_s, or under ancestral the sigma_down of
k-diffusion’s ancestral step with sigma_up of fresh noise added after.
The midpoint’s time comes from the schedule’s t_of_sigma, so this
integrates a GeneralizedNoiseScheduler.
KDPM2.init
Section titled “KDPM2.init”def init(x, times, process, *, key)KDPM2.step
Section titled “KDPM2.step”def step(x, t, t_next, denoised, eps, state, key, process, denoise)class LMS(order: int = struct.field(pytree_node=False, default=4))Linear multistep over dx/dsigma = (x - x_0) / sigma, k-diffusion’s
sample_lms and Diffusers 0.34.0’s LMSDiscreteScheduler: the last
order derivatives interpolated by the Lagrange polynomial through their
sigmas and integrated over the step, in closed form where Diffusers
quadratures; the order grows with the history. Integrates a
GeneralizedNoiseScheduler.
LMS.init
Section titled “LMS.init”def init(x, times, process, *, key)LMS.step
Section titled “LMS.step”def step(x, t, t_next, denoised, eps, state, key, process, denoise)class PNDM(skip_prk_steps: bool = struct.field(pytree_node=False, default=False))PNDM as Diffusers 0.34.0’s PNDMScheduler: Adams-Bashforth over eps with DDIM as the transfer.
PNDM is from Liu et al. (2022, arXiv 2202.09778). It is fourth order once
four outputs are available. The warmup is the paper’s,
three pseudo Runge-Kutta steps of four model evaluations each (the
stages at the interval’s midpoint and end), which seed the history with
each step’s first eps; under skip_prk_steps, the PLMS form Stable
Diffusion runs, the first step is a predictor-corrector pair (an Euler
step, eps re-read at its end, the step retaken with the mean) and the
orders grow from there. The schedule sets the transfer stride: ordinary
native grids use their adjacent interval, while published integer grids
retain their fixed training stride. Transfers cannot start at alpha = 0.
PNDM.init
Section titled “PNDM.init”def init(x, times, process, *, key)PNDM.step
Section titled “PNDM.step”def step(x, t, t_next, denoised, eps, state, key, process, denoise)class RK4()Classical fourth-order Runge-Kutta over dx/dsigma = eps on a variance-exploding schedule.
The half-step stages evaluate the model at the time the schedule maps
their sigma back to. This is FlaxDiff’s RK4Sampler.
RK4.init
Section titled “RK4.init”def init(x, times, process, *, key)RK4.step
Section titled “RK4.step”def step(x, t, t_next, denoised, eps, state, key, process, denoise)class TCD(eta: float = struct.field(pytree_node=False, default=0.3))Trajectory consistency sampling, as Diffusers 0.34.0’s TCDScheduler.
This is from Zheng et al. (2024, arXiv 2402.19159). The deterministic DDIM
step lands at (1 - eta) t_next, and the forward process noises it up to
t_next with fresh noise; this is the paper’s gamma-sampling with
gamma = eta. At eta 0 it
is DDIM; at eta 1 the step goes through the clean end of the schedule.
A tabulated schedule reads the intermediate time at its truncated
index, as the reference floors it. The last step of a walk lands on the
grid’s end itself and takes no noise.
TCD.init
Section titled “TCD.init”def init(x, times, process, *, key)TCD.step
Section titled “TCD.step”def step(x, t, t_next, denoised, eps, state, key, process, denoise)Consistency
Section titled “Consistency”class Consistency()Multistep consistency sampling, the update of Diffusers 0.34.0’s LCMScheduler.
This is Algorithm 1 of Song et al. (2023). The clean prediction is noised
again to the next level with fresh noise, x_s = alpha_s x_0 + sigma_s z,
and the last step keeps x_0 as it is. A latent consistency
model’s x_0 is the consistency function’s output, which
ConsistencyBoundary reads out of the model’s prediction.
Consistency.init
Section titled “Consistency.init”def init(x, times, process, *, key)Consistency.step
Section titled “Consistency.step”def step(x, t, t_next, denoised, eps, state, key, process, denoise)DPMSolverMultistep
Section titled “DPMSolverMultistep”class DPMSolverMultistep( order: int = struct.field(pytree_node=False, default=2), algorithm: Algorithm = struct.field(pytree_node=False, default='dpmsolver++'), solver_type: Literal['midpoint', 'heun'] = struct.field(pytree_node=False, default='midpoint'), lower_order_final: bool = struct.field(pytree_node=False, default=True), euler_at_final: bool = struct.field(pytree_node=False, default=False),)DPM-Solver and DPM-Solver++ as multistep integrators in lambda = log(alpha) - log(sigma).
The papers are Lu et al. (2022, arXiv 2206.00927) and arXiv 2211.01095.
This covers the four algorithms, three orders and two second-order forms
of Diffusers 0.34.0’s DPMSolverMultistepScheduler, with its defaults.
dpmsolver++ and sde-dpmsolver++ integrate the clean prediction, the
other two eps; the sde- forms add the noise term of the SDE solver.
With h = lambda_t - lambda_s0 over the step and D0, D1, D2 the finite
differences of the last outputs in lambda, the deterministic dpmsolver++
step is
x_t = (sigma_t / sigma_s0) x - alpha_t (e^-h - 1) D0 + c D1 + c_2 D2with each algorithm’s coefficients written as Diffusers writes them. The
first step has no history and is first order, the second at
most second. lower_order_final is Diffusers’ rule verbatim, which acts
only in a walk under 15 steps: first order on the last step and at most
second on the one before. euler_at_final makes the last step first
order whatever the length. A zero target forces the clean limit for
deterministic and ++ updates. The non-++ SDE first-order limit also
retains alpha_tsigma_s/alpha_s(noise-eps); it requires a finite source
alpha and a final order reduction. That endpoint is an equation-limit
extension: Diffusers refuses a literal zero-terminal non-++ config.
EDM uses the ++ algorithms over its existing process.
DPM-Solver++ 2M with no order taper is order=2, algorithm=“dpmsolver++”, solver_type=“midpoint”, lower_order_final=False, euler_at_final=False.
DPMSolverMultistep.init
Section titled “DPMSolverMultistep.init”def init(x, times, process, *, key)DPMSolverMultistep.step
Section titled “DPMSolverMultistep.step”def step(x, t, t_next, denoised, eps, state, key, process, denoise)DPMSolverSDE
Section titled “DPMSolverSDE”class DPMSolverSDE( depth: int = struct.field(pytree_node=False, default=MAX_BROWNIAN_DEPTH), seed: int | None = None,)Diffusers 0.34.0’s DPMSolverSDEScheduler, k-diffusion’s
sample_dpmpp_sde midpoint solver over a Brownian tree.
Each interval takes two ancestral first-order steps from its own start:
one to the geometric midpoint of sigma_t and sigma_s, where the model is
evaluated, and one to sigma_s with that midpoint’s clean prediction. Both
steps go down to k-diffusion’s sigma_down and add sigma_up of noise, and
both draw that noise from one Brownian path over the trajectory’s sigma
interval: the first over [sigma_t, sigma_mid] and the second over
[sigma_t, sigma_s], so the two are correlated exactly as nested
increments of one path. The source’s sampler transforms sigma with the
identity even though its own steps integrate -log(sigma), so the interval
widths are sigma differences.
depth resolves the root interval to (sigma_max - sigma_min) / 2**depth:
on a published VP table’s span of about 14.6 the default reaches 8.7e-7,
at or inside the reference tree’s own 1e-6 tolerance, and it is also where
a float32 position runs out of mantissa, so no deeper descent tells two
sigmas apart. A zero-sigma target has no ancestral step and lands on the
clean prediction.
The root interval is the schedule’s positive sigma domain, not the grid’s extremes, as the source builds its tree over every positive sigma it prepared, so a walk over a suffix of the grid keeps the whole grid’s path.
seed is the source’s noise_sampler_seed: the path is then fixed by
the checkpoint, independent of the walk’s key. A Torch seed names no JAX
stream, so it seeds this bridge rather than reproducing the reference tree.
It is the solver’s one pytree leaf, so a compiled walk takes another seed
as data; the rest is static.
DPMSolverSDE.init
Section titled “DPMSolverSDE.init”def init(x, times, process, *, key)DPMSolverSDE.step
Section titled “DPMSolverSDE.step”def step(x, t, t_next, denoised, eps, state, key, process, denoise)DPMSolverSinglestep
Section titled “DPMSolverSinglestep”class DPMSolverSinglestep( order: int = struct.field(pytree_node=False, default=2), algorithm: Literal['dpmsolver++', 'dpmsolver', 'sde-dpmsolver++'] = struct.field(pytree_node=False, default='dpmsolver++'), solver_type: Literal['midpoint', 'heun'] = struct.field(pytree_node=False, default='midpoint'), lower_order_final: bool = struct.field(pytree_node=False, default=False),)Diffusers 0.34.0’s grouped DPM-Solver updates from each group’s anchor.
A group of k model evaluations completes one k-th order update. Orders repeat [1, 2] or [1, 2, 3]; an incomplete final group uses lower order, as set_timesteps does in the reference. lower_order_final also lowers the final complete group, and a zero-sigma target forces final order 1. Source-domain checks use that effective order list.
At alpha=0, clean-prediction midpoint and deterministic Heun groups have finite limits. Noise-prediction groups above order 1 and third-order SDE Heun groups diverge; initialization rejects those source/grid pairs.
DPMSolverSinglestep.order_list
Section titled “DPMSolverSinglestep.order_list”def order_list(steps: int) -> list[int]Reference groups, completing an uneven final group at lower order.
DPMSolverSinglestep.init
Section titled “DPMSolverSinglestep.init”def init(x, times, process, *, key)DPMSolverSinglestep.step
Section titled “DPMSolverSinglestep.step”def step(x, t, t_next, denoised, eps, state, key, process, denoise)class Euler()The DDIM update written as an Euler step of the probability flow ODE. On a variance exploding schedule it is dx/dsigma = eps.
Euler.init
Section titled “Euler.init”def init(x, times, process, *, key)Euler.step
Section titled “Euler.step”def step(x, t, t_next, denoised, eps, state, key, process, denoise)EulerAncestral
Section titled “EulerAncestral”class EulerAncestral()Euler with k-diffusion’s ancestral noise injection (get_ancestral_step, eta 1).
Each step goes down to sigma_down, then adds fresh noise of standard
deviation sigma_up to bring the marginal back to sigma_s. It integrates a
GeneralizedNoiseScheduler.
EulerAncestral.init
Section titled “EulerAncestral.init”def init(x, times, process, *, key)EulerAncestral.step
Section titled “EulerAncestral.step”def step(x, t, t_next, denoised, eps, state, key, process, denoise)class Heun( s_churn: float = struct.field(pytree_node=False, default=0.0), s_tmin: float = struct.field(pytree_node=False, default=0.0), s_tmax: float = struct.field(pytree_node=False, default=float('inf')), s_noise: float = struct.field(pytree_node=False, default=1.0),)Heun’s second-order method: an Euler step, the derivative re-evaluated at its end, and their average.
This is Algorithm 2 of Karras et al. (2022).
s_churn, s_tmin, s_tmax and s_noise are the algorithm’s
stochasticity. At a sigma in [s_tmin, s_tmax] each step first raises the
noise level to sigma (1 + gamma), gamma = min(s_churn / N, sqrt(2) - 1)
for an N-interval walk, by adding fresh noise of standard deviation
s_noise sqrt(sigma_hat^2 - sigma^2), and then takes the Heun step from
there. The churn moves sigma directly, so it needs a variance-exploding
schedule, and a raised level past the schedule’s top has no model time, so
a sampling run that would churn there is refused. With churn, the step
evaluates the model at the raised level, and the evaluation sample made
at the grid point is unused, so the compiler removes it.
Diffusers 0.34.0’s HeunDiscreteScheduler limits the clean prediction of
both stages under clip_sample; that limit belongs to the process’s
conversion, SourceLimitedPrediction, so both evaluations here read the
limited prediction without the solver knowing about it.
Heun.init
Section titled “Heun.init”def init(x, times, process, *, key)Heun.step
Section titled “Heun.step”def step(x, t, t_next, denoised, eps, state, key, process, denoise)MultiStepDPM
Section titled “MultiStepDPM”class MultiStepDPM()A third order multistep integrator of dx/dsigma = eps on a variance
exploding schedule, from finite differences of the last three eps:
FlaxDiff’s MultiStepDPM, which despite the name is not DPM-Solver.
MultiStepDPM.init
Section titled “MultiStepDPM.init”def init(x, times, process, *, key)MultiStepDPM.step
Section titled “MultiStepDPM.step”def step(x, t, t_next, denoised, eps, state, key, process, denoise)RowKeys
Section titled “RowKeys”class RowKeys()One typed PRNG key per row of a batch, keys [B], for a solver whose draws are row by row
(RowSolver): a row’s draws are its own key’s, whatever rows sample beside it, as they are
sampled alone, while the model still runs once over the batch.
RowSolver
Section titled “RowSolver”class RowSolver(Solver[StateT], Protocol[StateT])A solver that draws each row by its own key (RowKeys): sample hands step_rows the
step’s row keys where it hands step one key.
RowSolver.step_rows
Section titled “RowSolver.step_rows”def step_rows( x, t, t_next, denoised, eps, state, keys: jax.Array, process, denoise, /,) -> tuple[jax.Array, StateT]step, row i drawing from keys[i] alone.
Solver
Section titled “Solver”class Solver(Protocol[StateT])The interface for one diffusion solver step and the state it keeps between steps.
StateT is that state: nothing for a one-step solver, the previous model
outputs for a multistep one. It is a type parameter, so each solver’s own
state type is checked at its call sites.
A solver is a pytree, as a pipeline call takes it as data: a
flax.struct.dataclass whose numbers are fields and whose settings are
struct.field(pytree_node=False), and a solver wrapping another holds it
as a field.
Solver.init
Section titled “Solver.init”def init(x, times, process, *, key) -> StateTReturn the initial state for sampling from x over the concrete grid times.
It checks the grid’s endpoint domains at compile time. key is the
root key of the whole sampling run, or a RowSolver’s RowKeys; only
DPMSolverSDE reads it, for its whole-trajectory Brownian tree. Each
step draws from the folded key passed to it.
Solver.step
Section titled “Solver.step”def step( x, t, t_next, denoised, eps, state, key, process, denoise, /,) -> tuple[jax.Array, StateT]Return the sample at t_next and the new state, from x and the model’s (denoised, eps) at t.
sample passes every argument by position, so a solver over another
algebra can name the pair for what it reads: the discrete solver takes
log-probabilities where a Gaussian one takes eps.
class UniPC( order: int = struct.field(pytree_node=False, default=2), solver_type: Literal['bh1', 'bh2'] = struct.field(pytree_node=False, default='bh2'), predict_x0: bool = struct.field(pytree_node=False, default=True), lower_order_final: bool = struct.field(pytree_node=False, default=True), disable_corrector: tuple[int, ...] = struct.field(pytree_node=False, default=()),)UniPC, a unified predictor and corrector in lambda, as Diffusers 0.34.0’s UniPCMultistepScheduler.
UniPC is from Zhao et al. (2023, arXiv 2302.04867). Its weights solve a
small linear system over the history’s positions. Each step first
corrects the sample the last predictor produced, with
the model output just read there and that predictor’s order, then
predicts the next sample from the corrected one. solver_type picks
B(h) as h (bh1) or e^h - 1 (bh2); predict_x0 integrates the clean
prediction, otherwise eps. lower_order_final caps the order by the
steps remaining, Diffusers’ rule, so the last step is first order;
disable_corrector names the step indices whose predictor’s output is
not corrected. The lowered clean-prediction terminal is alpha_t*x_0;
epsilon prediction uses the corrected sample with the pre-correction
epsilon. Initialization rejects an unlowered higher-order zero target.
At an alpha=0 source, bh1 and epsilon correctors diverge unless the
first correction is disabled. The finite infinite-node weights use the
same Vandermonde system with column scaling.
Every coefficient depends on the grid alone, so init solves them in
float64 for each interval and order, and a step reads
its interval’s row by where t sits on that grid and its order from
the state. A step’s t is a point of the grid init was given.
UniPC.init
Section titled “UniPC.init”def init(x, times, process, *, key)UniPC.step
Section titled “UniPC.step”def step(x, t, t_next, denoised, eps, state, key, process, denoise)