dew.diffusion.process
The diffusion process a model is trained and sampled with, and the denoiser that runs a model on it.
| Name | Summary |
|---|---|
Denoiser | Denoises with one model, its parameters and its conditions. |
DenoisingCondition | Text conditioning in the form the published model families read it. |
Process | Pairs a noise schedule with what the model predicts on it and how the loss is weighted. |
aligned_conditions | Return unconditional with each row’s own model inputs taken from conditions. |
Denoiser
Section titled “Denoiser”class Denoiser( process: Process, model: nn.Module, params: Variables, conditions: dict[str, Conditioning], unconditional: dict[str, Conditioning] | None, schedule: NoiseScheduler, opened: tuple[str, ...] = (),)Denoises with one model, its parameters and its conditions.
A call computes the model’s raw output at (x_t, t) and then the
process’s conversion of it into (x_0, epsilon). The two steps are
separate methods because a conversion is not always linear in the
output. Dynamic thresholding and sample clipping limit x_0, and a
consistency boundary computes a function of it, so combining two raw
outputs after conversion gives a different result from combining them
before. Guidance therefore calls raw_both, combines the raw outputs
and converts once, in the same order a source pipeline runs its
scheduler.
opened: tuple[str, ...]-
The collections its model’s calls open and return beside their outputs: what a watched step reads of the forward that trains (
raw_observed). A guidance’s prediction on an opened denoiser returns its pair and these (observed), through any transform of its own, as theGuidanceprotocol asks; none leaves every call’s output alone. schedule: NoiseScheduler-
The schedule
tis on: the process’s sampling schedule for a solver’s grid, its training schedule for a training objective’s draw.
Denoiser.convert
Section titled “Denoiser.convert”def convert(x_t, t, output) -> tuple[jax.Array, jax.Array]Return (x_0, epsilon) recovered from a raw model output at (x_t, t).
Denoiser.predict
Section titled “Denoiser.predict”def predict( x_t, t,) -> tuple[jax.Array, jax.Array] | tuple[tuple[jax.Array, jax.Array], Variables | None](x_0, epsilon) at (x_t, t) as an unguided walk predicts it, observed where the denoiser is
opened (observed).
Denoiser.observed
Section titled “Denoiser.observed”def observed( pair: tuple[jax.Array, jax.Array], sown: Variables | None,) -> tuple[jax.Array, jax.Array] | tuple[tuple[jax.Array, jax.Array], Variables | None]A prediction’s return: pair, and beside it what its model’s calls opened (opened), where
the denoiser opens any; the shape every guidance’s prediction returns.
Denoiser.spanning
Section titled “Denoiser.spanning”def spanning(t, t_next) -> DenoiserReturn this denoiser over the interval from t to t_next.
Every call of the returned denoiser passes the interval’s length in
model time to the model as the duration condition.
Denoiser.raw
Section titled “Denoiser.raw”def raw(x_t, t) -> jax.ArrayReturn the model’s raw output at (x_t, t) under the conditions.
Denoiser.raw_observed
Section titled “Denoiser.raw_observed”def raw_observed(x_t, t) -> tuple[jax.Array, Variables | None]The model’s raw output at (x_t, t) under the conditions, and what the call opened
(opened), None where it opens none.
Denoiser.raw_both
Section titled “Denoiser.raw_both”def raw_both(x_t, t) -> tuple[jax.Array, jax.Array]Return the conditional and the unconditional raw outputs.
Denoiser.raw_both_observed
Section titled “Denoiser.raw_both_observed”def raw_both_observed(x_t, t) -> tuple[tuple[jax.Array, jax.Array], Variables | None]Return the conditional and the unconditional raw outputs, and what their call opened.
Both come from one model call over the doubled batch. Raises
ValueError when the denoiser has no unconditional conditions.
DenoisingCondition
Section titled “DenoisingCondition”class DenoisingCondition( pooled: jax.Array | None = None, time_ids: jax.Array | None = None, guidance: jax.Array | None = None, mask: jax.Array | None = None,)Text conditioning in the form the published model families read it.
contextholds the token states.pooledis a pooled vector.time_idsare the size and crop ids that the XL towers add.guidanceis the distilled guidance value, which a guidance-embedded transformer takes as a model input instead of running two guided branches.maskis the[B, tokens]mask of the real token states in a right-paddedcontext, read by a family that excludes padded keys.
DenoisingCondition.aligned
Section titled “DenoisingCondition.aligned”def aligned(given: DenoisingCondition) -> DenoisingConditionReturn this conditioning with given’s distilled guidance value.
The guidance value belongs to the row and not to its caption, so a dropped or unconditional caption keeps the guidance scale the checkpoint samples at.
Process
Section titled “Process”class Process( schedule: NoiseScheduler, prediction: PredictionTransform, weighting: Weighting = ScheduleWeighting(), sampling: NoiseScheduler | None = None, interval: bool = False,)Pairs a noise schedule with what the model predicts on it and how the loss is weighted.
sampling is the schedule that inference integrates when it differs
from the training schedule. For example, EDM trains on log-normal sigmas
and samples on the Karras grid. None means sampling uses the training
schedule.
interval is whether the model predicts over an interval of time and
not at an instant, as MeanFlow’s average velocity and a shortcut model’s
step do. Such a model reads the interval’s length in model time as the
duration condition. At each step, sample gives the denoiser the
interval to the next grid point (Denoiser.spanning), so a
single-evaluation solver such as Euler covers the whole interval in
one step. A zero duration gives the instantaneous prediction.
Process.weight
Section titled “Process.weight”def weight(t) -> jax.ArrayProcess.rates
Section titled “Process.rates”def rates(t, *, like: jax.Array) -> tuple[jax.Array, jax.Array]Return the sampling schedule’s (alpha, sigma) at t, shaped to broadcast against like.
like is a [B, ...] state.
Process.variances
Section titled “Process.variances”def variances(t, *, like: jax.Array) -> tuple[jax.Array, jax.Array]Return the sampling schedule’s (alpha^2, sigma^2) at t (NoiseScheduler.variances),
shaped to broadcast against like.
Process.times
Section titled “Process.times”def times(steps: int) -> jax.ArrayReturn the descending grid of steps times, from T to 0, that a solver steps through.
A tabulated schedule cannot take more steps than it has entries, so
steps is capped at T there.
Process.noise
Section titled “Process.noise”def noise(key, shape) -> jax.ArrayDraw an array of shape from the sampling schedule’s Gaussian prior.
By default the prior’s standard deviation is that of the marginal at
T for unit-variance data. A schedule may declare a different scale
through prior_scale. The draw is float32’s whatever the default
dtype, as the samplers’ innovations are, so a float64 walk from a key
starts from the float32 walk’s numbers.
Process.denoiser
Section titled “Process.denoiser”def denoiser( model, params, conditions: Mapping[str, Conditioning], unconditional: Mapping[str, Conditioning] | None = None, *, schedule: NoiseScheduler | None = None, opened: Sequence[str] = (),) -> DenoiserReturn the denoiser of model under params with the given conditions.
The denoiser maps (x_t, t) to (x_0, epsilon) with t on
schedule, the sampling schedule unless a training objective passes
its training schedule for the t it drew. unconditional has the
same keys with the unconditional values, and classifier-free guidance
interpolates against it. opened names the collections its model’s
calls return beside their outputs (Denoiser.opened), a watched
step’s (dew.objectives.base.Step.sown).
aligned_conditions
Section titled “aligned_conditions”def aligned_conditions( conditions: Mapping[str, Conditioning], unconditional: Mapping[str, Conditioning],) -> dict[str, Conditioning]Return unconditional with each row’s own model inputs taken from conditions.
A keyword must name a conditioning record on both sides or on neither.
Array keywords, such as the spatial ones an inpainting source adds, pass
through unchanged. A keyword that is a record on one side only means the
caller paired two different conditionings, so it raises ValueError.