Skip to content

dew.nn.backbones.z_image

Z-Image’s single-stream transformer (S3-DiT), as Diffusers 0.40.0’s ZImageTransformer2DModel runs it for text-to-image.

Each 2x2 latent patch is one token. Each stream is padded to a multiple of 32 tokens with a learned pad token, and the image’s pad tokens sit at position (0, 0, 0). Two time-modulated refiner blocks run over the image and two unmodulated ones over the prompt, then the main blocks run over both, image first. Every block RMS-norms before and after the attention and the SwiGLU, and the tanh of the time modulation gates its residuals. The three-axis rotary places prompt token i at (1 + i, 0, 0) and patch (h, w) at (1 + P, h, w), where P is the padded prompt length. The angles are looked up in a float64 table rounded to float32, as the source does. The source pads a batch to its longest row and masks; here the prompt region is the conditioner’s budget rounded up to 32, masked the same way. The source takes time 1 - sigma and returns the negated flow, while this module takes Dew’s model time and returns Dew’s flow.

NameSummary
ZImageBlockRuns one ZImageTransformerBlock.
ZImageTransformerRuns Diffusers 0.40.0’s ZImageTransformer2DModel behind Dew’s model interface.

Flax module source

class ZImageBlock(
epsilon: float = 1e-05,
modulation: bool = True,
dtype: Dtype | None = None,
precision: PrecisionLike = None,
attention_impl: AttentionImpl = 'auto',
)

Runs one ZImageTransformerBlock.

With modulation, one projection of the time embedding gives the input scales (one plus the projection) and the output gates (its tanh) of the attention and the feed-forward. Without it, as in the prompt’s refiner blocks, the block has no time modulation.

def model_flops(lengths) -> float

Real-token QKV/output and gated MLP; pad rows stay outside MFU.

Flax module source

class ZImageTransformer(
in_channels: int = 16,
dim: int = 3840,
n_layers: int = 30,
n_refiner_layers: int = 2,
n_heads: int = 30,
norm_eps: float = 1e-05,
cap_feat_dim: int = 2560,
rope_theta: float = 256.0,
t_scale: float = 1000.0,
axes_dims: Sequence[int] = (32, 48, 48),
axes_lens: Sequence[int] = (1536, 512, 512),
dtype: Dtype | None = None,
precision: PrecisionLike = None,
attention_impl: AttentionImpl = 'auto',
)

Runs Diffusers 0.40.0’s ZImageTransformer2DModel behind Dew’s model interface.

__call__ takes NHWC latents, the model time the schedule supplies (the sigma times the training count) and a DenoisingCondition, and returns the flow. The condition’s context is the prompt states, padded on the right to a fixed budget, and its mask marks the real tokens.

text_keyword: str

Every call takes the caption as conditioning, which joins the image’s single stream.

def projections(input_channels: int, text_features: int) -> dict[str, tuple[int, int]]
def condition_lengths(lengths: np.ndarray, spans: np.ndarray) -> np.ndarray
def model_flops(lengths) -> float
def image_lengths(sample_shape: tuple[int, ...], rows: int) -> np.ndarray
def multimodal_flops(
lengths,
context_lengths,
input_channels: int,
text_features: int,
) -> float

Static patch/text maps, both refiner stacks and the joined stack.