References and attribution

The papers below explain methods that Dew uses. The projects below are Dew’s dependencies, reference implementations it is checked against, or sources of code it adapts. Citing a paper or project does not mean Dew reproduces all of its results or supports every model it covers. For Dew’s own interfaces, use the core API reference. For measurements with their hardware and revision, see benchmarks.

Diffusion and flow models

If you are new to diffusion, read DDPM first for the denoising objective. Then read EDM, which explains how noise levels, the network’s parameterization, training, and sampling relate to each other. Flow Matching describes a related way to learn continuous paths between two distributions. The other papers each cover a particular sampler, weighting rule, architecture, or guidance method.

Topic	Paper
Denoising diffusion	Denoising Diffusion Probabilistic Models
Implicit sampling	Denoising Diffusion Implicit Models
Learned variances and training choices	Improved Denoising Diffusion Probabilistic Models
Image synthesis and classifier guidance	Diffusion Models Beat GANs on Image Synthesis
Continuous-time score models	Score-Based Generative Modeling through Stochastic Differential Equations
Parameterization and sampling design	Elucidating the Design Space of Diffusion-Based Generative Models (EDM)
P2 loss weighting	Perception Prioritized Training of Diffusion Models
Pseudo numerical sampling methods	Pseudo Numerical Methods for Diffusion Models on Manifolds
Differential-equation solvers	DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps
Transformer diffusion backbones	Scalable Diffusion Models with Transformers (DiT)
Rectified-flow transformers	Scaling Rectified Flow Transformers for High-Resolution Image Synthesis
Continuous flow objectives	Flow Matching for Generative Modeling
Min-SNR loss weighting	Efficient Diffusion Training via Min-SNR Weighting Strategy
Guidance over part of the sampling path	Applying Guidance in a Limited Interval Improves Sample and Distribution Quality
Diffusion language modeling and a square-root schedule	Diffusion-LM Improves Controllable Text Generation

Dew’s simple_dit embeds the time step with EDM-style random Fourier features, where the original DiT uses sinusoidal embeddings. So simple_dit is a variant of the DiT architecture. How the time value is scaled belongs to the model and training convention you pick. The flow preset multiplies time by 1000, but Fourier features do not require that factor in general. The diffusion guide shows how a preset and a model fit together.

Representation learning and sequence models

Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture (I-JEPA) predicts the representations of hidden image regions from the visible context.
Revisiting Feature Prediction for Learning Visual Representations from Video (V-JEPA) applies the same feature prediction to video.
Simplified State Space Layers for Sequence Modeling (S5) describes the state-space layer that Dew’s SSM components follow.

The representation-learning guide explains the encoder, the predictor, the target encoder, and the masks before it gets to recipe settings.

Language-model post-training

Direct Preference Optimization: Your Language Model is Secretly a Reward Model derives DPO from preference data and a reference policy.
DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models introduces GRPO.
TRL and verl are the reference implementations for the small numerical comparisons recorded in post-training.

Those comparisons check fixed losses, gradients, and one chat-rendering case. They do not show that Dew learns end to end like TRL or verl, and they say nothing about agentic RL or production deployment.

Dependencies and adapted code

Dew uses JAX for arrays and transformations, Flax Linen for neural-network modules, Optax for optimizers, Orbax for checkpoints, and Grain for data loading. tyro turns the recipe dataclasses into command-line interfaces. Weights & Biases is an optional run tracker.

Depending on the loader you choose, the image and public-dataset paths use Albumentations, OpenCV, and TensorFlow Datasets. Transformers provides tokenizers and encoders, and safetensors provides the tensor file format. The offline byte-token and in-memory examples do not need all of these; installation lists the extras each workflow needs.

The Stable Diffusion Flax VAE is adapted from Hugging Face Diffusers v0.29.2, which is Apache-2.0. Parts of Dew’s attention blocks are also adapted from Diffusers’ Flax attention code. The InceptionV3 model used for FID is adapted mainly from jax-fid, which itself descends from the PyTorch/torchvision model. If you redistribute adapted code, keep the upstream attribution and license notices. The licenses of dependencies, checkpoints, and datasets are separate from Dew’s license.

facebookresearch/ijepa and facebookresearch/jepa are the reference code for JEPA masking and probes. Katherine Crowson’s k-diffusion and NVIDIA’s EDM implementation are the references for diffusion parameterizations, schedules, and solvers.

Tutorials and further reading

Sander Dieleman’s posts cover diffusion, typicality, guidance geometry, and noise schedules. Tony Duan’s Diffusion Models from Scratch works through the mathematics with small MNIST implementations and accompanying code.

I started my original FlaxDiff experiments from the Keras tutorials for DDPM by A_K Nain and DDIM by András Béres. They are still good introductions, though their APIs are not Dew’s. The FlaxDiff history page keeps those older experiments apart from current runs.

MaxText and Levanter are JAX projects for training language models. verl is for RL post-training and vLLM is for inference and serving. Each project documents its own supported models and deployment requirements.

Dew’s save_hf_layout writes model.safetensors and config.json into a directory in the Hugging Face layout. It does not translate tensor names or the model configuration. The separate save_pretrained_decoder API does translate for the decoder families it accepts, refuses model features it does not support, and writes the files of the tokenizer you give it next to the weights. Neither API guarantees that a given serving engine can run the export. Check the family-specific limits in language models, pass the tokenizer you trained with, and test the export in the program that will load it.