Skip to content

End-to-end runs

Five scripts under examples/ each run a whole job: they read data, train or load weights, and score or export the result. By default a script uses settings for real hardware. --smoke replaces them with the repository’s tiny fixtures, a few steps and one CPU device, and needs no network and no accelerator. tests/test_examples.py runs every smoke command below as a subprocess and checks the files it leaves.

examples/train_flowers_tpu.py trains a simple_dit denoiser from scratch on Oxford Flowers. One DiffusionRunConfig holds the model, the prepared ArrayRecords, CLIP text conditioning, CFG with a Heun solver, EMA and the validation metrics. config.train runs it over the whole slice under MeshSpec(fsdp=jax.device_count()), with a profiler window. At the end the script loads the run back with dew.pipeline, samples a grid of images and scores it with dew.eval.fid and dew.eval.clip_score.

First prepare the TFDS ArrayRecords once, then launch the same file on every worker with the TFDS version directory:

Terminal window
python -c "import tensorflow_datasets as tfds; tfds.builder('oxford_flowers102', \
data_dir='~/.cache/dew/datasets').download_and_prepare(file_format='array_record')"
python examples/train_flowers_tpu.py --data ~/.cache/dew/datasets/oxford_flowers102/2.1.1 --steps 200000
Terminal window
JAX_PLATFORMS=cpu python examples/train_flowers_tpu.py --smoke --out /tmp/flowers-smoke

The smoke run writes synthetic captioned records, conditions on the tiny CLIP fixture and trains for three steps. Both metrics read committed fixtures instead of downloading anything: the tiny CLIP tower, and the FID feature extractor named by --inception-weights, which is an InceptionV3 with every channel width cut to a sixteenth and randomly drawn parameters. For a real run, leave that flag unset and the script downloads the published checkpoint.

examples/sft_diffusion_gemma.py fine-tunes a DiffusionGemma checkpoint on chat data with a low-rank adapter. The script works in these steps:

  1. PretrainedBlockDecoder.load loads the base weights and their layouts.
  2. adapt(LoRA(...), key=0) adds low-rank factors to the projections that --modules names, with every base weight frozen.
  3. BlockDiffusionObjective trains the adapted model from the adapted variables, so only the factors change.
  4. Layout(host=("variables",)) keeps the train state in host memory between steps, so only the factors move to the device. For this the decoder is cloned with scan_layers=True: a host layout streams one layer per scan iteration, while a plain Python loop would have all its fetches hoisted onto the device together.
  5. source.adapter.save writes the PEFT adapter directory.
  6. The second half of the script loads the base weights again through dew.pipeline, reads the adapter onto them with LoRA.load, merges the factors into the weights and decodes a canvas.
Terminal window
python examples/sft_diffusion_gemma.py \
--model google/diffusiongemma-26B-A4B-it \
--chat data/tulu-3-sft.parquet --steps 2000
Terminal window
JAX_PLATFORMS=cpu python examples/sft_diffusion_gemma.py --smoke --out /tmp/dg-smoke

--chat is a Hub dataset id, a .jsonl file or a parquet file of conversations. The canvas preset reads it and renders each conversation with the checkpoint’s chat template. The conversations are in the messages column, or in the prompt column for rows in the verl layout. Each assistant turn becomes one row: the conversation before it, padded to --prompt-tokens, is the clean prompt, and the turn alone fills the canvases the denoiser scores, with EOS closing its last canvas. The smoke run writes three canned conversations, one of them two turns long, as JSONL and renders them with the fixture tokenizer’s own template.

examples/sft_gemma4.py trains every weight of a Gemma 4 text decoder on a Hub chat dataset. It packs conversations into windows, and LMObjective(loss_role=Role.ASSISTANT) counts the loss only on assistant targets. The trainer shards over the visible devices and accumulates micro-batches into one update. The run writes run.json next to its checkpoints, so PretrainedDecoder.from_run(run).save(directory) can write a Hugging Face directory that both transformers and Pretrained.load read. The run directory is <--out>/checkpoints/<name of --out>, which is the path the dew.eval command below reads.

Terminal window
python examples/sft_gemma4.py --model google/gemma-4-E2B \
--dataset allenai/tulu-3-sft-mixture --steps 4000 --out runs/gemma4-sft
python -m dew.eval --model dew --model_args run=runs/gemma4-sft/checkpoints/gemma4-sft \
--tasks hellaswag --limit 64

A run too large for one device loads sharded, its weights stored narrower: the mesh’s sizes and param_dtype are keys of --model_args, run=runs/gemma4-sft/checkpoints/gemma4-sft,fsdp=4,param_dtype=bfloat16, as TextGeneration.from_run takes its mesh and param_dtype (DewLM.from_run also takes a layout).

Terminal window
JAX_PLATFORMS=cpu python examples/sft_gemma4.py --smoke --out /tmp/gemma4-smoke

The script binds --dataset to a Hub Source and gives it to a Chat recipe. A local JSONL or Parquet file uses its path directly. The recipe renders each conversation with the checkpoint’s chat template; the source selects train[:N] when --rows N is set. The first --held-out conversations (512) are the validation set the final perplexity scores, and training reads only the rest. The smoke run writes canned JSONL conversations and fine-tunes the committed tiny Gemma 4 for two steps.

examples/train_rlvr.py trains Qwen2.5-0.5B-Instruct with GRPO on generated programming tasks. Each prompt asks for a Python program that reads two integers from stdin and prints a stated function of them. A completion’s reward is the fraction of three hidden test cases its program passes. The programs run in a SandboxFleet of processes with a wall clock, a CPU-time limit and a memory cap. Rollouts come from a rollout server one update ahead of the trainer, and the GRPO objective’s importance cap corrects for that one update of staleness. Post-training describes the pieces.

Terminal window
python examples/train_rlvr.py --backend native --steps 40 --out runs/rlvr-native
python examples/train_rlvr.py --backend vllm --vllm /path/to/vllm-env/bin/vllm --steps 40 --out runs/rlvr-vllm
python examples/train_rlvr.py --backend sglang --sglang /path/to/sglang-env/bin/sglang --steps 40 --out runs/rlvr-sglang
Terminal window
JAX_PLATFORMS=cpu python examples/train_rlvr.py --smoke --out /tmp/rlvr-smoke
JAX_PLATFORMS=cpu python examples/train_rlvr.py --smoke --turns 2 --out /tmp/rlvr-smoke-turns

The rollout backend is chosen with --backend:

  • native samples from Dew’s own Server in the training process and pushes weights to it in place.
  • vllm exports the checkpoint, starts a vLLM server on it with VLLM_SERVER_DEV_MODE=1, samples from it by token ids, and pushes weights by writing safetensors and asking vLLM to reload them. vLLM can live in its own environment; --vllm names its executable. --vllm-memory is vLLM’s share of the GPU, and XLA_PYTHON_CLIENT_MEM_FRACTION should leave it that much. On a 40 GB A100 the run fits at 0.12 for vLLM and 0.82 for JAX.
  • sglang does the same with an SGLang server (sglang serve) and pushes weights through its /update_weights_from_disk; --sglang names its executable. --sglang-memory is SGLang’s --mem-fraction-static, which SGLang takes as a fraction of the memory still free when it starts, after JAX has taken its share. On a 40 GB A100 the run fits at 0.75 for JAX and 0.8 for SGLang.

The run prints one line per update and writes rewards.json with each update’s reward, policy version and lag, plus, on vLLM and SGLang, the seconds each weight push took. --turns N gives each task up to N attempts through an in-process EnvironmentSource. An attempt that fails a test is told how many of the three passed and tries again, and the reward is the last attempt’s.

The smoke run trains the committed tiny Qwen2 for two updates on the native backend, and the second update trains on draws submitted one update earlier. Its eight-token budget cuts off every completion, and the example still scores and trains on those (truncation="score" on the RolloutScheduler), as a single-turn RLVR run should; an agentic run keeps the default "mask". The smoke run checks that the pieces connect. Learning needs the full run.

examples/evaluate_and_serve.py loads a run with dew.pipeline and writes one JSON report. The report holds:

  • perplexity through Evaluation.run, which is the trainer’s validation call without the optimizer or the tracker;
  • an lm-evaluation-harness suite through dew.eval.harness.DewLM;
  • CLIPScore of a diffusion run’s samples against their prompts, and FID against a directory of reference images;
  • a greedy continuation.

--openai-base-url and --ollama-host also send the same prompt to a served model, through the adapters in dew.inference.clients. Both SDKs are optional extras. If one is not installed, the report says so and the script carries on.

runs/shakespeare/checkpoints is the run, and data/shakespeare-val the held-out token store, that the commands in train_lm.py write.

Terminal window
python examples/evaluate_and_serve.py --run runs/shakespeare/checkpoints \
--tokens data/shakespeare-val --tasks hellaswag arc_easy --harness-limit 200 \
--image-run runs/flowers-tpu/checkpoints/flowers-256 \
--reference-images data/flowers-heldout
Terminal window
JAX_PLATFORMS=cpu python examples/evaluate_and_serve.py --smoke --out /tmp/eval-smoke

The smoke run first trains a byte-level model for two steps and then scores it. Given a diffusion run with --image-run instead, the smoke run scores that run’s CLIPScore and FID offline and trains no language model; outside a smoke run too, --image-run without --run scores the diffusion run alone. A smoke run has no held-out set, so the reference images are a second draw from the same run. That exercises the metric code but says nothing about the model.