dew.nn.derivatives
Declared derivatives: a function whose derivative is not its value’s own.
A surrogate (surrogate) evaluates one function and differentiates as a declared rule: a straight-through
rounding, a quantized product differentiated as its float one, an estimate of a gradient no autodiff of
the value gives (node perturbation), or a gradient a learning rule supplies (Objective.with_gradients).
Each is a derivative of Dew’s choosing, not of the value’s, so it is one declaration here, which the seam
registry reads as a surrogate (tests/test_derivative_seams.py) and no rewrite of the program’s algebra
replaces with the value’s own derivative.
A marked span (marked) is a computation carried as one call of its kind (Kind), which every transform
keeps a call of that kind, its tangent and cotangent too (docs/design/algebra.md §6.10). The algebra pass
reads its kind (dew.algebra.program): a protected span (Protected, an error-free transform’s) is kept as
it is, a surrogate’s bodies (Surrogate) and a kernel derivative’s glue (KernelGlue) are rewritten within
their call, and a derivative written to restructure algebra (Algebra) opens into the region around it.
A custom derivative is declared with its kind, which makes it one marked call of that kind:
declared_derivative (a reverse-mode rule, as jax.custom_vjp takes one) and declared_tangent (a tangent
rule, as jax.custom_jvp takes one), each of any kind, and surrogate (a value and its declared derivative).
Each lane moves its sites onto them (rows 4026 to 4030); then the slop check refuses jax.custom_vjp and
jax.custom_jvp outside this module.
| Name | Summary |
|---|---|
Algebra | A derivative written only to restructure algebra, which the pass rewrites freely and the engine retires once it derives the form itself. |
KernelGlue | A kernel derivative’s XLA around its launches: rewritten as any other math, its launches opaque. |
Kind | What a marked span is to the algebra pass, read as an object, never a name. |
Protected | A span the algebra pass keeps as it is: an error-free transform’s (dew.nn.precision), whose residual a reassociation would cancel. |
Surrogate | A surrogate’s bodies (surrogate): rewritten within, never related to autodiff’s derivative of its forward. |
TangentKind | The kinds a tangent rule is declared as (declared_tangent): a surrogate’s forward mode is surrogate’s alone, the one route whose rule reads the surrogate’s own value. |
bound_span | function as one marked call of kind (marked) over its arrays, passed by position or by name: its other arguments (a static factor, a dtype, a Python number, an option) and those at its static positions (a custom derivative’s nondiff_argnums, arrays among them, which its rule takes as constants) bound in, so a constant still splits where it is known; a call of no array, function itself. |
contracted | A custom derivative’s contract, declared where the derivative is made: test, the name of the one test function of tests/test_contracts_*.py that holds every order of it finite against its float64 truth and an independent plain formula. |
declared_derivative | primal with a declared reverse-mode rule (forward saving its residuals, backward taking them and the cotangent, as jax.custom_vjp.defvjp takes them, with symbolic_zeros as it takes it), one marked call of kind (bound_span): a kernel’s glue (KernelGlue), a rule its numerics need as written (Protected), a derivative of Dew’s choosing (Surrogate) or one written to restructure algebra (Algebra). |
declared_tangent | primal with a declared tangent rule (tangent(primals, tangents) returning the primal’s value and its tangent, as jax.custom_jvp.defjvp takes it, which reverse mode transposes; with symbolic_zeros, an operand no tangent reaches is a SymbolicZero there, as defjvp gives it), one marked call of kind (bound_span), as declared_derivative takes it; a surrogate’s tangent is surrogate’s, whose rule reads the surrogate’s own value, so a second order meets the rule again (TangentKind). |
marked | function as one marked call of kind (marked_call_p) over its arguments, a tree of arrays: it computes what function does, and every transform of it keeps a marked call of kind. |
marked_call_p | A marked span: a call of its body’s jaxpr (call_jaxpr) under its kind, whose JVP, linearization, transpose, batching, partial evaluation and checkpointing are JAX’s call primitive’s, each a marked call of the same kind; lowered by inlining its body, so it costs nothing compiled, inside a Pallas kernel’s body too (Dew’s JAX build lowers a call primitive there as its body). |
surrogate | The function whose value is forward(*operands), exactly, and whose derivative is the declared rule derivative(primals, tangents, value): the output’s tangent from the differentiable operands, their tangents and the value, linear in the tangents, so reverse mode transposes it and a higher order differentiates it. |
Algebra
Section titled “Algebra”class Algebra()A derivative written only to restructure algebra, which the pass rewrites freely and the engine retires once it derives the form itself.
KernelGlue
Section titled “KernelGlue”class KernelGlue()A kernel derivative’s XLA around its launches: rewritten as any other math, its launches opaque.
What a marked span is to the algebra pass, read as an object, never a name.
Protected
Section titled “Protected”class Protected()A span the algebra pass keeps as it is: an error-free transform’s (dew.nn.precision), whose
residual a reassociation would cancel.
Surrogate
Section titled “Surrogate”class Surrogate()A surrogate’s bodies (surrogate): rewritten within, never related to autodiff’s derivative of its
forward.
TangentKind
Section titled “TangentKind”The kinds a tangent rule is declared as (declared_tangent): a surrogate’s forward mode is surrogate’s
alone, the one route whose rule reads the surrogate’s own value.
bound_span
Section titled “bound_span”def bound_span( kind: Kind, function: Callable[..., Result], *, static: Sequence[int] = (),) -> Callable[..., Result]function as one marked call of kind (marked) over its arrays, passed by position or by name: its
other arguments (a static factor, a dtype, a Python number, an option) and those at its static positions
(a custom derivative’s nondiff_argnums, arrays among them, which its rule takes as constants) bound in,
so a constant still splits where it is known; a call of no array, function itself.
contracted
Section titled “contracted”def contracted(test: str, /) -> Callable[[Function], Function]A custom derivative’s contract, declared where the derivative is made: test, the name of the one
test function of tests/test_contracts_*.py that holds every order of it finite against its float64
truth and an independent plain formula. The function it decorates or wraps is returned unchanged;
tools/lint_slop.py (SLOP028) resolves the name and refuses a custom derivative without one. Its kind
is its maker’s (Kind), declared there alone.
declared_derivative
Section titled “declared_derivative”def declared_derivative( kind: Kind, primal: Callable[..., Value], forward: Callable, backward: Callable, *, nondiff_argnums: Sequence[int] = (), symbolic_zeros: bool = False,) -> Callable[..., Value]primal with a declared reverse-mode rule (forward saving its residuals, backward taking them and
the cotangent, as jax.custom_vjp.defvjp takes them, with symbolic_zeros as it takes it), one marked
call of kind (bound_span): a kernel’s glue (KernelGlue), a rule its numerics need as written
(Protected), a derivative of Dew’s choosing (Surrogate) or one written to restructure algebra
(Algebra). Forward mode refuses it as a bare jax.custom_vjp does.
declared_tangent
Section titled “declared_tangent”def declared_tangent( kind: TangentKind, primal: Callable[..., Value], tangent: Callable, *, nondiff_argnums: Sequence[int] = (), symbolic_zeros: bool = False,) -> Callable[..., Value]primal with a declared tangent rule (tangent(primals, tangents) returning the primal’s value and
its tangent, as jax.custom_jvp.defjvp takes it, which reverse mode transposes; with symbolic_zeros,
an operand no tangent reaches is a SymbolicZero there, as defjvp gives it), one marked call of
kind (bound_span), as declared_derivative takes it; a surrogate’s tangent is surrogate’s, whose
rule reads the surrogate’s own value, so a second order meets the rule again (TangentKind).
marked
Section titled “marked”def marked(kind: Kind, function: Callable[..., Result]) -> Callable[..., Result]function as one marked call of kind (marked_call_p) over its arguments, a tree of arrays: it
computes what function does, and every transform of it keeps a marked call of kind. An eager call
(_evaluating), which no program records, is function’s own: traced, every such call traced a program
a step’s steady state paid each step (a decision task’s eager scores, train 586).
marked_call_p
Section titled “marked_call_p”marked_call_p = create_call_primitive('marked_call')A marked span: a call of its body’s jaxpr (call_jaxpr) under its kind, whose JVP, linearization,
transpose, batching, partial evaluation and checkpointing are JAX’s call primitive’s, each a marked call of
the same kind; lowered by inlining its body, so it costs nothing compiled, inside a Pallas kernel’s body too
(Dew’s JAX build lowers a call primitive there as its body).
surrogate
Section titled “surrogate”def surrogate( forward: Callable[..., Value], derivative: Callable[..., Value], *, nondiff_argnums: Sequence[int] = (),) -> Callable[..., Value]The function whose value is forward(*operands), exactly, and whose derivative is the declared rule
derivative(primals, tangents, value): the output’s tangent from the differentiable operands, their
tangents and the value, linear in the tangents, so reverse mode transposes it and a higher order
differentiates it. The arguments at nondiff_argnums (static, as a dtype) take no tangent; each reaches
both forward and, ahead of the primals in the order of their positions, derivative, as JAX’s
custom_jvp passes them whatever order nondiff_argnums names them in.
The value a derivative reads is the surrogate’s own, so a derivative of a derivative meets the declared rule again wherever it differentiates the value (a straight-through rounding’s tangent is never rounded at a second order).