The Mask Is Not the Model: Auditing Prefix Invariance Catches Leaks in Zamba2 and Nemotron-H
A Korean startup team at VIDRAFT has discovered that the standard causal attention mask check in hybrid language models is nothing but a safety illusion: information leaks from the future slip through scans, normalization, and aggregation, and not one of 192 injected defects was caught by such a check.
The paper on arXiv (2608.22876) formalizes prefix invariance — the requirement that a token's representation at position t must not depend on inputs at position t + 1 and beyond. In pure transformers, causal attention masks enforce this, but modern architectures increasingly mix attention with state-space models (Mamba, Mamba-2), chunked scans, and convolutions. Each such operator has its own execution path, and the attention mask does not touch them.
The proposed audit runs in two forward passes with no training or gradients. Take two identical inputs differing only in the last token, run them independently, and compare activations layer by layer. If early-position representations change, causality is broken, and the layer where this first occurs is the culprit. The method localizes the leak to a specific layer in seconds.
In the experiment, 192 synthetic faults were injected across eight checkpoints of different architectures. Classic mask inspection detected none. The new audit found all 192 and pinpointed the exact layer. Moreover, static and dynamic analysis of chunked-scan code in transformers revealed the same defect in Zamba2 (Zyphra) and Nemotron-H (NVIDIA) — an inter-chunk axis error fixed via the reference implementation.
Why the Attention Mask Is No Longer Enough
For a long time, "causal mask = causal model" was an axiom. In decoder-only transformers, the mask truly closes all paths: attention is the only token-mixing mechanism. But hybrid models introduced parallel computational tracts. SSM recurrence, convolutional kernels, and chunked-scan operators process the sequence by their own rules. The attention mask simply isn't applied there — and future information can leak through any of these paths.
The authors demonstrate this on Nemotron-H: the chunked scan inside the attention block uses the wrong axis for accumulating state across chunks. The result — tokens from the next chunk influence representations in the current one. The attention mask remains perfectly correct because the leak doesn't go through attention.
How the Audit Works in Practice
The algorithm is cheap and portable:
- Prepare a pair of inputs: a base one and one with a modified last token.
- Run both through the model in inference mode.
- Compare hidden states layer by layer (e.g., L2 distance).
- The first layer with a non-zero difference is the prefix invariance violation point.
No fine-tuning, no gradients, works on any architecture where intermediate activations are accessible. The authors provide a reference implementation that fits on one page of code.
What This Means for Model Developers
If you train or use hybrid architectures (like Zamba, Nemotron-H, Mamba-2 with attention, or any attention + SSM + convolution mixes) — checking the attention mask gives a false sense of security. The VIDRAFT audit takes a couple of seconds and can save months of debugging strange generation artifacts that are actually leaks from the future.
The team has already integrated this check into their AX-RAY framework for continuous causality diagnostics. For everyone else — just add two forward passes to your CI pipeline before releasing a checkpoint.
Limitations and Next Steps
The method finds leaks but doesn't automatically classify their type — you need to look at the culprit layer's architecture. It also requires access to internal activations, which is unavailable in closed APIs (OpenAI, Anthropic). The authors plan to extend the approach to quantized and sparsified models, where numerical instability can masquerade as causal leaks.
Paper: The Mask Is Not the Model: Auditing Prefix Invariance in Attention, State-Space, and Hybrid Sequence Models (arXiv:2608.22876, August 24, 2026). Reference implementation and AX-RAY framework — in the VIDRAFT repository.
