Researchers at Mohamed bin Zayed University of Artificial Intelligence have released Flash-dLLM, a way to make diffusion language models generate text faster by moving less data through GPU memory and checking several proposed tokens together. In the team's tests, the method ran markedly faster than an earlier cache-based approach on math and coding tasks, though the fastest setting did not always produce the best answers.
Unlike a conventional language model that writes one token after another, a masked diffusion model fills in and revises missing tokens over several passes. That can expose work the GPU could do in parallel, but revisiting the sequence also means repeatedly reading and updating stored attention data. The researchers found that moving this data, known as the key-value cache, could consume much of the time saved by caching calculations.
Less cache traffic, more tokens accepted per pass
Flash-dLLM's first change combines several cache-update operations into one GPU kernel, so intermediate results need not be written to and read back from GPU memory. It also refreshes a limited set of previously decoded tokens that the model is attending to most closely, while serving other positions from the cache.
The second change lets the same diffusion model draft and verify candidate tokens without a separate verifier model. It looks at each candidate both as the proposed token and as a still-masked position; matching predictions that clear a confidence threshold can be accepted together. This reduces the number of revision passes while checking whether the parallel guesses agree with the model's masked view.
The team has released the implementation under an Apache 2.0 license.
Math and code benchmarks show faster decoding, with accuracy tradeoffs
On one NVIDIA A100 80GB GPU, using the same LLaDA-1.5 model and 512-token generation length, the researchers measured 210.6 generated tokens per second on five-shot GSM8K math questions with Flash-dLLM's cache and verification combined. Their rerun of Elastic-Cache reached 41.7 tokens per second under the same setup. The reported GSM8K accuracy was 83.02% for Flash-dLLM and 82.79% for Elastic-Cache.
On zero-shot HumanEval coding tasks at the same generation length, throughput was 185.6 versus 16.8 tokens per second; pass@1 was 40.24% versus 37.80%. These are the paper's controlled comparisons, not independent deployment results. Speed did not uniformly improve answer quality: at a 256-token HumanEval limit, the fastest Flash-dLLM configuration scored 39.63% pass@1, below the 43.29% of the most accurate configuration tested.
The results detailed here cover LLaDA-1.5 on structured math and code tasks. The authors have not tested this approach on continuous-space diffusion language models or open-ended dialogue and long-form writing, where confidence patterns may differ.
Sources
- Flash-dLLM research paper, arXiv:2609.26796
- Full paper, methods, results and limitations
- Official Flash-dLLM code repository
- Illustrative data-center image: Alexsandro Teixeira Ribeiro / Wikimedia Commons, CC BY-SA 4.0. It does not show the study's test system.
