Global Tech News For people figuring out what to build with AI.
AI

A Video Generator Skips Refinement for Tokens Predicted to Be Near Their Final Values

Diagram shows a video generator predicting which tokens are near their final values and refining the rest.

Researchers at KAIST and NAVER AI Lab report a way to speed up video generation by stopping repeated refinement for small pieces predicted to be near their final values, while continuing to refine pieces that are still changing.

A verifier predicts which tokens can skip later refinement

Autoregressive video models produce a stream in chunks, using earlier frames as context. In each chunk, the model represents the video as small space-and-time units called tokens and repeatedly refines their estimates. The tested generators refine every token in four passes.

The proposed method, Speculative Forcing, treats each intermediate estimate as a draft. A verifier predicts whether a token is already close to the value it would reach after all four passes. If so, the method fixes that token and skips its later refinement updates; the accepted token’s output remains available as context while the model refines the others. The underlying video generators stay frozen.

The researchers tested two verifiers. At the first pass, the training-free verifier looks for a close match between a draft and content in earlier generated chunks or reference frames. The paper treats such a match as a sign that a token is repeating familiar visual content and may need little more refinement. On later passes, it accepts an active token when its estimate changes little from the previous pass. Both tests use thresholds calibrated on held-out data; tokens that fail continue through the four-pass schedule. A small learned verifier instead uses the generator's internal features to estimate how far a draft remains from its four-pass result; the generator stays frozen while that verifier is trained.

Fewer passes increased throughput while benchmark scores stayed similar

On five-second videos generated by LongLive, an autoregressive video generator, the learned verifier averaged 1.68 refinement passes per token, down from four, while increasing measured throughput from 26.40 to 41.74 frames per second on one NVIDIA H100. The standard VBench total score was 83.08 for four-pass sampling and 83.21 with the verifier. The evaluation used VBench prompts and five random seeds per prompt.

On 30-second LongLive videos, the learned verifier averaged 1.72 passes per token and raised throughput from 24.5 to 37.8 frames per second. VBench-Long total scores were 82.69 and 82.81, respectively.

The authors also tested the Causal-Fast video model in Lingbot-World 2.0, a world model, on 200 held-out navigation scenes. The Causal-Fast video model generated 253 frames per scene as a virtual camera moved through each scene. Because each camera move can reveal new content, this test needed more refinement: the learned verifier averaged 2.55 passes per token. Throughput rose from 18.1 to 24.5 frames per second, while the six-dimension VBench quality aggregate was 82.62 for four-pass sampling and 82.65 with the verifier.

Speed depends on the generator and task; code remains under review

The authors say acceleration depends on how many tokens can safely stop early and on how much new content a task requires. Throughput was measured on one H100 using the video transformer’s forward time; it excludes video encoding and decoding.

The authors’ project page shows generated examples, while their GitHub repository says the code is under internal review and will be released later. The authors have not yet released code for readers to reproduce the reported results.

Sources

Image: Figure 1 from “Speculative Forcing for Efficient Autoregressive Video Generation” by Hyunwook Choi, Seungryong Kim, Dongyoon Han and Sangdoo Yun, via arXiv; CC BY 4.0. Used unmodified as an explanatory diagram, not as a separate test result.