A new video-to-code benchmark found that 18 multimodal coding models scored higher at recreating a scene’s appearance than its motion. Each model had to turn a video into executable code for a 3D scene that could be rendered back into a moving clip.
The paper introducing 4DCodeBench was posted to arXiv on Oct. 2. Each model received the same task prompt and was tested once on each of 200 scenes. The set includes 100 real-world videos and 100 synthetic scenes; only the synthetic scenes have complete 3D geometry and motion for ground-truth comparisons.
GPT-6 Astra [Max], the top overall model, averaged 0.91 across the benchmark’s static appearance and 3D-geometry measures, compared with 0.67 across its 2D- and 3D-motion measures. These are normalized family scores on a 0-to-1 scale, not percentages correct. The underlying tests differ, and full 3D comparisons are possible only for synthetic scenes.
The authors’ code breakdown found that 67% of solutions across the 18 models used analytic motion: formulas or chosen poses that directly set movement over time. Custom simulations, Blender’s physics tools and keyframes were other approaches; the authors say some solutions used additional methods, and the shares are rounded.
The benchmark measures how closely a reconstruction matches a recorded clip, not whether a model understands the forces behind it. A prescribed motion path can score well, and the paper does not test whether a model adapts when forces or starting conditions change.
Sources
Illustration: JZVFX, Wikimedia Commons, CC0. The frame is illustrative and is not part of 4DCodeBench.