Researchers at Georgia Tech report that a vision model trained to predict learned features from new camera views built useful 3D scene representations for downstream tasks, including simulated robot manipulation. Their method, SNAP, deliberately limits what its decoder can do, pushing more spatial information into the encoder. The paper was posted to arXiv on Oct. 2, and its record lists acceptance to NeurIPS 2026.
Novel-view synthesis asks a model to predict how a scene would look from another camera position. That task carries clues about depth and object layout, but a powerful decoder can reconstruct an image using its own visual shortcuts, leaving the encoder with features that transfer poorly to other tasks.
SNAP’s encoder combines DINOv3 features, learned visual descriptors for image patches, across multiple views. The encoder receives no camera poses. During training, a local decoder does receive the target camera pose and predicts that view’s DINOv3 features rather than raw pixels. Its target patches cannot attend to neighboring patches, so it has less room to fill in a plausible image by itself. Feature prediction instead pressures the encoder to carry scene structure. Training uses posed images but does not require dense 3D scene labels.
On the authors’ RealEstate10K point-correspondence benchmark, SNAP scored 15.48% on PCK@5, the share of predicted matches within five pixels, against 12.85% for VGGT, a geometry-supervised model. For evaluation, a separate dense matcher, RoMa v2, supplied the reference correspondences. SNAP used 256-pixel images in this test, versus 518-pixel images for VGGT. The comparison was not consistently in SNAP’s favor: on the paper’s Hypersim relative-depth test, VGGT’s reported error was 0.208 against SNAP’s 0.709; lower is better. The paper says VGGT’s number came from its off-the-shelf depth head, while SNAP’s depth readout was trained for the evaluation, so this is not a like-for-like depth-head comparison.
On three MimicGen simulated robot-manipulation tasks, the team tested camera views shifted away from the training viewpoints. The authors report that SNAP generally beat Adapt3R, a task-specific baseline, and tracked geometry-supervised VGGT. This is evidence from simulation, not a physical-robot deployment. The paper identifies dynamic scenes and broader tests of how performance scales as open questions.
Image credit: Figure 6 from Kaashyap et al., shared under CC BY-SA 4.0.