Four common image-text alignment scores in multimodal AI stayed high even after researchers replaced the models’ visual tokens—numerical features encoding image content—with size-matched random noise. Across 13 systems, accuracy on image-question tasks nevertheless fell by 38 to 50 percentage points.
The study tested models from five families, ranging from 0.5 billion to 72 billion parameters, on MMBench, a visual question-answering benchmark. Each model sends image features and question text through shared language-model layers. Researchers replaced the image features after the vision encoder and projector had mapped them into the model’s token space. Each substitute was an isotropic Gaussian vector matched to the original token’s norm, preserving scale while removing visual content.
Four scalar measures commonly used to compare internal image and text representations could not reliably distinguish the original tokens from the noise-treated ones. The authors traced this mismatch to shared feed-forward layers. A final projection maps both image and text features back into the model’s token space, emphasizing a few common output patterns; the representations can then look similar even when the image content is gone. Their similarity scores can therefore reflect the model’s shared architecture, not only genuine image-text interaction.
To separate those effects, the paper introduces the principal-angle gap. It compares the strongest shared direction between image and text features with the next strongest: a large gap means most measured overlap runs through one direction, while a smaller gap means the overlap is spread across several. The gap was larger for noise-treated inputs than for original images in all 13 models, and tracked accuracy more consistently as the visual signal was gradually weakened.
A real but question-irrelevant image exposed the metric’s limit. The gap still indicated structured visual geometry, while median answer accuracy fell five percentage points below the text-only condition. It can describe relationships inside a model, but it cannot show by itself whether image content helped answer a question. The study covers projector-based models and one benchmark.
Sources
- Primary paper: “The Alignment Illusion in Multimodal Large Language Models”
- Image description and license: Wikimedia Commons
- Image source paper: “Gemini: A Family of Highly Capable Multimodal Models”
Image credit: Gemini Team / Google, from “Gemini: A Family of Highly Capable Multimodal Models” (2023), via Wikimedia Commons, used unmodified under CC BY 4.0. The image illustrates multimodal input and is not a figure or result from the current study.