Perplexity has released two models that let text queries find relevant PDF pages while treating the pages as images, rather than first extracting their text with OCR. The weights are available under MIT licenses, and the smaller model can search a document index built by the larger one.
That approach targets a problem in conventional dense retrieval, where a system compresses a whole page into one numeric vector before comparing it with a query. A chart, table or page layout can contain details that are hard to preserve in a single summary. Perplexity's late-interaction models instead keep a 128-dimensional vector for each input token.
Each query token is compared with document tokens using MaxSim: the model finds its strongest token-level match on a page and combines those matches into a retrieval score. Different words in a question can therefore match different parts of a rendered page. Because the 0.6B and 9B versions share an embedding space, the larger model can build the image-page index while the smaller model encodes live queries.
On the public ViDoRe v3 visual-document benchmark, the model cards report nDCG@10 ranking scores of 62.3 for the 0.6B model and 65.2 for the 9B model. Perplexity also reports 63.5 when the 9B model indexes pages and the 0.6B model handles queries, compared with 62.3 when the smaller model does both. The ranking metric rewards relevant pages that appear higher among the first ten results. Perplexity says the asymmetric setup shifts extra computation to indexing while leaving the query encoder unchanged; these are company-run benchmark results, not a live-customer test.
Keeping vectors for each token increases index storage and candidate-scoring work compared with a single-vector system, and the cost grows with document length. The model cards also require text and image inputs to be encoded in separate batches; mixed text-and-image batches are not supported.
Sources
- Perplexity Research: "Multimodal embeddings beyond a single vector"
- Perplexity: pplx-embed-v2-late-0.6b model card
- Perplexity: pplx-embed-v2-late-9b model card
- Image: Nikola Tesla's 1914 patent diagram, public-domain scan. Illustrative technical-document page; not a Perplexity model image or benchmark result.
