Apple's LensVLM-9B shrinks long documents into images, then expands only the pages that matter
LensVLM: Compressing long context as images, expanding only relevant pages
Apple released LensVLM-9B, a 9B vision language model that compresses long text into images and selectively expands only relevant pages to their uncompressed form using learned tools. Built on Qwen3.5-9B, it offers 5x, 10x, and 15x compression options. The model is available on Hugging Face under Apple's ML research license, with code on GitHub and a paper on arXiv.
LensVLM is a 9B Vision Language Model (VLM) that scans compressed images of text, then selectively expands only the relevant pages to their uncompressed form via learned tools.
- himata4113
I always found it weird that we don't have glacial type input for llms or any kind of active-working memory.
There's no reason why we shouldn't be able to expose active relevant information that is only relevant for the next request: current agents running, time, etc.
There's also no reason why we shouldn't have a cheaper lossy input which uses way less bytes per token - see deepseek flash 4.1.
- rao-v
I really like this approach! I sort of think of the vision encoder here as an expensive high fidelity RAG encoder.
The thing I’d love to do with a system like this is train it to be KV cache ordering independent (ie permutation invariant at the page level). Basically each page’s KV cache should be understandable by the model in any ordering - which would allow you to go one step further and treat the KV cache of the vision encoded page as the chunk for the model to reason over.
Then all these zoom in for more detail tricks will extend naturally.
- taylorfinley
Oh My Pi has done this for a while now, they call it Snap compact.