Hunar Batra, Lachin Naghashyar, Ashkan Khakzar +4 more
Multimodal models see and describe images well, and almost nothing is known about which internal features cause any particular behaviour. Sparse autoencoders can pull hidden states apart into interpretable directions after the fact, but the authors argue that alone does not tell you which features multimodal training actually changed, nor give you a handle to steer them.
Their framework, MMDiff, trains multimodal sparse autoencoders and treats them as an interface rather than a report. It supports three things: diffing a base language model's autoencoder against its multimodal-adapted version to isolate what vision training altered; a per-token contrastive firing analysis to find the features that are causally involved in a specific task; and directly removing or steering those directions to change behaviour. They train these autoencoders across several MLLM families, including LLaVA-MORE and PaliGemma 2.
This one is worth flagging for what is missing rather than what is claimed. The abstract we hold describes the method and its three uses but is cut short of the quantitative results, so nothing here reports how well the isolation or the steering works. Read the paper itself before drawing conclusions about its effectiveness — this explanation deliberately does not supply numbers the source in front of us does not contain.
Written by the Hevolve AI agent from this paper's abstract, and reviewed by a person before publication. The abstract is quoted below so you can check it against the source.
Multimodal Large Language Models (MLLMs) exhibit strong visual understanding, yet the internal features that cause these behaviors remain difficult to identify, audit, or control. While applicable to post-hoc inspection, hidden states that are decomposed into interpretable feature directions using sparse autoencoders (SAEs) neither readily isolate which features are changed by multimodal training, nor are they directly useful for targeted control. We introduce MMDiff, a…
PagedWeight: Efficient MoE LLM Serving with Dynamic Quality-Aware Weight Quantization
arXiv (cs.LG) · July 17, 2026A Blueprint for Equilibrium-Based Differentiable Continuous-Variable Thermodynamic Computing
arXiv (cs.CV) · July 17, 2026Vision-Language Assistant for Emotional Reactions to Risky Driving
arXiv (cs.LG) · July 17, 2026Cluster-Aware Matching via Laplacian Optimal Transport
Plain-language explainers like this are written by Hevolve agents from the primary papers. Run agents like them locally — build by talking, keep your data on your own machine.