Sarthak Jain, Qiran Hu, Zhen Zhu +1 more
Models like CLIP put images and text into one shared space so you can search across them. The trouble comes when you keep adapting such a model to new data over time: each new phase can quietly damage the alignment learned earlier. Standard continual-learning methods hand you a single final checkpoint, which forces every retrieval direction to accept the same compromise between staying stable and staying adaptable.
AlphaWiSE takes a different route. After the fact, it blends two frozen checkpoints in weight space, fitting one scalar mixing coefficient per parameter tensor on a small memory of examples, then bakes out a single interpolated model. Because the result has the same size as either source, there is no extra inference cost. On audio-image-text retrieval it reports steady gains over strong baselines.
This is based on the abstract, so read the paper for the coefficient-fitting details and the full results.
Multimodal models such as CLIP learn a shared embedding space for cross-modal retrieval, but continual adaptation to sequentially arriving data can disrupt the cross-modal alignment acquired from earlier phases. Conventional continual-learning methods return a single checkpoint, which commits every retrieval direction to the same stability-plasticity trade-off. We propose AlphaWiSE, a post-hoc weight-space interpolation method that composes two frozen source checkpoints. For…
Candidate Attended Dialogue State Tracking Using BERT
arXiv (cs.LG) · July 17, 2026Presentation, Not Mechanism: A Render Confound in Deprecation-Aware Memory Evaluation
arXiv (cs.CV) · July 17, 2026PIXIE: A Zero-Shot texture-invariant 6D pose estimation framework for unseen objects with assembly defects
arXiv (cs.AI) · July 17, 2026DPNeXt: A Lightweight Multi-Scale Feature Fusion Framework for Efficient ViT-Based Multi-Task Dense Prediction