Brain decoding often works by matching fMRI responses to the internal representations of an AI model, and CLIP, with its joint image-and-text space, has become a favorite target. But CLIP was never built to line up with brains, so some of what it encodes is shortcut features that don't correspond to anything the brain does, which caps the alignment.

The question here is simple: would an adversarially robust version of CLIP align better? Robust training strips out brittle shortcut features and keeps more perceptually meaningful ones, which might sit closer to real neural activity. Testing this by swapping only the target representation, on two fMRI datasets, the robust variants consistently improved image retrieval and zero-shot classification. Attribution analysis showed robust and standard models weight features quite differently, suggesting robustness reorganizes what the representation cares about.

Based on the abstract, so the paper will have the datasets and metrics spelled out.