In medical AI that reads both images and text, getting the answer right isn't the whole job; a clinician also has to trust the reasoning behind it. This paper steps back from the leaderboard and looks at nine systems from a gastrointestinal endoscopy challenge to see which design choices actually produce reliable behavior, not just high scores.

The honest finding is that a good answer and good reasoning don't always travel together. Lightweight fine-tuning of pretrained models scores well on the challenge, but those wins don't reliably turn into faithful, complete clinical explanations. Systems that force structured reasoning and point explicitly at their evidence behave more dependably across question types, though the authors stress this is a correlation they observed, not something isolated with controlled ablations. They argue for evaluation that moves past word-overlap scores toward evidence-linked explanations, leakage-aware data handling, and basic robustness and calibration checks.

This is a retrospective analysis summarized from the abstract, so the paper carries the per-system detail and caveats.