Offline reinforcement learning has become a popular way to squeeze treatment recommendations out of historical patient data, no fresh trials required. This paper asks a blunt question: can we trust the results? The author borrows covariate balance diagnostics, a standard tool from causal inference for spotting hidden confounding and model misspecification, and turns them on existing offline RL studies.

The finding lands in an awkward but honest place. One of two things is true: either these studies carry a serious risk of bias, or the balance metrics themselves are too weak to judge them. Either way the conclusion is the same, current offline RL treatment studies cannot be called statistically robust. The paper closes by sketching what more careful applications would need.

This is a diagnostic, agenda-setting piece rather than a fix, so read the full paper for how the diagnostics are applied and what robustness would actually require.