This article explores how advanced AI models can generate detailed image descriptions and reasoning without actual image input, a phenomenon called mirage reasoning. It highlights vulnerabilities in these models, particularly in medical contexts, and introduces B-Clean, a method for better evaluating multimodal AI systems by minimizing non-visual inference.

• A model topped a chest X-ray QA benchmark while being completely blind to the actual images, exposing "mirage reasoning" where models fabricate plausible visual descriptions from text priors alone.
• Explicitly prompting models to guess without assuming an image was present caused performance to drop sharply, showing benchmark scores were inflated by non-visual inference rather than real image understanding.
• The proposed B-Clean method strips out textual cues that let models cheat their way to correct answers, aiming to force genuine visual grounding in evaluations—critical for high-stakes fields like medical AI.