Unsolvable Problem Detection: Evaluating Trustworthiness of Vision Language Models shines light on a new challenge for VLMs called Unsolvable Problem Detection (UPD) in VQA tasks. It identifies the models’ struggle to withhold answers to unsolvable problems and offers insights for improving reliability (Atsuyuki Miyai et al.).

• Delves into three settings within UPD, including Absent Answer Detection and Incompatible Answer Set Detection.
• Reveals that leading VLMs like GPT-4V and LLaVA-Next-34B have difficulty with UPD benchmarks, indicating a need for improvements.
• Explores both training-free and training-based solutions to tackle UPD.

By spotlighting the UPD challenge, this paper emphasizes the need for VLMs not only to provide correct solutions but also to recognize their limitations. This insight is crucial in developing more sophisticated AI that is capable of discerning when to abstain from incorrect problem-solving, thereby improving trust in AI systems.