In a recent paper titled ‘How well do LLMs cite relevant medical references? An evaluation framework and analyses’, researchers propose a framework for assessing the relevance of sources cited by AI in supporting medical claims. Here’s the essence of their findings:

• Using GPT-4 for source validation showed an 88% concordance rate with medical doctors’ assessments.
• An automated pipeline, SourceCheckup, reviewed over 40K pairs of statements and sources generated by various LLMs, revealing that a significant portion (around 50% to 90%) of responses were not fully supported.
• Their dataset of medical questions and expert annotations has been open-sourced for ongoing evaluations.

Key Highlights:

• Exploration of LLMs’ ability to accurately provide medical citations.
• Introduction of SourceCheckup for automated evaluation.
• An open-source dataset for future research.

Given the potential risks associated with incorrect medical information, this study is crucial for improving regulatory frameworks and trust in AI-assisted medical decision-making.