This research assesses the capability of Large Language Models (LLMs), such as Meditron, Llama2, and Mistral, to support medical personnel. The study particularly focuses on LLMs providing interactions which may help to correct or validate physicians’ decisions.

• Experiments on PubMedQA questions demonstrate the influence of prompt engineering on LLM accuracy.
• Highlights the beneficial role of LLM feedback in medical settings, providing corrective guidance.
• Suggests further need for research in refining LLM suggestions for practical and reliable medical aid. The implications of LLMs in healthcare could be substantial, contributing to improved accuracy in medical decision-making. As LLMs evolve, their integration in clinical environments promises a collaborative landscape where AI complements human expertise, enhancing outcomes.