OpenMedLM: Prompt engineering can out-perform fine-tuning in medical question-answering with open-source large language models

In the recent publication OpenMedLM, researchers showcase a novel prompting platform that elevates the performance of open-source (OS) large language models (LLMs) within the medical field. The study detailed how various OS LLMs handled medical benchmarks, employing tactics like zero-shot, few-shot, and chain-of-thought prompting, as well as ensemble voting strategies. The results were impressive:

• Achieved 72.6% accuracy on the MedQA benchmark, surpassing previous models by 2.4%.
• Attained 81.7% accuracy on the MMLU medical-subset, marking the first OS LLM to break the 80% threshold.
• Highlighted emergent medical-specific properties in OS LLMs, previously undocumented.
• Underlined the efficiency and potential cost-effectiveness of prompt engineering over extensive model fine-tuning.

This underpins the growing importance of accessible LLMs tailored for medical applications. The practical implications could span developing countries and underfunded healthcare systems. Further, it underscores the untapped potential in prompt engineering, prompting more research to explore this area.