When to Retrieve: Teaching LLMs to Utilize Information Retrieval Effectively

This paper discusses a new training methodology for Large Language Models (LLMs) that helps them decide when to utilize an external information retrieval (IR) system, and when to rely on their internal parametric memory. The process involves training LLMs to signal their uncertainty by generating a special token, indicating the need for external information. This approach has been tested on the PopQA dataset, showing enhanced performance under different configurations (always retrieving, always using memory, or using a popularity threshold).
Key Insights:
- Training LLMs to detect their knowledge limits improves response accuracy.
- Adapt-LLM model exhibits high accuracy levels using internal memory alone.
- Provides a scalable method to optimize LLM responses based on question popularity.
This improved training approach presents a significant step forward in optimizing LLM usage for effective and accurate information retrieval, based on the contextual requirement of questions.
Personalized AI news from scientific papers.