The paper introduces EvoKD, a method that employs active learning in the knowledge distillation process to improve small domain models’ performance using LLMs. It identifies the student model’s weaknesses and iteratively synthesizes new samples, resulting in better adaptations for various NLP tasks. To delve deeper, visit the publication.

• Develops an iterative model-improvement mechanism integrating active learning.
• Utilizes LLMs to generate challenging samples for the student model’s weaknesses.
• Demonstrates effectiveness across text classification and named entity recognition tasks.

This method showcases the potential of active learning in making smaller models more robust and efficient, thereby presenting a significant step forward in the realm of knowledge distillation methodologies.