The study introduces an innovative technique for distilling Large Language Models (LLMs) into smaller models tailored for specific applications. Leveraging LLMs to generate labels and rationales for unlabeled data, the process simplifies training student models, emphasizing cost-saving and minimal human intervention.

• Utilizes zero-shot prompting to draw rationales from teacher models.
• Reduces dependency on handcrafted examples, lowering token count requirements.
• Emphasizes multi-task training where student models mimic teacher predictions and rationales.
• Enables significant cost-savings without sacrificing performance.

This research points towards a cost-effective and streamlined future for developing tailored AI systems, crucial for widespread AI adoption and functionality in resource-constrained environments. Delve into the efficient distillation process