Read more about the study here. The work introduces a method to harvest massive amounts of instruction data naturally existing on the web, significantly altering LLM fine-tuning practices. Employing a series of processes such as document recall, pair extraction, and refinement, the MAmmoTH2 project hails as a leap towards more efficient LLM enhancement.

• Efficient harvesting of 10 million instruction-response pairs.
• Reduces the need for human intervention and costly data sourcing.
• Utilizes open-source LLMs for refining data, increasing performance on reasoning tasks like MATH and GSM8K.

This initiative not only showcases a shift away from traditional LLM fine-tuning techniques but also offers a substantial benchmark in improving reasoning and chatbot performances through harvested instruction data. The MAmmoTH2-Plus version, in particular, sets new standards in LLM efficacy.