Summary of Enhancing Code Generation Performance of Smaller Models by Distilling the Reasoning Ability of LLMs

• Large Language Models (LLMs), through ‘Chain-of-Thought’ prompting, have significantly advanced code generation by creating complex ‘solution plans’.
• Smaller models struggle to emulate this reasoning capability, impacting their code generation skills adversely.
• The CodePLAN framework seeks to distill LLMs’ reasoning prowess into smaller models using a multi-task learning approach focusing on both code and solution plan generation.
• Employing backward reasoning and plan sampling strategies, the framework aims for high-quality solution plans, and initial tests indicate a 130% performance increase compared to traditional fine-tuning.

This paper underscores the importance of enabling smaller models to harness the reasoning abilities of their larger counterparts. Such knowledge transfer could democratize access to powerful AI capabilities, making them available in more cost-effective and resource-constrained environments. Further research can investigate how well these distilled abilities generalize to diverse programming languages and paradigms.