Eye-in-hand imitation learning systems often grapple with compounding execution errors that arise from states unseen during expert demonstrations. Diffusion Meets DAgger (DMD) is a novel approach that leverages the power of diffusion models to create samples that cover these outliers, thereby eschewing the need for exhaustive new sample collection. The recent research on Franka Research 3’s non-prehensile pushing tasks demonstrates that DMD significantly outperforms naive behavior cloning and even bests NeRF-based augmentation with surprising results: an 80% success rate with just 8 expert demonstrations.

Main insights:

• DMD incorporates diffusion model advancements to fill gaps in expert demonstrations.
• Achieves high success rates with minimal expert input.
• Surpasses traditional augmentation techniques.

This research emphasizes the importance of creatively integrating diffusion models into imitation learning, potentially reducing resource overhead while boosting performance.