In AutoAD III: The Prequel – Back to the Pixels, the team focuses on enhancing Audio Description (AD) generation for movies through the creation of specialized training datasets and a cutting-edge model architecture. Key contributions include:

• Development of two innovative methods for building AD datasets aligned with video data.
• Introduction of a Q-former-based architecture that integrates large language models with pre-trained visual encoders for generating AD directly from raw video.
• Proposal of new evaluation metrics tailored to better match human assessment in AD quality.

This research is significant because:

• It addresses the critical gap in the availability of high-quality AD resources and models, fostering inclusivity in media consumption.
• The methodologies introduced can dramatically improve the quality and realism of automated AD, which could be extended to other multimedia applications, making it a cornerstone for future multimedia AI systems.