In ‘Large Multilingual Models Pivot Zero-Shot Multimodal Learning across Languages’, researchers propose the MPM paradigm, demonstrating that large language models preconditioned on English image-text data can adapt to other languages impressively well.

• Unveiled the MPM training paradigm for non-English multimodal model training.
• VisCPM is presented as a practice of MPM in image-to-text and text-to-image generation.
• Achieved state-of-the-art performance in Chinese without relying on native data.
• Outperformed native language models in zero-shot conditions.
• Models, codes, and weights available at OpenBMB/VisCPM.

Read More

This research is a game-changer for multilingual AI, offering a method to efficiently leverage multimodal learning in various languages. The advancement benefits languages typically underrepresented in AI, expanding the scope of AI’s applicability and inclusivity.