The study presents a ‘Browse and Concentrate’ method aimed at comprehensively fusing multimodal content prior to input into LLMs, addressing modality isolation issues. This method significantly improves scene comprehension involving multiple images, with adjustments and training specifically designed to enhance multimodal integration.

• Introduced the two-phase ‘Browse and Concentrate’ paradigm.
• Developed targeted training strategies for better multi-image input processing.
• Demonstrated performance enhancements in multi-image scenarios. This innovative approach paves the way for more effective handling and comprehension of complex multimodal inputs within various AI applications, crucial for advancing AI capacities in dealing with real-world data.