The paper Probing Multimodal LLMs as World Models for Driving provides critical insights into the application of LLMs to autonomous driving. The study examines how well these models handle dynamic scenarios through visual inputs and identifies significant limitations.

Results Summary:

• Models perform well in interpreting single images but fail in creating coherent narratives over time.
• Demonstrates considerable inaccuracies in modeling vehicle dynamics and interactions with other road agents.
• Introduces DriveSim, a simulator for creating varied driving scenarios, and a corresponding dataset.

This research highlights the need for improved foundation models to ensure they are truly effective in real-world applications like autonomous driving. By addressing these gaps, future models may become more reliable in dynamic and complex environments.