What happens when you harness the collective forecasting capability of LLMs? Philipp Schoenegger and colleagues answer this question in their paper titled ‘Wisdom of the Silicon Crowd: LLM Ensemble Prediction Capabilities Match Human Crowd Accuracy’. The study explored the ensemble approach using twelve LLMs to predict future events.

• Findings show that LLM ensembles can match the forecasting accuracy of a large crowd of human forecasters.
• Their aggregated predictions on binary questions rival those from human participants in a three-month forecasting tournament.
• The study also found an ‘acquiescence effect,’ where predictions leaned towards a positive outcome.
• Introducing human cognitive outputs to LLMs further improved their forecasting accuracy.

This research is a testament to the potential of LLMs in becoming substantial tools for societal applications, mirroring human collective wisdom. The findings represent a significant step towards utilizing AI for complex forecasting tasks and suggest that collaboration between human strategies and machine intelligence can further refine prediction models.