In the paper Wisdom of the Silicon Crowd: LLM Ensemble Prediction Capabilities Match Human Crowd Accuracy, researchers experiment with the concept of ‘wisdom of the crowd’ by aggregating the predictions from a crowd of twelve LLMs, comparing them with those from a human tournament. Interestingly, they found that LLMs can replicate the ‘wisdom of the crowd’ to rival the accuracy of human predictions by a significant margin.

• The ensemble approach uses twelve LLMs to forecast future events.
• LLM predictions are compared with those from 925 human forecasters.
• The aggregated LLM crowd statistically matches the human crowd accuracy.
• Further research includes optimizing the ensemble mix and exploring the effects of augmenting LLM predictions with human inputs.

The paper provides promising evidence for the use of LLMs in decision-making and forecasting, suggesting that with proper aggregation methods, AI could supplement or even replace traditional human crowds in forecasting tasks.