Nov 29, 05:01 PM

Study in Nature Shows LLMs Achieve 86% Accuracy in Neuroscience Predictions Using BrainBench, Outperforming Experts by 20%

A recent study published in 'Nature Human Behavior' reveals that large language models (LLMs) significantly outperform human experts in predicting neuroscience experimental outcomes. The research indicates that LLMs achieved an accuracy rate of 86%, compared to 63% for human experts, marking a 20 percentage point advantage. The study utilized a fine-tuned Mistral 7B model, although other models also demonstrated effective performance. Researchers employed BrainBench, a tool for comparing real and fabricated study abstracts, to validate their findings. This advancement suggests that LLMs can synthesize vast amounts of scientific literature to enhance research capabilities in neuroscience.

#Nature Human Behavior #Mistral #BrainBench

Written with ChatGPT (GPT-4o mini).

Sources

John Nosta@JohnNosta
2 years ago
🧐 Do you think that LLMs might be the ULTIMATE COGNITIVE ENHANCER? #AI #LLMs #cognition #brain https://t.co/E6HoGAM0OB
Neuroscience News@NeuroscienceNew
2 years ago
AI Outperforms Experts in Predicting Study Outcomes Large language models (LLMs) can predict neuroscience study outcomes more accurately than human experts, according to a new study. Using BrainBench, a tool comparing real and fabricated study abstracts, researchers found that… https://t.co/dETsXFzveO
Ethan Mollick@emollick
2 years ago
New study shows LLMs outperform neuroscience experts at predicting experimental results in advance of experiments (86% vs 63% accuracy). They use a fine-tuned Mistral 7B but other models worked too. Suggests LLMs can integrate scientific knowledge at scale to support research https://t.co/mqKsYYWMvm

Additional media

Image #1 for story study-nature-shows-llms-achieve-86-accuracy-neuroscience-predictions-using-20-b78a1594

Image #2 for story study-nature-shows-llms-achieve-86-accuracy-neuroscience-predictions-using-20-b78a1594

Image #3 for story study-nature-shows-llms-achieve-86-accuracy-neuroscience-predictions-using-20-b78a1594

Image #4 for story study-nature-shows-llms-achieve-86-accuracy-neuroscience-predictions-using-20-b78a1594

Image #5 for story study-nature-shows-llms-achieve-86-accuracy-neuroscience-predictions-using-20-b78a1594

Image #6 for story study-nature-shows-llms-achieve-86-accuracy-neuroscience-predictions-using-20-b78a1594