Mar 29, 10:30 PM

2024: Google's AI Outperforms Humans in Fact-Checking, 20x Cheaper

Google and its DeepMind division, in collaboration with Stanford University, have announced a significant advancement in evaluating the factuality of content generated by large language models (LLMs) in 2024. The research introduces a new method called Search-Augmented Factuality Evaluator (SAFE), which leverages LLMs and Google Search to assess and verify the factuality of long-form content. This approach has demonstrated that LLMs can achieve superhuman performance in fact-checking, outperforming human annotators in both cost and reliability, being 20x cheaper than human annotators. The study also introduced LongFact, a prompt set designed to benchmark the factuality of LLMs across various domains. The findings suggest that larger models tend to be more factual and that LLMs can serve as efficient and cost-effective annotators. The research evaluated thirteen popular LLMs, including Gemini, GPT, and Claude, providing a comprehensive dataset and an evaluation metric that considers both precision and recall. This breakthrough could significantly impact the field of AI and fact-checking, offering a more scalable and accurate method for verifying information.

#Google #DeepMind #Stanford University #Google Search #LongFact #Gemini #GPT #Claude

Written with ChatGPT (GPT-4).

Sources

Additional media

Image #1 for story 2024-google-s-ai-outperforms-humans-fact-checking-20x

Image #2 for story 2024-google-s-ai-outperforms-humans-fact-checking-20x

Image #3 for story 2024-google-s-ai-outperforms-humans-fact-checking-20x