Jul 10, 03:13 PM

Advancements in LLMs Focus on Reducing Hallucinations by 10%

Recent advancements in the field of Large Language Models (LLMs) have focused on addressing the issue of hallucinations, where models generate information not grounded in their training data. Lilian Weng has released a technical blog on 'Extrinsic Hallucinations in LLMs,' which discusses sources, detection, and mitigation strategies for these hallucinations. The Lookback Lens method, developed by Yung-Sung Chuang and colleagues, uses attention maps to detect and mitigate contextual hallucinations. Additionally, NVIDIA's Weight-Decomposed Low-Rank Adaptation (DoRA) has shown to outperform existing methods in fine-tuning LLMs and Vision Language Models (VLMs). Another notable technique is DoLa, which focuses on the last layers of a model to prioritize factually correct tokens. These advancements highlight the ongoing efforts to improve the reliability and accuracy of LLM outputs. A discussion on these topics is scheduled for 8 PM IST/7:30 AM PST. The new method has shown a 10% reduction in contextual hallucinations in the XSum summarization task.

#Large Language Models #Lilian Weng #Extrinsic Hallucinations in LLMs #Lookback Lens #NVIDIA #Vision Language Models #DoLa #XSum

Written with ChatGPT (GPT-4o).

Sources

Additional media

Image #1 for story advancements-llms-focus-on-reducing-hallucinations-10

Image #2 for story advancements-llms-focus-on-reducing-hallucinations-10

Image #3 for story advancements-llms-focus-on-reducing-hallucinations-10

Image #4 for story advancements-llms-focus-on-reducing-hallucinations-10

Image #5 for story advancements-llms-focus-on-reducing-hallucinations-10

Image #6 for story advancements-llms-focus-on-reducing-hallucinations-10

Image #7 for story advancements-llms-focus-on-reducing-hallucinations-10

Image #8 for story advancements-llms-focus-on-reducing-hallucinations-10

Image #9 for story advancements-llms-focus-on-reducing-hallucinations-10