Oct 25, 05:00 PM

Salesforce AI Introduces PROVE: New RAG Evaluation Framework Based on Sub-Question Coverage

Salesforce AI Research has introduced a novel evaluation framework for Retrieval-Augmented Generation (RAG) systems based on sub-question coverage. This new benchmarking paradigm, called Programmatic VLM Evaluation (PROVE), aims to enhance the evaluation of Visual Language Models (VLM) responses to open-ended queries. Additionally, the research highlights improvements in context recall by 95% with an adapted embedding model, and the importance of chunking in RAG for boosting accuracy, preserving context, and speeding up processing. Various approaches to optimizing RAG systems, such as SmartRAG and graph-based RAG, are also discussed, emphasizing the role of semantic chunking in maintaining document structure and context.

#Salesforce AI Research #Programmatic VLM Evaluation #Visual Language Models #SmartRAG

Written with ChatGPT (GPT-4o).