Sources
Maxime Labonne
I reviewed the dark field of LLM-as-a-judge so you don't have to. Here are the key findings. Model Performance Variability LLMs show inconsistent performance across datasets and tasks. No single model dominates all scenarios. GPT-4 generally leads, with open-source models like… https://t.co/qaRNIgGxsn
Alexander DoriaAmazing synthesis of the new potential of LLM-as-a judge/synthetic data to enchance information retrieval systems. And the best methods to get there, especially alignment/calibration on your real life queries. https://t.co/ysiAQ4qFtd
Stuart Reid We recently benchmarked a whole bunch of small LLMs on a variety of small tasks. And, honestly, they sucked. Is there a trick to getting these things to work? The larger LLMs did exceptionally well. And the prompts were not very long.
Additional media
