Scale Introduces New Private SEAL Leaderboard to Improve LLM Evaluations

Scale has introduced a new public leaderboard aimed at improving the evaluation of large language models (LLMs). This leaderboard addresses key issues in current evaluation methods, such as contamination of evaluation sets during model training and the quality of human raters. The leaderboard requires that models can only be featured the first time an organization encounters the prompts, ensuring integrity. This initiative, which includes a private SEAL leaderboard, is seen as a significant step in enhancing the quality of evaluations and benchmarks in the AI industry.