Recent discussions among AI researchers highlight the ease of gaming benchmarks for large language models (LLMs) such as ChatGPT. Experts point out that training on paraphrased examples of test sets can lead to artificially high scores. LMSys, a gold standard for LLM benchmarking, is also susceptible to manipulation despite its rigorous standards. Critics argue that benchmarks like MMLU are not trustworthy, and emphasize the need for well-curated and secret test sets to maintain the integrity of evaluations. Trusted third-party evaluations, such as those from Scale AI, are recommended for more reliable assessments. This issue has come into focus again in light of the 'Reflection' saga, where a 13B model was able to outperform GPT-4.