BigCodeBench, a new benchmark designed to evaluate large language models (LLMs) on practical and challenging programming tasks, has been introduced. Unlike simpler benchmarks like HumanEval and MBPP, BigCodeBench tests LLMs on more realistic and comprehensive coding scenarios, including open code and math LLMs. This initiative is led by Terry Yue Zhuo and aims to address the saturation of basic coding benchmarks by state-of-the-art (SOTA) LLMs. Current top models, including GPT-4 and new models from DeepSeek AI, achieve around 50% success on these tasks, which involve a wide range of tool/library calls. The benchmark highlights that approximately 40% of tasks remain unsolved by SOTA models, emphasizing the need for more robust evaluation standards in the field of AI and machine learning.