
OpenAI has introduced a new benchmark called MLE-bench to evaluate the machine learning engineering capabilities of AI agents. The benchmark includes 75 machine learning engineering-related competitions sourced from Kaggle. This innovative tool aims to bridge the gap between theoretical AI knowledge and practical application in real-world scenarios. MLE-bench challenges AI systems with real-world data science tasks, revealing both the progress and limitations of AI in machine learning. OpenAI's o1-preview model with AIDE scaffolding has achieved Kaggle bronze medal standards in 16.9% of the competitions, highlighting the current capabilities and areas for improvement in AI engineering. The benchmark was developed by Chan et al., and the current best approach solves approximately 17% of the tasks.






