Sources
Loading...
Additional media
Loading...

OpenAI's o1-preview model has been evaluated extensively in a recent study, revealing its strong performance across various tasks. The model demonstrates an 83.3% success rate in solving complex competitive programming problems. It also excels in generating coherent and accurate radiology reports, performing high school-level mathematical reasoning tasks, and handling chip design tasks. Despite its high performance, the o1-preview model is noted to be slow and expensive. The evaluation, detailed in the DevQualityEval v0.6 and a 280-page PDF, provides insights into the opportunities and challenges of using OpenAI's o1-preview and o1-mini models for generating quality code.

