Terence Tao, one of the most gifted living mathematicians and a Fields medalist, has evaluated the mathematical abilities of the AI model GPT-o1. Tao described the AI's performance as comparable to that of a 'mediocre, but not completely incompetent, graduate student.' This assessment marks an improvement over previous models, which Tao found to be closer to 'actually incompetent' graduate students. Additionally, GPT-o1 scored 120 on an IQ test. Tao's review suggests that with one or two more iterations, GPT-o1 could become a serious research tool.
Does one have to be a genius to do maths? - Terence Tao: https://t.co/4Ji86PH9n4 https://t.co/hDy4DyKAei
1/n Someone had GPT-o1 take an IQ test; it scored a 120. I guess good enough for a not completely incompetent graduate student. https://t.co/SmC6WKudW4
Terence Tao is giving commentary on o1's math capabilities on Mastodon, and has mixed but overall optimistic takeaways World's foremost expert is seriously evaluating a model vs. his PhD students w/ mixed results. Rate of improvement is incredible. https://t.co/CsGf5weNX2 https://t.co/tjqJVT8dxt