Apple researchers released a paper in early June titled “The Illusion of Thinking,” contending that leading large language models—including OpenAI’s o1 and o3, DeepSeek-R1 and Claude 3.7 Sonnet—solve complex problems mainly through pattern matching rather than genuine reasoning. The study reported a sharp drop-off in accuracy on multi-step tasks, arguing that current evaluation benchmarks overstate machine reasoning capabilities. Within days, multiple teams issued rebuttals. A response paper, “The Illusion of the Illusion of Thinking,” co-authored by Anthropic’s Claude Opus, says Apple’s experiments misattributed errors that stemmed from token-limit truncation, rigid scoring methods and the inclusion of unsolvable puzzles. The authors claim that when output length constraints are removed or answers are compressed, model performance rebounds, undermining Apple’s conclusion that reasoning collapses. Other researchers echoed the criticism, while some analysts—including AI commentator Gary Marcus—found the counter-arguments unconvincing and noted mathematical mistakes in several follow-up studies. The episode has opened a broader debate over how to design tests that distinguish genuine reasoning from statistical pattern matching, highlighting the unsettled state of measuring cognitive capabilities in rapidly advancing AI systems.
1. Chinese scientists confirm AI capable of spontaneously forming human-level cognition "The result shows that LLMs are not "stochastic parrots." Instead, these models have an internal understanding of real-world concepts much like humans." Lets have a look: https://t.co/L4PXAszZmU
AI responds to the Apple paper, claiming AI is an illusion Wondering how much of the original Apple paper that only had human authors was written by AI 😂 It would be silly for researchers not to have AI write at least 75-80% of the paper anyway 🤷 https://t.co/fb3naozEta
A look at seven rebuttals to Apple's paper on limitations of Large Reasoning Models, and why none make a compelling case (@garymarcus / Marcus on AI) https://t.co/0tmjZCKZFq https://t.co/Dd7x0yZ9Mb https://t.co/ZOzeer1FAj