May 14, 12:19 PM

Sakana AI Open-Sources Continuous Thought Machines, Achieves 72.47% ImageNet Accuracy in Human-Like AI Reasoning

Sakana AI, a Tokyo-based artificial intelligence startup co-founded by former Google AI scientists, has introduced a new neural network architecture called Continuous Thought Machines (CTM). The CTM is designed to mimic human-like step-by-step reasoning by leveraging neuron-level timing, neuron-specific memory, and synchronization, allowing each artificial neuron to retain a short history of its previous activity and decide when to activate next. Unlike traditional Transformer-based models, which process inputs in fixed, parallel layers, CTMs unfold computation over internal steps, or 'ticks', enabling dynamic adjustment of reasoning depth and duration based on task complexity. This approach allows the model to allocate computation dynamically, providing interpretability in how reasoning steps are formed and decisions are made. Early demonstrations show that CTMs can perform tasks such as image classification, 2D maze solving, and reinforcement learning with competitive accuracy. On the ImageNet-1K benchmark, CTM achieved 72.47% top-1 and 89.89% top-5 accuracy. The model's sequential reasoning and natural calibration provide transparency, allowing researchers to observe the reasoning process over time. Sakana AI has open-sourced the CTM implementation on GitHub, including training scripts, pretrained checkpoints, and analysis tools, to encourage further research and experimentation. The company states its goal is to eventually achieve levels of competency that rival or surpass human brains. CTM remains a research architecture, with further progress needed in optimization and hardware efficiency before it is ready for enterprise deployment.

#Sakana AI #Google AI #Continuous Thought Machines #Transformer #ImageNet #GitHub

Written with ChatGPT (GPT-4).