GensynAI Report 'GPT@home' Highlights Decentralized Training as Future of AI

A new report from GensynAI highlights the growing trend of decentralized training for AI models. The report, titled 'GPT@home: Why the Future of Training is Decentralized,' discusses the feasibility of training large AI models over the world's edge devices. Innovations such as DiLoCo, SWARM, lo-fi, DisTrO, DiPaCo, and DMoE are reducing inter-node communication and handling unreliable devices, making decentralized training a reality. PrimeIntellect published OpenDiLoCo and NousResearch built DisTrO, reducing inter-GPU communication by 1000x. The report emphasizes that decentralized training could potentially scale to the level of Bitcoin, which uses 150 TWh/year—100 times more than the largest AI clusters. The findings suggest that decentralized training could lay the foundation for a more open and accessible model training ecosystem.