Sources
Loading...
Additional media
Loading...
DeepNewz, mobile.
People-sourced. AI-powered. Unbiased News.
Highly comprehensive report on decentralized training, and why it’s positioned to be one of the most important innovations of our lifetime. Great piece by @_jamico & the team at @gensynai. https://t.co/M8OfRgg9sd
Time is flying fast and so is DeAI development! Let's revisit the ideas after 3 months #1 Decentralized training becomes real: @PrimeIntellect published OpenDiLoCo and @NousResearch built DisTrO (Distributed Training Over the Internet) reducing inter-GPU communication by 1000x… https://t.co/YnFQnAilff
Today's AI trends "lay the foundation for a more open and accessible model training ecosystem"; lots of details showing why in this new @gensynai report https://t.co/k0dDBGOTVR
A new report from GensynAI highlights the growing trend of decentralized training for AI models. The report, titled 'GPT@home: Why the Future of Training is Decentralized,' discusses the feasibility of training large AI models over the world's edge devices. Innovations such as DiLoCo, SWARM, lo-fi, DisTrO, DiPaCo, and DMoE are reducing inter-node communication and handling unreliable devices, making decentralized training a reality. PrimeIntellect published OpenDiLoCo and NousResearch built DisTrO, reducing inter-GPU communication by 1000x. The report emphasizes that decentralized training could potentially scale to the level of Bitcoin, which uses 150 TWh/year—100 times more than the largest AI clusters. The findings suggest that decentralized training could lay the foundation for a more open and accessible model training ecosystem.
