First Decentralized AI Training Project Launches Globally with 10B LLM

The launch of the first decentralized training project on a global scale marks a significant milestone in the world of artificial intelligence. This initiative allows anyone to contribute compute resources, enabling the training of large language models (LLMs) without the need for massive data centers. The project is entirely open-source and has been compared to models like Llama, with the potential to pretrain better models through community contributions. This approach leverages distributed GPUs across the world, challenging traditional norms of compute-memory-storage-network in data centers. Recent advancements such as DiLoCo demonstrate the feasibility of this decentralized method. The project includes the first-ever open-source decentralized training of a 10B LLM on open compute.