Nous Research has released a preliminary report on DisTrO (Distributed Training Over-the-Internet), a new system that significantly reduces communication requirements for training AI models. The DisTrO method enables decentralized training by lowering bandwidth needs by 1,000 to 10,000 times, including an 857x reduction in bandwidth requirements, with potential for up to 3000x in pre-training and 10,000x in fine-tuning. This advancement addresses the latency problem in decentralized training, allowing for efficient, low-latency training of large neural networks over slow internet connections. The open-source nature of this project is expected to foster collaboration and innovation in AI, potentially democratizing access to powerful AI model training.