Sources
bayes
it would appear ai decentralization is accelerating. this is early work, and we'll see how it in particular pans out, but I'm really happy to see such strong, concrete steps from Nous. it's also amazing that they're already training a 15B param model with it. bullish https://t.co/SsQqYzbUiw
Rohan Paul This is just so beautiful. People worldwide connect their computers to train a 15B parameter language model over the internet and live-stream the process. And Nous Research's DisTrO tech reduces the communication between GPUs by 10,000x https://t.co/XooWvIKjmH
Patrick Hsu Distributed training over the internet! https://t.co/qZh5dGrsuy
Additional media
