Sources
lior.eth (Lior Messika)
last year, I made the case for pipeline parallelism as the next frontier for distributed training. today, @gensynai introduces SkipPipe, a massively optimized and infinitely scalable pipeline-parallel training method. machine learning is going through a renaissance, led by… https://t.co/MVEDj1G5aM
Jeff Amico SkipPipe is a massive breakthrough in decentralized model training. It is communication efficient but also highly scalable, unlike data parallel methods. We're excited to open source it today 🦾 https://t.co/Uuy0WywQma
gensyn Introducing SkipPipe SkipPipe is a new communication-efficient, pipeline-parallel training method. It reduces distributed training time by up to 55% and is scalable to theoretically infinite model size. Today, we're open sourcing it to push the frontier of decentralised ML. https://t.co/pP4tKQMfsR
