Sources
Ryan Morrison
Soon we’ll have 100mb models as capable as today’s 1b models and at that point we can just embed intelligence in an app. I recently trained a 200mb TinyGPT model on a coffee recipe database and it can generate passable recipes from a simple ingredient prompt. For now it will be… https://t.co/IHweuzhZr2
Tamay Besiroglu I think this is an important and surprising result. Data movement bottlenecks could constrain efficient (high utilization) scaling of AI training runs to ~100x beyond current top models, with a hard limit around ~1000x due to latency constraints. https://t.co/bJ8gPkOEuK https://t.co/qnM0Khe9Lr
- Tamay Besiroglu
I think this is an important and surprising result. Data movement bottlenecks could constrain efficient scaling of AI training runs to ~100x beyond current models, with a hard limit around ~1000x due to latency constraints. https://t.co/wjlyR6hkcg https://t.co/qnM0Khe9Lr
