Sources
Tom Dörr
"... why this operation is called "attention" in the first place - it comes from attending to words of a source sentence ..." https://t.co/3B6N3XQl3m
Corry WangReally interesting account of how the attention mechanism in LLMs was invented, told for the first time in public IMO the most interesting thing here is that if Dzmitry hadn’t invented attention in September 2014, someone else would’ve done it anyways a few months later https://t.co/apIoMSVBqR
Jean de Nyandwi History of "attention"(the idea that was popularized/packaged nicely by Transformer paper) and how everything came together, piece by piece. https://t.co/1NfizXkGFs
Additional media
