Sources
Bindu Reddy
Quantization Is All You Need SOTA LLMs are too large to run on laptops. Quantization is a technique used to reduce LLMs' computational and memory requirements and create a smaller version of the model. Quantization is central to OSS progress It involves converting the model's… https://t.co/bAnJLnONsW
fly51fly[CL] Apple Intelligence Foundation Language Models https://t.co/O4KChBcb35 - The models introduced, AFM-on-device and AFM-server, are foundation language models designed to power Apple Intelligence features efficiently and responsibly. AFM-on-device is a ~3B parameter… https://t.co/kh6u4OmzgK
Irina Rish Can one achieve SOTA LLM performance at a much lower bitsize (=>memory/inference costs) than current (post-training) quantization? YES! - by training ternary LLMs - a "sweet spot" between underperforming binary and costly ful-precision ones. Happy to announce our recently… https://t.co/siObHApQ9P
