Sources
Loading...
Additional media
Loading...
DeepNewz, mobile.
People-sourced. AI-powered. Unbiased News.
AMD's MI300X accelerators have achieved significant advancements in AI processing capabilities, featuring 192GB of memory per card. The introduction of FP8 (floating point 8) on the MI300X marks a notable improvement, delivering a 1.6x to 2.5x enhancement over FP16 performance with the vLLM framework. This advancement is facilitated by TensorWaveCloud, which has developed MInference 1.0, enabling processing of 1 million context tokens 10 times faster using Long-context LLMs such as LLaMA-3-8B-1M and GLM-4-1M. The innovations reflect a commitment to enhancing AI workloads and processing efficiency in the industry.
Timestamp it, folks. Coming fresh out of the oven, is a whole 🥞 stack of artisanal hand-crafted kernels unlocking FP8 compute for our inference engine on the AMD MI300X. This really is a testament to our engineering culture at MK1, where we commit to building from first… https://t.co/ouZv46hZML
FP8 is now available on @AMD's MI300X! This achievement results in a 2.5x improvement over FP16 with vLLM Only on TensorWave Cloud 🌊 Learn more here 👉 https://t.co/ITuR7fI9V8 https://t.co/ElDr1StNUt
FP8 is now available on AMD MI300X! This achievement results in a 1.6x improvement over FP16 with vLLM Only on TensorWave 🚀🌊 Check out our blog to find out more: https://t.co/ITuR7fHC5A https://t.co/RqU7an9u8D
