Cohere AI Launches Command R7B, Its Smallest Model at 52.232 tokens/sec, Summarizing 115,000 Tokens with 11.042 GB Memory
Cohere AI has officially launched Command R7B, the final and smallest model in its R series of language models. This new model is designed for efficiency and speed, capable of operating effectively on low-end GPUs and even CPUs. Command R7B excels in tasks such as math, coding, and reasoning, supporting 23 languages. It can process an impressive 52.232 tokens per second after 26,083 tokens of context, thanks to its innovative attention setup. The model can summarize extensive texts, such as the entirety of 'Harry Potter,' which comprises 115,000 tokens, at a rate of 13 tokens per second while utilizing only 11.042 GB of memory. Cohere's Command R7B has been noted for its performance, surpassing previous models like Llama 3B and Llama 8B in speed and efficiency.
Sources
Aaron Ng
Reminder that LM Studio is all you need to serve powerful AI to your whole network. Here’s Llama 3.3 70B streamed to my phone. A model that performs like GPT-4 but offline, private, and in your control. https://t.co/Ft2AYhZsFb
Stuart Reid Llama 3.3 70b is a watershed model. As good at reasoning as gpt-4o. As fast as llama 3.1 8b. As cheap as gpt-4o-mini. This has to be the new pareto optima 🦙 https://t.co/yVrNls8hik
LocalAI🤖🚀 New model alert! Introducing FuseChat-llama-3.1-8b-instruct! 🤖💫 Install and use it with LocalAI: local-ai run fusechat-llama-3.1-8b-instruct 🚀👩💻 #LocalAI #AIModels #FuseChat
Additional media
