Sources
Loading...
Additional media
Loading...
DeepNewz, mobile.
People-sourced. AI-powered. Unbiased News.
Google has released two new open-source models, Gemma 27B and 9B. The Gemma 27B model has garnered significant attention for its performance, reportedly surpassing LLaMA 3 70B on the LMSYS benchmark. This model has been evaluated on the Chatbot Arena by human raters and has achieved notable ELO scores. Despite its impressive performance, some experts believe that it does not significantly advance the open-source model field. However, its efficiency and permissive license make it accessible for a wide range of users. The Gemma 27B model is also compared to other models like Claude Sonnet, GPT-4o, Meta 400B, and Nemotron 340B.
Evaluation of Gemma 2 9B and 27B on LMSYS Chatbot Arena "Gemma 2 27B and 9B Instruction Tuned models were evaluated on the Chatbot Arena (Chiang et al., 2024) in blind side by side evaluations by human raters against other state of the art models. We report ELO scores in Figure… https://t.co/rM5If4DyuZ
Jaw dropping result by Gemma for a 27B model, with permissive license! Look at Nemotron with 340B below it. Don't get me wrong, Nvidia released a good model too. But the efficiency of Gemma... Lot of people can run this one. https://t.co/Bcfb3wxv54 https://t.co/2xe5ZGgfmd
Gemma 27B Is Now The Top Open Source Model On LymSys?! I was reading this wrong in the morning, but it seems like Gemma 27B is the top open-source model. Human eval arena IMO, this is a highly gamed benchmark, and we will know a lot more about this model once we have the… https://t.co/cRAeo2c75v