Sources
Rohan PaulChina's Xiaohongshu(Rednote) released its dots.llm open source AI model Type: A MoE model with 14B activated and 142B total parameters trained on 11.2T tokens. No Synthetic Data during Pretraining Training Stages: Pretraining and SFT. Architecture: Multi-head Attention with https://t.co/OsDBKxnmHP
Tiezhen WANGIt's hard to argue if RED note or Zhihu hold the most valuable social media content in Chinese. But this dots.llm1.inst from RED note looks quite good. - 142B MoE, 14B active parameters - Trained on 11.2T tokens - base model released - MIT license - 32k content length https://t.co/lUNjpmDSl5
Chubby♨️China's Rednote Open-source dots.llm performance & cost. When it comes to price-performance-ration China is top performing https://t.co/DyDAz4aiDo
Additional media






