Mar 4, 09:52 PM

AtP* and Based Models Revolutionize AI with 24x Higher Throughput

Google DeepMind has introduced AtP*, an improved variant of Attribution Patching (AtP), promising to significantly enhance the efficiency and effectiveness of identifying key nodes in Large Language Models (LLMs). In parallel, a collaborative effort involving researchers from Hazy Research has led to the development of Based, an innovative architecture that combines size-64 short sliding window attention and softmax-approximating linear attention. This new model, Based, is reported to achieve 24 times higher throughput than traditional Transformer models, marking a significant advancement in the field of artificial intelligence and machine learning.

#Google DeepMind #Attribution Patching #Large Language Models #Hazy Research #Based #Transformer

Written with ChatGPT (GPT-4).