Jan 3, 10:17 AM

Meta Introduces Scalable Memory Layer Technology, Enhancing Language Models by 128 Billion Parameters and One Trillion Tokens with Explicit Working Memory

Meta's research team, Meta FAIR, has introduced advancements in memory layer technology aimed at enhancing the capabilities of language models. The new scalable memory layer can increase a language model's capacity by up to 128 billion parameters, equivalent to one trillion tokens, without significantly increasing computational demands. This development is seen as a solution to the limitations of context length in existing models. Additionally, Meta AI has proposed a novel approach called Explicit Working Memory (EWE), which aims to improve factuality in long-form text generation by integrating a working memory system. These innovations are part of ongoing efforts to enhance the reasoning abilities of language models and address challenges in training and inference.

#Meta #Meta FAIR #Meta AI #Explicit Working Memory

Written with ChatGPT (GPT-4o mini).

Sources

Additional media

Image #1 for story meta-introduces-scalable-memory-layer-technology-enhancing-language-models-128-94db79c5

Image #2 for story meta-introduces-scalable-memory-layer-technology-enhancing-language-models-128-94db79c5

Image #3 for story meta-introduces-scalable-memory-layer-technology-enhancing-language-models-128-94db79c5