Oct 19, 02:20 PM

NVIDIA Unveils Normalized Transformer (nGPT) for 4-20x Faster Training and Improved Stability of Large Language Models

NVIDIA researchers have introduced a new architecture called the Normalized Transformer (nGPT), which utilizes a hypersphere-based design to enhance the training speed and stability of large language models (LLMs). The nGPT architecture reportedly achieves training speeds that are 4 to 20 times faster than previous models. This advancement is expected to significantly improve convergence speed in transformer models, as noted by various experts in the field. The introduction of nGPT marks a notable development in the ongoing evolution of AI and machine learning technologies.

#NVIDIA #Normalized Transformer

Written with ChatGPT (GPT-4o mini).

Sources

Vlad Ruso PhD@vlruso
2 years ago
Nvidia AI Introduces the Normalized Transformer (nGPT): A Hypersphere-based Transformer Achieving 4-20x Faster Training and Improved Stability for LLMs https://t.co/2VwuxQhKa3 #nGPT #AITraining #NaturalLanguageProcessing #MachineLearning #NVIDIA #ai #news #llm #ml #research #… https://t.co/tmWmdwV0S9
Marktechpost AI Research News ⚡@Marktechpost
2 years ago
Nvidia AI Introduces the Normalized Transformer (nGPT): A Hypersphere-based Transformer Achieving 4-20x Faster Training and Improved Stability for LLMs Researchers from NVIDIA propose a novel architecture called the Normalized Transformer (nGPT), which incorporates… https://t.co/8x2V1N5GB6
Chubby♨️@kimmonismus
2 years ago
New Transformer architecture modifications from NVIDIA researchers - nGPT: A hypersphere-based Transformer achieving 4-20x faster training and improved stability for LLMs. (Via Reddit) https://t.co/Ot1NFS4afy

Additional media

Image #1 for story nvidia-unveils-normalized-transformer-ngpt-4-20x-faster-training-improved-large-93b9ac86

Image #2 for story nvidia-unveils-normalized-transformer-ngpt-4-20x-faster-training-improved-large-93b9ac86