Mar 20, 04:19 AM

Huggingface Adds GaLore: 20% VRAM Reduction, 82.5% Optimizer Memory Cut

Huggingface Transformers has officially integrated GaLore, a new feature that allows users to finetune models with 20% less VRAM usage. This integration, supported by contributions from the Huggingface team and community members, enables the combination of GaLore with ORPO using Axolotl, promising more efficient model training. Further enhancements and integrations are planned, aiming to reduce the optimizer state memory footprint by 82.5% without compromising performance. This development is expected to facilitate state-of-the-art (SOTA) results on consumer-grade hardware, making advanced model training more accessible.

#Huggingface Transformers #GaLore #Huggingface #ORPO #Axolotl

Written with ChatGPT (GPT-4).