Researchers at Apple have introduced a new machine learning method called Cut Cross-Entropy (CCE), which computes the cross-entropy loss without the need to materialize logits for all tokens into global memory. This innovation significantly reduces the memory footprint of loss computation, decreasing it from 24GB to just 1MB for the Gemma 2 2B model. The method aims to enhance the efficiency of large-vocabulary language models, addressing the challenges posed by traditional loss computation methods. Additionally, a separate study titled 'A Case for Soft Loss Functions' by Uma et al. highlights the advantages of using soft labels for training in computer vision and other AI tasks through crowd annotations.