Recent advancements in machine learning have demonstrated significant improvements in the efficiency and cost of training models. A notable example includes training a diffusion model for just $1800, significantly lower than traditional costs. Additionally, a BERT model was trained to outperform the original using a single GPU in a single day. Another achievement involved training a BERT model up to the quality of T5 using only 6GB of text, which is 745 times less data than usual, distilled from the Pile, in under 60 hours. Furthermore, a CLIP competitive model was trained on just 1/400 of the data typically required. These developments, in contrast to the $3 billion spent on training alone, indicate a substantial reduction in resources needed for high-quality model training.