Generative AI models are facing significant challenges due to their reliance on tokens, which are a fundamental aspect of their internal processing environments. Unlike humans, these models do not process text in a linear or holistic manner, but instead break it down into tokens. This tokenization process can lead to strange behaviors and limitations in understanding languages, mathematics, and other complex tasks. Notable AI models such as Gemma and GPT-4o are affected by these issues, which hamper their performance and ability to generate accurate and coherent outputs. TechCrunch's Kyle Wiggers has explored these limitations in detail.