Sources
Loading...
Additional media
Loading...
DeepNewz, mobile.
People-sourced. AI-powered. Unbiased News.
Ilya Sutskever, co-founder of OpenAI and Safe Superintelligence, recently stated to Reuters that the results from scaling up pre-training, a phase where AI models learn from vast amounts of unlabeled data, have plateaued. This sentiment is echoed by other industry insiders who report challenges in achieving performance gains from scaling pre-training. As a result, companies like OpenAI, Google, and DeepMind are shifting their focus from pre-training to inference-time compute, which is significantly more expensive. Innovations in hardware and optimization are being explored to make AI more efficient and curb rising costs. Despite these challenges, many AI researchers remain optimistic about the future of scaling AI, though they acknowledge a more linear improvement path ahead.
AI: Enterprise Search, the other 'AI Search' Opportunity. RTZ #540 ...Glean with an early lead vs industry leaders https://t.co/Y2ORLIlPCr #Tech #AI @OpenAI $MSFT $NVDA $GOOG $AMZN $META $AAPL $TSLA @perplexity_ai @glean https://t.co/hP1ou9blrf
Scaling LLMs hits diminishing returns when benchmarks become the goal, not the tool. Spending billions for a 1% gain on artificial metrics misses the point. The real breakthrough will come when we solve reliability, not just raw power. #AI #LLMs #OpenAI #Cursor
LLM SCALING - WALL OR NO WALL? The big question is, have LLMs hit a wall? Short answer - It depends on the benchmarks. We can continue to scale LLMs if we invent more challenging benchmarks. The problem is that we are beginning to saturate useful benchmarks. Once you get to…
