OpenAI and others seek new path to smarter AI as current methods hit hurdles

Artificial intelligence companies like OpenAI are facing significant challenges in developing larger language models. To overcome these hurdles, they are exploring innovative training techniques that mimic human thought processes, aiming to improve the efficiency and effectiveness of AI models.
Reuters:
‎

A dozen AI scientists, researchers and investors told Reuters they believe that these techniques, which are behind OpenAI’s recently released o1 model, could reshape the AI arms race, and have implications for the types of resources that AI companies have an insatiable demand for, from energy to types of chips.
After the release of the viral ChatGPT chatbot two years ago, technology companies, whose valuations have benefited greatly from the AI boom, have publicly maintained that “scaling up” current models through adding more data and computing power will consistently lead to improved AI models.
But now, some of the most prominent AI scientists are speaking out on the limitations of this “bigger is better” philosophy.
Ilya Sutskever, co-founder of AI labs Safe Superintelligence (SSI) and OpenAI, told Reuters recently that results from scaling up pre-training – the phase of training an AI model that uses a vast amount of unlabeled data to understand language patterns and structures – have plateaued…
[R]esearchers are exploring “test-time compute,” a technique that enhances existing AI models during the so-called “inference” phase, or when the model is being used. For example, instead of immediately choosing a single answer, a model could generate and evaluate multiple possibilities in real-time, ultimately choosing the best path forward.
This method allows models to dedicate more processing power to challenging tasks like math or coding problems or complex operations that demand human-like reasoning and decision-making…
“This shift will move us from a world of massive pre-training clusters toward inference clouds, which are distributed, cloud-based servers for inference,” Sonya Huang, a partner at Sequoia Capital, told Reuters.

‎
MacDailyNews Take: We live in interesting times.
Once AGI becomes smarter than Einstein, ask it to design chips for itself with greater energy efficiency that offer the same or better performance or, better yet, have it explain how to build large-scale fusion reactors.
After that:
In three years, Cyberdyne will become the largest supplier of military computer systems. All stealth bombers are upgraded with Cyberdyne computers, becoming fully unmanned. Afterwards, they fly with a perfect operational record. The Skynet Funding Bill is passed. The system goes online… Human decisions are removed from strategic defense. Skynet begins to learn at a geometric rate. It becomes self-aware at 2:14 a.m. Eastern time, August 29th. In a panic, they try to pull the plug. — The Terminator

We are currently about 1/4th of the way to being sustainable with Substack subscriptions.
Not a bad start!
Please tell your Apple-loving friends about MacDailyNews on Substack and, if you’re currently a free subscriber, please consider $5/mo. or $50/year to keep MacDailyNews going. Just hit the subscribe button. Thank you!

Read on Substack

‎
Please help support MacDailyNews — and enjoy subscriber-only articles, comments, chat, and more — by subscribing to our Substack: macdailynews.substack.com. Thank you!
Support MacDailyNews at no extra cost to you by using this link to shop at Amazon.
The post OpenAI and others seek new path to smarter AI as current methods hit hurdles appeared first on MacDailyNews.