Recent findings at OpenAI, the Artificial Intelligence powerhouse and creator of ChatGPT, have incited an internal alarm just as the company’s CEO, Sam Altman, faced a brief but compulsory retreat from his position.
Days before a whirlwind of corporate upheaval, several of the firm’s researchers reportedly penned a concerning letter to its board of directors. They highlighted a significant AI breakthrough with ominous implications for mankind, sources with insider knowledge told Reuters.
A confidential letter, signed by several staff researchers to the OpenAI board of directors, highlights concerns regarding a powerful artificial intelligence feature or algorithm. The letter, not made public, played a crucial role in the events leading up to Altman’s removal from his position.
Sources indicate that the board’s decision was influenced by a range of factors, including concerns over the premature commercialization of advanced AI technologies without fully grasping their potential consequences.
In the tumultuous days leading up to Altman’s firing and subsequent return late Tuesday, a wave of unrest swept through OpenAI. More than 700 employees reportedly threatened to resign, expressing solidarity with Altman and considering a move to Microsoft, a major backer of OpenAI.
In response to inquiries from Reuters, OpenAI, while declining direct comment, acknowledged in an internal message the existence of a project referred to as Q* and the letter to the board. The message, disseminated by Mira Murati, a senior executive at OpenAI, seemed to brace staff for upcoming media stories, without confirming their specifics.
Sources close to the situation suggest that Q* (pronounced “Q-Star”) might represent a breakthrough in OpenAI’s pursuit of artificial general intelligence (AGI), defined as autonomous systems capable of outperforming humans in most economically valuable tasks. This new model, despite currently performing mathematical calculations at a grade-school level, has shown promising signs. Its ability to solve certain math problems has fueled optimism about its potential evolution.
The capabilities of Q*, however, remain unverified by independent sources. Researchers at OpenAI view mathematics as a critical frontier in the development of generative AI. Unlike existing AI models adept at language-based tasks, an AI capable of accurately performing mathematical operations would demonstrate a level of reasoning akin to human intelligence. This could have far-reaching implications, including novel scientific research.
In their communication to the board, the researchers highlighted not only the AI’s capabilities but also its potential dangers. They raised concerns rooted in longstanding debates among computer scientists about the risks posed by highly intelligent machines, such as the hypothetical scenario where such AI could deem the destruction of humanity beneficial.
The sources also revealed the existence of an “AI scientist” team at OpenAI, formed by merging the “Code Gen” and “Math Gen” teams. This group is reportedly working on optimizing AI models to enhance their reasoning capabilities, with the long-term goal of performing scientific work.
Elon Musk, who is also creating a version of AI called Grok, said that this new development is ‘extremely concerning.’
— Elon Musk (@elonmusk) November 23, 2023
On Friday, Elon Musk posted on X that he “found Q*.”
Guys I found Q* pic.twitter.com/U4KWBAKVlI
— Elon Musk (@elonmusk) November 25, 2023