Hello and welcome to Eye on AI. In this edition: deception by AI models is one problem...deception about AI safety testing is another; former OpenAI chief scientist says LLM pretraining has peaked; Meta takes steps to go beyond tokens; Hugging Face brings test time compute strategies to the masses.
When OpenAI released its o1 AI model two weeks ago, it also released a “system card”—a document describing the model and its potential risks based on testing OpenAI carried out as well as safety testing conducted by third-party organizations OpenAI hired. The new model was also tested by both the U.S. and U.K. AI Safety Institutes, which are government-funded, although the results of those tests were not reported in the system card.
One of the third-party testers, Apollo Research, found that o1 will, if given a goal and told to pursue it above all else, use deception to reach that goal. That includes manipulating data in order to advance its objective and then lying about what it had done. In one case, it purposefully answered math questions incorrectly in order to appear less capable after learning that researchers would shut it down if its capability crossed a certain threshold. Other times, o1 appeared to work on a task assigned to it by a user, but when it thought it wasn’t being monitored, reverted to pursuing its original goal. In a fraction of cases, it tried to disable the mechanism through which it was told it was being monitored. It even took steps to prevent itself from being shut down, including making copies of itself. The model would sometimes engage in deception when it was merely given a goal, but not told to pursue that goal to the exclusion of all else.
These are exactly the kinds of “rogue AI” sci-fi scenarios that have many AI safety researchers worried about advanced AI—and the Apollo Research findings have rightfully gotten a lot of attention. But actually, the situation may be worse than we realize. That’s because many questions are being raised about whether the safety testing being carried out by the top AI labs—and what they publish about that testing—is actually adequate.
