The companies developing the world's most powerful artificial intelligence systems are increasingly warning that the technology could become a tangible danger to humanity without stronger safeguards, while the competitive pressure to build more capable models makes slowing down a difficult choice for any one company.
The chorus of warnings from inside OpenAI and Anthropic is growing. There, scientists and executives are calling for governments, competitors and independent institutions to establish limits that individual laboratories may be unwilling to impose on themselves.
