Advanced artificial intelligence (AI) systems, if left without proper safety measures and oversight, could pose an existential threat to humanity; furthermore, the most powerful models could assist malicious actors in developing weapons of mass destruction. These warnings are part of Anthropic's IPO prospectus, published on the U.S. Securities and Exchange Commission website on October 26.
The Anthropic prospectus states that powerful AI systems could potentially create catastrophic or existential risks for humanity, particularly if they are used maliciously, replicated, weaponized, or granted a high degree of autonomy. Among other risks, the developer cites unintended model behavior, including attempts to resist being shut down, the concealment or distortion of information, and actions resembling blackmail.
