AI Safety Researchers Resign from OpenAI, Anthropic
Senior AI safety researchers at both OpenAI and Anthropic have resigned, citing concerns over the rapid, engagement-driven deployment of AI products. The departures reportedly stem from fears that commercial pressures are overriding calls for slower, more responsible scaling and sufficient consumer protections.
- Jan Leike, former co-lead of OpenAI's "Superalignment" team, resigned stating that the company's "safety culture and processes have taken a backseat to shiny products". He expressed concern that the company is not on a trajectory to be able to steer and control AI systems much smarter than humans. - Daniel Kokotajlo, another former OpenAI researcher on the governance team, quit due to a loss of confidence that the company would "behave responsibly around the time of AGI" (Artificial General Intelligence). Some former employees have faced highly restrictive non-disparagement agreements that could threaten the loss of vested equity for criticizing the company. - In a similar vein, Mrinank Sharma, who led the Safeguards Research Team at Anthropic, resigned with a public letter warning that "the world is in peril" from a series of interconnected crises, including AI and bioterrorism. He noted the constant "pressures to set aside what matters most" within the industry. - Anthropic was founded in 2021 by seven former OpenAI employees, including CEO Dario Amodei, with an explicit focus on AI safety and has positioned itself as a public benefit corporation. Unlike OpenAI, which has seen numerous high-profile departures, all of Anthropic's co-founders remain with the company. - The resignations at OpenAI follow the dissolution of its "Superalignment" team, which was tasked with research on the long-term risks of superintelligent AI. This has raised questions about the company's commitment to its original safety-focused mission. - Concerns from departing researchers are not just about long-term existential risks. For example, former OpenAI researcher Zoë Hitzig resigned over the company's decision to introduce advertising into ChatGPT, comparing it to the mistakes made by Facebook in prioritizing engagement over user well-being. - The backdrop to these departures is an accelerated pace of development in the AI industry, with both OpenAI and Anthropic releasing numerous major model updates and new products over the past year. This rapid scaling has intensified the debate around the trade-offs between competitive pressures and responsible deployment.
