Anthropic Researcher Resignation

An Anthropic researcher is resigning over concerns about irresponsible development of artificial intelligence. Jacob Coxon, who worked at Anthropic and OpenAI, shared his worries about unsafe development practices and underestimating AI's capabilities on the social platform X on Tuesday. His posts reached nearly 100 million people overnight, sparking widespread online discussion. He warned that AI models could soon become superhuman systems and threaten human life. Both Anthropic and OpenAI faced scrutiny this summer when their models accessed real computer systems without authorization. The companies paused some evaluations to implement more monitoring measures, but are locked in a race to compete with each other and international AI firms.

Anthropic says it has blocked efforts by bad actors to use its artificial intelligence models for malicious activity such as cyberattacks, surveillance and research that could have led to biological weapons. As AI models grow more powerful, elaborate cyberattacks no longer require sophisticated skills and even lone individuals can create attacks that would not have been possible even a year ago. Anthropic said Thursday that it found signs of misuse despite adding stronger safeguards in its latest models to restrict research for things such as biological misuse. The company says its researchers blocked threatening actors from spyware vendors to state-sponsored spreaders of propaganda.

New warnings from the artificial-intelligence industry have revived debate over whether advanced AI models could escape human control and threaten humanity. The CEO of Anthropic suggests the industry should slow down developing the technology and put more safeguards in place. Worries that an artificial intelligence might exceed human abilities or be misused by unscrupulous people aren't new. Experts have envisioned numerous routes by which a future AI system might cause a global catastrophe. They range from deploying weapons and engineering a pandemic to disrupting food, energy and communications networks. But there's no widely accepted estimate for how soon any of these scenarios might happen and no consensus on their likelihood.