OpenAI disbands mission alignment team, which focused on ‘safe’ and ‘trustworthy’ AI development

OpenAI has disbanded a squad that focused connected — arsenic the institution itself described — ensuring that its AI systems are “safe, trustworthy, and consistently aligned with quality values.” At the aforesaid time, the team’s erstwhile person has been fixed a caller relation arsenic the company’s “chief futurist.”

OpenAI confirmed to TechCrunch that the team’s members person present been assigned to different roles. The quality was first reported by Platformer.

The disbanded squad successful question, which appears to person been formed successful September of 2024, was the startup’s interior portion dedicated to moving connected alignment. That’s a wide tract of interest wrong the manufacture that seeks to guarantee that AI acts successful accordance with quality interests.

“We privation these systems to consistently travel quality intent successful complex, real-world scenarios and adversarial conditions, debar catastrophic behavior, and stay controllable, auditable, and aligned with quality values,” a post from OpenAI’s Alignment Research blog declares.

An OpenAI job posting for the Alignment squad describes it arsenic being dedicated to AI probe that focuses connected “developing methodologies that alteration AI to robustly travel quality intent crossed a wide scope of scenarios, including those that are adversarial oregon high-stakes.”

In a blog post published Wednesday, Josh Achiam, the erstwhile caput of OpenAI’s Alignment team, explained his caller relation arsenic the company’s Chief Futurist. “My extremity is to enactment OpenAI’s ngo — to guarantee that artificial wide quality benefits each of humanity — by studying however the satellite volition alteration successful effect to AI, AGI, and beyond,” Achiam wrote.

Achiam noted that, successful his caller role, helium would beryllium collaborating with Jason Pruet, a physicist from OpenAI’s method staff.

A spokesperson for OpenAI said that remainder of the Alignment squad — a radical of six oregon 7 radical — had subsequently been reassigned to antithetic parts of the company. The spokesperson couldn’t accidental wherever precisely the squad members had been assigned, but said that they were engaged successful akin enactment successful those roles. It was besides unclear whether Achiam would person a caller squad arsenic portion of his “futurist” role.

The spokesperson attributed the disbanding of the squad to the kinds of regular reorganizations that hap wrong a fast-moving company.

OpenAI antecedently had what it called a “superalignment team”—which was formed successful 2023 focused connected studying semipermanent existential threats posed by AI — but that squad was disbanded successful 2024.

Achiam’s personal website inactive lists him arsenic caput of Mission Alignment astatine OpenAI, and describes him arsenic being funny successful ensuring that the “long-term aboriginal of humanity is good.” His LinkedIn profile shows helium had served arsenic Head of Mission Alignment since September of 2024.