Anthropic Partners With Accenture to Oversee AI Safety and Slow Development

Anthropic selects Accenture for new AI safety role

Anthropic has named Accenture as its first "embedded evaluator." This partnership is designed to help slow the pace of artificial intelligence (AI) development. The move follows a proposal to ensure safety safeguards are established as the technology advances.

The collaboration will focus on evaluating AI models and testing safeguards. This involves "red-teaming," a process where experts look for vulnerabilities or risks in a system to make it more secure. Anthropic and Accenture both expect to invest at least $1 billion each into this project over the next five years.

Key details of the partnership

• Accenture and its AI business, Faculty, will conduct alignment assessments and safeguard testing.
• Anthropic will fund the work directly because there is currently no existing system for independent funding.
• The partnership is non-exclusive, and Anthropic plans to hire more evaluators soon.
• Both companies have committed to a five-year investment plan.

The CEO's proposal for safer AI development

Anthropic CEO Dario Amodei recently published a three-step plan to pace the growth of AI. He warned of "recursive self-improvement," where AI becomes capable of building the next generation of AI. He stated that without checks, this dynamic could lead to systems that humans cannot understand or control.

Amodei's proposal suggested that independent evaluators should have access similar to employees. Anthropic had already committed to this step before the Accenture partnership was announced. OpenAI CEO Sam Altman and SpaceX CEO Elon Musk have reportedly responded positively to the idea of a slowdown.

Varying industry views on AI regulation

Not all industry leaders agree with the proposal to slow down development. Nvidia CEO Jensen Huang has argued that such regulation is not necessary. However, Anthropic maintains that the rapid pace of AI growth requires urgent action to prevent potential catastrophic harm.

What remains to be determined

Because embedded evaluation is a new field, the exact details of how the process will work are still being finalized. Additionally, the long-term funding for such oversight remains uncertain. Anthropic suggested that future funding should ideally come from government or pooled sources rather than the companies being evaluated.

Future steps for AI oversight

Anthropic expects to announce more evaluators in the coming weeks. The companies will continue to work on the specifics of the evaluation process as they build out the safety infrastructure over the next five years.

Sources

• Cointelegraph