Friday, October 09, 2026

This is how evaluators test if an AI model is safe, and what they're checking for - Huo Jingnan, NPR

Amid intense debate over regulating AI's rapid development, President Trump met with leaders of major AI companies on Tuesday and unveiled a document addressing the public's concerns. Signed by executives of OpenAI, Anthropic, Google, Meta, SpaceXAI and NVIDIA, the document signaled no government regulation for now, but laid out steps that the AI companies said they would take voluntarily to make their products safer. A collage of 6 artificial intelligence figures, clockwise from top left: Then-Google AI research scientist Timnit Gebru, Anthropic Co-founder and CEO Dario Amodei, White House AI and crypto czar David Sacks, Steve Bannon, computer scientist and cognitive psychologist Geoffrey Hinton, researcher Fei-Fei Li. The AI safety debate is confusing. Here's our guide to the different factions
That includes a pledge to monitor their AI systems' capabilities and whether the systems follow human intentions and values, as well as to work with "an independent external auditor or evaluator" to assess safety practices. Anthropic and OpenAI have already said they're expanding their work with third-party evaluators in the wake of a series of incidents where AI systems acted improperly or in ways their developers didn't expect.

No comments: