AI · Official
Anthropic Gives Independent Evaluator METR Staff-Level Access to Its Systems
Anthropic News (community mirror) ·
At a glance
Anthropic has begun giving METR, a nonprofit evaluation team, access close to that of its own employees. The move is meant to let outside organizations verify Anthropic's AI models and safety practices. METR plans to publish its risk assessment results.
Key points
- Six METR researchers use company equipment, office space, and internal tools, working at a level similar to Anthropic's own risk evaluation team.
- METR may publish analysis of Anthropic's risk levels, incidents, and management practices.
- Anthropic cannot block publication, but it can redact certain categories of information, such as confidential intellectual property.
- METR is also evaluating with SecureBio and Accenture, and is in talks with Hugging Face and others.
Why it matters
It is hard to trust an AI company's own claims that it is safe, so it matters that outside organizations check directly. This is similar to how an outside regulator examines a bank.
Original source
An update on independent evaluators
Anthropic News (community mirror) · Oct 15, 09:00 KST
Read on Anthropic News (community mirror) ↗AI summarized the original article. Check the original for accuracy. Copyright belongs to Anthropic News (community mirror).
More from this field
- AWS Quick Adds Chat-Built Flows, Updated Web App, and Teamwork Among Agents
AWS Quick added updates that let a team turn one person's work routine into a shared, repeatable process. Flows can now be built and changed entirely in chat, and they can run from the web, desktop, or mobile app. Agents working together can also give more complete answers.
Amazon AWS Machine Learning blog · Nov 21, 10:06 KST - Amazon Bedrock Web Search Adds Five Languages and Country-Based Results
Amazon's Web Search tool for Bedrock, which gives AI agents cited, up-to-date web results, now supports Spanish, French, German, Italian, and Japanese, plus non-US English. A new optional country field can also steer results toward sources from a specific country. Existing customers get the language update automatically, and pricing stays the same in every language.
AWS What's New · Nov 21, 09:17 KST - AWS Adds Shared Cache Skill to Cut Self-Hosted LLM Cost and Latency
AWS wrote about a shared cache that lets self-hosted AI models reuse saved work across servers, aiming to cut cost and wait time. The post shows how a coding agent can help set it up. Amazon says it can cut inference cost by up to 40% and first-token delay by up to 80%.
Amazon AWS Machine Learning blog · Nov 21, 09:14 KST - Can AI Really Find Black Friday TV Deals? A BBC Test Says Not Yet
A BBC team tested four popular AI chatbots to see if they could find a good Black Friday deal on a 55-inch TV. The results were mixed, with many links broken or out of date.
BBC News · Nov 21, 09:06 KST