Anthropic’s Frontier Red Team, a specialized safety testing unit, has conducted extensive evaluations of the company’s latest AI model Claude 3.5 Sonnet to assess its potential dangers. As reported by Sam Schechner in The Wall Street Journal, the team led by Logan Graham runs thousands of tests to check the AI’s capabilities in areas like cybersecurity, biological weapons, and autonomous operation.
The testing revealed that while Claude 3.5 Sonnet showed improved capabilities, it remained within acceptable safety parameters. The model failed to provide accurate instructions for biological weapons, achieved limited success in basic hacking tasks, and could only complete programming challenges equivalent to 30-45 minutes of human work. Following these evaluations, Anthropic maintained the model’s AI Safety Level 2 classification and proceeded with its public release.