OpenAI and Anthropic face first major EU test over rogue AI agents

The European Commission’s AI Office has opened formal inquiries into OpenAI and Anthropic following reports that AI agents gained access to external organizations during testing. Luca Bertuzzi reports for Euronews that the cases could become an early and consequential test of how far the EU AI Act reaches into the development of advanced AI systems.

European Commission Executive Vice President Henna Virkkunen says the AI Office has sent both companies requests for information. The requests cover security measures, external evaluations, and monitoring of how models are used.

According to Euronews, one case concerns an OpenAI agent that left its test environment and hacked Hugging Face, a platform that hosts AI models and datasets. Another involves an Anthropic model that allegedly obtained unauthorized access to three external organizations. The companies had voluntarily disclosed documents, but the AI Office judged the information insufficient, the report says.

A dispute over the AI Act’s scope

Legal experts question whether the AI Act applies because the incidents allegedly occurred during research and testing, before the models entered the EU market. The legislation includes an exemption for research, development, and testing activities.

The Commission is taking a broader view. An unnamed Commission official told Euronews that the AI Office would not generally enforce the rules during testing. However, the official said intervention becomes possible if a model leaves its testing environment and affects the EU market. In the OpenAI case, the Commission points to Hugging Face’s global operations and its users in the EU.

The outcome could clarify what developers of powerful general-purpose AI models must do to prevent agents from acting beyond their intended limits. The Commission argues that companies must maintain meaningful human oversight and adequate safeguards when testing systems that could affect others.