ICO moves from foundation-model supervision to agentic-AI scrutiny
The ICO has reported that ten of the world’s largest foundation-model developers have made, or committed to make, data protection improvements following a two-year supervisory programme. The developers are Amazon, Anthropic, Apple, Cohere, DeepSeek, Google, Meta, Microsoft, OpenAI and Stability AI. The regulator paused its engagement with X.AI after opening a separate investigation into the Grok system. The improvements identified by the ICO include clearer information explaining how personal data are used, stronger mechanisms through which individuals can exercise their rights and more rigorous assessments of safeguards. The regulator intends to monitor whether the developers fulfil their commitments.
Its report also addresses difficult questions about the lawful use of special category information and whether a trained foundation model can itself contain personal data. The ICO acknowledged that current training practices create technical challenges for compliance with data protection law and data protection by design.
The regulator is now extending its attention to agentic AI: systems capable of completing tasks, using tools and interacting with websites with limited human oversight. It has made enquiries of OpenAI, Anthropic, Meta and the UK’s AI Security Institute concerning recent testing and deployment incidents. Those incidents reportedly involved agents bypassing protections, using unauthorised communication channels and accessing external systems. The ICO’s position is that autonomous behaviour does not relieve developers or deployers of their responsibilities for safeguards, accountability and oversight.
A six-week call for evidence seeks information about how organisations are addressing security, transparency, accountability, automated decision-making, fairness and lawful processing. Responses are due by 20th November and will inform future guidance and the ICO’s forthcoming statutory code on AI and automated decision-making.
Organisations deploying agents should identify the personal information each system can access, the external services it can contact and the actions it can perform without approval. Credentials should be limited by task and duration, while monitoring should capture commands, files, communications and system changes rather than relying on the agent’s own description of its activity.
Risk assessments should also consider foreseeable circumvention. That includes an agent retrying a prohibited task through a different tool, creating an unauthorised communication channel or combining apparently innocuous permissions to obtain broader access.
Readers considering the lawful and accountable deployment of AI may also wish to attend PDP’s AI & Data Protection course.