EU GMP Annex 22 and the revised Annex 11: what the drafts ask of AI systems

What was published, and its status

What draft Annex 22 covers

The scope section sets the boundaries. The draft applies to static models, ones that do not change once deployed, with deterministic outputs. It says dynamic models that keep learning in use, and models whose outputs are probabilistic and not identical for the same input, should not be used in critical GMP applications. It says the document does not apply to generative AI and large language models, and that such models should not be used in critical GMP applications.

A critical application is one with a direct impact on patient safety, product quality, or data integrity. For non-critical applications, the draft allows other kinds of models, including generative AI, provided qualified personnel are responsible for checking the outputs. This is sometimes described as a ban on generative AI in GMP. It is narrower than that: it keeps generative AI out of critical decisions and puts a qualified person in charge of it elsewhere.

What it requires for critical applications

• Intended use. A description of what the model does, the data it works on, and the responsibilities of the human operator.
• Acceptance criteria. Performance criteria at least as high as the performance of the process the model replaces.
• Independent test data. Test data kept separate from training data, and people who have seen the test data not involved in training.
• Explainability. A record of the features that drove a decision, for example through methods such as SHAP, LIME, or heat maps.
• Confidence. A confidence score logged for each prediction, so low-confidence outputs can be handled appropriately.
• Controlled change. A retrained or changed model introduced only through change control, with an impact assessment.

What the revised Annex 11 and Chapter 4 add

The revised Annex 11 draft strengthens lifecycle management for computerized systems, makes quality risk management mandatory throughout and adds data integrity and security controls. The revised Chapter 4 draft covers documentation across paper, digital, and hybrid formats and introduces risk-based data governance. Together they update the ground Annex 22 sits on: any AI system is also a computerized system, and its outputs are documentation.

What it means for AI used near GMP work

For systems that assemble evidence, draft records, or answer questions for a qualified person to check, the draft points in a clear direction:

1. Decide, for each use, whether it is a critical application in the Annex 22 sense, and document why.
2. Where generative AI is involved, keep it to non-critical uses, and define the qualified person who checks each output.
3. Record how each output was produced, from which sources, so the check is meaningful.
4. Surface uncertainty rather than hiding it; the draft's confidence requirement reflects the same principle.
5. Put changes to models, prompts, and workflow steps under change control.

These are the controls agentic solutions must be built around: approved sources, source lineage, execution records, surfaced uncertainty, and expert approval as a defined step. Agentic AI in GxP environments: a practical guide covers them in more detail.