Agentic AI in GxP environments: A practical guide

What an AI agent is, in operational terms

An AI agent is a system that works toward a defined goal by taking a series of steps: finding information, calling tools, checking results, and producing an output. In pharmaceutical operations, the useful version of that idea is narrow. The agent does a specific job the operation has defined, against sources the operation has approved, and hands its work to a person who decides.

Consider a deviation raised on a batch. Before anyone can investigate the cause, someone has to assemble the batch record, the in-process results, the raw material lots and their certificates, the equipment logs, and any related deviations. An agent can do that assembly, compare the batch with similar batches, check the evidence against the relevant procedures, and draft a summary of what it found and what it could not find. The investigator reviews the evidence, tests the explanations, and determines the cause.

That division of labor is the whole idea: agents take on the assembly work, and experts keep the judgment.

Agents, copilots, and chat assistants

A chat assistant is useful for drafting and exploring. It is not a controlled process. The governed agent is the version that belongs near GMP work, because every element of its work can be inspected.

What regulators have said so far

No regulator has published a final rule specific to AI agents in GMP manufacturing. Several documents set the direction, and a quality organization should know what each one covers.

Two points are easy to miss. First, the Annex 22 draft does not forbid generative AI in GMP; it keeps it out of critical applications, the ones with a direct impact on patient safety, product quality, or data integrity, and puts a qualified person in charge of checking it elsewhere. Second, FDA's draft guidance is about AI that produces information supporting a regulatory decision, such as a submission. Its scope excludes AI used for operational efficiencies like internal workflows. That does not mean operational AI is unregulated; it means the existing GMP framework, including the predicate rules and Part 11, is where the expectations come from.

The post in this guide, What FDA's draft AI guidance asks for, and what it leaves out, covers the FDA draft in detail.

Why "GxP compliant AI" is the wrong question

Vendors often describe software as "GxP compliant." The claim does not hold up, and quality teams know it. Compliance is a property of a regulated process and the organization running it. A tool can make that process easier to control and to demonstrate, but it cannot make the process compliant by itself.

An analogy helps. A restaurant earns its health grade as a whole. A supplier can sell the restaurant excellent knives that are easy to clean, and that helps, but the knives are not graded; the kitchen is. Software in a GxP environment is the same. The accurate claims are that it is built for GxP environments, that it supports validated processes, and that it gives the operation the records it needs to show how work was done.

Where an individual workflow is validated for a specific intended use, that validation belongs to the operation and that use. It does not transfer to a platform as a whole.

The controls a quality team should expect

Whatever the regulatory status of a given document, the controls that make agentic work inspectable are consistent. A quality organization evaluating any agentic system should expect each of these.

• Approved sources and tools. The agent is constrained to the data and tools approved for the task and nothing else.
• Source lineage. Every result keeps its path back to the original record, table, field or note.
• Execution records. Each run is recorded with an owner: the sources used, the evidence assembled, the uncertainty surfaced, the draft prepared, the review, and the approval.
• Surfaced uncertainty. Missing or conflicting information is shown rather than papered over, so an unsupported conclusion does not pass silently.
• Access control. Who can run, change, and approve each workflow is defined and enforced.
• Expert approval. A qualified person reviews and approves the result as a defined step, and that approval is recorded.
• Controlled change. Changes to models, prompts, tools, and workflow steps are versioned and assessed before they are used.

These are the controls Agentic Solutions are built around. The customer's systems remain the source of truth, and the execution record is available for the quality team to inspect.

Audit trails and execution records

21 CFR 11.10(e) requires secure, computer-generated, time-stamped audit trails of operator entries and actions that create, modify, or delete electronic records. An audit trail answers "who changed this record, and when."

Agentic work needs a second kind of record. An execution record answers "how was this result produced": which sources the agent was allowed to use, what it retrieved, which rules it applied, what it could not find, what it drafted, and who approved it. The two records work together. The audit trail protects the integrity of the record; the execution record makes the reasoning behind a result inspectable.

Human in the loop versus expert approval

"Human in the loop" is often used to mean that a person is somewhere nearby. That is not enough in GMP work. A signature on a result the reviewer could not realistically check is not a review.

Expert approval is more specific. The workflow defines the point at which a qualified person must review, what they are shown, what they are approving, and what happens if they reject it. The reviewer sees the draft beside its evidence and the uncertainty the agent surfaced. Their decision is recorded as part of the run. The Annex 22 draft points the same way: it asks that the operator's role and responsibilities be described as part of the intended use.

The value does not stop at approval. When agents take on the assembly, the reviewer's time goes into the questions that need expertise: whether an explanation holds, what the finding means for the process, and what to do next.

Validating and changing an agentic workflow

Validation principles do not change because a workflow uses AI. GAMP 5 Second Edition (2022) emphasizes critical thinking and a risk-based approach scaled to the intended use, and ISPE's AI guide extends that to AI-enabled systems. For an agentic workflow, a risk-based approach usually means:

1. Define the intended use and whether the application is critical in the Annex 22 sense.
2. Identify the risks: wrong source, missed evidence, incorrect rule, unsupported conclusion.
3. Test the workflow on representative cases, including cases where evidence is missing, and confirm the uncertainty is surfaced.
4. Confirm the approval step, the access controls and the execution record work as intended.
5. Put changes to models, prompts, tools, and steps under change control, with an impact assessment before release.

The Annex 22 draft expects that a retrained model is introduced only as a controlled change. The same discipline applies to a new model version or a revised prompt in an agentic workflow.

Where to start

Most operations start with a job where the assembly work is heavy and the decision is clearly human: digitizing and reviewing records, assembling evidence for investigations, or characterizing raw material variability. These are useful, bounded, and easy to inspect. In an ideal design, each job builds context that the next job can reuse, because every solution runs on the same platform and Context Layer.