Clinical / Method Note
Evaluating Governed Agents
Evaluate an agent in the workflow it will use, with its authority and review responsibilities defined.
Sources checked
Purpose
Start with the task, the people affected and the decisions an agent may influence. The FDA and EMA principles emphasize context of use, proportionate assessment, human and AI interaction, clear limitations and continued monitoring. FDA and EMA AI principles.
What Your Team Reviews
What may the agent do?
Name the permitted inputs, available actions, prohibited actions and points requiring human approval. Include the expected response when a source is unavailable or a request exceeds the agreed scope.
What do realistic tests reveal?
Assess correct outcomes and important failures, including unsupported findings, missed issues and incomplete handoffs. Record the configuration and test conditions so a reviewer can understand what was assessed.
How is the human decision made?
Confirm that reviewers can inspect sources, challenge a proposal and record the next action. Agree who responds to failures and which changes require another assessment.
Outputs to Agree
A scope and responsibility record, evaluation results, unresolved limitations, a named acceptance decision and an operating review plan. Include triggers for reassessment after changes to the model, sources, tools or intended use. NIST’s AI RMF describes testing before deployment and evaluation during operation. NIST AI RMF 1.0.
Confirm the deliverables and acceptance responsibilities in the engagement scope.
Limits
Passing a selected test set does not prove every future action will be correct. A named reviewer is one control within a broader process. Qualification, validation and release decisions remain within your organization’s applicable procedures.
Sources
- FDA and EMA: Guiding Principles of Good AI Practice in Drug Development
January 2026 joint principles, PDF. Guiding principles do not constitute approval of a platform.
- NIST: Artificial Intelligence Risk Management Framework 1.0
NIST AI 100-1, January 2023. A voluntary framework; version 1.0 is under revision.
Your next step
Define the task and its review.
Scope one agent workflow, its permitted actions and the people responsible for acceptance.
Discuss an Agent Workflow Explore Governed Agent Workflows