The American Medical Association has introduced a governance framework to ensure the safe and effective integration of agentic AI in healthcare. This framework emphasizes that agentic AI systems must be evidence-based and maintain high performance to execute clinical tasks, addressing the need for accountability and oversight in their deployment.
Four key questions will steer the AMA’s strategy for managing AI agents in clinical tasks, addressing permitted actions and accountability. These focus on which procedures AI tools should handle and under what circumstances, alongside identifying who bears responsibility for their outcomes.
Governance Framework
The framework offers guidance on how the AMA plans to regulate agentic AI as it expands in medical settings, such as ambient documentation, diagnostics, and patient monitoring. Agentic AI can function independently, executing detailed tasks that generative AI cannot manage alone.
Because these tools perform multi-step processes without human review, an incorrect or biased judgment might cause significant patient harm before a clinician can intervene. The AMA’s new framework will support collaboration with other groups to ensure AI agents are deployed safely and responsibly in areas affecting patient care.
Industry Response
Industry associations are exploring ways to mitigate agentic AI risks, highlighting physicians’ critical role in its development and implementation. The AMA’s framework outlines four questions it will evaluate when reviewing clinical AI agents: What clinical role is the AI performing? What confirms its capability, and what occurs if it fails? What actions should the AI be authorized to take, and under what conditions? Who is liable for the risks and outcomes?
According to the framework, “As AI capabilities advance, evidence, authority, oversight and accountability must remain aligned in service of better patient care.” The new guidance comes amid an explosion of agentic AI tools in healthcare, with about 28% of surveyed healthcare leaders saying their organization has already deployed agentic AI tools, and 44% reporting conducting pilots or proof-of-concept projects.
Most respondents anticipate agentic AI will significantly reshape clinical workflows. Yet 72% indicate these tools are sometimes introduced without IT approval, revealing a supervision gap. Insufficient oversight could pose major hurdles for healthcare leaders, with over half citing compliance breaches and unauthorized access to health data as primary concerns.
According to legal and consulting firm Manatt, 47 states introduced over 250 AI bills impacting healthcare in 2025. As the use of agentic AI in healthcare continues to grow, the AMA’s governance framework aims to ensure that these tools are used in a way that prioritizes patient care and safety.
