Workshop B

When the Algorithm Disagrees: Teaching Clinical Judgment in the Age of AI

John R. Weinstein, PhD, MS

Physician Assistant Program, Chobanian & Avedisian School of Medicine

Room L203

Artificial intelligence (AI) is increasingly embedded in health care across settings, including predictive analytics, medical imaging interpretation, documentation, and clinical support. While algorithms rival human clinicians, their utility is constrained by several limitations, including a lack of explainability and algorithmic bias. Beyond clinical accuracy, AI raises urgent regulatory concerns.

Despite their complexity and ubiquity, most trainees and practicing clinicians receive little formal instruction in interpreting, evaluating, or integrating AI outputs. Consequently, AI increases clinical performance only slightly and not to the level of AI alone, highlighting that the central challenge is not merely tool accuracy but effective human–AI integration.

Even if the long-term impact of AI remains uncertain, these tools are already shaping care. This workshop addresses that gap by focusing on professional judgment in the presence of algorithmic input. Under a decision-theoretic lens, AI integration is a problem of action under uncertainty; when tools are imperfect but unavoidable, failing to prepare learners to engage with them is not a neutral omission, but a higher-risk strategy. Participants will engage with practical examples and leave with a structured, transferable approach to begin to teach this skill.

Target Audience:

Health professions educators across disciplines, including medicine, residency programs, and genetic counseling. No prior AI expertise required.

Learning Objectives:

Upon completion of this workshop, learners will be able to:

  • Analyze how AI outputs influence decision-making under uncertainty
  • Identify common pitfalls in human–AI interaction
  • Apply a structured framework to teach learners how to evaluate and integrate AI tools critically
  • Design a brief, case-based teaching activity incorporating AI into their own program context
Session Outline:
  • Context (5 minutes): Overview of AI use across health settings and limited formal training.
  • Case Activity (20 minutes): In small groups, participants work through realistic cases drawn from clinical AI use. Groups discuss how (or whether) the AI influences their reasoning.
  • Debrief (10 minutes): Facilitated discussion highlighting variability in decisions, trust calibration, bias, and consequences of error.
  • Educational Implications (10 minutes): Brief review of challenges in clinician–AI integration and a decision-theoretic framing that not preparing learners to engage with AI carries risk.
  • Instructional Template & Example (10 minutes): Introduction of a teaching template. Brief illustration of how it was applied longitudinally in the PA program.
  • Group Application Activity (15 minutes): Participants design a teaching activity using the template.
  • Share-Out & Closing (5 minutes): Selected groups share their activity. Workshop concludes with a practical framework for teaching decision-making with AI.