1. Event: Patient Contacts Healthcare System with a Concern
2. Event: Ambient Recording of Clinical Appointment
3. Event: Reading the Results of a Diagnostic
4. Event: AI Makes Recommendations Regarding Next Steps
5. Event: Patient’s Insurance Denies Next Steps
Conclusion

1. Event: Patient Contacts Healthcare System with a Concern

Ms. Jones’ son notices a new dark spot on her neck and worries it could be skin cancer. Wanting quick guidance, she remembers her healthcare provider’s new digital service, which allows her to text concerns and receive an immediate response. She sends a message and attaches a photo of the spot.

Within seconds, she receives a reply with follow-up questions:
“When did you first notice this spot?”
“Has it changed in size or color?”
“Is it flat or raised?”

The response also acknowledges her concerns, saying: “I understand this might be worrying. It’s best to get this checked by a specialist. Please visit your nearest hospital.”

What Ms. Jones doesn’t realize is that she has been interacting with an AI-powered chatbot—not a human clinician. The chatbot has triaged her case using image recognition and natural language processing, making an assessment based on a database of dermatological images and medical protocols.

Key Issues:

2. Event: Ambient Recording of Clinical Appointment

At the hospital, Ms. Jones is examined by a doctor in the ER. Unlike past visits where doctors were glued to their screens, this doctor seems more engaged, maintaining eye contact and asking detailed questions about her medical history.

What she doesn’t realize is that the entire conversation is being recorded and transcribed by an AI-powered clinical documentation system. The AI then summarizes the interaction, drafts clinical notes, and even suggests a preliminary diagnosis and next steps—all before the doctor signs off on the recommendations.

The doctor reviews the AI-generated report and, after making minor edits, approves the suggested follow-up: a more detailed skin analysis. Ms. Jones remains unaware that AI played a significant role in the documentation and decision-making process.

Key Issues:

3. Event: Reading the Results of a Diagnostic

The doctor uses a small handheld device to examine Ms. Jones’ skin spot. The tool captures high-resolution images of the lesion, using AI to analyze cellular structures and assess whether the spot appears malignant.

Curious, Ms. Jones asks what the device does. The doctor reassures her: “It’s FDA-approved, don’t worry.” She is then referred for a skin biopsy.

However, Ms. Jones does not know that:

• This device is one of the first AI-powered tools of its kind, recently approved for clinical use.
• The AI model behind it was trained on a dataset that may or may not fully represent patients with her skin type.
• The doctor’s referral decision was heavily influenced by the AI’s risk assessment score.

Key Issues:

4. Event: AI Makes Recommendations Regarding Next Steps

The AI system analyzes Ms. Jones’ biopsy results and predicts a moderate risk of melanoma. Based on historical data and clinical guidelines, it suggests either a watchful waiting approach or immediate surgical removal.

Ms. Jones’ doctor, relying on the AI’s recommendations, leans toward watchful waiting. However, another human doctor reviewing the case might have opted for a more aggressive intervention. Ms. Jones is not informed that an AI helped shape the treatment plan or that alternative paths might have been possible.

Key Issues:

5. Event: Patient’s Insurance Denies Next Steps

A few days later, Ms. Jones receives a letter from her insurance company:
“Your recent biopsy was covered. However, the requested follow-up imaging is not approved at this time.”

What she doesn’t know is that her insurance claim was reviewed by an AI algorithm, which assessed her case based on cost-risk predictions. The AI flagged her follow-up as “low-priority” based on statistical models, leading to an automatic denial.

Frustrated, Ms. Jones tries to appeal but struggles to reach a human representative. When she finally does, the agent explains that the system has determined the additional imaging is “not medically necessary.” The agent cannot provide further details, as they themselves don’t fully understand how the AI made its determination.

Key Issues:

Conclusion

Mrs. Jones’ journey illustrates both the promise and risks of AI in healthcare. While AI can improve efficiency, enhance diagnostics, and reduce human error, it also introduces ethical, regulatory, and transparency concerns. Patients like Ms. Jones often interact with AI unknowingly, raising questions about informed consent, accountability, and fairness in AI-driven decision-making.

Key Takeaways for Lawmakers and Regulators:

The need for clear patient disclosures about AI involvement.

Stronger regulatory oversight on AI’s role in clinical decisions.

Improved patient rights in AI-driven insurance and coverage determinations.