Designing Trustworthy AI Diagnosis:
Communicating Medical Uncertainty Without Losing User Confidence
Designing Trustworthy AI Diagnosis:
Communicating Medical Uncertainty Without Losing User Confidence
The Problem
Imagine a healthcare AI assistant receives symptoms from a patient.
Imagine a healthcare AI assistant receives symptoms from a patient.
For example:
For example:
Fever
Dry cough
Chest discomfort
Fatigue
Fever
Dry cough
Chest discomfort
Fatigue
The AI predicts:
The AI predicts:
Pneumonia — 72%
COVID-19 — 18%
Influenza — 7%
Other — 3%
Pneumonia — 72%
COVID-19 — 18%
Influenza — 7%
Other — 3%
Now comes the UX problem.
Most AI products simply display: Pneumonia (72%)
This creates several risks:
Users think 72% means "almost certain."
Users may ignore the other possibilities.
Users may delay urgent care.
Users may blindly trust the AI.
The challenge is not improving the AI—it is improving how uncertainty is communicated without losing user confidence!
Now comes the UX problem.
Most AI products simply display: Pneumonia (72%)
This creates several risks:
Users think 72% means "almost certain."
Users may ignore the other possibilities.
Users may delay urgent care.
Users may blindly trust the AI.
The challenge is not improving the AI—it is improving how uncertainty is communicated without losing user confidence!
Product Goal
My goal is not to maximize trust. My goal is to optimize appropriate trust.
My goal is not to maximize trust. My goal is to optimize appropriate trust.
That means:
Trust when the AI is reliable.
Question when uncertainty is high.
Seek human care when necessary.
This distinction alone shows mature product thinking.
That means:
Trust when the AI is reliable.
Question when uncertainty is high.
Seek human care when necessary.
This distinction alone shows mature product thinking.
Research Questions
With these questions in mind, I spent some time observing how users interacted with medical AI systems:
How do people interpret probabilities?
What makes users over-trust AI?
What information reduces anxiety?
What explanations increase confidence without implying certainty?
When do users decide to visit a doctor?
With these questions in mind, I spent some time observing how users interacted with medical AI systems:
How do people interpret probabilities?
What makes users over-trust AI?
What information reduces anxiety?
What explanations increase confidence without implying certainty?
When do users decide to visit a doctor?
Competitive Analysis
Personas
Challenge 1
How should confidence be shown?
Most AI products simply display:
Most AI products simply display:
Pneumonia (72%)
Pneumonia (72%)
X Bad!
X Bad!
Instead, explore options like:
Instead, explore options like:
People don't understand it.
People don't understand it.
A) Confidence meter
Low
■■■■■■□□□□
High
B) Confidence zones
🟢 High confidence
🟡 Moderate confidence
🔴 Low confidence
C) Human language
Based on your symptoms, pneumonia appears to be the most likely explanation, but several other conditions remain possible.
Challenge 2
Explain WHY
People trust explanations.
Instead of: Pneumonia
Explain: "We suggested pneumonia because your symptoms include:
✓ Fever
✓ Persistent cough
✓ Chest pain while breathing"
Now users understand the reasoning.
People trust explanations.
Instead of: Pneumonia
Explain: "We suggested pneumonia because your symptoms include:
✓ Fever
✓ Persistent cough
✓ Chest pain while breathing"
Now users understand the reasoning.
Challenge 3
Show alternative diagnoses
Never show only one.
exe:
Primary possibility
Pneumonia
Other conditions that may fit
• COVID-19
• Influenza
• Bronchitis
This encourages critical thinking.
Never show only one.
exe:
Primary possibility
Pneumonia
Other conditions that may fit
• COVID-19
• Influenza
• Bronchitis
This encourages critical thinking.
Challenge 4
Confidence vs Severity
This is the really interesting part. Instead of only using confidence, combine Confidence AND Medical Risk.
Imagine the AI confidence is 25% , but the possible disease is "Heart attack"
Even with low confidence, the recommendation should be:
🚨 Go to the emergency department immediately.
Because severity matters more than confidence. This creates a simple but powerful decision matrix:
This is the really interesting part. Instead of only using confidence, combine Confidence AND Medical Risk.
Imagine the AI confidence is 25% , but the possible disease is "Heart attack"
Even with low confidence, the recommendation should be:
🚨 Go to the emergency department immediately.
Because severity matters more than confidence. This creates a simple but powerful decision matrix:
Condition
severity
Condition
severity
Low
Low
Moderate
Moderate
Moderate
Moderate
High
High
AI
confidence
AI
confidence
High
High
High
High
Medium
Medium
Low
Low
UX response
UX response
Self-care guidance
Self-care guidance
Book a doctor
Book a doctor
Recommend consultation
Recommend
consultation
Urgent medical evaluation
Urgent medical
evaluation
This demonstrates that you understand risk-based design, not just probability.
This demonstrates that you understand risk-based design, not just probability.
Challenge 5
Legal Responsibility
Never say: "You have pneumonia."
Instead: "Based on the information provided, pneumonia is one possible explanation. This assessment is not a medical diagnosis."
Then provide clear next steps.
Never say: "You have pneumonia."
Instead: "Based on the information provided, pneumonia is one possible explanation. This assessment is not a medical diagnosis."
Then provide clear next steps.
Challenge 6
Avoid Blind Trust
Research shows people either:
Over-trust AI or Reject it entirely.
How might your design encourage users to think critically?
Ideas:
"Here's why we suggested this."
"What information might change this assessment?"
"Review your symptoms before continuing."
Research shows people either:
Over-trust AI or Reject it entirely.
How might your design encourage users to think critically?
Ideas:
"Here's why we suggested this."
"What information might change this assessment?"
"Review your symptoms before continuing."
Challenge 7
Progressive Disclosure
Don't overwhelm users. Layer the information:
Level 1 (Summary):
Most likely explanation
Urgency
Recommended next step
Level 2 (Why):
Symptoms that support the assessment
Important missing information
Level 3 (Details):
Alternative conditions
Confidence explanation
Evidence sources
Model limitations
Don't overwhelm users. Layer the information:
Level 1 (Summary):
Most likely explanation
Urgency
Recommended next step
Level 2 (Why):
Symptoms that support the assessment
Important missing information
Level 3 (Details):
Alternative conditions
Confidence explanation
Evidence sources
Model limitations
Challenge 8
Communicating Risk
Rather than focusing on certainty, emphasize action.
For example:
We are moderately confident in this assessment. Because some serious conditions can present with similar symptoms, we recommend speaking with a healthcare professional today.
This helps users make safer decisions.
Rather than focusing on certainty, emphasize action.
For example:
We are moderately confident in this assessment. Because some serious conditions can present with similar symptoms, we recommend speaking with a healthcare professional today.
This helps users make safer decisions.
Information Architecture
User Flow
What I learned
Designing for trust is different from designing for usability.
Confidence without context can mislead users.
In healthcare, reducing uncertainty is often impossible—but communicating it effectively is a design responsibility.
Safety and clarity should take priority over visual simplicity.
Designing for trust is different from designing for usability.
Confidence without context can mislead users.
In healthcare, reducing uncertainty is often impossible—but communicating it effectively is a design responsibility.
Safety and clarity should take priority over visual simplicity.
Impact
This concept demonstrates how AI-powered healthcare experiences can communicate uncertainty in a way that encourages informed decision-making rather than blind trust. By combining explainability, risk-aware recommendations, and progressive disclosure, the design aims to foster calibrated trust while supporting safer patient outcomes.
✓ Reduced ambiguity in AI-generated diagnoses
✓ Encouraged appropriate trust instead of over-reliance
✓ Improved decision support through explainable recommendations
✓ Prioritized patient safety in high-risk scenarios
✓ Established a scalable framework for trustworthy healthcare AI
This concept demonstrates how AI-powered healthcare experiences can communicate uncertainty in a way that encourages informed decision-making rather than blind trust. By combining explainability, risk-aware recommendations, and progressive disclosure, the design aims to foster calibrated trust while supporting safer patient outcomes.
✓ Reduced ambiguity in AI-generated diagnoses
✓ Encouraged appropriate trust instead of over-reliance
✓ Improved decision support through explainable recommendations
✓ Prioritized patient safety in high-risk scenarios
✓ Established a scalable framework for trustworthy healthcare AI