What Happened
An Associated Press investigation found that OpenAI's Whisper speech-to-text model routinely fabricates content, inserting invented medications like 'hyperactivated antibiotics,' racial commentary, and violent statements into transcripts. Researchers found hallucinations in as many as eight of ten transcriptions in one study, with roughly 40% of fabrications judged harmful. Despite OpenAI's warnings against high-risk use, tens of thousands of clinicians across dozens of health systems used Whisper-based tools that deleted the original audio, making errors impossible to verify.
Impact
Fabricated text risked entering official medical records for millions of patient visits with no source audio retained, raising patient-safety and liability concerns industry-wide.
How to Prevent This
- Retain original audio recordings so transcripts can be audited against source
- Require clinician review and sign-off on AI transcripts before entry into medical records
- Prohibit deployment of models in high-risk domains their maker explicitly warns against
- Benchmark hallucination rates on domain-specific audio before procurement
- Add automated anomaly detection for content absent from the audio, especially medications and diagnoses