What Happened
During GPT-5's launch livestream, OpenAI presented benchmark bar charts whose heights did not match the numbers they represented. In one, GPT-5's 52.8 percent score was drawn nearly twice as tall as o3's 69.1 percent, and other bars with very different scores were shown at identical heights. Observers noted the irony of botched visuals during a segment touting reduced hallucination, and speculated the charts may have been AI-generated and unchecked.
Impact
The misleading charts undercut OpenAI's own accuracy messaging and drove widespread criticism of the rollout; Sam Altman publicly called it a 'mega chart screwup.' The reputational stumble compounded other launch-day complaints about model behavior and the new router.
How to Prevent This
- Human-verify every benchmark visual against source numbers before public presentation
- Never publish AI-generated charts without a proofread of axes, scales, and bar heights
- Standardize chart tooling that binds bar geometry directly to underlying data values
- Add a marketing/comms review gate for launch materials making quantitative claims
- Publish machine-readable benchmark data so third parties can independently check charts