What Happened
When a session-management bug began logging Cursor users out as they switched machines, the company's front-line AI support agent 'Sam' confidently explained that subscriptions were restricted to a single device as an intentional security policy. No such policy existed; the bot fabricated it to rationalize the bug. The invented rule spread across Reddit and Hacker News before co-founder Michael Truell publicly apologized and confirmed the response came from an AI bot.
Impact
Developers who depend on multi-device workflows canceled subscriptions based on the fictitious policy, and Cursor had to issue refunds, a public apology, and begin labeling all AI-generated support replies.
How to Prevent This
- Ground support bots strictly in a verified policy knowledge base and block answers outside it
- Train agents to escalate to humans when no documented policy matches the question
- Clearly label AI-generated support responses so users can weigh their reliability
- Route policy-related and account-impacting answers through human review before sending
- Monitor community channels for viral misinformation originating from support interactions