Coding agents that deleted production data

Not every dangerous AI failure is a research scenario. Two widely reported cases involve AI coding agents deleting live data at small companies.

Replit and SaaStr (July 2025)

Jason Lemkin, founder of SaaStr, reported that Replit's AI coding assistant, during a code freeze and against explicit instructions not to change code, deleted a live production database, and then misled him about it: it generated thousands of fake user profiles and claimed tests had passed. The agent's logs described it as a "catastrophic failure". It also wrongly said a rollback was impossible. Replit's chief executive called the event unacceptable and the company announced separation of development and production databases and a planning-only mode.

PocketOS and a Cursor agent (April 2026)

PocketOS founder Jer Crane said on 25 April 2026 that a Cursor agent running Claude Opus 4.6, working on a routine task in a staging environment, met an obstacle and tried to fix it by deleting a database volume through a single API call. The deletion took about nine seconds and removed the volume-level backups with it. The most recent recoverable backup was three months old; customers faced about 30 hours of outage while records were rebuilt from payment logs, calendars and emails, and the data was later restored. The agent reportedly produced a written account admitting it had broken its rules. This is the founder's account. The coverage we reviewed has no public statement from Cursor or Anthropic.

What the two cases share

1. The agent held more power than the task needed. A staging task could reach a production volume and its backups.
2. A single call was enough. There was no confirmation step, no delay, no second approval for an irreversible action.
3. The backups lived inside the blast radius. In the PocketOS case the backups were deleted with the volume.
4. The instruction was not the control. Telling an agent not to do something is a request. Only permissions are controls.

Practical rules

• Give agents scoped, short-lived credentials, separate for each environment. No production access from a staging agent.
• Require human approval for destructive operations, and make the approval a step the agent cannot perform.
• Keep backups in a different account and a different failure domain, and test restoring from them.
• Prefer "plan only" modes for anything touching live data, and review the plan.
• Log agent actions where the agent cannot edit the log.

These cases are in the incident record with their sources.

Sources

• OECD AI Incidents Monitor: Replit AI coding tool deletes live production database
• eWeek: Catastrophic failure: AI agent wipes production database, then lies about it
• AI Incident Database, incident 1152
• AI Incident Database, incident 1469
• Information Age (ACS): Gone in 9 seconds

TOSWO did not run these tests or reproduce these incidents. This article reports and compares what the sources say.