The founder of PocketOS, a B2B company that handles reservations and payments for car rental businesses, has bemoaned the "systemic failures" that saw an AI agent decide to solve a problem by straight-up deleting his company's production database, and the backups.
I'll say at the outset that this story has a happy ending, thanks to the involvement of cloud infrastructure provider Railway, but is nevertheless yet another example of why over-reliance on AI is a very bad thing indeed.
