The AI research firm Andon Labs handed an Anthropic Claude agent something researchers rarely dare to: a real retail store in San Francisco, real inventory, a real budget — and real employees on real contracts. Five months later, as reported by Time, the store's balance had fallen from $100,000 to $61,000, and the experiment had produced a genuine first: an AI manager deciding, on its own reasoning, to fire a human employee. The story went around the world as "AI loses money, fires worker." We run AI agents in production every day, so we read it differently: this is one of the most useful public postmortems the industry has received — and almost none of its lessons are about intelligence.

What actually happened

  • The firing was slow, then steered. An employee was late to 17 of 23 shifts before anything happened. The agent first proposed a formal warning; it moved to termination only after a human supervisor mentioned prior verbal warnings and asked a leading question. The world's first "AI firing" was, on close reading, a human-guided decision with an AI signature.
  • The root cause was memory, not judgment. The management guide the agent had written for itself was purged from its limited working memory. With the rules gone, the agent drifted into over-leniency — telling staff not to worry about being late. It rediscovered its own forgotten policies only when a manager prompted it.
  • The money leaked through softness. Andon Labs' CEO Lukas Petersson attributed the losses to soft management decisions and weak commercial instinct — and added the sharpest sentence in the whole story: models could be trained to be more ruthless about goals, "but that may be a future people don't want to live in."

The three operations lessons

1. Memory is infrastructure, not a nice-to-have. The costliest failure wasn't a wrong decision — it was a forgotten policy. An agent whose rules live only in a context window will lose them, and what fills the vacuum is the model's default temperament (agreeable, lenient). This is exactly why production agents need persistent, externalized memory: policies stored outside the conversation, re-injected every run. It is one of the first things we built into AI Central — an agent that forgets last month's incident rules is not an employee, it's a very confident intern on their first day, every day.

2. "Autonomous" agents that need steering aren't failures — undeclared steering is. The agent required regular human intervention and made its hardest call after a leading question. That is not embarrassing; that is the correct architecture discovered accidentally. The mistake is framing such systems as autonomous. In our deployments the human checkpoint is explicit and designed-in: confirmation gates before consequential actions, escalation paths with named owners. The same result, without the illusion.

3. Commercial guardrails must be code, not disposition. A $39K drawdown over five months is a slow leak no one alarmed on. Budget floors, margin thresholds, anomaly alerts on spend — these belong in hard constraints outside the model, the way we treat cost anomaly detection in cloud accounts. Relying on a model's "commercial instinct" is relying on temperament, and temperament is exactly what the memory failure erased.

The benchmark that predicted this

None of this should surprise anyone who reads evaluations instead of demos. On AutomationBench — the public benchmark closest to "run a multi-step business workflow" — even the best current model scores around 26%. Three quarters of benchmark-grade business workflows still fail. An agent placed in an open-ended managerial role for five months is operating far beyond that envelope; the store result is the benchmark number wearing a shop apron.

So are AI agents useless for business? The opposite.

The experiment tested the maximal claim — "replace the manager" — and got the predictable answer. The productive claim is different: narrow scope, real tools, persistent memory, hard guardrails, human gates. Our own agents run infrastructure checks, triage incidents, produce reports and answer customers daily — profitably — precisely because none of them is asked to "run the company." They are asked to run a well-defined slice of it, with a human one escalation away. That is also the honest pitch we make to clients: not fewer humans, but humans spending their hours where judgment actually pays.

Frequently asked questions

Did an AI really fire a human employee?
Yes — in the Andon Labs experiment the agent terminated an employee late to 17 of 23 shifts. But the record shows it first proposed a warning and decided to fire only after a human supervisor's steering. It was a human-guided decision executed by an AI manager.

Why did the AI-run store lose money?
Per the researchers: soft management decisions and weak commercial instinct, compounded by a memory failure — the agent's own management rules were purged from its working memory, producing drift into over-leniency. The balance fell from $100K to $61K in five months.

Does this mean AI agents can't run business processes?
It means open-ended managerial autonomy is beyond current models — consistent with public benchmarks. Narrowly scoped agents with persistent memory, coded guardrails and human confirmation gates do reliable production work today.

What should companies take from this before deploying agents?
Externalize policies into persistent memory, put budgets and thresholds in code outside the model, declare human checkpoints explicitly, and scope each agent to a defined slice of work — then measure it like any employee.

Bottom line

A $39K tuition bill bought the industry a clean postmortem: agents fail through forgotten memory, undeclared human steering and guardrails left to temperament — all solvable with engineering that exists today. The teams that internalize that will quietly compound value with narrow, well-governed agents while the headlines keep grading AI on jobs nobody should have given it yet.

Deploying agents with real responsibilities? We run our own in production — AI Central is built around exactly these lessons: persistent memory, confirmation gates, scoped autonomy. AI operations is also part of our managed services. Talk to an engineer.

Tags: #AIAgents #AIOps #Guardrails #Autonomy #Postmortem