AI doesn't fail because the model is bad. It fails because there's nothing underneath it

Norbert Rosenwinkel Posted on May 31 AI doesn't fail because the model is bad. It fails because there's nothing underneath it # dotnet # ai # programming # eventsourcing There's a question every system runs into the moment it goes to production and starts doing real things: what exactly happened, in what order, against what data — and can you prove it? AI is just making that question very loud right now. Picture the case that gets more likely with every tool-using agent: a support agent — not a human, an LLM with tool access — cancels a subscription, issues a refund, fires off three follow-up emails. The next day the customer says: I never cancelled. Now answer the question above. In most codebases the honest answer is: you can see the current state of the database (subscription cancelled), but not the path that got it there. A few log lines the next refactor will overwrite. No reliable record of which actor acted on behalf of which customer. And undoing it means hand-writing a correction and hoping you catch every side effect. That's not a model problem. GPT wasn't "wrong." The problem sits one layer down. The AI part is now the easy part Two years ago the model was the hard part. Today you wire up an agent that calls tools, makes plans, and takes actions in ten minutes. The demo looks fantastic — and that's exactly the trap. A demo doesn't move anything real. The moment something touches real state in production , the problems no better prompt will solve show up: State : what was the situation when the decision was made? A CRUD table only knows now . History : which steps led to the outcome? Without a record — gone. Attribution : who or what acted, and authorized by what? Reversibility : a wrong action — how do you take it back cleanly ? Trust : can someone quietly rewrite the record afterwards? This isn't actually an AI problem And here's the part that matters to me more than all the agent hype: none of this is new, and none of it is AI-specific. A webhoo