Making AI-Generated Code Fail Gracefully

Nick Valencia Posted on May 31 Making AI-Generated Code Fail Gracefully # python # ai # ux # errors Making AI-Generated Code Fail Gracefully If your app generates code with an LLM and executes it, you already know the dirty secret: it fails a lot. Not catastrophically — just wrong method names, bad assumptions about state, off-by-one stuff. The kind of errors a human would fix in 10 seconds. The question is what your user sees when that happens. The Problem Version 1 of my app showed users raw Python tracebacks when a generated script failed. Something like: Script execution failed: Traceback (most recent call last): File "", line 3, in items = timeline.GetItemsInTrack("video", 1) AttributeError: 'Timeline' object has no attribute 'GetItemsInTrack' The LLM got the method name wrong — it's GetItemListInTrack, not GetItemsInTrack. An easy fix. But my users are video editors, not Python developers. That traceback means nothing to them except "it broke." The Fix: Silent Self-Correction Instead of showing the error, I send it back to the LLM with context: "The previous script failed with: AttributeError: 'Timeline' object has no attribute 'GetItemsInTrack'. Generate a corrected script." The LLM sees its own mistake, fixes the method name, and the corrected script runs. The user sees: "Hmm, let me try that a different way..." Then 2 seconds later: "✓ Set opacity to 50% on 12 clips" They never see the error. It just works on the second attempt. The Implementation (High Level) The retry loop is simple: LLM generates a script Script fails (validation or execution) Send the error message back to the LLM as a new prompt LLM generates a corrected script Try again (up to 2 retries) If all retries fail, show a friendly message suggesting simpler commands The key insight: LLMs are surprisingly good at fixing their own mistakes when you show them the exact error. The success rate on retry is much higher than the first attempt because the error message narrows the solution space. Fr