When the LLM Refuses: A Fallback Chain That Salvages Most Refusals

sm1ck Posted on May 31 • Originally published at honeychat.bot When the LLM Refuses: A Fallback Chain That Salvages Most Refusals # llm # python # ai HoneyChat Engineering (2 Part Series) 1 Inworld TTS Paralinguistic Tags Don't Work — Here's What Does 2 When the LLM Refuses: A Fallback Chain That Salvages Most Refusals Every production LLM app eats false-positive refusals. A user asks something perfectly fine, the safety filter trips, the model emits two sentences of "I can't help with that," and your UI shows a wall. Do that a few times and the user leaves. We've measured this on HoneyChat — Telegram-native AI companion, ~300 DAU, 17 languages. Across a normal day, somewhere between 2% and 8% of model calls land in a refusal or finish_reason="content_filter" state. Most of those are not actually problematic content — they're the model being twitchy about edge phrasing, polysemous words, or roleplay framing. The pattern below recovers about 70% of them. HoneyChat LLM routing at a glance ( core/llm.py , plan-gated via OpenRouter): Tier(s) Pace Primary model (OpenRouter slug) free / basic / premium natural qwen/qwen3-235b-a22b-2507 free / basic / premium instant / explicit deepseek/deepseek-v4-flash vip / elite any google/gemini-3.1-flash-lite-preview Emergency content_filter fallback chain ( GEMINI_CONTENT_FILTER_FALLBACK_CHAIN ): x-ai/grok-4.20 → an open roleplay-tuned model. The rescue chain below is what feeds traffic into that fallback only when it's actually needed. Three steps, in order of cost. Step 0: Don't trigger it in the first place Free, and where most posts on this topic stop. Two things: Tighten the safety knobs the provider exposes. For Gemini via OpenRouter, that's safety_settings in the extra body. Default is BLOCK_MEDIUM_AND_ABOVE on four categories; for roleplay/chat traffic we lower them via a helper called _maybe_inject_gemini_safety_off() : extra_body = { " safety_settings " : [ { " category " : " HARM_CATEGORY_HARASSMENT " , " thr