RAG Explained for Beginners: How AI Assistants Stop Making Things Up

aashna mahajan Posted on May 31 • Originally published at dev.to RAG Explained for Beginners: How AI Assistants Stop Making Things Up # rag # ai # software # mcp I once submitted an essay with three citations that I hadn't personally verified. The AI had suggested them, and they sounded right. None of them existed. That's not a quirk or a bug — it's exactly how LLMs work. And once you understand why, a technique called RAG starts to make a lot of sense. AI assistants are remarkably good at sounding right. The model isn't lying — it's doing its best with what it knows. The problem is that what it knows has limits, and it doesn't always know where those limits are. Ask one about a recent event, a niche regulation, or anything from a source it's never seen — and it fills the gap anyway. Confidently. That's the gap RAG was built to close. Once you understand how it works, you'll have a much clearer picture of why some AI tools are genuinely reliable and others are just very convincing guessers. Here's what's actually going on. First, What's the Problem? Large language models (LLMs)—the technology powering AI assistants like ChatGPT and Claude—are trained on vast amounts of data from across the internet. That training gives them a remarkable ability to reason, summarize, and generate content. But it also comes with some real limitations: They have a knowledge cutoff. An LLM trained last year doesn't know what happened last month. They can hallucinate. When they don't know something, they don't say "I don't know"—they generate a confident-sounding answer anyway. Wrong facts, fake statistics, invented sources. All delivered with a straight face. They don't know your specific sources. Think of a software engineer asking an AI assistant about their company's internal API documentation, deployment runbooks, or architecture decisions. None of that is in the training data. The model has never seen it — and it will still try to answer. The model isn't lying — it's generatin