RAG chatbot vs traditional chatbot

A RAG (retrieval-augmented generation) chatbot retrieves relevant chunks from your knowledge base at answer time and lets an LLM compose the reply. A traditional chatbot walks a hard-coded decision tree. RAG scales; decision trees don't.

Capability	RAG chatbot	Traditional chatbot
Answer source	Live retrieval from your docs	Hand-authored branches
Handles unseen questions	Yes — if the answer is in the corpus	Falls to a generic fallback
Update effort	Re-index changed docs	Rewrite the flow
Cost model	LLM tokens + embeddings	Build-time authoring

Frequently asked questions

Does RAG hallucinate?

Less than a naked LLM. RAG grounds the model in retrieved chunks and instructs it to only answer from those chunks. Hallucinations still happen when the corpus is thin or contradictory — good chunking and eval loops matter.