At least once a month, a prospective client tells us they need to fine tune a model on their company data. When we ask what problem they're solving, the answer is almost always some version of: 'We want the AI to know our documentation, our policies, our products.' That's a retrieval problem, not a fine tuning problem. And the difference is worth tens of thousands of dollars and months of schedule.
What each one actually does
Retrieval augmented generation (RAG) gives a model access to your knowledge at answer time. Your documents are indexed in a vector database. When a question arrives, the most relevant passages are retrieved and handed to the model as context. The model 'knows' your content the way a sharp consultant knows the binder you just handed them. And it can cite exactly where each answer came from. Fine tuning, by contrast, changes the model's weights. It's for teaching behavior, not facts: a specific output format, a house style, a specialized task the base model performs poorly. Fine tuning on your knowledge base is a poor way to make a model factual about your business. The knowledge blurs together, goes stale the day your docs change, and can't cite sources.
The decision test we use
Ask one question: when your content changes, should the system's answers change too? If yes. Policies, product docs, prices, procedures. You want RAG, because retrieval reflects updates the moment they're indexed. A fine tuned model would need retraining every time your knowledge moves. Reach for fine tuning only when you can name a behavior the base model gets wrong after serious prompt engineering: a strict output schema it keeps breaking, domain shorthand it misreads, a tone it can't hold. Even then, we usually run RAG underneath and fine tune on top.
What this means for your budget
A production RAG pilot. One process, evaluation harness, human in the loop controls. Typically ships in four to eight weeks. Fine tuning adds data preparation, training runs, evaluation, and a retraining pipeline you'll maintain forever. That's justified when behavior is genuinely the problem. It's an expensive detour when what you actually needed was a good retrieval index. The honest summary: start with RAG, measure, and let the gaps. Not the hype. Tell you whether fine tuning earns its cost.