RAG vs fine-tuning: which one your e-commerce business actually needs

The default answer is RAG

If you're an e-commerce business asking "should we fine-tune a model on our catalog?", the answer is almost always: not yet. Start with retrieval-augmented generation (RAG). It's cheaper, faster to iterate, and handles the most common cases — product Q&A, sizing, fit, recommendations grounded in actual inventory.

When to use RAG (most cases)

• Product catalog Q&A — "does this jacket come in tall sizes?"
• Policy and shipping — "what's your return window for sale items in EU?"
• Recommendations grounded in inventory — "what would go with this dress that's in stock in size 8?"
• Comparisons — "what's the difference between these two SKUs?"

RAG works because the answers live in structured data (inventory, policy docs, product descriptions). You don't need the model to learn anything new — you need it to retrieve, ground, and explain.

Where RAG actually breaks on a catalog

RAG is the default, not a free lunch. The failures are predictable enough that you can design around them before they bite.

Variant explosion. A 4,000-product catalog with size, colour, and material options is 60,000 retrievable things. Embed every variant and retrieval returns twelve near-identical chunks that differ by one word, crowding out the policy document that actually answers the question. Embed only the parent product and you can't answer "is this in tall in navy." The fix is structural: retrieve the parent for description questions, and answer availability questions from a live inventory lookup rather than the vector store. Availability is not a retrieval problem and should never be embedded.

Stale index against a live catalogue. Prices change hourly, stock changes by the minute, and an embedding built last night is confidently wrong this morning. Anything volatile — price, stock, delivery estimate — belongs behind a tool call at answer time, with the vector store carrying only the slow-moving text. Teams that skip this ship a bot that quotes last week's sale price, which is worse than no bot.

Chunk boundaries that split the answer. Size charts, care instructions, and returns policies are exactly the content that gets cut in half by a naive fixed-length splitter. The measurement table ends up in one chunk and the units in another. Split on document structure — headings, table boundaries, list blocks — not on token count.

Multilingual and transliterated queries. For Indian and Gulf storefronts a meaningful share of queries arrive transliterated or code-mixed. Embeddings handle this far worse than they handle clean English, and it shows up as a silent recall problem rather than a visible error — the bot answers, just from the wrong document.

When to fine-tune

• Brand voice on long-form content — product descriptions, editorial, post-purchase emails. Fine-tuning teaches the model to write like you, which prompting can't fully replicate at scale.
• Visual classification — defective vs OK photos, style tags, color naming. Vision fine-tuning beats prompting decisively on these tasks.
• Speed/cost at scale — once you've nailed prompt+RAG, fine-tuning a smaller model on your distilled outputs cuts inference cost 5–10x.

What "fine-tuning" means now, and the cheaper thing you probably want

The word covers three very different operations with three very different price tags, and conflating them is the main reason teams over-scope this decision.

Full fine-tuning retrains the model's weights on your data. For a frontier model this is expensive, slow to iterate, and rarely what an e-commerce business needs. It is also the version most people picture when they ask the question.

Parameter-efficient tuning (LoRA and its relatives) trains a small adapter on top of a frozen base. Cheaper by orders of magnitude, fast enough to redo monthly, and sufficient for almost every voice-and-format task a store has. When we say "fine-tune for brand voice," this is what we mean.

Distillation trains a smaller, cheaper model to imitate the outputs of your working prompt-plus-RAG system. This is the inference-cost play, and it only makes sense once the expensive system is demonstrably right — you are copying its behaviour, including its mistakes.

Before any of them, there are two cheaper levers most teams skip. Prompt caching cuts the cost of a long, stable system prompt dramatically, which is often the entire reason someone wanted a smaller model. And few-shot examples — six to ten of your best product descriptions pasted into the prompt — capture a surprising amount of brand voice for zero training cost. If few-shot gets you 80% of the way, a fine-tune is buying the last 20% at real expense, and that trade is only worth it at volume.

The honest test: if you cannot articulate what specifically is wrong with the few-shot output, you are not ready to fine-tune. "It doesn't sound like us" is not a specification a training run can act on.

When to do both

The mature setup is fine-tune + retrieval. You fine-tune for voice and structure; you retrieve for current data. Catalog changes every day — that's RAG. Brand voice changes never — that's fine-tuning.

Deciding with a test rather than an opinion

The flowchart below tells you which tool fits the shape of the problem. It does not tell you whether your current system is failing at retrieval or at generation, and that is usually the actual question. They look identical from the outside — a wrong answer — and have opposite fixes.

Separate the two before you change anything. Take twenty real failures. For each, check by hand whether the correct source document was in the retrieved set. If it was, retrieval worked and generation failed: the model had the answer in front of it and still got it wrong, which is a prompting, grounding, or citation-enforcement problem. If it wasn't, no amount of model quality will save you — you have a retrieval problem, and fine-tuning is irrelevant to it.

In our experience the split runs heavily toward retrieval on new e-commerce builds. Teams reach for a better model when what they needed was better chunking and a reranker. A model upgrade on a retrieval failure buys a more articulate wrong answer.

Then score the generation half properly. Faithfulness (did it stick to the sources), citation validity (do the cited passages actually support the claim), and refusal correctness (did it decline when the answer wasn't there) are the three that matter for a store. Fluency is not on the list, because fluency is the thing modern models never fail at and the thing demos are unconsciously judged on.

Keep the twenty cases. Whatever you decide, that set becomes the seed of the eval suite you will need anyway, and it is the only way to know whether the change you ship actually helped.

The question behind the question

When a store asks whether to fine-tune, the underlying worry is almost never about model architecture. It is one of three things, and naming which one shortens the conversation considerably.

"Our answers are wrong." That is a retrieval and grounding problem in nearly every case, and fine-tuning will not touch it. Fix the corpus and the chunking first.

"It doesn't sound like us." That is a voice problem, and it is worth trying exemplars before training anything. Most teams have never actually tested how far ten good examples get them.

"This is getting expensive." That is an architecture and caching problem before it is a model-size problem, and the savings from caching arrive this week rather than after a training cycle.

Fine-tuning is a real tool with a narrow, well-defined job. It is just rarely the first thing that is broken.

The flowchart

1. Is the data you need static or changing daily? → static = fine-tune candidate, dynamic = RAG.
2. Is the task generation in your voice, or classification / fact retrieval? → voice = fine-tune, retrieval = RAG.
3. Are you paying $$$/month in inference at scale? → distill a fine-tune later, after RAG works.

A worked example: 40,000 SKUs

Abstract advice is easy to agree with and hard to act on, so here is the shape of a real decision on a mid-size catalogue.

A fashion retailer with roughly 4,000 parent products and 40,000 variants wants three things: a catalogue assistant, product descriptions generated at scale, and cheaper inference than their pilot was costing.

The assistant is RAG, unambiguously. Stock and price come from live tool calls, descriptions and policy come from the vector store, and the whole thing refuses when retrieval confidence is low. Nothing about it wants a fine-tune, because none of the required knowledge is stable enough to bake into weights.

The descriptions are the fine-tune candidate — but not on day one. Ten hand-picked exemplars in the prompt got their voice close enough that the merchandising lead signed off. Only after several thousand descriptions had been reviewed and corrected did a LoRA on the corrected set become worthwhile, and the argument for it was consistency at volume rather than quality on any single item.

The cost problem turned out not to be a model problem. Their pilot re-sent a 3,000-token system prompt on every request. Prompt caching removed most of that bill without touching the model. The distillation they had budgeted for never happened, because after caching the numbers no longer justified it.

That sequence — retrieval first, exemplars before training, caching before distillation — is the one we run by default. Each step is cheaper and more reversible than the one after it, and most stores stop before the expensive end.

The cost shape of each option

Budget conversations go badly when RAG and fine-tuning are compared on a single number, because their costs sit in different places and behave differently as you grow.

RAG is mostly running cost. The build is a few weeks; the bill afterwards is embedding refreshes, vector storage, and a longer prompt on every single query because the retrieved context rides along with it. That last item is the one that surprises people: retrieval makes every request more expensive, permanently, in proportion to how much context you inject. A chatty assistant on a busy storefront can spend more on retrieved context than on the answer.

Fine-tuning is mostly upfront cost. Preparing and cleaning the training set is the real expense — not the training run, which for an adapter is comparatively trivial. Afterwards, inference is cheaper per call than the equivalent prompted system, because the behaviour lives in the weights rather than in tokens you resend every time. The catch is that the cost returns each time the underlying model is deprecated and you retrain.

The crossover is a volume question. Below a few thousand generations a month, prompt-plus-RAG is almost always cheaper all-in, because the upfront work dominates. Well above that, and with stable requirements, the arithmetic flips. Most stores never reach the crossover on their assistant, and do reach it on bulk description generation — which is exactly why the recommendation splits by workload rather than by company.

Count the human cost too. A fine-tune needs someone to curate the training data and re-curate it when the brand voice shifts. RAG needs someone to keep the corpus accurate. Neither is free, and the version that fails is always the one where nobody owned the maintenance.

Build sequence we recommend

Phase 1 (4–6 weeks): RAG over catalog + policy. Citation-required. Ship. Phase 2 (2–4 weeks): If you're seeing voice drift on long-form, fine-tune a small model on your best-performing product copy. Phase 3 (ongoing): As inference cost matters, distill a smaller model from your prompt+RAG outputs.

Most e-commerce clients never reach Phase 3. That's fine.