Context Windows vs Retrieval When to Use Which

Bigger context windows didn't kill RAG — they changed the trade. The decision is no longer "stuff or retrieve." It's "what goes in the context, and how it gets there."

The recurring claim that large context windows make RAG obsolete is wrong. A large window lets you stuff more in; it doesn't make stuffing the right answer. Retrieval still wins on cost, precision, and the long tail — and the trade is now sharper, not gone.

Strip it down:

• **Stuffing.** Put everything in the context. Cheap to build, expensive per request, imprecise on long contexts, and quality drops as the context grows.
• **Retrieval.** Put the right things in the context. More to build, cheaper per request, more precise, and quality holds as the corpus grows.

The trade is cost, precision, and scalability — not "which works."

Stuffing wins when the context is small and known: a few documents, a fixed set of instructions, a short conversation. The overhead of retrieval isn't justified when everything fits and the cost is low.

• Small, known context.
• Fixed instructions.
• Short conversations.

Retrieval wins when the corpus is larger than the window, when cost matters, when precision matters, and when the long tail can't be stuffed. The bigger the window, the more retrieval lets you use it well — the two are complements, not substitutes.

• Corpus larger than the window.
• Cost-sensitive workloads.
• Precision-sensitive queries.
• Long-tail knowledge.

The decision is "what goes in the context, and how it gets there" — a combination of stuffing (fixed, small, known) and retrieval (variable, large, precise). Bigger windows make the combination more powerful, not less.

Context windows and retrieval are complements, not substitutes. Stuff the small and known, retrieve the large and variable — and the bigger the window, the more retrieval lets you use it well.