What it means
Training data is the text a model learned from during pre-training, before its knowledge cutoff. It is what an LLM knows by default, separate from anything it retrieves live at query time. Models build confidence about brand-attribute associations ("cheap", "durable", "fast") from this data, and those associations shape answers even before live search runs. Retrieval, through RAG, is the second surface that can add or correct what training data already encoded.
Why it matters
Brands that were widely covered and well-regarded before a model's cutoff start with an edge: The model already associates them with the right attributes. A newer entrant cannot rewrite training data, so its only path to the answer is retrieval, which makes RAG the more democratic surface for challengers. Model developers also filter pre-training data by quality and authority, so weak or thin sources are less likely to be encoded at all.
Picture two project-management tools. The incumbent launched in 2015 and is baked into the model's training data, so it gets named in "best project management software" prompts unprompted. The 2024 challenger only appears when the model runs a live search and finds a strong, fact-dense comparison page. Retrieval is the challenger's whole game.
How to use this knowledge
Check what the model already believes. Ask LLMs to describe your brand with no live search and note which attributes are right, wrong, or missing.
Win the retrieval surface if you are new. Publish authoritative, well-structured pages so live retrieval can pull you into answers training data ignores.
Strengthen the associations you want. Run targeted on-page and off-page campaigns around the attributes you need the model to learn.
Stay crawlable. If crawlers cannot reach you, you enter neither the training data nor the retrieval pool.
Growth Memo guidance
LLMs are biased by training data. Models develop confidence scores for brand-attribute relationships (eg, "cheap", "durable", "fast") independent of real-time retrieval. These pre-existing associations influence citation likelihood even when your content enters the candidate pool.
OpenAI (and most likely other model developers as well) filter pre-training data by both quality and authority: "At the pre-training stage, we filtered our dataset mix for GPT-4 ... and removed these documents from the pre-training set."
The pre-requisite to being visible in LLMs is, of course, their ability to crawl your site, whether that's directly or through training data.
RAG (Retrieval Augmented Generation) — The live retrieval surface that lets newer brands enter answers their training data exposure cannot.
Primary bias — The brand-attribute associations a model holds from training data before grounding in live results.
Knowledge cutoff — The date after which a model has no native knowledge, defining the edge of its training data.
Hallucination — What can happen when training data is thin or wrong and no live source corrects it.
Topical authority — A signal model developers use to filter and weight training data by quality.