What it means
An open source Large Language Model (LLM) is a model whose weights are released publicly, so you can download it, run it on your own infrastructure, and fine-tune it on your own data. It is the alternative to a closed model you reach only through a provider's API. The tradeoff is control: You decide where the model runs and what it trains on, and you also carry responsibility for the quality of what it produces.
Why it matters
Hosting your own model does not remove the core risk Kevin flags with any LLM: Hallucination. Models make things up in a convincing way, and older or smaller ones do it more often. A Stanford study found hallucination rates between 69% and 88% for legal queries, and broad estimates run 15 to 20% for an older model like GPT-3 versus about 3% for newer ones. When you self-host an open source model, that error rate is yours to measure and contain.
Say a mid-market B2B SaaS company self-hosts an open source LLM to draft first-line support replies. It looks cheaper than an API until a spot check shows 16% of answers contain a fabricated detail. After adding retrieval grounding and human review on billing topics, the fabrication rate on sampled replies drops from 16% to 4%, and the model is safe to ship.
How to use this knowledge
Pick an open source model only when control, cost at scale, or data residency justify the operational load of running it yourself. Benchmark it against a closed API on your own prompts before committing, and measure hallucination on a labeled sample rather than trusting vendor claims. Ground outputs in retrieval and keep a human in the loop for high-stakes answers, because a self-hosted model moves the accuracy burden onto you.
Growth Memo guidance
The biggest risk of using generative AI and, at the same time, the biggest factor holding even wider adoption back is hallucination. LLMs can make things up in a very convincing way. — The state of generative AI for SEO (PT.2)
The key point is to not trust LLM output blindly, just like we shouldn't trust everything we read on the internet. — The state of generative AI for SEO (PT.2)
Hallucination — the fabricated-output risk you inherit and have to measure when you self-host a model.
Fine-tuning — the main reason to choose open weights, since you can train the model on your own data.
Retrieval-augmented generation (RAG) — grounds a model's answers in your sources to cut hallucination on factual queries.
Proprietary LLM — the closed, API-only alternative you access through a provider instead of hosting yourself.