GenAI & LLM foundations
What Is an LLM?
Large language models power almost every modern AI product. Here is what is actually happening under the hood.
A large language model (LLM) is a system trained on enormous amounts of text to predict the most likely next piece of text, given everything that came before it. Repeated millions of times, that simple mechanism produces surprisingly coherent writing, reasoning, and code.
Every AI engineer needs a working mental model of how LLMs behave, because most of the craft of AI engineering is compensating for their specific strengths and weaknesses.
Key takeaways
How an LLM actually works, briefly
During training, an LLM is shown huge volumes of text and learns statistical patterns about which words tend to follow which. At inference time, it uses those patterns to generate text one piece at a time, each new piece chosen based on everything generated so far.
There is no built-in fact database and no true understanding — just an extremely sophisticated pattern completion engine, which is powerful but explains both its strengths and its failure modes.
The vocabulary every AI engineer needs
Tokens are the chunks of text a model processes — roughly a word or part of a word. The context window is the maximum amount of text (in tokens) a model can consider at once. Temperature controls how random or deterministic outputs are.
These three concepts explain most practical LLM behavior: why very long documents get cut off, why the same prompt sometimes gives different answers, and why some outputs feel more "creative" than others.
Where LLMs need help
LLMs can state incorrect information with total confidence — a failure mode called hallucination — because they optimize for plausible-sounding text, not verified truth. They also cannot access real-time information or private data on their own.
This is exactly why retrieval-augmented generation, tool calling, and evaluation exist: they compensate for what a raw LLM cannot do reliably on its own.
Put this into practice