AI Principles
The AI Stack, From First Principles
An open textbook in six parts. Every chapter runs the same circuit: the plain-English intuition, the real mechanism, where it breaks, and why it matters economically. Start anywhere — the chapters cross-reference.
8 chapters
AI Principles
What is a token?
A token is the basic unit of text an AI model reads and writes —not a word, not a letter, but a chunk. Models don't see language; they see tokens.
What is an embedding?
An embedding turns a token into apoint in space— a list of numbers — arranged so thatmeaning becomes distance. Similar things sit close together, and relationships become directions you can do arithme
What is a neural network?
A neural network is a stack ofweighted sums. Each unit multiplies its inputs by weights, adds them up, and passes the result through a simple bend. Learning is nothing more thanadjusting those weights
What is attention?
Attention is the move that lets every tokenlook at every other tokenand decide which ones matter for it right now. It's how a model figures out that "it" refers to "the animal" — and it's the heart of
What is a transformer?
A transformer is theassembly. Take embeddings, attention, and a small neural network, wire them into a repeatableblock, and stack that block dozens of times. That stack is what an LLMis.
Parameters & weights
A parameter is a singlelearned number— one weight in one of the model's matrices. A large model is billions of them, and together those numbersarethe model. "175 billion parameters" means 175 billion
How models are pretrained
Pretraining is the long, brutally expensive first phase where a model readstrillions of tokensof text and, by predicting the next one over and over, teaches itself language, facts, and reasoning — wit
What is an LLM?
A large language model is atransformertrained to do one thing:predict the next token. Run that prediction over and over, feeding each guess back in, and the next-token machine becomes a writer.
4 chapters
Hardware
Why AI needs GPUs
A neural network is, underneath, one operation repeated trillions of times:matrix multiplication. GPUs win because they do that one thing with thousands of cores at once — while a CPU, built for a few
Memory, HBM & the memory wall
Chips can now do arithmetic far faster than they can fetch the numbers to work on. That widening gap is thememory wall— and it's why AI accelerators are defined by theirhigh-bandwidth memory (HBM)as m
The compute supply chain
Behind everyGPUis a chain that runs from a Dutch lithography machine to a Taiwanese fab to a power substation — and nearly every link is held byone to three companies. It's the most concentrated suppl
Quantization & distillation
A frontier model is enormous. Two techniques shrink it to run cheaply:quantizationstores each weight in fewer bits, anddistillationtrains a small "student" to copy a big "teacher" — keeping most of th
4 chapters
Infrastructure
What is inference?
Training builds the model once.Inferenceis running it — the forward passes that turn your prompt into an answer, every single time you hit send. It happens in two very different phases, and it is the
Latency, throughput, tokens/sec
How fast is an AI model? The honest answer is a formula. Becausedecodeis memory-bound, a model's speed is set by how fast it can stream itsweightsout of memory —tokens per second ≈ bandwidth ÷ model b
KV cache & batching
Two tricks make serving LLMs affordable. TheKV cachestops the model recomputing the whole prompt every step;batchingreuses one expensive weight-load across many users at once. Together they turn a was
The data-center cluster
A frontier model is too big for any singleGPU. It runs acrossthousands of them, wired together into racks, pods, and buildings — drawing tens of megawatts. At this scale the network, not the chip, bec
7 chapters
Foundation
Differences between LLMs
GPT, Claude, Gemini, Llama, Mistral, DeepSeek — almost all of them are thesame transformer recipeyou just learned. What actually separates them lives on a handful of axes: scale, data, architecture, a
Fine-tuning & RLHF
Apretrainedbase model knows language but has no manners — it just continues text. Fine-tuning and RLHF are the two phases that turn that raw predictor into a model thatfollows instructions, stays help
Multimodal models
A model that "sees" doesn't need a new brain. An image is just chopped into patches, each patch turned into atokenin thesame vector spaceas words — and fed into thesame transformer. Everything becomes
Scaling laws
The single most consequential fact in modern AI: modellossfalls as a smoothpower lawas you add compute, data, andparameters. On a log-log plot it's a straight line — predictable enough to bethundreds
Safety, evals & guardrails
Deploying AI means managing its failure modes —hallucination, jailbreaks, data leakage, and prompt injection. No single filter is enough, so you stack imperfect layers. It works well for most threats,
The foundation-model labs
A handful of organizations build the frontiermodelsthat nearly everyone else buildson. The club is small for a reason:scaling lawsturn compute into a capital barrier only a few can clear.
RAG vs fine-tune vs prompt
You've now met three ways to bend a model to your needs:promptingchanges its context,RAGfeeds it knowledge, andfine-tuningchanges its weights. Choosing well is most of applied AI — and the rule is sim
8 chapters
Orchestration
What is RAG?
A model's knowledge is frozen attraining timeand blind to your private or recent data.Retrieval-augmented generationfixes that without retraining: find the relevant documents, put them in theprompt, a
Agents & tool use
A model on its own can only write text. Put itin a loop with tools— search, a calculator, code, an API — and it canact: take a step, read the result, decide the next step, and repeat until the task is
Multi-agent & what comes next
The frontier isn't a bigger model — it's manyagentscoordinating: an orchestrator delegating to workers, agents calling agents, whole workflows automated. The promise is enormous. So is the problem:rel
The tool landscape
The AI toolchain looks chaotic — hundreds of products, all changing monthly. But it resolves into astack of layers, each a category with a durable purpose even as the tools inside it churn. And every
Vector search
Under everyRAGsystem is one hard problem: find the handful ofvectorsnearest a query, among millions or billions. Comparing them all is too slow — soapproximate nearest-neighborsearch trades a sliver o
Prompting
A prompt is a program — written in plain English, for a machine whose only instinct is tocontinue text. You don't change theweights; you change thecontextyou condition them on. Done well, that's enoug
The context window
The context window is a model'sworking memory— the number of tokens it can hold and attend to at once. Everything inside it the model can use; everything outside it, it never sees. And making it bigge
Evals
You cannot improve what you don't measure — and AI outputs are variable and subjective, so measuring is hard.Evalsare test suites for AI: a fixed set of cases with known-good answers, run on every cha
3 chapters
Application
Choosing a model
The most common and most expensive mistake in applied AI is reaching for the bestmodelwhen you needed the cheapest one thatclears the bar. The right rule is nearly the opposite of the instinct: match
Cost optimization
AI serving cost isn't one number to shave — it's astack of independent levers that multiply. Each cuts cost by a factor; stacked, they compound. This is the operational discipline that turns a token t
The economics of a token
Strip away the narrative and all of AI's economics reduce to one unit: thetoken. What it costs to produce, what it sells for, and how many you must sell to repay thetrainingbill. This is theCircuit'sc
Originally from
First Principles by Michael J. Richardson · CC-BY 4.0 ·
Hosted at TheCatch.AI / Tech Stack / AI Principles