The tool landscape
The AI toolchain looks chaotic — hundreds of products, all changing monthly. But it resolves into a stack of layers, each a category with a durable purpose even as the tools inside it churn. And every layer maps to a chapter you've just read.
01The answer, then the intuition
Learn the layers, not the logos
The tool landscape feels overwhelming because people present it as a list of products. It isn't — it's a stack. From the model at the bottom to the guardrails at the top, each layer does one job, and whatever product is popular this quarter, the job stays the same. Learn the layers and the churn stops mattering: when a tool changes, you already know what slot it fills.
Better still, this stack is the book. Every layer is a chapter you've read — which means you already understand the whole toolchain from first principles. Click through it:
The AI application stack — click a layer
Each layer is a category; the tools are examples, not endorsements. Each maps to a chapter.
02Mechanics
The layers, top to bottom
- Model & provider. The foundation model and how you call it — a closed API or a self-hosted open-weight model.
- Serving & inference. What actually runs the model efficiently — batching and quantization engines, or a managed endpoint.
- Gateway & routing. A layer in front of many models for routing, fallback, and unified billing.
- Retrieval & vector DB. The searchable memory that grounds answers in your data.
- Orchestration & agents. The frameworks that chain prompts, tools, and agent loops into a workflow.
- Evals & observability. The measurement and tracing that tell you if any of it works.
- Guardrails. The safety filters wrapping the whole thing.
The practical advice falls straight out of this shape: pick each layer by need, keep the interfaces clean so you can swap a tool without a rewrite, and don't marry a framework — the categories are stable, the products are not. Understanding beats tooling, because understanding is what tells you which tool you actually need.
04The math
expand ▾An app is a composition
04The math
expand ▾An app is a composition
An AI application is literally a composition of the layers — the output of one feeds the next, top wrapping bottom:
Read inside-out: route the request to a model, retrieve grounding context, generate conditioned on both, then guard the output. Because it's a composition, each layer is independently swappable — change the retriever without touching the router — which is exactly why clean interfaces matter more than any single tool. The math of the stack is the math of function composition: understand each function, and you understand the whole system, regardless of which library implements each one.
05The code
expand ▾The whole book, in one pipeline
05The code
expand ▾The whole book, in one pipeline
Every chapter, composed into a single AI app. The tools are stubs; the shape is the point.
app.py
def route(task): # ch31 — cheapest model that clears the bar
return "mid" if task["easy"] else "frontier"
def retrieve(query): # ch21 / ch25 — grounding via vector search
return ["Refunds within 30 days."]
def generate(model, ctx, q): # ch20 — prompt, conditioned on context
return f"[{model}] {q} -> grounded in {ctx[0]}"
def guard(text): # ch33 — safety filter on the output
return text if "leak" not in text else "[blocked]"
def app(task): # the composition (ch34)
model = route(task)
ctx = retrieve(task["q"])
draft = generate(model, ctx, task["q"])
return guard(draft)
print(app({"q": "What's the refund window?", "easy": True}))
print(app({"q": "Draft a legal brief.", "easy": False}))
# [mid] What's the refund window? -> grounded in Refunds within 30 days.
# [frontier] Draft a legal brief. -> grounded in Refunds within 30 days.07Going deeper
expand ▾The primary sources
07Going deeper
expand ▾The primary sources
LangChain / LangGraph · orchestration & agent frameworks.
vLLM · the open serving engine behind much of the batching layer.
Model Context Protocol (MCP) · an open standard for the tool/agent layer.
Awesome-LLMOps · a maintained map of the whole tooling landscape.
"First Principles — how AI actually works", chapter "The tool landscape", by The Catch. CC-BY 4.0. Hosted at TheCatch.AI / Tech Stack / AI Principles.