The Kit

Accelerators

Nvidia H100 — what it is: the workhorse GPU of the 2023–2025 buildout; Hopper architecture, 80GB HBM3, ~3.35 TB/s bandwidth, ~989 TFLOPS dense BF16 (SXM) ⚑ unverified, NVLink 4 at 900 GB/s ⚑ unverified. The chip most frontier models of that window were trained on, and the de facto unit of account for AI compute ("how many H100s?"). · when it matters: it is the baseline. Rental pricing, benchmark comparisons, and capacity announcements are all denominated in H100-equivalents; as newer chips ship, H100s cascade down into the rental market, so it is also the chip a reader is most likely to actually rent ⚑ unverified.

Nvidia H200 — what it is: the same Hopper compute die with the memory system upgraded: 141GB HBM3e at 4.8 TB/s ⚑ unverified. Zero additional FLOPs. · when it matters: it is the cleanest proof of the memory wall in a product catalog — Nvidia added only memory and got up to ~1.9× on LLM inference ⚑ unverified, because inference is bandwidth-bound. Choose it over H100 when the model (or the KV cache) doesn't fit in 80GB.

Nvidia B200 / GB200 — what it is: Blackwell generation — two reticle-limit dies fused into one logical GPU, 192GB HBM3e, ~8 TB/s bandwidth ⚑ unverified, FP4 inference support, NVLink 5 at 1.8 TB/s ⚑ unverified, ~1,000–1,200W ⚑ unverified. Sold increasingly as the GB200 NVL72 rack: 72 GPUs + Grace CPUs in one liquid-cooled NVLink domain, ~120kW per rack ⚑ unverified. · when it matters: it marks the shift from "buy a chip" to "buy a rack" — the sellable unit of AI compute is now a 1.5-tonne liquid-cooled system, which reshapes data-center design, power planning, and who can even take delivery.

AMD MI300X / MI325X — what it is: AMD's flagship accelerator; chiplet-based, 192GB HBM3 at 5.3 TB/s (MI300X) ⚑ unverified, more memory than the contemporaneous H100/H200. Runs the ROCm software stack rather than CUDA. · when it matters: it is the only merchant-silicon alternative with real deployments (Microsoft, Meta, OpenAI have all publicly taken volume ⚑ unverified) — which makes it the live test of whether Nvidia's moat is the chip or the software. On paper the memory advantage should win inference workloads; in practice ROCm's maturity gap is the recurring complaint ⚑ unverified. Watch AMD's actual data-center revenue vs. its guidance as the honest scoreboard.

Google TPU (v5p, v6e/Trillium) — what it is: Google's in-house systolic-array accelerator, deployed in "pods" of thousands of chips on a dedicated optical interconnect (ICI, with optical circuit switching) ⚑ unverified. Gemini models train on it; Apple has used TPU pods for its foundation models ⚑ unverified. · when it matters: it is the proof that a hyperscaler can escape the Nvidia tax at frontier scale — but only by owning the entire vertical stack. Rentable only through Google Cloud, so it is an architecture lesson and a cost lever, not a market product.

Broadcom — what it is: not an accelerator brand but the silent partner behind most custom ones — co-designer and supplier of the serdes/networking IP for Google's TPU and other hyperscaler ASICs (reported: Meta, ByteDance, OpenAI's custom chip program ⚑ unverified), plus the Tomahawk/Jericho switch silicon underpinning AI Ethernet fabrics. · when it matters: every hyperscaler that wants off Nvidia's roadmap routes through Broadcom, making its "AI revenue" line one of the best public proxies for the custom-silicon rebellion. Also the networking side of the Nvidia-vs-Ethernet war.

The manufacturing chokepoints

TSMC — what it is: the foundry. ~90% of leading-edge logic capacity ⚑ unverified; fabricates Nvidia, AMD, Google, Amazon, Apple silicon; owns the CoWoS advanced-packaging capacity that has repeatedly been the binding constraint on global GPU supply ⚑ unverified. Concentrated in Taiwan, with US/Japan fabs ramping years behind ⚑ unverified. · when it matters: always. TSMC's capex plans and packaging expansion schedule are the supply forecast for AI compute. Its earnings calls are the closest thing this layer has to ground truth, and its geography is the single largest concentrated risk in the entire AI economy.

ASML — what it is: the sole manufacturer of EUV lithography machines — the only tools that can print leading-edge chips. ~€200M per EUV system, ~€350–400M for High-NA ⚑ unverified; ships tens of units per year ⚑ unverified. A literal monopoly at the leading edge, and the primary instrument of chip export-control policy. · when it matters: on multi-year horizons (its order book foreshadows fab capacity ~2–3 years out) and in every geopolitical scenario — control of ASML's shipping manifest is control of who gets to make advanced chips.

GPU-cloud rental — where a reader actually touches this layer

The practical on-ramp: nobody reading this buys an H100; everybody can rent one for a few dollars an hour. Pricing moves weekly — see the live board. [asset: B20 GPU price/availability]

Lambda— what it is: AI-focused GPU cloud (plus workstations/servers for buyers); on-demand and reserved H100/B200-class instances and clusters, with an ML-ready software image. · when it matters: the straightforward "I need real GPUs today without hyperscaler procurement" option — solid default for training runs and fine-tuning at team scale.

CoreWeave— what it is: the flagship "neocloud" — a former crypto-mining operation rebuilt into one of the largest Nvidia GPU fleets outside the hyperscalers, now a public company with multi-billion-dollar contracts (Microsoft, OpenAI ⚑ unverified). Kubernetes-native, cluster-scale. · when it matters: serious scale — thousands of interconnected GPUs with InfiniBand — and simultaneously the bubble-watch specimen: its debt-financed, Nvidia-entangled model (chips as loan collateral ⚑ unverified) makes it the purest publicly traded bet on GPU demand staying above GPU supply.

RunPod— what it is: developer-oriented GPU cloud — per-second billing, containerized pods, serverless GPU endpoints, a community marketplace tier alongside secure data-center capacity. · when it matters: the hobbyist-to-startup sweet spot: fine-tune a 7B model, host a Stable Diffusion endpoint, run inference bursts — minutes from signup to a running GPU, at prices well under hyperscaler list ⚑ unverified.

Vast.ai— what it is: the spot market — a brokerage matching renters with idle GPUs from data centers and individuals; bidding, interruptible instances, wildly heterogeneous hardware from RTX 4090s to H100s. · when it matters: absolute lowest cost-per-FLOP for fault-tolerant, non-sensitive workloads — and, read as a ticker, the closest thing to a live spot price for GPU compute. When Vast's 4090/H100 rates move, the supply-demand balance moved. [asset: B20 GPU price/availability]