H100, H200, B200, B300, GB200, GB300, and Vera Rubin. What they are and when each one is the right choice. By XIRR Advisors, brokers of colocation space and reserved GPU compute capacity. First published April 2026, updated October 2026.
When you strip away the marketing and the naming conventions, choosing an NVIDIA data center GPU comes down to three practical questions. How much memory sits on the chip? How fast can that memory move data? How much power does the chip draw under load? The right chip for you is the one that matches your workload, not the one with the loudest headlines.
For most organizations in 2026, no single chip is the answer. A practical setup blends a few of them based on what each is built for.
H200 is the workhorse for serving large language models in production. Its extra memory lets one GPU hold larger models and serve more users.
B300 is the chip you reach for once your serving workload includes reasoning models, the kind that walk through a problem step by step before answering. Those generate far longer outputs than a standard chatbot reply, and they need the extra memory and throughput B300 provides.
B200 is the chip for serious training, including fine-tuning your own large models or running production training pipelines.
GB200 and GB300 NVL72 racks are full rack-scale systems (72 GPUs wired together to behave like one very large accelerator) for organizations training very large models from scratch. These are sometimes called "trillion-parameter" models. Frontier AI labs and hyperscalers operate at this scale. Most companies do not.
Vera Rubin is NVIDIA's next platform after Blackwell. It comes as an 8-GPU server (Rubin NVL8) and as a full rack (Vera Rubin NVL72), and every form of it is liquid cooled. If your roadmap extends past 2026, it is worth reserving capacity early, because next-generation supply is allocated months in advance and tight allocations have already become the norm.
The harder problem is rarely the chip. It is finding the right capacity on the right terms. The right data center, the right cooling, the right contract length, and the right operator. That is what XIRR Advisors does. This guide covers the chips. We help you find the capacity.
Before we go further, here are the acronyms used throughout this guide.
HBM3, HBM3e, HBM4, HBM4E are generations of High Bandwidth Memory, the very fast memory stacked next to the GPU die.
FP4, FP8, FP16 are floating-point number precisions used for AI math. FP4 means 4-bit. Lower precision means more operations per second, with some loss of accuracy. Modern AI models tolerate FP8 and FP4 well.
TFLOPS, PFLOPS, ExaFLOPS measure compute throughput. TFLOPS means trillion floating-point operations per second. PFLOPS is a thousand TFLOPS. ExaFLOPS is a thousand PFLOPS.
Dense and sparse describe how a compute figure is counted. NVIDIA quotes some figures with sparsity, a shortcut that skips part of the math and gives a higher number. This guide uses the dense figures, without the shortcut, unless it says otherwise.
TDP stands for Thermal Design Power, measured in watts. It is roughly how much heat the chip produces under load and therefore how much cooling it needs.
NVLink is NVIDIA's high-speed link between GPUs and CPUs. NVLink-C2C is a chip-to-chip variant.
NVL72 and NVL8 are NVIDIA's system names. NVL72 is a rack of 72 NVLink-connected GPUs that behave like one large accelerator. NVL8 is a server with 8 GPUs. Older material calls the Vera Rubin rack NVL144: NVIDIA renamed it NVL72 in December 2025.
HGX is NVIDIA's reference server platform that hardware partners (Supermicro, Dell, HPE, and others) build production servers around. DGX is NVIDIA's own branded system.
LLM means Large Language Model. RAG means Retrieval-Augmented Generation. SLA means Service Level Agreement.
GPUaaS means GPU-as-a-Service, an on-demand rental model. SMB means small or mid-sized business.
NVIDIA names each generation of data center GPU after a scientist. Inside each generation, there is usually a base chip and a mid-cycle refresh.
| Generation | Years | Base Chip | Refresh | Rack Form |
|---|---|---|---|---|
| Hopper | 2022 to 2024 | H100 | H200 | HGX H100 / H200 |
| Blackwell | 2024 to 2026 | B200 | B300 (Blackwell Ultra) | HGX B200 / B300 |
| Grace Blackwell | 2025 to 2026 | GB200 | GB300 | NVL72 (72 GPUs per rack) |
| Vera Rubin | 2026 onward | Rubin | Rubin Ultra (announced 2025) | Rubin NVL8 server / Vera Rubin NVL72 rack |
Four generations of NVIDIA data center GPUs, from Hopper to Vera Rubin. Each generation adds compute, and most steps add memory and bandwidth. The strategic question is whether your workload needs the headroom, or whether the previous generation does the job.
The pattern is consistent. Each generation brings more compute and more power per GPU, and most steps add memory and bandwidth as well. The strategic question is whether your workload genuinely needs the headroom, or whether the previous generation does the job. Each step up also asks more of the building: more power in each rack, and liquid cooling for the rack-scale systems and for everything in the Rubin generation.
Each chip below follows the same four-part structure. What it is, what is inside, what it is best for, and when it is the right choice.
What it is. NVIDIA's 2022 flagship and the first chip designed natively for transformer training. Built on the Hopper architecture.
What is inside. 80 GB of HBM3 memory at 3.35 TB/s of bandwidth. About 2 PFLOPS of FP8 compute, or about 4 PFLOPS on NVIDIA's headline figure, which counts sparsity. Up to 700 W[1][10].
Best for. Mainstream training, fine-tuning, computer vision, recommendation systems, and any production workload where cost matters more than peak performance.
When you would reach for H100. When you are running steady-state production AI workloads on models below 70 billion parameters. When you want hardware that every provider and every software stack already supports. When models you train or fine-tune already fit comfortably inside 80 GB. When budget discipline matters more than the latest specs.
What it is. The 2024 mid-cycle refresh of H100. Same Hopper compute engine, dramatically expanded memory.
What is inside. 141 GB of HBM3e memory at 4.8 TB/s of bandwidth. The same FP8 compute as H100. Up to 700 W[2].
Best for. Inference at scale, particularly serving LLMs in production. The extra memory lets one GPU host larger models and serve more concurrent users.
When you would reach for H200. When you serve LLMs to end users and dollar-per-token economics drive your business. When you have outgrown H100 for inference because models no longer fit on a single 80 GB GPU. When you want a mature, widely supported platform for inference. When you run long-context inference and need the memory headroom but not yet the compute jump that Blackwell delivers.
What it is. NVIDIA's 2025 flagship and the workhorse of the current generation. A two-die package fused into a single GPU.
What is inside. About 180 GB of HBM3e memory per GPU at 8 TB/s of bandwidth. NVIDIA lists 1,440 GB for its eight-GPU DGX B200 system, and its rack figures work out to 186 GB per GPU[10]. Native FP4 support. NVIDIA projects 3x the training performance and 15x the inference performance of its previous-generation DGX H100 system. Power is widely reported at about 1,000 W per GPU[3].
Best for. Frontier training at scale, plus modern inference workloads where you want to consolidate larger H100 fleets onto fewer, denser nodes.
When you would reach for B200. When you are running serious training programs on models above 100 billion parameters. When power and rack-space efficiency matter, since one B200 replaces several H100s. When you are deploying new AI infrastructure today and want a useful runway before the next refresh. When you are launching multimodal or long-context inference workloads where H200 latency starts to bottleneck.
What it is. The mid-cycle refresh of Blackwell, available from NVIDIA's partners since the second half of 2025. Designed by NVIDIA explicitly for reasoning and agentic AI.
What is inside. 288 GB of HBM3e memory at 8 TB/s of bandwidth. Roughly 15 PFLOPS dense FP4 compute, a 1.5x lift over B200. 1,400 W TDP[4]. NVIDIA's own 8-GPU server, DGX B300, is air cooled and draws about 14 kW, 15 kW at most[10].
Best for. Reasoning model inference, agentic AI deployments, and the leading edge of foundation-model training. The chip-of-record for models that generate thousands of tokens before producing an answer.
When you would reach for B300. When you launch reasoning or agentic AI products with long, multi-step outputs. When inference contexts exceed 128,000 tokens. When real-time agent latency defines product quality. When you build sovereign or national AI infrastructure today.
What it is. A "superchip" pairing one Grace CPU with two Blackwell GPUs over NVIDIA's NVLink-C2C link. Sold primarily as part of a rack-scale system.
What is inside. Each GB200 superchip carries 372 GB of HBM3e memory and 16 TB/s of bandwidth. The standard configuration is the GB200 NVL72 rack, which packs 72 B200 GPUs and 36 Grace CPUs, with 13.4 TB of GPU memory, into one liquid-cooled rack that behaves, from the software's point of view, like one extremely large accelerator[6][10].
Best for. Trillion-parameter model training, hyperscaler deployments, and teams operating at the foundation-model frontier where the workload is large enough to keep an entire rack saturated.
When you would reach for GB200. When you train foundation models from scratch and training cycles consume thousands of continuous GPU-hours. When workloads saturate an entire rack for weeks or months at a time. When you need many GPUs to work as one, for jobs that would otherwise be split across separate servers over slower links. When you are committing to rack-scale capacity today and reasoning workloads do not yet call for GB300.
What it is. The same GB200 superchip design, upgraded with B300 GPUs. The current rack-scale standard for new builds.
What is inside. The GB300 NVL72 rack holds 72 Blackwell Ultra GPUs and 36 Grace CPUs, with 20 TB of GPU memory. It delivers approximately 1.1 ExaFLOPS of dense FP4 compute, roughly 1.5x a GB200 NVL72[4], and draws up to 142 kW. Its GPUs and CPUs are liquid cooled and its other components are air cooled[10].
Best for. Reasoning-model training and inference at rack-scale, where GB200's memory and throughput leave tokens-per-dollar on the table.
When you would reach for GB300. When you train frontier reasoning or agentic models from scratch and the longer outputs and larger KV caches break GB200 economics. When inference fleets serve reasoning models at rack-scale and having about 50% more memory than GB200 translates directly into tokens-per-dollar. When training and inference share one fleet at sustained high utilization. When you are committing to new rack-scale capacity today and there is no reason to choose the prior generation.
What it is. NVIDIA's next platform after Blackwell, named after astronomer Vera Rubin. The Rubin GPU pairs with a new Arm-compatible CPU called Vera.
What is inside. 288 GB of HBM4 memory per GPU at 19.2 TB/s of bandwidth. Rubin comes in two forms. Rubin NVL8 is an 8-GPU server. Vera Rubin NVL72 is a full rack with 72 Rubin GPUs and 36 Vera CPUs, 20.7 TB of HBM4 memory, and about 2.5 ExaFLOPS of dense FP4 compute, roughly 2.3x a GB300 NVL72[10].
Power and cooling. Every form of Rubin is liquid cooled. There is no air-cooled Rubin. NVIDIA's preliminary figure for its 8-GPU server, DGX Rubin NVL8, is about 24 kW. The Vera Rubin NVL72 rack runs on warm water, 45°C (113°F) going in, which lets a building do without chillers[10]. NVIDIA has not published the power of a Rubin GPU or of the Vera Rubin NVL72 rack. Industry reporting, not NVIDIA, puts the GPU near 2.3 kW and the rack between 190 and 230 kW.
Best for. Workloads that have outgrown Blackwell Ultra: frontier training and next-generation reasoning inference. A Vera Rubin rack holds 72 GPUs, the same number as a GB300 rack, with faster memory and more than twice the compute.
When you would reach for Vera Rubin. When the next generation of reasoning models pushes context and output lengths past what Blackwell can serve efficiently. When you are planning capacity that will run for years and want the generation that is just starting. When your site, or your operator's, can supply liquid cooling at the density Rubin needs.
A note on names. Older material calls the Vera Rubin rack NVL144. NVIDIA renamed it NVL72 in December 2025, because the old number counted the two chips inside each GPU and the new one counts the GPUs. It is the same rack. "VR200", "R100" and "R200" are trade shorthand, not NVIDIA product names. At its 2025 conference NVIDIA said systems built on a Rubin Ultra refresh were due in the second half of 2027[8]. The NVIDIA product pages we checked in October 2026 do not list it and give no specifications, so this guide gives none.
Figures are rounded to communicate orders of magnitude. Rows for a single chip give figures per GPU. Rows for an NVL72 rack give figures for the whole rack of 72 GPUs. Where NVIDIA has not published a figure, the table says so.
| Chip or Rack | Architecture / Year | Memory | Bandwidth | FP4 (dense) | Power | Best For |
|---|---|---|---|---|---|---|
| H100 | Hopper, 2022 | 80 GB HBM3 | 3.35 TB/s | n/a (FP8: ~2 PF) | Up to 700 W per GPU | Proven training and inference |
| H200 | Hopper, 2024 | 141 GB HBM3e | 4.8 TB/s | n/a (FP8: ~2 PF) | Up to 700 W per GPU | LLM inference at scale |
| B200 | Blackwell, 2025 | About 180 GB HBM3e | 8 TB/s | ~10 PF | About 1,000 W per GPU (reported) | Frontier training, modern inference |
| B300 | Blackwell Ultra, 2025 to 2026 | 288 GB HBM3e | 8 TB/s | ~15 PF | 1,400 W per GPU | Long-context, reasoning, agentic AI |
| GB200 NVL72 rack | Grace and Blackwell, 2025 | 13.4 TB per rack | 576 TB/s per rack | 720 PF per rack | About 132 kW per rack (our planning estimate) | Rack-scale training |
| GB300 NVL72 rack | Grace and Blackwell Ultra, 2025 to 2026 | 20 TB per rack | Up to 576 TB/s per rack | 1,080 PF per rack | Up to 142 kW per rack | Trillion-parameter training, sovereign clusters |
| Rubin | Vera Rubin, 2026 | 288 GB HBM4 | 19.2 TB/s | ~35 PF | Not published by NVIDIA (about 2,300 W reported) | Next-generation training and reasoning inference |
| Vera Rubin NVL72 rack | Vera Rubin, 2026 | 20.7 TB per rack | 1,400 TB/s per rack | 2,520 PF per rack | Not published by NVIDIA (190 to 230 kW reported) | Frontier training and reasoning inference at rack scale |
The chip decides more than performance. It decides which buildings can host it. Servers with eight GPUs, up to B300, can run on air, which more existing data centers can provide. The rack-scale systems, and everything in the Rubin generation, need liquid cooling.
| System | Power | Cooling |
|---|---|---|
| 8-GPU H100 or H200 server | About 10 kW | Air |
| 8-GPU B200 server | About 14 kW | Air |
| 8-GPU B300 server (DGX B300) | About 14 kW, 15 kW at most | Air |
| GB200 NVL72 rack | About 132 kW | Liquid, with air for the smaller parts |
| GB300 NVL72 rack | Up to 142 kW | Liquid, with air for the smaller parts |
| Rubin NVL8 server (DGX Rubin NVL8) | About 24 kW | Liquid only |
| Vera Rubin NVL72 rack | Not published by NVIDIA (190 to 230 kW reported) | Liquid only, on warm water |
B300 and GB300 figures are NVIDIA's. The Rubin NVL8 figure is NVIDIA's preliminary one[10]. H100, H200, B200 and GB200 figures are our planning estimates. The Vera Rubin rack range is industry reporting, not an NVIDIA figure.
One more requirement comes from NVIDIA itself. Its reference design for GB300 clusters calls for a data center built to Uptime Institute Tier III design standards (NVIDIA writes it Tier 3), which means any part of the site can be serviced without shutting it down.
For the full picture, see Liquid vs Air Cooling for AI GPUs: Rack Density Explained. For the step from GB300 to Rubin, see GB300 vs Vera Rubin: What Changes for the Data Center.
Each profile below uses the same structure. The default recommendation, the upgrade trigger, and the conditions under which you would reach for top-tier rack-scale or next-generation silicon.
Default: B200 for training, H200 for production inference.
Upgrade to B300 when you launch reasoning or agentic models, or when serving workloads exceed 128,000 token contexts.
Move to GB200 or GB300 NVL72 when you train foundation models from scratch and your training cycles consume thousands of continuous GPU-hours.
Reserve Vera Rubin when your next training run or serving stack actually needs Rubin's faster memory and extra compute, not when the calendar turns.
Default: H100 or H200 in carrier-neutral colocation. Sufficient for RAG, internal copilots, computer vision, fraud detection, demand forecasting, and recommendation systems.
Upgrade to B200 when memory pressure forces multi-GPU sharding for what should be single-GPU inference, or when consolidation saves meaningful power and rack space.
Move to B300 when reasoning latency starts to impact SLA commitments, or when you deploy agentic AI applications that generate long outputs.
Move to GB200 or GB300 racks when you train your own foundation models in-house and training cycles run continuously at rack-scale, a threshold most enterprises never cross.
Default: H100 or H200 capacity from a GPUaaS provider. Proven hardware, with no infrastructure to run yourself.
Upgrade to B200 when your product genuinely requires it. Examples include fine-tuning open-weight models above 70 billion parameters, real-time multimodal applications, or workloads where H200 latency is a documented bottleneck.
Reach for B300 when you launch a reasoning or agentic feature where end-user latency drives retention, and H200 or B200 cannot keep up.
Use GB200, GB300, or Rubin when a single training run genuinely needs an NVLink-connected rack as one accelerator.
Default today: GB300 NVL72 racks. A common choice for new national AI infrastructure.
Add B300 standalone systems when the use case is targeted, such as a national language model, a single agency or ministry's reasoning workload, or a classified inference pilot, and you do not yet need rack-scale density.
Step up to GB200, GB300 or Vera Rubin NVL72 racks when sovereign workloads such as national reasoning models, multilingual foundation models, classified inference, and defense and intelligence applications require rack-scale density. Match the generation to the workload: GB200 for general-purpose training, GB300 for reasoning at scale, Vera Rubin for the next horizon.
Expect to operate a mixed fleet: B300 standalone for targeted workloads, GB200 or GB300 NVL72 as the rack-scale default depending on whether reasoning is the primary workload, and Vera Rubin reserved for the next horizon.
Most organizations do not have a chip problem. They have a capacity problem. The right answer is rarely a single chip. A practical setup is a portfolio:
Once you know which chip fits your workload, the next problem is finding the actual capacity. The right data center, the right contract length, the right cooling, and the right operator on the right terms. That is what XIRR Advisors does. We broker colocation space and reserved GPU compute capacity for clients who would rather build their products than spend months negotiating with providers.
XIRR Advisors is an independent infrastructure desk. We broker two things: Tier III colocation space and power for hardware you own, and reserved GPU compute capacity (H100/
contact@xirradvisors.com
Stay ahead of the colocation and GPU capacity markets. Every Monday morning, XIRR Advisors publishes a curated brief on supply, pricing, operator news, and the deals shaping AI infrastructure.
Long: $NVDA, $IREN, $CRWV, $NBIS, $CIFR, $NUAI, $APLD, $HUT, $MSFT, $AMZN. Not investment advice.