AI Infrastructure Foundations
(AI-INFRA 100-01)
The shared vocabulary and architecture of an AI GPU cluster
A conceptual walkthrough of the AI cluster stack, from the GPU node to the fabrics to the management plane that operates a fleet of them. The goal is a working mental model and precise vocabulary, not hands-on execution: you leave able to name every component, place it in the stack, and explain what it is there for.
Description
The workload drives the architecture
Training and inferencing as distinct workload shapes, why a GPU rather than a CPU, and the chain of needs a single workload creates.
Memory and storage
HBM against conventional DDR, what a workload reads and writes, and the checkpoint burst that shapes how AI storage is sized.
Beyond one GPU: NVLink and the rack
Why one GPU is not enough, GPUs running in lockstep, NVLink as the rack-internal GPU interconnect, and the rack as the unit of deployment. Node anatomy across both rack-scale and baseboard form factors.
The four fabrics
The RoCE v2 compute fabric across racks, the converged fabric carrying storage and management and tenant traffic, why the DPU exists as a distinct device from the SuperNIC, and where each fabric lives.
One cluster becomes a fleet
The out-of-band network nobody touches, the two BMCs in every GPU node, and the management cluster that provisions the GPU clusters.
Provisioning and lifecycle
The chain from k0rdent through Metal3/Ironic down to Redfish on each BMC, plus what the GPU Operator and Network Operator take over once a node is running Kubernetes.
Validation and failure domains
UFM and Fabric Manager as continuous fabric validation, blast radius by stack layer, and where to look first when a job slows down or a node drops out.
Scale, and why AI infrastructure differs
Patterns that work at ten nodes and break at a thousand, declarative operations as the default, mapping operator tasks to the component and tool that perform them, and a consolidated comparison against conventional datacenter practice.
FORMAT
90 minutes, instructor-led
Conceptual and architectural
No hands-on lab
Who Should Attend
INFRASTRUCTURE ENGINEER
You have run servers, switches, and racks for years and are now being handed GPU clusters. You need the vocabulary and the architecture before you touch the hardware: what a GPU tray is, why there are four fabrics instead of one, and why two BMCs sit in every node.
DEVOPS, DC OPS, PRODUCT, AND FIELD ROLES
You need to follow and contribute to a technical conversation about GPU topology, RDMA fabric, and cluster automation without implementing it yourself. Analogies and layered diagrams carry the technical slides, and no Linux or scripting background is assumed.
Lab Requirements
General datacenter literacy: servers, racks, switches
Basic networking vocabulary: IP, TCP, VLAN
Awareness that GPUs accelerate machine learning
No RDMA, DPU, or Kubernetes experience assumed
