AI Cluster Provisioning
(AI-INFRA 100-04)
Bare-metal management with k0rdent, Metal3/Ironic, and the operator stack
A hands-on course that takes physical servers to a GPU cluster an ML team can use, on one methodology: deploy a stage, then validate it before moving on. You enroll bare-metal hosts, provision a Kubernetes cluster declaratively, layer the GPU Operator and Network Operator onto it, and run an acceptance test that certifies the result for handoff. The cluster is deliberately mixed, two workers carry GPUs and one does not, because a real fleet is never one shape of node. The course closes by handing you a real provisioning failure and having you descend the layers to its root cause before the answer is revealed.
Description
Enroll and inspect hosts
Provisioning as four explicit gates, why AI clusters are declared rather than built by hand, and the k0rdent object model a request flows through. The BareMetalHost lifecycle and the Redfish verbs running underneath it, with inspection data as the first validation gate.
Lab: enroll and inspect hosts
Provisioning the cluster
What a single kubectl apply sets in motion, and the three watch points that tell you whether it is working: hardware state in Ironic, infrastructure readiness in CAPI, and the k0rdent rollup. Declared state as the contract the system reconciles against.
Lab: provision the GPU cluster
Operators, targeting, and handoff
MultiClusterService as fleet delivery. Deploy the GPU Operator and prove it, deploy the Network Operator and prove bytes actually move. Target GPU workloads to GPU-carrying nodes and everything else away from them, confirm the network path differs correctly by node type, and run the acceptance test.
3 labs: operator stack, workload targeting, acceptance smoke test
Reading a provisioning failure
Descending the layers on a fault you triage yourself: the ClusterDeployment tells you it is not progressing, BareMetalHost status names the error type, the events carry the message, and the Ironic conductor log shows the outbound Redfish calls where the real cause sits.
Lab: reading a provisioning failure
From bare metal to handoff
What the lab built, how it maps onto the machines on your own floor, and how the deploy-and-validate methodology carries from four nodes to thousands: conductor throughput, precompiled driver images, rollout, and blast radius.
FORMAT
3 hours, instructor-led
Short briefing, then terminal time
Real provisioning waits are used as teaching time
Mixed cluster: GPU and non-GPU workers
Hosted lab environment, nothing to install
Who Should Attend
INFRASTRUCTURE ENGINEER
You are responsible for turning racks of GPU servers into clusters other teams consume. You are comfortable with kubectl and YAML. What you need is the declarative provisioning chain end to end, the validation gate at each stage, and the diagnostic path to follow when a host will not come up.
PLATFORM AND SRE TEAMS
You operate the management cluster and own the templates that define what a provisioned cluster looks like. The template as a fleet contract, MultiClusterService rollout, and blast radius when a template changes are the parts of this course aimed at you.
Lab Requirements
Kubernetes basics: kubectl get, describe, apply
Resources, namespaces, DaemonSets
Reading YAML, and jq
Comfort tailing pod logs
Conceptual grasp of the k0rdent, Metal3/Ironic, Redfish chain
What the GPU Operator and Network Operator are for
