AI Networking Fundamentals
Collective communication, RDMA, and rail topology explained from the operator's side, including why adding nodes to a training job can make it slower.
advanced· 4 minRDMAInfiniBand
Pillar
AI Infrastructure
The physical and platform layer beneath every model: accelerators, GPU cluster topology, RDMA and collective communication, storage for training and inference, schedulers, capacity planning, power and cooling.
What this section covers
Consolidated guides rather than a page per concept. Each area below is one resource, or will be, rather than a cluster of near-duplicates.
Accelerators
Memory capacity, memory bandwidth, and numeric formats — and why bandwidth rather than FLOPS usually sets your inference throughput.
Interconnect and networking
NVLink, RDMA, InfiniBand, and Ethernet for AI. At multi-node scale the fabric, not the accelerator, is normally the limit.
Storage for training and inference
Three different access patterns — dataset streaming, checkpoint bursts, and weight loading — with three different failure modes.
Capacity, power, and scheduling
Rack power density, cooling, quota, and the schedulers that decide whether expensive hardware sits idle.
Resources
Resources in this pillar
Every resource states its author, its review status, and whether the procedures in it were executed or only reviewed.
GPU Infrastructure for AI
What determines accelerator performance in production: memory capacity versus bandwidth, interconnect topology, and the checks that find a misplaced workload.
advanced· 6 minGPUNVIDIA
Storage Architecture for AI
Dataset streaming, checkpoint bursts, and model loading place completely different demands on storage. Designing for one and getting the others wrong is the usual outcome.
advanced· 4 minStorageNVMe
Where this connects
No layer of the stack is operated in isolation. These are the sections you are most likely to need next.
Cloud & OpenStackThe control plane that turns this hardware into capacity anyone can schedule against.Go →DevOps & PlatformDevice plugins, scheduling, and the orchestration layer that hands accelerators to workloads.Go →ResearchWhere we publish measurement rather than assertion about this layer.Go →
Newsletter
Inside The AI Stack Brief
A practical weekly briefing on AI engineering, infrastructure, production operations, and the technologies powering the AI stack.
One email a week. No sponsorship placements inside the technical sections. Unsubscribe in one click.