The stack decides the machine.
Your scheduler, fabric and serving layer all make assumptions about the hardware underneath. Buying without those assumptions in mind produces a cluster that works while underdelivering.
The scheduler has opinions about your hardware.
Hardware gets chosen first and the software stack gets treated as something to sort out afterwards. That order causes real cost. Your scheduler, your orchestration layer, your storage architecture and your serving framework each impose requirements on the hardware beneath them, and discovering those requirements after the purchase order means either living with a compromise or buying twice.
The relationship runs both ways. A serving framework with efficient memory management changes how much GPU memory you actually need, which changes the SKU, which changes the price. Teams that size hardware without knowing what will run on it consistently over-buy.
Five questions worth answering before the order.
For scheduling, Slurm covers what a traditional HPC scheduler expects from node uniformity and interconnect, while Kubernetes deals with why bare metal matters more than it does for ordinary workloads and where the virtualisation overhead actually lands.
For the fabric, InfiniBand sets out when the premium is justified and when Ethernet is genuinely sufficient. On the data side, storage for AI explains why AI storage requirements look nothing like general enterprise storage, and why under-specifying it leaves expensive accelerators idle.
For distributed workloads, Ray rewards a balance of GPU and CPU rather than maximum accelerator density. And for production serving, vLLM has changed inference sizing enough that estimates made before it are usually wrong in your favour.
Asked first, answered straight.
Frequently, and by more than people expect. Serving frameworks with efficient memory handling can reduce the accelerator memory you need, and schedulers impose their own requirements on node uniformity and interconnect.
For large multi-node training, usually. For inference, fine tuning and smaller jobs, high-speed Ethernet is often sufficient, and the saving is substantial. The deciding factor is how much cross-node communication your workload actually generates.
Bare metal, for anything performance-sensitive at scale. The virtualisation overhead is real, and the variability introduced by shared infrastructure is harder to plan around than the raw performance difference.
Slurm
Homogeneous nodes, fast fabric and shared storage. Partition planning at purchase time.
Kubernetes
Bare metal control, predictable node shapes and multi-tenant partitioning.
InfiniBand
When it beats Ethernet, and the transceiver arithmetic that breaks budgets.
Storage for AI
Parallel filesystems sized on sustained throughput, not enterprise IOPS.
Ray
Balanced nodes rather than GPU-maximised ones, because pipelines are part CPU-bound.
vLLM and serving
Memory for KV cache at concurrency, and why geography beats specification.
Working through this on a real requirement?
Twenty minutes with the desk, no pitch and no quote at the end of it. Tell us roughly what you need and we will come back within one business day.
Sent. We are on it.
Your enquiry has landed with the desk. Acknowledged within one hour.