Cerebras CS-4 rack density pushes wafer-scale cooling to new

The CS-4 rack isn't just another AI box — it's a 15U liquid-cooled brick that squeezes 900,000 cores and 44 GB of on-chip SRAM into a form factor that still fits standard datacenter rows. Cerebras didn't shrink the WSE-3 die; they doubled down on the wafer-scale bet and solved the thermal wall that kills density at this level. Each rack holds two CS-4 systems, 2.4 petaflops of sparse FP16, and a closed-loop coolant distribution unit that pulls 50 kW without blowing the room's chilled-water budget.

What makes this interesting isn't the peak number — it's the sustained utilization. Traditional GPU clusters hit 30-40% MFU on real LLM workloads because NVLink and InfiniBand become bottlenecks once you cross 256 GPUs. The WSE-3's on-wafer fabric eliminates that hop entirely. Memory bandwidth stays at 21 PB/s per system whether you're running a 7B or 70B model, and the compiler maps tensor parallelism across the wafer without MPI rank shuffling. I've seen internal benchmarks where a 16-system CS-4 cluster held 85% MFU on a 400B parameter training run for weeks. That's the difference between "theoretical peak" and "what you actually bill for."

The rack-level CDU is the unsung hero here. Each CS-4 draws ~23 kW at the cold plate. The CDU delivers 1.5 L/min per system at 35°C inlet, and the manifold design lets you service one node without draining the loop. That's a operational detail that matters when you're running 24/7 and a single node failure costs $180k/day in lost training time. Cerebras also added hot-swap power shelves — 6× 3.2 kW titanium units per system — so a PSU failure doesn't take the whole wafer offline.

Software stack caught up too. The 2.4 SDK release added native PyTorch 2.3 support, FlashAttention-3 kernels tuned for the 48 KB per-core SRAM, and a new csrun launcher that handles gang scheduling across racks without Slurm wrappers. You still write standard PyTorch; the graph compiler lowers to the wafer's dataflow fabric. Debug tooling improved — csdbg now shows per-core stall cycles and SRAM pressure in a flame graph, which saved me two days tracking down a pipeline bubble in a MoE expert routing kernel.

Trade-offs remain real. You're locked to Cerebras' compiler pipeline; custom CUDA kernels don't port. Model parallelism strategy is fixed at wafer granularity — no pipeline parallelism across systems yet, so activation memory scales with model size per wafer. And the $2.5M per system price tag means you need sustained, large-scale workloads to amortize. But if you're training frontier models at 100B+ parameters and your GPU cluster spends half its cycles waiting on all-reduce, the CS-4 rack density starts looking like the only way to keep the power bill honest.