Uber’s Fix for a Server Fleet Where No Two Machines Run the Same Speed

Uber’s infrastructure team published a problem on October 13 that sounds simple until you sit with the number: across the roughly 5 million CPU cores running Odin, Uber’s stateful workload orchestration platform, raw single-core performance varies by close to 100% from the slowest host to the fastest. Same workload, same core count, nearly double the throughput depending on which rack it landed on.

That’s the ordinary cost of running infrastructure for more than a few years. Odin spans both on-prem data centers and public cloud, old x86 generations next to new ARM silicon, hardware bought in different years under different procurement cycles. Nobody runs a fleet of identical machines at Uber’s scale. The interesting part isn’t that the variance exists. It’s what scheduling a workload “by core count” does to you once it does.

The failure mode of counting cores

If a scheduler treats a core as a core, it allocates the same share of “compute” to a workload regardless of which physical host it lands on. Put that workload on a fast host and you’ve overallocated: you’re paying for headroom nobody uses. Put it on a slow host and you’ve underallocated: the workload gets starved exactly when it needs capacity. For stateless services this mostly washes out in aggregate, since requests spread across many hosts. For stateful services like Kafka and Cassandra, where a given partition or replica is pinned to specific hosts and can’t just be rescheduled onto whatever’s free, it doesn’t wash out. It shows up as uneven latency, uneven replication lag, and operators manually working around hosts they’ve learned to distrust.

The fix: stop measuring in cores

Uber’s answer is the Normalized Compute Unit, or NCU. The mechanism is almost aggressively simple: benchmark every host type against a fixed baseline, assign it a Host Performance Factor, and multiply raw CPU usage by that factor before it ever reaches the scheduler. An older server type is the baseline, set at 100. The newest ARM hosts come in at 127. A recent x86 generation lands at 151. The newest x86 hosts benchmark at 196, essentially double the baseline server’s throughput per core.

Once usage is expressed in NCUs instead of raw cores, the Odin scheduler is making allocation decisions in units that actually mean the same thing everywhere in the fleet. A workload asking for a given number of NCUs gets an allocation that performs the same regardless of whether it lands on a 100-factor host or a 196-factor host; the scheduler just hands out more or fewer physical cores to make the normalized number match.

What it bought them

For Kafka specifically, Uber reports an 8% reduction in total allocated cores at the same load, plus a tightening of CPU utilization variance across server types from an 8-percentage-point spread down to 3 points. Read that correctly: this isn’t an 8% cost cut from a clever trick. It’s 8% of cores that used to be allocated purely as a safety margin against the worst-case host, now freed up because the scheduler knows which hosts are actually fast and which are actually slow. That’s capacity that existed the whole time. It was just locked up as insurance against not knowing.

Why this generalizes past Uber

Most teams at Uber’s scale eventually hit this exact wall, and most reach for the same two options first, and both are bad. Option one: buy only identical hardware, forever, which is a procurement constraint nobody actually sticks to once a vendor changes a product line or a better price shows up. Option two: overprovision everywhere to cover the worst host, which is the safety-margin-on-every-box tax Uber just partially eliminated. The NCU approach is really a third option: stop trying to make the fleet homogeneous, and instead make the unit of measurement homogeneous. That’s a cheaper problem to solve, and it’s one you can solve incrementally. Benchmark new hardware as it arrives, assign it a factor, done.

The catch, which Uber’s writeup doesn’t dwell on, is that a Host Performance Factor is a single number standing in for a multi-dimensional thing. Real workloads don’t bottleneck on raw CPU alone. Memory bandwidth, cache size, and I/O characteristics differ across hardware generations too, and a workload that’s cache-sensitive won’t necessarily scale with a CPU-benchmark-derived factor the way a CPU-bound workload will. Uber’s own numbers (8% fewer cores, not 20% or 30%) suggest the team knows this and is treating NCU as a meaningful improvement on ignoring the problem entirely, not a complete solve. Worth keeping in mind if you’re tempted to copy the approach wholesale: benchmark the factor against the workloads you actually run, not just raw CPU throughput.

If you operate a fleet that’s accumulated more than one hardware generation, which is most fleets past year three, the practical takeaway is to check whether your scheduler is quietly assuming a core is a core. If it is, you’re either overpaying for safety margin or shipping uneven performance to whichever service draws the short straw on host assignment.

Source: Uber Engineering, “Ensuring Similar Computational Resources to Stateful Workloads” (October 13, 2026)