Random — Load Balancing

Overview

What this concept solves

Random load balancing is exactly what it sounds like: for each request, pick a server uniformly at random. No counter, no state, no coordination. It sounds too crude to work — and over a handful of requests it is lumpy — but as volume grows the law of large numbers drags the distribution toward a perfectly even split, just like round robin.

Its real superpower shows up in distributed systems. Round robin needs a shared counter to stay coordinated across many load-balancer instances. Random needs nothing shared at all: a thousand independent balancers each flipping their own coin still produce an even global distribution, because uniform randomness composes. That zero-coordination property is why random (and its smarter cousin, power-of-two-choices) underpins so many large-scale systems.

Mechanics

How it works

One line of logic

Pick an index in [0, N) uniformly and forward the request there:

text

target = floor(random() * N)

Each server has probability 1/N of being chosen on every request, independent of every other request. The expected share for each is exactly 1/N — 25% with four servers.

Lumpy now, even later

Because picks are independent, short runs cluster: you'll see S2, S2, S2 in a row, and one server can be 40% ahead after twenty requests. This is normal variance, not a bug. The deviation from the ideal 1/N share shrinks on the order of 1/√n — so it's loud at n=20 and nearly silent at n=10,000. The prototype's 'Max deviation from 25%' stat makes this concrete: watch it fall each time you add another 100 requests.

Weighted random

Random extends to uneven fleets the same way round robin does: give each server a weight and pick proportionally (draw a number in [0, ΣW) and find which server's cumulative band it lands in). That's weighted random — the stateless cousin of weighted round robin.

Interactive prototype

Run it. Break it. Tune it.

Sandboxed simulation embedded right in the page. No setup, no install.

simulation › Random

About this simulation

The load balancer flips a 4-way coin for every request — p = ¼ per server, no memory of past picks. The 'Recent picks' strip shows the lumpiness up close; the 'Max deviation from 25%' stat shrinks as you pile on requests. Hit '+100 instant' a few times to watch it converge.

Hands-on

Try these on your own

Open the prototype above, run each experiment, predict the answer, then verify.

try 01

Feel the lumpiness

Click 'Send one' about fifteen times and read the 'Recent picks' strip. You'll almost certainly see a server repeat two or three times in a row, and the handled counts will look uneven. That short-run clustering is expected — independent coin flips cluster.

try 02

Watch it converge

Now hammer '+100 instant' five or six times. The 'Max deviation from 25%' stat falls steadily — often into the low single digits — and the 'Actual vs target' row creeps toward 25% / 25% / 25% / 25%. That's the law of large numbers doing round robin's job without a counter.

try 03

Lumpy vs even, side by side

Reset, send just 8 requests, and note the max deviation (likely 10–25%). Reset again and '+100 instant' twice for ~200 requests, and compare. Same algorithm, dramatically tighter distribution — deviation shrinks roughly like 1/√n as n grows.

In practice

When to use it — and what you give up

When to reach for it

Many uncoordinated load balancers — client-side balancing in RPC libraries, or a fleet of edge proxies that can't cheaply share a counter.
You want round-robin fairness without shared state — random matches it in the limit with literally no bookkeeping.
As the base for smarter schemes — 'power of two choices' picks two servers at random and routes to the less busy one, getting most of least-connections' benefit while staying nearly stateless.
High request volume — the more traffic, the closer random gets to a perfect split, so it's ideal exactly where load balancing matters most.

Not for low volume or long-lived connections

With few requests, random's variance is real — one backend can genuinely run hot. And like round robin it's blind to request duration, so it's a poor fit for long-lived or wildly variable connections. When those matter, move to least-connections or power-of-two-choices.

Pros

Zero shared state — scales to any number of independent load balancers with no coordination.
Matches round robin's even distribution as request volume grows.
Trivial and O(1): a single random draw, no counters to maintain or synchronize.
Foundation for power-of-two-choices, which gets near-least-connections quality almost for free.

Cons

Lumpy over short runs — real short-term imbalance from normal variance.
Blind to request cost and server load, exactly like round robin.
No guarantee of even counts at low volume; a small service can see a genuinely overloaded backend.
Depends on a decent PRNG; a biased random source skews the whole distribution.

Reference

Code & further reading

A minimal reference implementation and pointers worth bookmarking.

random.ts

// Uniform random: one draw, no state, no coordination.
class RandomBalancer {
  constructor(private servers: string[]) {}
  pick(): string {
    const i = Math.floor(Math.random() * this.servers.length);
    return this.servers[i];
  }
}

// Weighted random: draw into the cumulative-weight band.
class WeightedRandomBalancer {
  private total: number;
  constructor(private pool: { id: string; weight: number }[]) {
    this.total = pool.reduce((s, p) => s + p.weight, 0);
  }
  pick(): string {
    let r = Math.random() * this.total;
    for (const p of this.pool) {
      if ((r -= p.weight) < 0) return p.id;
    }
    return this.pool[this.pool.length - 1].id; // float-rounding guard
  }
}

References & further reading

7 sources

Knowledge check

Did the prototype land?

Quick questions, answers revealed on submit. No scoring saved.

question 01 / 03

Over a very large number of requests, how does random's distribution compare to round robin's?

question 02 / 03

What is random load balancing's key advantage over round robin in a distributed system?

question 03 / 03

Roughly how does random's deviation from the ideal even split behave as the number of requests n grows?

0/3 answered