power of two choices

TIL about the Power of Two Choices algorithm used in load balancing, queuing theory and distributed systems in general.

The main takeaway is that it is better to pick two queues at random and choose the one with the least amount of work than finding the best queue and sending a workload there. Why? If we are dealing with many decision makers (as in distributed load balancers for example) and each one choose the best queue disregarding each other choices then all their choices will go to the same queue, overwhelming it.

NGINX implements this algorithm for their load balancers for example, check it out!