Load balancing algorithm

[akvadrako]

I see that linkerd v1 had a bunch of configurable load balancing algorithms. I'm specifically interested in the having requests go to the least-loaded pod.

I can't find any docs on what kind of load balancing linkerd2 is using. Based off the code is seems to be hard-coded to EMWA. Is this accurate?

[hawkw]

I can't find any docs on what kind of load balancing linkerd2 is using. Based off the code is seems to be hard-coded to EMWA. Is this accurate?

This is correct. Linkerd 2 uses the power-of-two-choices (P2C) load balancing algorithm, with exponentially-weighted moving averages (EWMA) as the load estimate. These load estimates are based on latency rather than a request count.

It would certainly be possible to provide a least-loaded load metric to the P2C load balancer, as a replacement for the EWMA latency load metric. We haven't done this largely because there hasn't really been demand for it in the past; latency-based EWMA load balancing provides the lowest latency in most cases. Do you have a specific use-case that requires a least-loaded load estimate?

[akvadrako]

My goal is to fail gracefully and have it easy to understand, so each backend pod will only handle a specific number of simultaneous connections. A least-loaded algorithm will ensure that if a new request is rejected, it's because all the pods are at capacity.

The requests can stream back large amounts of data, taking minutes to complete, so I'm not sure how latency will be measured in a relevant way.