Load Balancing Strategies
After the RPC client gets multiple providers from service discovery, it must choose one for each call—load balancing. Common strategies: random, round-robin, least active, consistent hash. This article explains when each fits and tradeoffs with a comparison table.
Overview
- Random: Choose one at random; simple, roughly even when instance count is high. Short-term imbalance possible.
- Round-robin: Choose in order; even distribution. Slow instances can pile up if performance varies.
- Least active: Choose the instance with the fewest in-flight requests; slow instances get fewer calls, fast ones more. Suited when instance performance differs.
- Consistent hash: Hash a parameter (e.g. userId) onto a ring; same parameter always maps to the same instance. Suited for session stickiness or cache locality. Consider rehash impact on scale-up/down.
- Weighted: Any strategy can be weighted; higher weight for stronger instances means more traffic.
Example
Example 1: Strategy comparison
| Strategy | Pros | Cons | Best for |
|---|---|---|---|
| Random | Simple, stateless | Short-term imbalance | Homogeneous instances |
| RoundRobin | Even distribution | Slow instances can bottleneck | Homogeneous instances |
| LeastActive | Adapts to slow instances | Requires active count tracking | Varying instance performance |
| ConsistentHash | Stickiness, locality | Rehash on scale | Stateful, cache locality |
Example 2: Weighted round-robin
- Instance A weight 3, B weight 1. Round-robin sequence: A A A B A A A B …; A gets about 75% of traffic.
Example 3: Dubbo configuration
XML<dubbo:reference loadbalance="leastactive"/> <!-- Or random, roundrobin, consistenthash -->
Core Mechanism / Behavior
- Random / RoundRobin: Stateless; no coordination. Easy to implement and reason about.
- LeastActive: Requires per-instance in-flight count. Adds a bit of state but better adapts to performance differences.
- ConsistentHash: Virtual nodes improve distribution. On instance add/remove, only some keys remap.
Key Rules
- For homogeneous, stateless instances use Random or RoundRobin; for varying performance use LeastActive.
- For stickiness (same user → same instance) use ConsistentHash; be aware of rehash on scale.
- Combine with health checks so unhealthy instances are excluded from load balancing.
What's Next
See RPC Fundamentals and Dubbo Architecture. See CAP and High Availability for multi-instance deployment.