Load Balancing Algorithms

Load balancing algorithms are essential for distributing incoming network traffic across multiple servers, ensuring optimal utilization of resources, preventing server overload, and enhancing the performance and reliability of applications. Various load-balancing algorithms exist, each with its characteristics.

Important Topics for the Load Balancing Algorithms

Static Load Balancing Algorithms
Dynamic Load Balancing Algorithms
- Least Connection Method Load Balancing Algorithm
- Least Response Time Method Load Balancing Algorithm

Load balancing algorithms can be broadly categorized into two types: Dynamic load balancing and Static load balancing.

1. Static Load Balancing Algorithms

Static load balancing involves predetermined assignment of tasks or resources without considering real-time variations in the system. This approach relies on a fixed allocation of workloads to servers or resources, and it doesn’t adapt to changes during runtime.

Types of Static Load Balancing Algorithms

Round Robin
Weighted Round-Robin
Source IP hash

1.1. Round Robin Load Balancing Algorithm

The Round Robin algorithm is a simple static load balancing approach in which requests are distributed across the servers in a sequential or rotational manner. It is easy to implement but it doesn’t consider the load already on a server so there is a risk that one of the servers receives a lot of requests and becomes overloaded.

For example:

Imagine you have a group of friends, and you want to share a bag of candies equally with all of them. You give one candy to each friend in a circle, and then you start over. This is like Round Robin – making sure everyone gets a fair share.

We need to implement a basic Round Robin load balancing algorithm. The goal is to distribute incoming requests evenly among a list of servers. The first request goes to the first server, the second one goes to the second server, the third request goes to the third server and it continues further for all the requests.

Below is the implementation of the Round Robin Load Balancing Algorithms:

Java

import java.util.ArrayList;

import java.util.List;
 
class LoadBalancer {

    private List<String> servers;

    private int currentIndex;
 
    public LoadBalancer(List<String> servers) {

        this.servers = new ArrayList<>(servers);

        this.currentIndex = 0;

    }
 
    public String getNextServer() {

        String nextServer = servers.get(currentIndex);

        currentIndex = (currentIndex + 1) % servers.size();

        return nextServer;

    }
}
 
public class RoundRobinExample {

    public static void main(String[] args) {

        // Sample list of servers

        List<String> serverList = new ArrayList<>();

        serverList.add("Server1");

        serverList.add("Server2");

        serverList.add("Server3");
 
        // Create a load balancer with the server list

        LoadBalancer loadBalancer = new LoadBalancer(serverList);
 
        // Simulate requests to the load balancer

        for (int i = 0; i < 10; i++) {

            String nextServer = loadBalancer.getNextServer();

            System.out.println("Request " + (i + 1) + ": Routed to " + nextServer);

        }

    }
}

Output

Request 1: Routed to Server1
Request 2: Routed to Server2
Request 3: Routed to Server3
Request 4: Routed to Server1
Request 5: Routed to Server2
Request 6: Routed to Server3
Request 7: Routed to Serve...

Here is the explanation of the above code:

Load Balancer Class (LoadBalancer):
- Maintain a list of server names.
- Implement a method (getNextServer) that returns the next server in a round-robin fashion.
- Keep track of the current index to determine the next server.
Main Class (RoundRobinExample):
- Create a list of server names to be used in the load balancer.
- Instantiate a LoadBalancer object with the list of servers.
- Simulate a series of requests by repeatedly calling the getNextServer method.
- Print the server to which each request is routed.

Usecases of Round Robin Load Balancing Algorithm:

Simple and fair distribution of incoming requests among servers.
Suitable for scenarios where servers have similar processing capabilities.
Prevents overloading a single server while ensuring all servers get a share of the workload.

Benefits and Drawbacks of Weighted Round Robin Load Balancing Algorithm:

Benefits:
- Simplicity: Easy to implement and understand.
- Fairness: Ensures that each server gets an equal share of the load.
Drawbacks:
- Unequal Capacities: Doesn’t consider the varying capacities of servers; treats all servers equally.
- Predictability: May not be optimal for scenarios with heterogeneous server capacities.

1.2. Weighted Round Robin Load Balancing Algorithm

The Weighted Round Robin algorithm is also a static load balancing approach which is much similar to the round-robin technique. The only difference is, that each of the resources in a list is provided a weighted score. Depending on the weighted score the request is distributed to these servers.

Servers with higher weights are given a larger proportion of the requests.
The distribution is cyclic, similar to the round-robin technique, but with each server receiving a number of requests proportional to its weight.
In a basic Weighted Round Robin implementation, there may not be an explicit limit on the number of requests each server can handle.
The limitation comes from the capacity of each server, and the algorithm assumes that each server can handle its allocated proportion of requests.
If a server reaches its processing capacity, it may start rejecting or queuing additional requests, depending on the server’s specific behavior.
There may be scenarios where weights can be adjusted dynamically based on server health, load, or other factors.

For example:

let’s say your friends have different levels of candy cravings. You want to be fair, so you give more candies to the friend who loves them the most. Weighted Round Robin does something similar – it gives more tasks to the friends who can handle them better.

Let’s say you have three servers with weights: Server1 (weight 0.3), Server2 (weight 0.2), and Server3 (weight 0.1). The total weight is 0.3 + 0.2 + 0.1 = 0.6. During each cycle, Server1 would receive 0.3/0.6 (50%) of the requests, Server2 would receive 0.2/0.6 (33.33%), and Server3 would receive 0.1/0.6 (16.67%).

Below is the implementation of the Weighted Round Robin Load Balancing Algorithms:

Java

import java.util.ArrayList;

import java.util.List;

import java.util.Random;
 
class WeightedRoundRobinBalancer {

    private List<Server> servers;

    private int[] cumulativeWeights;

    private int totalWeight;

    private int currentIndex;

    private Random random;
 
    public WeightedRoundRobinBalancer(List<Server> servers) {

        this.servers = new ArrayList<>(servers);

        this.totalWeight = calculateTotalWeight(servers);

        this.cumulativeWeights = calculateCumulativeWeights(servers);

        this.currentIndex = 0;

        this.random = new Random();

    }
 
    private int calculateTotalWeight(List<Server> servers) {

        int totalWeight = 0;

        for (Server server : servers) {

            totalWeight += server.getWeight();

        }

        return totalWeight;

    }
 
    private int[] calculateCumulativeWeights(List<Server> servers) {

        int[] cumulativeWeights = new int[servers.size()];

        cumulativeWeights[0] = servers.get(0).getWeight();

        for (int i = 1; i < servers.size(); i++) {

            cumulativeWeights[i] = cumulativeWeights[i - 1] + servers.get(i).getWeight();

        }

        return cumulativeWeights;

    }
 
    public Server getNextServer() {

        int randomValue = random.nextInt(totalWeight);

        for (int i = 0; i < cumulativeWeights.length; i++) {

            if (randomValue < cumulativeWeights[i]) {

                currentIndex = i;

                break;

            }

        }

        return servers.get(currentIndex);

    }
 
    // Inner class representing a server with a weight

    static class Server {

        private String name;

        private int weight;
 
        public Server(String name, int weight) {

            this.name = name;

            this.weight = weight;

        }
 
        public String getName() {

            return name;

        }
 
        public int getWeight() {

            return weight;

        }

    }
}
 
public class WeightedRoundRobinExample {

    public static void main(String[] args) {

        // Sample list of servers with weights

        List<WeightedRoundRobinBalancer.Server> serverList = new ArrayList<>();

        serverList.add(new WeightedRoundRobinBalancer.Server("Server1", 3));

        serverList.add(new WeightedRoundRobinBalancer.Server("Server2", 2));

        serverList.add(new WeightedRoundRobinBalancer.Server("Server3", 1));
 
        // Create a weighted round-robin load balancer with the server list

        WeightedRoundRobinBalancer balancer = new WeightedRoundRobinBalancer(serverList);
 
        // Simulate requests to the load balancer

        for (int i = 0; i < 10; i++) {

            WeightedRoundRobinBalancer.Server nextServer = balancer.getNextServer();

            System.out.println("Request " + (i + 1) + ": Routed to " + nextServer.getName());

        }

    }
}

Output

Request 1: Routed to Server1
Request 2: Routed to Server2
Request 3: Routed to Server2
Request 4: Routed to Server3
Request 5: Routed to Server3
Request 6: Routed to Server3
Request 7: Routed to Serve...

Here is the explanation of the above code:

WeightedRoundRobinBalancer Class:
- Manages a list of servers with weights.
- Calculates total and cumulative weights during initialization.
- Provides a method (getNextServer) to retrieve the next server based on weighted round-robin.
- Contains an inner class (Server) representing a server with a name and weight.
WeightedRoundRobinExample Class:
- Demonstrates the usage of the WeightedRoundRobinBalancer.
- Creates a sample list of servers with weights.
- Instantiates a WeightedRoundRobinBalancer object.
- Simulates requests and prints the server to which each request is routed.

Usecases of Weighted Round Robin Load Balancing Algorithm:

Differentiate server capacities by assigning weights.
Allocate a proportionally higher share of the workload to servers with higher capacities.
Useful when servers have varying processing power or resources.

Benefits and Drawbacks of Weighted Round Robin Load Balancing Algorithm:

Benefits:
- Capacity Consideration: Accounts for different server capacities by assigning weights.
- Flexibility: Can be adjusted to handle varying workloads effectively.
Drawbacks:
- Complexity: More complex than simple Round Robin.
- Maintenance: Requires adjusting weights as server capacities change.

1.3. Source IP Hash Load Balancing Algorithm

The Source IP Hash Load Balancing Algorithm is a method used in network load balancing to distribute incoming requests among a set of servers based on the hash value of the source IP address. This algorithm aims to ensure that requests originating from the same source IP address are consistently directed to the same server.

How Source IP hash load balancing algorithm works:

Step-1: A client initiates a connection by sending a request to a load balancer.
Step-2: The load balancer intercepts the incoming request.
Step-3: The load balancer extracts the source IP address from the incoming request.
Step-4: The load balancer applies a hash function to the source IP address. The hash function generates a fixed-size hash value based on the input, ensuring that similar source IP addresses result in the same hash value.
Step-5: The load balancer uses the hash value to determine which server in the backend pool should handle the request. This is typically done by mapping the hash value to a server index.
Step-6: The load balancer forwards the request to the server that corresponds to the determined index.
Step-7: The selected server processes the request and sends the response back to the client through the load balancer.

If the load balancer is configured for session persistence, it ensures that subsequent requests from the same source IP address are consistently directed to the same server. This is beneficial for applications that require maintaining session information or state.

For example:

Think of your friends coming to your house, and you want to remember who gets which toy every time they visit. IP Hash is like remembering which friend played with which toy last time, so you always give them the same one.

We need to implement a load balancing algorithm that distributes incoming requests across a set of servers based on the hash of the source IP address. The goal is to ensure that requests coming from the same source IP address are consistently routed to the same server.

Below is the implementation of the Source IP Hash Load Balancing Algorithms:

Java

import java.util.HashMap;

import java.util.Map;
 
class SourceIpHashLoadBalancer {

    private Map<String, String> ipToServerMap;
 
    public SourceIpHashLoadBalancer() {

        this.ipToServerMap = new HashMap<>();

    }
 
    public void addServer(String serverName) {

        // Add server to the mapping

        ipToServerMap.put(serverName, serverName);

    }
 
    public String getServerForIp(String sourceIp) {

        // Calculate hash of the source IP

        int hash = sourceIp.hashCode();
 
        // Get the list of available servers

        String[] servers = ipToServerMap.keySet().toArray(new String[0]);
 
        // Map the hash value to a server index

        int serverIndex = Math.abs(hash) % servers.length;
 
        // Return the selected server

        return servers[serverIndex];

    }
}
 
public class SourceIpHashLoadBalancerExample {

    public static void main(String[] args) {

        // Create a source IP hash load balancer

        SourceIpHashLoadBalancer loadBalancer = new SourceIpHashLoadBalancer();
 
        // Add servers to the load balancer

        loadBalancer.addServer("Server1");

        loadBalancer.addServer("Server2");

        loadBalancer.addServer("Server3");
 
        // Simulate requests with different source IPs

        String[] sourceIps = {"192.168.1.1", "10.0.0.1", "172.16.0.1"};
 
        for (String sourceIp : sourceIps) {

            String selectedServer = loadBalancer.getServerForIp(sourceIp);

            System.out.println("Request from " + sourceIp + " routed to " + selectedServer);

        }

    }
}

Output

Request from 192.168.1.1 routed to Server2
Request from 10.0.0.1 routed to Server3
Request from 172.16.0.1 routed to Server1

Here is the explanation of the above code:

SourceIpHashLoadBalancer Class:
- Fields:
  - ipToServerMap: A mapping of server names to server names (used for consistent hashing).
- Methods:
  - addServer: Adds a server to the load balancer.
  - getServerForIp: Calculates the hash of the source IP and determines the server to handle the request.
SourceIpHashLoadBalancerExample Class:
- Demonstrates the usage of the SourceIpHashLoadBalancer.
- Creates an instance of the load balancer, adds servers, and simulates requests from different source IPs.

Usecases of Source IP Hash Load Balancing Algorithm:

Ensure session persistence by consistently routing requests from the same IP address to the same server.
Suitable for scenarios where maintaining state information on a specific server is crucial.
Helps in scenarios where connection persistence is important, such as in web applications with user sessions.

Benefits and Drawbacks of Source IP Hash Load Balancing Algorithm:

Benefits:
- Consistency: Ensures requests from the same source IP always go to the same server, maintaining session state.
- Predictability: Useful when connection persistence is critical.
Drawbacks:
- Limited Distribution: May lead to uneven load distribution if certain source IPs are more active.
- Scaling Challenges: Adding or removing servers may disrupt session persistence.

2. Dynamic Load Balancing Algorithms

Dynamic load balancing involves making real-time decisions about how to distribute incoming network traffic or computational workload across multiple servers or resources. This approach adapts to the changing conditions of the system, such as variations in server load, network traffic, or resource availability.

Types of Dynamic Load Balancing Algorithms

Least Connection Method
Least Response Time Method

The choice between dynamic and static load balancing depends on the characteristics of the system, the nature of the workload, and the desired level of adaptability. Dynamic load balancing is often favored in dynamic, high-traffic environments, while static load balancing may be suitable for more predictable scenarios.

2.1. Least Connection Method Load Balancing Algorithm

The Least Connections algorithm is a dynamic load balancing approach that assigns new requests to the server with the fewest active connections. The idea is to distribute incoming workloads in a way that minimizes the current load on each server, aiming for a balanced distribution of connections across all available resources.

To do this load balancer needs to do some additional computing to identify the server with the least number of connections.
This may be a little bit costlier compared to the round-robin method but the evaluation is based on the current load on the server.

For example:

Imagine you’re at a playground, and some kids are playing on different swings. You want to join the swing with the fewest kids so that it’s not too crowded. Least Connection is like choosing the swing with the least number of kids already on it.

We need to implement a Load Balancing Algorithm that distribute incoming requests across a set of servers and should aim to minimize the number of active connections on each server by directing new requests to the server with the fewest active connections. This ensures a balanced distribution of the workload and prevents overload on individual servers.

Below is the implementation of the Least Connection Method Load Balancing Algorithms:

Java

import java.util.ArrayList;

import java.util.HashMap;

import java.util.List;

import java.util.Map;
 
class LeastConnectionLoadBalancer {

    private Map<String, Integer> serverConnections;
 
    public LeastConnectionLoadBalancer() {

        this.serverConnections = new HashMap<>();

    }
 
    public void addServer(String serverName) {

        // Add a server to the load balancer with 0 initial connections

        serverConnections.put(serverName, 0);

    }
 
    public String getServerWithLeastConnections() {

        // Find the server with the least active connections

        int minConnections = Integer.MAX_VALUE;

        String selectedServer = null;
 
        for (Map.Entry<String, Integer> entry : serverConnections.entrySet()) {

            if (entry.getValue() < minConnections) {

                minConnections = entry.getValue();

                selectedServer = entry.getKey();

            }

        }
 
        // Increment the connection count for the selected server

        if (selectedServer != null) {

            serverConnections.put(selectedServer, minConnections + 1);

        }
 
        return selectedServer;

    }
}
 
public class LeastConnectionLoadBalancerExample {

    public static void main(String[] args) {

        // Create a Least Connection load balancer

        LeastConnectionLoadBalancer loadBalancer = new LeastConnectionLoadBalancer();
 
        // Add servers to the load balancer

        loadBalancer.addServer("Server1");

        loadBalancer.addServer("Server2");

        loadBalancer.addServer("Server3");
 
        // Simulate requests and print the server to which each request is routed

        for (int i = 0; i < 10; i++) {

            String selectedServer = loadBalancer.getServerWithLeastConnections();

            System.out.println("Request " + (i + 1) + ": Routed to " + selectedServer);

        }

    }
}

Output

Request 1: Routed to Server1
Request 2: Routed to Server2
Request 3: Routed to Server3
Request 4: Routed to Server1
Request 5: Routed to Server2
Request 6: Routed to Server3
Request 7: Routed to Serve...

Here is the explanation of the above code:

LeastConnectionLoadBalancer Class:
- Fields:
  - serverConnections: A map that tracks the number of active connections for each server.
- Methods:
  - addServer: Adds a server to the load balancer with an initial connection count of 0.
  - getServerWithLeastConnections: Determines the server with the least active connections and increments its connection count.
LeastConnectionLoadBalancerExample Class:
- Main Method:
  - Creates an instance of the LeastConnectionLoadBalancer.
  - Adds servers to the load balancer.
  - Simulates requests and prints the server to which each request is routed based on the least connection algorithm.

Usecases ofLeast Connection Load Balancing Algorithm:

Distribute incoming requests to servers with the fewest active connections.
Prevent overloading busy servers and distribute the load evenly.
Suitable for scenarios where the number of active connections correlates with server load.

Benefits and Drawbacks of Least Connection Load Balancing Algorithm:

Benefits:
- Balanced Load: Distributes traffic to servers with the fewest active connections, preventing overloading.
- Dynamic: Adapts to changing server workloads.
Drawbacks:
- Ignored Capacities: Ignores server capacities; a server with fewer connections may still have less capacity.
- Sticky Sessions: May not be suitable for scenarios requiring session persistence.

2.2. Least Response Time Method Load Balancing Algorithm

The Least Response method is a dynamic load balancing approach that aims to minimize response times by directing new requests to the server with the quickest response time.

It considers the historical performance of servers to make decisions about where to route incoming requests, optimizing for faster processing.
The dynamic aspect comes from the continuous monitoring of server response times and the adaptive nature of the algorithm to route incoming requests to the server with the historically lowest response time.

For example:

Picture yourself at a snack bar where you can order food from different servers. You notice that some servers are faster than others. You choose the server that seems to serve food the quickest each time you go. Least Response Time is like picking the server with the shortest line.

We need to implement a Load Balancing Algorithm that distribute incoming requests across a set of servers and should aim to minimize the response time by directing new requests to the server with the least accumulated response time. This ensures a balanced distribution of the workload and helps optimize the overall system performance.

Below is the implementation of the Least Response Time Load Balancing Algorithms:

Java

import java.util.HashMap;

import java.util.Map;
 
class LeastResponseLoadBalancer {

    private Map<String, Long> serverResponseTimes;
 
    public LeastResponseLoadBalancer() {

        this.serverResponseTimes = new HashMap<>();

    }
 
    public void addServer(String serverName) {

        // Add a server to the load balancer with 0 initial response time

        serverResponseTimes.put(serverName, 0L);

    }
 
    public String getServerWithLeastResponseTime() {

        // Find the server with the least accumulated response time

        long minResponseTime = Long.MAX_VALUE;

        String selectedServer = null;
 
        for (Map.Entry<String, Long> entry : serverResponseTimes.entrySet()) {

            if (entry.getValue() < minResponseTime) {

                minResponseTime = entry.getValue();

                selectedServer = entry.getKey();

            }

        }
 
        // Increment the response time for the selected server

        if (selectedServer != null) {

            serverResponseTimes.put(selectedServer, minResponseTime + 1);

        }
 
        return selectedServer;

    }
}
 
public class LeastResponseLoadBalancerExample {

    public static void main(String[] args) {

        // Create a Least Response load balancer

        LeastResponseLoadBalancer loadBalancer = new LeastResponseLoadBalancer();
 
        // Add servers to the load balancer

        loadBalancer.addServer("Server1");

        loadBalancer.addServer("Server2");

        loadBalancer.addServer("Server3");
 
        // Simulate requests and print the server to which each request is routed

        for (int i = 0; i < 10; i++) {

            String selectedServer = loadBalancer.getServerWithLeastResponseTime();

            System.out.println("Request " + (i + 1) + ": Routed to " + selectedServer);

        }

    }
}

Output

Request 1: Routed to Server1
Request 2: Routed to Server2
Request 3: Routed to Server3
Request 4: Routed to Server1
Request 5: Routed to Server2
Request 6: Routed to Server3
Request 7: Routed to Serve...

Here is the explanation of the above code:

LeastResponseLoadBalancer Class:
- Fields:
  - serverResponseTimes: A map that tracks the accumulated response time for each server.
- Methods:
  - addServer: Adds a server to the load balancer with an initial response time of 0.
  - getServerWithLeastResponseTime: Determines the server with the least accumulated response time and increments its response time.
LeastResponseLoadBalancerExample Class:
- Main Method:
  - Creates an instance of the LeastResponseLoadBalancer.
  - Adds servers to the load balancer.
  - Simulates requests and prints the server to which each request is routed based on the least response time algorithm.

Usecases of Least Response Time Load Balancing Algorithm:

Minimize overall system response time by directing new requests to servers with the least accumulated response time.
Optimize system performance by distributing workload based on historical response times.
Suitable for scenarios where minimizing response time is a critical requirement.

Benefits and Drawbacks of Least Response Time Load Balancing Algorithm:

Benefits:
- Optimized Performance: Directs traffic to servers with the quickest response times, optimizing overall system performance.
- Adaptable: Adjusts to changes in server responsiveness over time.
Drawbacks:
- Historical Bias: Heavily influenced by past response times, may not always reflect current server capabilities.
- Complex Implementation: Requires tracking and managing historical response times.

Article Tags :

System Design

Load Balancer