Load Distribution

This article describes Load Distribution and how to set it up.

1 Introduction

    Load distribution or load balancing (these terms will be used interchangeably in this document) mean that some process (front-end) acts as a distributor of HTTP/HTTPS requests and send different requests off to different servers, or back-ends.

    Throughout this article, a cluster will mean a number of cluster enabled Web-sites. Each cluster enabled Web-site will be refered to as a cluster node.


    Figure 1: The LoadBalancer distributes client requests to the cluster nodes.

    Bundled with the Orion Application Server is the LoadBalancer which can be used to distribute HTTP/HTTPS requests to different cluster nodes.

    The LoadBalancer uses the multicast address 230.0.0.2:27512 to receive notification from cluster nodes in order to dynamically add and remove available nodes.


    Figure 2: All cluster nodes with a cluster id value of 1 or higher uses the multicast address 230.0.0.2:27512 to notify the LoadBalancer that they are operational.

    If a cluster node known by the LoadBalancer is gracefully stopped, it will notify the LoadBalancer that it is no longer available. If request is sent to a cluster node that isn't responding, the LoadBalancer will route the request to another cluster node in its list.

    1.1 Normal behaviour

      The default behaviour of the LoadBalancer is as follows:

      • If a new request is made from an IP that has not connected to the Web-module before and that has no session associated with it, it will send off the user to a random cluster node.

      • If a request is made from an IP that has already been "talking" to the Web-module, the request will get sent to the same cluster node as the previous request, unless it is specified that requests will not be routed based on IP (-dontUseIP as command line option or use-ip="false" in load-balancer.xml).

      • If a request is made within a Keep-Alive socket, the request will get sent to the same cluster node as the previous request, unless you have specified that Keep-Alives should not be used (-dontUseKeepalives as command line option or use-keepalives="false" in load-balancer.xml

      • If a request if made from a user with a session, the request will get sent to the primary cluster node for that session (ie the node that the session was created on). If the primary cluster node for the session does not respond, the request will be sent to another cluster node in the same cluster island. Since the state has been replicated the other cluster nodes has the same user state.

      • If a request to the proper cluster node fails, the request will be sent to another cluster node.

      • If a cluster node notifies the LoadBalancer that it is available, the LoadBalancer will add the cluster node to its list of operatinal cluster nodes

      • If a cluster node notifies the LoadBalancer that it is unavailable, the LoadBalancer will remove the cluster node from its list of operational cluster nodes

      • If no cluster nodes are operational, the LoadBalancer will display a 503 Service Unavailable message

    1.2 Requirements

      In order for LoadBalancer to work, the following requirements must be met:

      • LoadBalancer must be able to use multicast

      • LoadBalancer needs to be able to reach all cluster nodes in order to distribute/redirect requests

      • LoadBalancer needs to be able to communicate with clients over the specified address/port

      • LoadBalancer needs to be able to receive data on multicast 230.0.0.2:27512 to hear when cluster nodes are available

      • Any cluster node needs to be able to send data on multicast 230.0.0.2:27512 to inform the LoadBalancer of their availability

      • Any cluster node needs to be configured with a cluster-island value greater than 0 in order to inform the LoadBalancer of their availablility

2 Starting the LoadBalancer

    This section describes how to start the LoadBalancer.

    The LoadBalancer is started using the following command in the Orion installation directory:


            java [system properties} -jar loadbalancer.jar [options]
    Listing 1: Starting the LoadBalancer

    The LoadBalancer specific [system properties] are -Dhttp.cluster.debug and -Dcluster.debug.

    The available options are:

      -config [path] - used to specify the path to a Loadbalancer configuration file. Details are available here .

      -host [name/ip] - name/ip address to listen to for HTTP connections on (ALL by default). Details are available here .

      -port [number] - port to listen to for HTTP connections (80 by default). Details are available here .

      -dontUseSessionID - tells the balancer not to scan for Servlet session IDs as a means of identifying clients. Details are available here .

      -dontUseIP - tells the balancer not to route connections based on IP if no session exists/is scanned for. Details are available here .

      -dontUseKeepAlives - tells the balancer not to maintain keep-alives. Details are available here .

      -minimumIsland [number] - the minimum island to add to alive servers list. Details are available here .

      -maximumIsland [number] - the maximum island to add to alive servers list. Details are available here .

      -selectiontype [random|first]- how to select servers, the default is random, options are: 'random', 'first'. Details are available here .

      -debug - log information to the console regarding the progress of the load balancer. Details are available here .

    Each of these options is described in separate sections.

    When the LoadBalancer is started, the following message should be displayed in the console:


    Balancer initialized...
    Listing 2: LoadBalancer startup message.

3 Starting the LoadBalancer using a configuration file

    By default, the LoadBalancer tries to load a configuration file named config/load-balancer.xml. If such a file exists, the LoadBalancer tries to use it.

    It is also possible to specify a specific configuration file by starting the LoadBalancer with a command similar to the one stated in listing 3 below.


        java -jar loadbalancer.jar -config [path]
    Listing 3: Starting the LoadBalancer

    Where [path] should be replaced with the path to a loadbalancer configuration file.

    The LoadBalancer will output a notification to the console whenever a cluster node register itself with it.

4 Specifying name/ip to listen to

    By default, the LoadBalancer listens to all available name/ip addresses for incoming HTTP/HTTPS requests.

    To configure this value either start LoadBalancer with a command like in listing 4 below or by specifying this value for the host attribute in the <load-balancer> tag of the used configuration file as described in listing 5 below.


        java -jar loadbalancer.jar -host 127.0.0.1
    Listing 4: Example of starting the LoadBalancer with the -host option.


        <load-balancer
            ...
            host="127.0.0.1"
            ...
        >
        ...
        </loadbalancer>
    Listing 5: Example of host configuration in LoadBalancer configuration file.

5 Specifying port to listen to

    By default, the LoadBalancer listens to port 80 for incoming HTTP/HTTPS requests.

    To configure this value either start LoadBalancer with a command like in listing 6 below or by specifying this value for the port attribute in the <load-balancer> tag of the used configuration file as described in listing 7 below.


        java -jar loadbalancer.jar -port 8080
    Listing 6: Example of starting the LoadBalancer with the -port option.


        <load-balancer
            ...
            port="8080"
            ...
        >
        ...
        </loadbalancer>
    Listing 7: Example of port configuration in LoadBalancer configuration file.

6 Ignoring SessionID

    By default, the LoadBalancer scans for the Servlet Session ID to identify clients. Clients are identified in order for the LoadBalancer to select the correct backend Server for handling the clients request.

    To configure the LoadBalancer to ignore the Servlet Session ID either start LoadBalancer with a command like the example in listing 8 below or set the value of the attribute use-session-id to false in the <load-balancer> tag of the used configuration file as described in listing 9 below.


        java -jar loadbalancer.jar -dontUseSessionID
    Listing 8: Example of starting the LoadBalancer with the -dontUseSessionID switch.


        <load-balancer
            ...
            use-session-id="false"
            ...
        >
        ...
        </loadbalancer>
    Listing 9: Example of use-session-id configuration in LoadBalancer configuration file. Possible values are "true" and "false", with the default being "true".

    If the LoadBalancer cannot identify clients by the usage of their Servlet Session IDs, the LoadBalancer scans for the clients IP to identify clients.

7 Ignoring Clients IP

    By default, if the LoadBalancer is configured to ignore Servlet Session IDs, the LoadBalancer scans for the clients IP to identify clients. Clients are identified in order for the LoadBalancer to select the correct backend Server for handling the clients request.

    To configure the LoadBalancer to ignore the clients IP either start LoadBalancer with a command like the example in listing 10 below or set the value of the attribute use-ip to false in the <load-balancer> tag in the used configuration file as described in listing 11 below.


        java -jar loadbalancer.jar -dontUseIP
    Listing 10: Example of starting the LoadBalancer with the -dontUseIP switch.


        <load-balancer
            ...
            use-ip="false"
            ...
        >
        ...
        </loadbalancer>
    Listing 11: Example of use-ip configuration in LoadBalancer configuration file. Possible values are "true" and "false", with the default being "true".

8 Disallowing keep-alives

    By default, the LoadBalancer allows for clients to reuse http connections per the HTTP 1.1 specification.

    To configure the LoadBalancer to disallow keep-alives either start LoadBalancer with a command like the example in listing 12 below or set the value of the attribute use-keep-alives to false in the <load-balancer> tag in the used configuration file as described in listing 13 below.


        java -jar loadbalancer.jar -dontUseKeepAlives
    Listing 12: Example of starting the LoadBalancer with the -dontUseKeepAlives switch.


        <load-balancer
            ...
            use-keep-alives="false"
            ...
        >
        ...
        </loadbalancer>
    Listing 13: Example of use-keep-alives configuration in LoadBalancer configuration file. Possible values are "true" and "false", with the default being "true".

9 Setting the minimum cluster island number

    By default, the LoadBalancer will manage all nodes on all cluster islands. It is sometimes needed to be able to specify that a certain loadbalancer should only handle some cluster islands. This is done by specifying the minimum and/or maximum cluster island numbers that the LoadBalancer should listen to.

    To configure the minimum cluster island number the LoadBalancer should listen to, either start LoadBalancer with a command like the example in listing 14 below or set the value of the attribute minimum-island to the appropriate value in the <load-balancer> tag in the used configuration file as described in listing 15 below.


        java -jar loadbalancer.jar -minimumIsland 5
    Listing 14: Example of starting the LoadBalancer with the -minimumisland option.


        <load-balancer
            ...
            minimum-island="5"
            ...
        >
        ...
        </loadbalancer>
    Listing 15: Example of minimum-island configuration in LoadBalancer configuration file.

10 Setting the maximum cluster island number

    By default, the LoadBalancer will manage all nodes on all cluster islands. It is sometimes needed to be able to specify that a certain loadbalancer should only handle some cluster islands. This is done by specifying the minimum and/or maximum cluster island numbers that the LoadBalancer should listen to.

    To configure the maximum cluster island number the LoadBalancer should listen to, either start LoadBalancer with a command like the example in listing 16 below or set the value of the attribute maximum-island to the appropriate value in the <load-balancer> tag in the used configuration file as described in listing 17 below.


        java -jar loadbalancer.jar -maximumIsland 10
    Listing 16: Example of starting the LoadBalancer with the -maximumisland option.


        <load-balancer
            ...
            maximum-island="10"
            ...
        >
        ...
        </loadbalancer>
    Listing 17: Example of maximum-island configuration in LoadBalancer configuration file.

11 Setting the server-selection behaviour

    By default, the LoadBalaner will select a cluster node at random when the client first accesses a cluster island. Optionally the LoadBalancer can be configured to select the first registered for a certain cluster island instead.

    To configure the server-selection behaviour the LoadBalancer should use, either start LoadBalancer with a command like the example in listing 18 below or set the value of the attribute selection-type to the appropriate value in the <load-balancer> tag in the used configuration file as described in listing 17 below.


        java -jar loadbalancer.jar -selectionType first
    Listing 16: Example of starting the LoadBalancer with the -selectionType option.


        <load-balancer
            ...
            selection-type="first"
            ...
        >
        ...
        </loadbalancer>
    Listing 17: Example of selection-type configuration in LoadBalancer configuration file. Possible values are "random" and "first", with the default being "random".

12 Configuring cluster islands

    It is possible to add initial islands that should be added to the list of avaialble islands during LoadBalancer initialization. Such entries can be added to the body of the <load-balancer> tag in the used configuration file as described in listing 18 below.


        <load-balancer ... >
            ...
            <island id="1">
                ...
            </island>
            ...
        </loadbalancer>
    Listing 18: Example of adding a initial island.

    Each island node normally contains a number of <backend-server> nodes (see below that describes initial cluster nodes that should be part of this island.

13 Configuring cluster island nodes

    It is possible to add initial cluster nodes to the initial islands that should be added to the list of available cluster nodes during LoadBalancer initialization. Such entries can be added to the body of the <island> tags in the used configuration file as described in listing 19 below.


        <load-balancer ... >
            ...
            <island id="1">
                <backend-server host="asite" port="80"/>
                ...
            </island>
            ...
        </loadbalancer>
    Listing 18: Example of adding a initial cluster island node.

    This feature should be avoided as it leaves the system static, preventing dynamic removals/readings. If the cluster-island value of the cluster nodes web-site.xml is different from the value specified in the LoadBalancer configuration file, it could lead to unexpected behaviour if the LoadBalancer has the cluster node listed for two different islands (a secondary when the cluster node manages to register itself with the LoadBalancer), making it very possible that the clients state is not available on the cluster node.

    Although new cluster nodes will be added dynamically, the hard coded ones will remain in the list even if they are going down. The feature should be viewed as a useful tool during development and should be avoided in production environments.

14 Balancing HTTPS sites

    The LoadBalancer can be used to balance back-end sites using SSL. In order to do so the LoadBalancer must be told to use the clients IP for client identification. The reason for this is that the LoadBalancer will pass through the requests without looking into them, as decrypting the message would be too time-consuming.


    Figure 3: All back-end cluster nodes are running in HTTPS mode, the LoadBalancer forwards requests to the back-ends without looking at the actual request.

    To use the LoadBalancer to balance back-end sites using SSL the LoadBalancer has to be told to use the clients IP address instead of session ID as identification, as described here.

15 Debugging

    This section describes common techniques that can be used to debug HTTP-clustering.

    15.1 Tracing

      In order to get trace information from the cluster services, start the Orion Server with the cluster.debug and http.cluster.debug system properties set to true, as exemplified in listing 7 below.


          java -Dhttp.cluster.debug=true -Dcluster.debug=true -jar orion.jar -console
      Listing 7: Starting a Orion server with cluster debug system properties turned on

      The cluster.debug property will list common debug information to the cosole.

      The http.cluster.debug property will list HTTP cluster debug information to the console.

16 Troubleshooting

    This section describes common problems and possible solutions.

    16.1 Wrong IP address used for web-site

      It is not reliable to get the IP address of the local host on all platforms. It is therefore recommended that the value for the host attribute is set correctly. This issue is normally encountered when the LoadBalancer is having troubles identifying the web-site.

      If this issue is encountered during development, it might be usefull to temporarily hardcode the back-ends into the LoadBalancers configuration file as described here.

    16.2 Firewall issues

      Make sure that any firewalls allows for the LoadBalancer to listen to multicast on 230.0.0.2:27512 and that the cluster-enabled Web-modules are allowed to send on the same multicast address/port.

      Using a simple multicast sender/receiver test application allows for testing the multicast address and make sure that packets can be both sent and received.

      If this issue is encountered during development, it might be usefull to temporarily hardcode the back-ends into the LoadBalancers configuration file as described here.

    16.3 Proxies/Gateways issues

      As multicast packets are known to bog down networks if allowed to roam free, most proxies and gateways won't let multicast packets pass. Make sure that the multicast packets can reach all nodes in the cluster as well as the LoadBalancer and configure any proxies/gateways accordingly.

      If this issue is encountered during development, it might be usefull to temporarily hardcode the back-ends into the LoadBalancers configuration file as described here.

Copyright © 2005 IronFlare AB