JBoss.orgCommunity Documentation

SIP Load Balancer User Guide

The Guide to the JBoss Communications SIP Load Balancer

by Douglas Silas, Jean Deruelle, Vladimir Ralev, Ivelin Ivanov, and Jared Morgan

Abstract

JBoss Communications is the first and only open source VoIP platform certified for JAIN SLEE 1.1 and SIP Servlets 1.1 compliance. JBoss Communications SIP Load Balancer is a SIP and HTTP Load Balancer to be used in a JBoss Communications platform cluster, to increase SIP and HTTP performance, and achieve High Availability and Fault Tolerance on those protocols.


This manual uses several conventions to highlight certain words and phrases and draw attention to specific pieces of information.

In PDF and paper editions, this manual uses typefaces drawn from the Liberation Fonts set. The Liberation Fonts set is also used in HTML editions if the set is installed on your system. If not, alternative but equivalent typefaces are displayed. Note: Red Hat Enterprise Linux 5 and later includes the Liberation Fonts set by default.

Four typographic conventions are used to call attention to specific words and phrases. These conventions, and the circumstances they apply to, are as follows.

Mono-spaced Bold

Used to highlight system input, including shell commands, file names and paths. Also used to highlight key caps and key-combinations. For example:

The above includes a file name, a shell command and a key cap, all presented in Mono-spaced Bold and all distinguishable thanks to context.

Key-combinations can be distinguished from key caps by the hyphen connecting each part of a key-combination. For example:

The first sentence highlights the particular key cap to press. The second highlights two sets of three key caps, each set pressed simultaneously.

If source code is discussed, class names, methods, functions, variable names and returned values mentioned within a paragraph will be presented as above, in Mono-spaced Bold. For example:

Proportional Bold

This denotes words or phrases encountered on a system, including application names; dialogue box text; labelled buttons; check-box and radio button labels; menu titles and sub-menu titles. For example:

The above text includes application names; system-wide menu names and items; application-specific menu names; and buttons and text found within a GUI interface, all presented in Proportional Bold and all distinguishable by context.

Note the > shorthand used to indicate traversal through a menu and its sub-menus. This is to avoid the difficult-to-follow 'Select Mouse from the Preferences sub-menu in the System menu of the main menu bar' approach.

Mono-spaced Bold Italic or Proportional Bold Italic

Whether Mono-spaced Bold or Proportional Bold, the addition of Italics indicates replaceable or variable text. Italics denotes text you do not input literally or displayed text that changes depending on circumstance. For example:

Note the words in bold italics above username, domain.name, file-system, package, version and release. Each word is a placeholder, either for text you enter when issuing a command or for text displayed by the system.

Aside from standard usage for presenting the title of a work, italics denotes the first use of a new and important term. For example:


The SIP Load Balancer is used to balance the load of SIP service requests and responses between nodes in a SIP Server cluster, such as JBoss Communications JAIN SLEE or SIP Servlets. Both JBCP servers can be used in conjunction with the SIP Load Balancer to increase the performance and availability of SIP services and applications.

In terms of functionality, the SIP Load Balancer is a simple stateless proxy server that intelligently forwards SIP session requests and responses between User Agents (UAs) on a Wide Area Network (WAN), and SIP Server nodes, which are almost always located on a Local Area Network (LAN). All SIP requests and responses pass through the SIP Load Balancer.

Configuring the SIP load balancer and the two SIP Servlets-enabled Server nodes is described in Configuring the JBoss Communications SIP Load Balancer and SIP Server Nodes.

Procedure 1.1. Configuring the JBoss Communications SIP Load Balancer and SIP Server Nodes

  1. Configure lb.properties Configuration Properties File

    Configure the SIP Load Balancer's Configuration Properties file by substituting valid values for your personal setup. Example 1.1, “Complete Sample lb.properties File” shows a sample lb.properties file, with key element descriptions provided after the example. The lines beginning with the pound sign are comments.

    Example 1.1. Complete Sample lb.properties File

    # Mobicents Load Balancer Settings
    # For an overview of the Mobicents Load Balancer visit http://docs.google.com/present/view?id=dc5jp5vx_89cxdvtxcm
    # The Load balancer will listen for both TCP and UDP connections
    
    
    
    # The binding address of the load balancer. This also specifies the 
    # default value for both internalHost and externalHost if not specified separately.
    host=127.0.0.1
    
    # The binding address of the load balancer where clients should connect (if the host property is not specified)
    #externalHost=127.0.0.1
    
    # The SIP port from where servers will receive messages
    # delete if you want to use only one port for both inbound and outbound)
    internalPort=5065
    
    # The SIP port used where clients should connect
    externalPort=5060
    
    # The binding address of the load balancer where SIP application servers should connect (if the host property is not specified)
    #internalHost=127.0.0.1
    
    # The RMI port used for heartbeat signals
    rmiRegistryPort=2000
    
    # The HTTP port for HTTP forwarding
    # if you like to activate the integrated HTTP load balancer, this is the entry point
    httpPort=2080
    #If no nodes are active the LB can redirect the traffic to the unavailableHost specified in this property,
    #otherwise, it will return 503 Service Unavailable
    #unavailableHost=google.com
    
    # If you are using IP load balancer, put the IP address and port here
    #externalIpLoadBalancerAddress=127.0.0.1
    #externalIpLoadBalancerPort=111
     
    # Requests initited from the App Servers can route to this address (if you are using 2 IP load balancers for bidirectional SIP LB)
    #internalIpLoadBalancerAddress=127.0.0.1
    #internalIpLoadBalancerPort=111
    
    # The addresses in the SIP LB Via headers can be either the real addresses or those specified in the external and internal IP LB addresses
    useIpLoadBalancerAddressInViaHeaders=false
    
    # Designate extra IP addresses as serer nodes
    #extraServerNodes=222.221.21.12:21,45.6.6.7:9003,33.5.6.7,33.9.9.2
    
    # Call-ID affinity algortihm settings. This algorithm is the default. No need to uncomment it.
    #algorithmClass=org.mobicents.tools.sip.balancer.CallIDAffinityBalancerAlgorithm
    # This property specifies how much time to keep an association before being evitcted.
    # It is needed to avoid memory leaks on dead calls. The time is in seconds.
    #callIdAffinityMaxTimeInCache=500
    #The following attribute specified the policy after failover. If set to true all calls from the failed node
    #will go to a new healthy node (all calls to the same node). If set to false the calls will go to random new nodes.
    #callIdAffinityGroupFailover=false
    
    # Uncomment to enable the consistent hash based on Call-ID algorithm.
    #algorithmClass=org.mobicents.tools.sip.balancer.HeaderConsistentHashBalancerAlgorithm
    # This property is not required, it defaults to Call-ID if not set, cna be "from.user" or "to.user" when you want the SIP URI username
    #sipHeaderAffinityKey=Call-ID
    #specify the GET HTTP parameter to be used as hash key
    #httpAffinityKey=appsession
     
    # Uncomment to enable the persistent consistent hash based on Call-ID algorithm.
    #algorithmClass=org.mobicents.tools.sip.balancer.PersistentConsistentHashBalancerAlgorithm
    # This property is not required, it defaults to Call-ID if not set
    #sipHeaderAffinityKey=Call-ID
    #specify the GET HTTP parameter to be used as hash key
    #httpAffinityKey=appsession
     
    #This is the JBoss Cache 3.1 configuration file (with jgroups), if not specified it will use default
    #persistentConsistentHashCacheConfiguration=/home/config.xml
     
    # Call-ID affinity algortihm settings. This algorithm is the default. No need to uncomment it.
    #algorithmClass=org.mobicents.tools.sip.balancer.CallIDAffinityBalancerAlgorithm
    # This property specifies how much time to keep an association before being evitcted.
    # It is needed to avoid memory leaks on dead calls. The time is in seconds.
    #callIdAffinityMaxTimeInCache=500
    
    # Uncomment to enable the consistent hash based on Call-ID algorithm.
    #algorithmClass=org.mobicents.tools.sip.balancer.HeaderConsistentHashBalancerAlgorithm
    # This property is not required, it defaults to Call-ID if not set, cna be "from.user" or "to.user" when you want the SIP URI username
    #sipHeaderAffinityKey=Call-ID
    #specify the GET HTTP parameter to be used as hash key
    #httpAffinityKey=appsession
    
    # Uncomment to enable the persistent consistent hash based on Call-ID algorithm.
    #algorithmClass=org.mobicents.tools.sip.balancer.PersistentConsistentHashBalancerAlgorithm
    # This property is not required, it defaults to Call-ID if not set
    #sipHeaderAffinityKey=Call-ID
    #specify the GET HTTP parameter to be used as hash key
    #httpAffinityKey=appsession
     
    #This is the JBoss Cache 3.1 configuration file (with jgroups), if not specified it will use default
    #persistentConsistentHashCacheConfiguration=/home/config.xml
    
    
    #If a node doesnt check in within that time (in ms), it is considered dead
    nodeTimeout=5100
    #The consistency of the above condition is checked every heartbeatInterval milliseconds
    heartbeatInterval=150
    
    
    #JSIP stack configuration.....
    javax.sip.STACK_NAME = SipBalancerForwarder
    javax.sip.AUTOMATIC_DIALOG_SUPPORT = off
    # You need 16 for logging traces. 32 for debug + traces.
    # Your code will limp at 32 but it is best for debugging.
    gov.nist.javax.sip.TRACE_LEVEL = 0
    
    // Specify if message contents should be logged.
    gov.nist.javax.sip.LOG_MESSAGE_CONTENT=false
    
    gov.nist.javax.sip.DEBUG_LOG = logs/sipbalancerforwarderdebug.txt
    gov.nist.javax.sip.SERVER_LOG = logs/sipbalancerforwarder.xml
    gov.nist.javax.sip.THREAD_POOL_SIZE = 64
    gov.nist.javax.sip.REENTRANT_LISTENER = true
    
                

    host

    Local IP address, or interface, on which the SIP load balancer will listen for incoming requests.

    externalPort

    Port on which the SIP load balancer listens for incoming requests from SIP User Agents.

    internalPort

    Port on which the SIP load balancer forwards incoming requests to available, and healthy, SIP Server cluster nodes.

    rmiRegistryPort

    Port on which the SIP load balancer will establish the RMI heartbeat connection to the application servers. When this connection fails or a disconnection instruction is received, an application server node is removed and handling of requests continues without it by redirecting the load to the lie nodes.

    httpPort

    Port on which the SIP load balancer will accept HTTP requests to be distributed across the nodes.

    internalTransport

    Transport protocol for the internal SIP connections associated with the internal SIP port of the load balancer. Possible choices are UDP, TCP and TLS.

    externalTransport

    Transport protocol for the external SIP connections associated with the external SIP port of the load balancer. Possible choices are UDP, TCP and TLS. It must match the transport of the internal port.

    externalIpLoadBalancerAddress

    Address of the IP load balancer (if any) used for incoming requests to be distributed in the direction of the application server nodes. This address may be used by the SIP load balancer to be put in SIP headers where the external address of the SIP load balancer is needed.

    externalIpLoadBalancerPort

    The port of the external IP load balancer. Any messages arriving at this port should be distributed across the external SIP ports of a set of SIP load balancers.

    internalIpLoadBalancerAddresst

    Address of the IP load balancer (if any) used for outgoing requests (requests initiated from the servers) to be distributed in the direction of the clients. This address may be used by the SIP load balancer to be put in SIP headers where the internal address of the SIP load balancer is needed.

    internalIpLoadBalancerPort

    The port of the internal IP load balancer. Any messages arriving at this port should be distributed across the internal SIP ports of a set of SIP load balancers.

    extraServerNodes

    Comma-separated list of hosts that are server nodes. You can put here alternative names of the application servers here and they will be recognized. Names are important, because they might be used for direction-analysis. Requests coming from these server will go in the direction of the clients and will not be routed back to the cluster.

    algorithmClass

    The fully-qualified Java class name of the balancing algorithm to be used. There are three algorithms to choose from and you can write your own to implement more complex routing behaviour. Refer to the sample configuration file for details about the available options for each algorithm. Each algorithm can have algorithm-specific properties for fine-grained configuration.

    nodeTimeout

    In milliseonds. Default value is 5100. If a server node doesnt check in within this time (in ms), it is considered dead.

    heartbeatInterval

    In milliseconds. Default value is 150 milliseonds. The hearbeat interval must be much smaller than the interval specified in the JAIN SIP property on the server machines - org.mobicents.ha.javax.sip.HEARTBEAT_INTERVAL

  2. Configure logging

    The SIP load balancer uses Java Logging as a logging mechanism. You can configure it through a property file and specify the property file to be used by using the following command -Djava.util.logging.config.file=./lb-logging.properties. Please refer to JDK logging for more informationon how to configure the Java logging.

All User Agents send SIP messages, such as INVITE and MESSAGE, to the same SIP URI (the IP address and port number of the SIP Load Balancer on the WAN). The Load Balancer then parses, alters, and forwards those messages to an available node in the cluster. If the message was sent as a part of an existing SIP session, it will be forwarded to the cluster node which processed that User Agent's original transaction request.

The SIP Server that receives the message acts upon it and sends a response back to the SIP Load Balancer. The SIP Load Balancer reparses, alters and forwards the message back to the original User Agent. This entire proxying and provisioning process is carried out independent of the User Agent, which is only concerned with the SIP service or application it is using.

By using the Load Balancer, SIP traffic is balanced across a pool of available SIP Servers, increasing the overall throughput of the SIP service or application running on either individual nodes of the cluster. In the case of a JBCP server with </distributed> capabilities, load balancing advantages are applied across the entire cluster.

The SIP Load Balancer is also able to failover requests mid-call from unavailable nodes to available ones, thus increasing the reliability of the SIP service or application. The Load Balancer increases throughput and reliability by dynamically provisioning SIP service requests and responses across responsive nodes in a cluster. This enables SIP applications to meet the real-time demand for SIP services.

In addition to the SIP load balancing, there are several options for coordinated or cooperative load balancing with other protocols such as HTTP.

Typically, a JBoss Application Server will use apache HTTP server with mod_jk, mod_proxy, mod_cluster or similar extension installed as an HTTP load balancer. This apache-based load balancer will parse incoming HTTP requests and will look for the session ID of those requests in order to ensure all requests from the same session arrive at the same application server.

By default, this is done by examining the jsessionid HTTP cookie or GET parameter and looking for the jvmRoute assigned to the session. The typical jsessionid value is of the form <sessionId>.<jvmRoute>. The very first request for each new HTTP session does not have a session ID assigned; the apache routes the request to a random application server node.

When the node responds it assigns a session ID and jvmRoute to the response of the request in a HTTP cookie. This response goes back to the client through apache, which keeps track of which node owns each jvmRoute. Once the very first request is served this way, the subsequent requests from this session will carry the assigned cookie, and the apache load balancer will always route the requests to the node, which advertised itself as the jvmRoute owner.

Instead of using apache, an integrated HTTP Load Balancer is also available. The SIP Load Balancer has a HTTP port where you can direct all incoming HTTP requests. The integrated HTTP load balancer behaves exactly like apache by default, but this behavior is extensible and can be overridden completely with the pluggable balancer algorithms. The integrated HTTP load balancer is much easier to configure and generally requires no effort, because it reuses most SIP settings and assumes reasonable default values.

Unlike the native apache, the integrated HTTP Load Balancer is written completely in Java, thus a performance penalty should be expected when using it. However, the integrated HTTP Balancer has an advantage when related SIP and HTTP requests must stick to the same node.

The SIP/HTTP Load Balancer exposes an interface to allow users to customize the routing decision making for special purposes. By default there are three built-in algorithms. Only one algorithm is active at any time and it is specified with the algorithmClass property in the configuration file.

It is up to the algorithm how and whether to support distributed architecture or how to store the information needed for session affinity. The algorithms will be called for every SIP and HTTP request and other significant events to make more informed decisions.

The following is a list of the built-in algorithms:

org.mobicents.tools.sip.balancer.CallIDAffinityBalancerAlgorithm

This algorithm is not distributable. It selects nodes randomly to serve a give Call-ID extracted from the requests and responses. It keeps a map with Call-ID -> nodeId associations and this map is not shared with other load balancers which will cause them to make different decisions. For HTTP it behaves like apache.

org.mobicents.tools.sip.balancer.HeaderConsistentHashBalancerAlgorithm

This algorithm is distributable and can be used in distributed load balancer configurations. It extracts the hash value of specific headers from SIP and HTTP messages to decide which application server node will handle the request. Information about the options in this algorithms is available in the balancer configuration file comments.

org.mobicents.tools.sip.balancer.PersistentConsistentHashBalancerAlgorithm

This algorithm is distributable and is similar to the previous algorithm, but it attempts to keep session affinity even when the cluster nodes are removed or added, which would normally cause hash values to point to different nodes.

org.mobicents.tools.sip.balancer.ClusterSubdomainAffinityAlgorithm

This algorithm is not distributable, but supports grouping server nodes to act as a subcluster. Any call of a node that belongs to a cluster group will be preferentially failed over to a node from the same group. To configure a group you can just add the subclusterMap property in the load balancer properties and listing the IP addresses of the nodes. The groups are enclosed in parentheses and the IP addresses are separate by commas as follows:

subclusterMap=( 192.168.1.1, 192.168.1.2 ) ( 10.10.10.10,  20.20.20.20,  30.30.30.30)

The nodes specified in a group do not have to alive and nodes that are not specified are still allowed to join the cluster. Otherwise the algorthim behaves exactly as the default Call-ID affinity algorthim.

Revision History
Revision 3.0Thu Jun 11 2009Jared Morgan
Second release of the "parameterized" documentation.
Revision 2.0Fri Mar 06 2009Douglas Silas
First release of the "parameterized", and much-improved JBCP documentation.