Skip to main content

Multihoming with SailFin - Traffic separation

Posted by rampsarathy on August 26, 2009 at 10:45 PM PDT

High availability in SailFin can be achieved by deploying a cluster of instances and configuring the load balancer and the replication modules as per the user's needs. Apart from the basic configuration of these modules, SailFin (2.0) also allows users to separate the intra-cluster traffic (resulting from the load-balancer, replication and the group management service modules) from the external traffic, which allows users to maintain/configure their network in way that best suits their traffic needs. Traffic separation also allows the users to plan their network and augment certain parts of it when required. This following steps describes how SailFin 2.0 can be configured on multiple interfaces (IP addresses), The instructions assume that the user wants to separate the cluster internal traffic (CLB and GMS only) from the external SIP/Http traffic (from the UAs).

Machine setup:

In order to separate the traffic, the machines should have atleast 2 IP addresses, which ideally would belong to different networks. There are different ways of multi-homing a system which are out of scope of the discussion here. For the sake of simplicity we would assume the machine on which this configuration is created has 2 IP addresses which are on different networks (one may not be reachable from the other). We will call the first IP as the external ip and the second one as internal IP. The objective is to expose the external IP (through a h/w load balancer) to the UAs,so that all the traffic from the UAs would be through them. The internal IP is used only by the SailFin cluster instances for the intra-cluster communication.

On some machines (especially the ones that are dual-stack enabled), it is mandatory to configure the multicast routing  rule.
E.g # route add -net 224.0.0.0 netmask 240.0.0.0 dev eth2

Configuration :

Create  a cluster of N instances where each instance is running on a separate machine, N being 3 in the example below. Let us call the cluster mh-cluster

The following commands have to be executed to achieve  traffic separation for mh-cluster,



Step 1:

Create the property tokens for the external listener (corresponds to the external IP), which would be the public address of that machine, The tokens are used because the external address of every machine would be different and these would be resolved based on the machine specific values that we would configure later.

 

These listeners exist by default in the configuration, we are just modifying the address property.

> asadmin set --user admin --port 4848 --passwordfile passwordfile  mh-cluster-config.sip-service.sip-listener.sip-listener-1.address=\${EXTERNAL_LISTENER_ADDRESS}

> asadmin set --user admin --port 4848 --passwordfile passwordfile  mh-cluster-config.http-service.http-listener.http-listener-1.address=\${EXTERNAL_LISTENER_ADDRESS}

 

> asadmin set --user admin --port 4848 --passwordfile passwordfile  mh-cluster-config.sip-service.sip-listener.sip-listener-2.address=\${EXTERNAL_LISTENER_ADDRESS}

 

 > asadmin set --user admin --port 4848 --passwordfile passwordfile  mh-cluster-config.http-service.http-listener.http-listener-2.address=\${EXTERNAL_LISTENER_ADDRESS}

 

 
Step 2:

Set the listener type of the public listeners to "external". This denotes that these listeners should be used only for handling UA traffic and not by the clb for proxying.

> asadmin set --user admin --port 4848 --passwordfile passwordfile  mh-cluster-config.http-service.http-listener.http-listener-1.type=external

 > asadmin set --user admin --port 4848 --passwordfile passwordfile  mh-cluster-config.sip-service.sip-listener.sip-listener-1.type=external

> asadmin set --user admin --port 4848 --passwordfile passwordfile  mh-cluster-config.http-service.http-listener.http-listener-2.type=external

 > asadmin set --user admin --port 4848 --passwordfile passwordfile  mh-cluster-config.sip-service.sip-listener.sip-listener-2.type=external



Step 3:

Create the system properties corresponding to the tokens that would be used for IP address resolution in the respective instances

 

INTERNAL_LISTENER_ADDRESS would be used by the internal listeners that are created in the next step.

> asadmin create-system-properties --user admin --port 4848 --passwordfile passwordfile --target mh-cluster EXTERNAL_LISTENER_ADDRESS=0.0.0.0:INTERNAL_LISTENER_ADDRESS=0.0.0.0

 

> asadmin create-system-properties --user admin --port 4848 --passwordfile  passwordfile --target server EXTERNAL_LISTENER_ADDRESS=0.0.0.0:INTERNAL_LISTENER_ADDRESS=192.168.2.11

Step 4 :

Create new listeners that will be used by clb for prxying fe-be traffic, this is done by setting  the type of the listener as "internal"

> asadmin create-http-listener --user admin --port 4848 --passwordfile passwordfile --target mh-cluster --listeneraddress 0.0.0.0 --defaultvs server --listenerport 28080 internal-http-listener

> asadmin create-sip-listener --user admin --port 4848 --passwordfile  passwordfile --target mh-cluster --siplisteneraddress 0.0.0.0 --siplistenerport 25060 internal-sip-listener

Modify the address attribute so that it points to the internal address property

> asadmin set --user admin --port 4848 --passwordfile passwordfile  mh-cluster-config.sip-service.sip-listener.internal-sip-listener.address=\${INTERNAL_LISTENER_ADDRESS}

> asadmin set --user admin --port 4848 --passwordfile passwordfile  mh-cluster-config.http-service.http-listener.internal-http-listener.address=\${INTERNAL_LISTENER_ADDRESS}

 

> asadmin set --user admin --port 4848 --passwordfile passwordfile  mh-cluster-config.sip-service.sip-listener.internal-sip-listener.type=internal

>asadmin set --user admin --port 4848 --passwordfile  passwordfile  mh-cluster-config.http-service.http-listener.internal-http-listener.type=internal

Step 5:

Configure GMS bind address so that GMS communication happens through a specific interface

# Note that this workaround is required because the GMS in DAS does not bind to the specified address if this (default-cluster) is not present.

> asadmin set --user admin --port 4848 --passwordfile passwordfile  default-cluster.property.gms-bind-interface-address=\${INTERNAL_LISTENER_ADDRESS}

> asadmin set --user admin --port 4848 --passwordfile passwordfile  mh-cluster.property.gms-bind-interface-address=\${INTERNAL_LISTENER_ADDRESS}

Step 6:

Configure the IP addresses of the cluster instances

> asadmin create-system-properties --user admin --port 4848 --passwordfile passwordfile --target instance101 EXTERNAL_LISTENER_ADDRESS=10.12.152.29:INTERNAL_LISTENER_ADDRESS=192.168.2.1

 >asadmin create-system-properties --user admin --port 4848 --passwordfile passwordfile --target instance102 EXTERNAL_LISTENER_ADDRESS=10.12.152.39:INTERNAL_LISTENER_ADDRESS=192.168.2.4

> asadmin create-system-properties --user admin --port 4848 --passwordfile passwordfile --target instance103 EXTERNAL_LISTENER_ADDRESS=10.12.152.58:INTERNAL_LISTENER_ADDRESS=192.168.2.5

 

Once all the above commands have executed succesffuly , please restart the nodeagents and cluster for the changes to take effect, restart of cluster is required because changing the type (only the type attribute) of a listener dynamically is not supported.

 

Verify (using netstat) if the listeners are bound to the correct IPs.

Step 7 (optional) :

There might be a h/w load balancer that fronts this entire SailFin cluster, which is typically used for spraying the sip traffic to the individual instances. And when a request is sent out from SailFin, its the address of this h/w load balancer that has to be put in the contact and via headers, this would enable the client to reach the load balancer when it sends a response after address resolution.

 

This address of the load balancer has to be configured in the cluster so that the instances can pick it up when they are creating an outgoing request. One way to do this would be to configure it under the sip-container-external-sip-address attribute, but this would mean that there can only be one load balancer that is fronting all the listeners. To make this configuration more flexible in 2.0, now every listener (that is external) can take the external-sip-address and port attributes,

 

This can be configured the following way

> asadmin set --user admin --port 4848 --passwordfile passwordfile  mh-cluster-config.sip-service.sip-listener.sip-listener-1.external-sip-address=

> asadmin set --user admin --port 4848 --passwordfile passwordfile  mh-cluster-config.sip-service.sip-listener.sip-listener-1.external-sip-address=

Related Topics >>