(ve):HA Experimental

  • This page was last modified on February 2, 2011, at 14:40.
The (mt) Community Wiki is a collaborative project. Any (mt) Media Temple customer or employee may contribute. Not all articles and/or content have been tested for accuracy by (mt) Media Temple.

For officially moderated and tested articles, be sure to visit our KnowledgeBase.

From (mt) Community Wiki

Contents

(ve) HA

This is an experimental article and cannot be directly supported by (mt) Media Temple. Please see the (mt) State of Support. For this to work, (mt) will need to switch you to a different networking model that allows multi-cast and you will lose your (ve):Parallels Power Panel.

We're going to create a Virtual IP (VIP) that can be shared among many boxes and will load-balance HTTP traffic on this VIP using HAProxy

For more information:

Clustering http://www.clusterlabs.org/wiki/Documentation

HAProxy http://haproxy.1wt.eu/download/1.3/doc/configuration.txt

Ubuntu 10.04 (ve) instructions

Installation

install/configure 

sed -i 's/archive/us.archive/g' /etc/apt/sources.list
apt-get update
apt-get install openais pacemaker haproxy

Enable corosync on startup 

sed -i 's/=no/=yes/g' /etc/default/corosync

Disable haproxy on startup, corosync will start it 

sed -i 's/ENABLED=.*/ENABLED=1/g' /etc/default/haproxy
update-rc.d -f haproxy remove

Add SHM, necessary for initial corosync configuration. 

sed -i 's/exit 0//g' /etc/rc.local
echo "mount -o nosuid,nodev /dev/shm" >> /etc/rc.local
mount -o nosuid,nodev /dev/shm

Edit corosync config file, should be identical on both nodes 

vi /etc/corosync/corosync.conf

Look for "interface" configuration. bindnetaddr has to be the NETWORK address, so for IP 205.186.155.100/21, the network address is: 205.186.152.0 

interface {
# The following values need to be set based on your environment
ringnumber: 0
bindnetaddr: YOUR_NETWORK_ADDRESS
mcastaddr: 226.94.1.5
mcastport: 5405
}

Start service on both nodes 

/etc/init.d/corosync start

Check the status of the cluster, you should see all of your nodes 

crm status

Configure Cluster

Once you apply the config on one node it gets replicated to another cluster node automatically

Assign variables export $VIP=YOURVIPNUMBER

Disable Stonith 

crm configure property stonith-enabled=false

Add VIP 

crm configure primitive vip1 ocf:heartbeat:IPaddr2 params ip=$VIP cidr_netmask=21 op monitor interval=30s

Set Quorum to ignore for a two-node cluster 

crm configure property no-quorum-policy=ignore

Prevent resource from auto-moving from another node recovery and stick to the current one 

crm configure rsc_defaults resource-stickiness=100

Get HAProxy resource on all boxes participating in the cluster 

wget -O /usr/lib/ocf/resource.d/heartbeat/haproxy http://github.com/russki/cluster-agents/raw/master/haproxy
chmod 755 /usr/lib/ocf/resource.d/heartbeat/haproxy

Configure HAProxy resource 

crm configure primitive haproxy ocf:heartbeat:haproxy params conffile=/etc/haproxy/haproxy.cfg op monitor interval=30s

Ensure that HAProxy and VIP stay on the same server 

crm configure colocation haproxy-with-vip INFINITY: haproxy vip1

Ensure that HAProxy starts after VIP 

crm configure order haproxy-after-vip mandatory: vip1 haproxy

Enable encryption

  • generate entropy, will take anywhere from 1-5 minutes to generate enough entropy, can do a lot of IO on the same server to help generate it

in one terminal 

corosync-keygen

In another terminal, generate enough IO to create entropy 

while /bin/true; do dd if=/dev/urandom of=/tmp/100 bs=1024 count=100000; for i in {1..10}; do cp /tmp/100 /tmp/tmp_$i_$RANDOM; done; rm -f /tmp/tmp_* /tmp/100; done

Copy over authkey to all members of the HA cluster 

scp /etc/corosync/authkey root@INSERT_IP_OF_YOUR_SERVER:/etc/corosync

Enable encryption on all members of the HA cluster & restart corosync 

sed -i 's/secauth: off/secauth: on/g' /etc/corosync/corosync.conf; /etc/init.d/corosync restart

Check the status of the cluster, you should see all of your nodes 

crm status

Configure HAProxy

The file needs to live on all members of the HA cluster that HAProxy can failover to HAProxy can only listen on the VIP All real webservers can only listen on their real IPs, not 0.0.0.0:80, otherwise they will take over HAProxy on VIP as well. So basically, hardcode your "Listen" to the real ip of your box

Really basic config, HAProxy can be as complicated as you want it to be

Edit config 

/etc/haproxy/haproxy.cfg

Replace VIP, REAL_IP1 & REAL_IP2 variables 

global
log 127.0.0.1   local0
log 127.0.0.1   local1 notice
#log loghost    local0 info
maxconn 4096
#chroot /usr/share/haproxy
user haproxy
group haproxy
daemon
#debug
#quiet

defaults
log     global
mode    http
option  httplog
option  dontlognull
retries 3
option redispatch
maxconn 2000
contimeout      5000
clitimeout      50000
srvtimeout      50000

listen  webfarm VIP:80
mode http
#stats enable
#stats auth YOUR_STAT_USER:STAT_PASS
#stats uri /hastats?stats
cookie  SERVERID rewrite
balance roundrobin
option forwardfor
server  webA REAL_IP1:80 cookie a check inter 2000 rise 2 fall 5
server  webB REAL_IP2:80 cookie a check inter 2000 rise 2 fall 5

restart HAProxy on the cluster 

crm  resource restart

Useful Commands

To obtain a list of the available resource classes, run 

crm ra classes

To find all the OCF resource agents provided by Pacemaker and Heartbeat, run 

crm ra list ocf heartbeat
crm ra list ocf pacemaker

find the node on which the IP address is running. 

crm resource status vip1

bring node out of HA commission 

crm node standby

bring node into HA commision 

crm node online

move resource to another box 

crm resource move haproxy server2.com

clear the service startup errors 

crm resource cleanup WebSite

stop resource 

crm  resource stop  haproxy

delete resource 

crm delete primitive haproxy

delete one of the properties 

crm configure delete cli-prefer-img_haproxy
Retrieved from "http://wiki.mediatemple.net/w/(ve):HA_Experimental"