How to Configure Keepalived, Virtual IP Failover, and VRRP High Availability on Ubuntu Server

Maintaining high availability (HA) across enterprise Linux web servers, API gateways, database clusters, and reverse proxies requires eliminating single points of failure (SPOFs) at the network layer. Even if backend application processes are clustered across redundant nodes, clients connecting to a single static IP address face immediate downtime if that specific host suffers a hardware failure, power loss, or kernel panic. To guarantee seamless service continuity, infrastructure architects implement Virtual Router Redundancy Protocol (VRRP) through Keepalived.

Keepalived provides robust high-availability clustering and load-balancer health checking for Linux platforms. By binding a floating Virtual IP (VIP) across two or more Ubuntu Server nodes, Keepalived actively negotiates cluster state. If the primary MASTER node drops VRRP heartbeat advertisements, the BACKUP node detects the failure within milliseconds, claims the VIP, updates adjacent network switches via gratuitous ARP frames, and seamlessly assumes client traffic without connection resets. This operational guide details how to install and configure Keepalived on Ubuntu Server, script dynamic health checks, manage split-brain safeguards, and test zero-downtime failover transitions.

Architectural Overview: VRRP State Machine & Floating Virtual IPs

Understanding the underlying VRRP broadcast mechanism and election state machine ensures robust cluster design:

Cluster Entity	Operational Role	Network Behaviour	State Criteria
MASTER Node	Active traffic receiver; holds the Virtual IP (VIP)	Broadcasts VRRP heartbeats to multicast 224.0.0.18 every 1 second	Highest priority value (e.g., 101); healthy local daemon checks
BACKUP Node	Passive standby node; monitors MASTER heartbeat	Listens silently on VRRP multicast; prepares to claim VIP	Lower priority value (e.g., 100); transitions to MASTER upon missing 3 consecutive heartbeats
Virtual IP (VIP)	Shared public or internal service endpoint	Bound to local NIC alias (e.g., eth0:vip); migrates on failover	Advertised to network switches via gratuitous ARP (GARP) upon state change
Health Check Script	Local service liveness monitor (e.g., Nginx, HAProxy)	Executes periodically; adjusts node priority dynamically	Returns exit code 0 (success) or non-zero (failure lowers priority)

Prerequisites and Environment Verification

• Infrastructure: Two Ubuntu Server 22.04 LTS or 24.04 LTS instances attached to the same Layer 2 broadcast domain (VLAN).
• IP Topology Example:
  • Node 1 (MASTER): Physical IP 192.168.10.11 (interface eth0)
  • Node 2 (BACKUP): Physical IP 192.168.10.12 (interface eth0)
  • Shared Floating Virtual IP (VIP): 192.168.10.100
• Administrative Privileges: Root or sudo access on both nodes.
• Kernel Socket Binding: Ability to bind non-local IP addresses to permit services to listen on the VIP before it transitions.

Step 1: Enabling Non-Local IP Binding in the Linux Kernel

To allow reverse proxies (such as Nginx or HAProxy) to bind to the floating Virtual IP even on the standby BACKUP node where the IP is not yet active, enable ip_nonlocal_bind in the sysctl configuration:

# Execute on BOTH Node 1 and Node 2
echo "net.ipv4.ip_nonlocal_bind=1" | sudo tee -a /etc/sysctl.d/99-keepalived.conf
sudo sysctl --system

Step 2: Installing Keepalived on Both Ubuntu Nodes

Install the Keepalived daemon from the standard Ubuntu package repositories:

# Execute on BOTH Node 1 and Node 2
sudo apt update && sudo apt install -y keepalived

Step 3: Configuring the MASTER Node (Node 1)

Create the primary Keepalived configuration file at /etc/keepalived/keepalived.conf:

sudo nano /etc/keepalived/keepalived.conf

Populate the file with the following configuration:

global_defs {
    router_id lb01
    enable_script_security
    script_user root
}

# Define health check script to monitor local service (e.g., Nginx)
vrrp_script chk_service {
    script "/usr/bin/killall -0 nginx"
    interval 2
    weight 2
    fall 2
    rise 2
}

# Define the VRRP instance
vrrp_instance VI_STATIC {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 101
    advert_int 1

    authentication {
        auth_type PASS
        auth_pass Secr3tVrrpPass!
    }

    virtual_ipaddress {
        192.168.10.100/24 dev eth0 label eth0:vip
    }

    track_script {
        chk_service
    }
}

Step 4: Configuring the BACKUP Node (Node 2)

On Node 2, open the Keepalived configuration:

sudo nano /etc/keepalived/keepalived.conf

Populate the backup configuration with matching virtual_router_id and authentication, but with state BACKUP and a lower base priority (100):

global_defs {
    router_id lb02
    enable_script_security
    script_user root
}

vrrp_script chk_service {
    script "/usr/bin/killall -0 nginx"
    interval 2
    weight 2
    fall 2
    rise 2
}

vrrp_instance VI_STATIC {
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1

    authentication {
        auth_type PASS
        auth_pass Secr3tVrrpPass!
    }

    virtual_ipaddress {
        192.168.10.100/24 dev eth0 label eth0:vip
    }

    track_script {
        chk_service
    }
}

Step 5: Starting Keepalived and Validating VIP Binding

Enable and start the Keepalived service across both cluster nodes:

# Execute on BOTH nodes
sudo systemctl enable --now keepalived

Verifying Active VIP on MASTER

Run the ip addr command on Node 1 (MASTER):

ip addr show eth0

The output will reflect both the physical IP (192.168.10.11) and the virtual secondary IP: inet 192.168.10.100/24 scope global secondary eth0:vip.

Running the same command on Node 2 (BACKUP) will confirm that only 192.168.10.12 is present. Node 2 remains in standby mode listening for advertisements.

Step 6: Testing Automatic Failover and Recovery

Scenario A: Controlled Service Failure Simulation

1. On Node 1 (MASTER), terminate the tracked service:

   sudo systemctl stop nginx
   

2. Within 4 seconds (two failed checks × 2-second interval), chk_service fails, dropping Node 1’s effective priority below Node 2.
3. Inspect system logs on Node 2:

   sudo journalctl -u keepalived -n 20 --no-pager
   

   You will observe: VRRP_Instance(VI_STATIC) Transition to MASTER STATE, followed by Sending gratuitous ARP on eth0 for 192.168.10.100.
4. Run ip addr show eth0 on Node 2; the VIP 192.168.10.100 is now bound to Node 2.
5. Restart the service on Node 1: sudo systemctl start nginx. Node 1 regains its higher priority and preemptively claims the VIP back.

Scenario B: Total Host Outage (Kernel / Power Failure)

Simulate an abrupt host failure by cutting network connectivity on Node 1: sudo ip link set eth0 down. Node 2 misses three heartbeats (3 seconds), promotes itself to MASTER, and restores VIP connectivity instantly.

Firewall Rules for VRRP Multicast Traffic (UFW)

If Uncomplicated Firewall (UFW) is active on your Ubuntu servers, it will block VRRP multicast traffic by default, causing both nodes to claim the VIP simultaneously (split-brain). Whitelist VRRP traffic:

# Allow incoming VRRP protocol (protocol number 112) from the peer node
sudo ufw allow in proto vrrp from 192.168.10.12 to 224.0.0.18 comment 'Allow VRRP from Node 2'

# On Node 2, allow from Node 1:
sudo ufw allow in proto vrrp from 192.168.10.11 to 224.0.0.18 comment 'Allow VRRP from Node 1'

Troubleshooting Common Keepalived Failures

Issue 1: Split-Brain Condition (Both Nodes Show MASTER State)

Root Cause: Firewall rules or cloud hypervisor security groups (e.g., AWS Security Groups or Proxmox MAC filters) are dropping IP protocol 112 / multicast traffic, preventing Node 2 from seeing Node 1’s advertisements.

Remediation: Verify firewall rules. If deploying in cloud environments that do not support multicast (e.g., AWS EC2), configure unicast peer communication inside vrrp_instance:

unicast_src_ip 192.168.10.11
unicast_peer {
    192.168.10.12
}

Issue 2: Keepalived Fails to Start with “Insecure script_user” Warning

Root Cause: Modern Keepalived enforces strict script security. Tracking scripts owned by non-root users or writable by group/others are blocked.

Remediation: Ensure tracking scripts reside in root-owned paths with chmod 700, and confirm enable_script_security is declared inside global_defs.

Issue 3: MAC Address Caching Delays on Hardware Switches

Root Cause: Managed core switches fail to update their CAM tables promptly upon VIP failover.

Remediation: Configure Keepalived to broadcast multiple redundant gratuitous ARP packets upon transition by adding vrrp_garp_master_repeat 5 and vrrp_garp_master_refresh 60 to global_defs.