How to Configure Corosync, Pacemaker, and pcs for High-Availability Cluster Resource Management on Ubuntu Server

Modern enterprise infrastructure demands continuous operational uptime. When critical services—such as relational database engines, reverse proxies, or messaging brokers—experience unexpected hardware faults or operating system panics, automated failover mechanisms must instantaneously transfer IP endpoints and stateful workloads to surviving standby nodes without manual administrative intervention.

In the Linux ecosystem, the definitive open-source standard for high-availability orchestration combines Corosync (the cluster messaging and membership layer) and Pacemaker (the intelligent Cluster Resource Manager). Managed through the modern pcs command-line interface, this clustering stack monitors daemon health, manages floating Virtual IP addresses (VIPs), enforces resource colocation rules, and handles fencing (STONITH) to eliminate split-brain risk. This guide provides an end-to-end operational walkthrough for deploying and configuring a robust Pacemaker/Corosync cluster on Ubuntu Server.

---

High-Availability Architecture: The Clustering Stack

A production Linux HA cluster consists of three symbiotic layers operating cooperatively across interconnected nodes:

Clustering Layer	Software Daemon	Core Architectural Responsibility
Messaging & Membership	corosync	Maintains node connectivity using the Kronosnet (knet) protocol. Handles heartbeat discovery, node joins, and quorum calculation.
Resource Manager (CRM)	pacemakerd	Orchestrates resource states. Evaluates dependencies, initiates service start/stop sequences, and monitors service health via Resource Agents (OCF/systemd).
Configuration & Control	pcsd / pcs	A unified CLI and daemon wrapper that synchronises cluster configuration files (corosync.conf and CIB XML) across all nodes simultaneously.
Fencing / Isolation	stonith	“Shoot The Other Node In The Head.” Uses hardware IPMI, iLO, or cloud fencing agents to power down malfunctioning nodes and prevent split-brain disk corruption.

---

Cluster Topology & Prerequisites

This deployment establishes a two-node active/passive high-availability cluster serving a floating virtual IP address and a managed Nginx web service on Ubuntu Server (22.04 LTS or 24.04 LTS).

Node Hostname	Private IP (Internal Network)	Subnet / Gateway	Assigned Role
ha-node01	192.168.20.11	192.168.20.0/24	Primary Cluster Node
ha-node02	192.168.20.12	192.168.20.0/24	Secondary Cluster Node
cluster-vip	192.168.20.50	Floating VIP	Shared Service Endpoint

---

Step 1: Network Configuration and Host Resolution

Corosync and Pacemaker require low-latency, deterministic name resolution across all participating cluster nodes. Ensure static host mappings are present in /etc/hosts on both nodes:

sudo nano /etc/hosts

Append the cluster resolution records:

192.168.20.11    ha-node01
192.168.20.12    ha-node02

Next, configure firewall rules using UFW to permit Corosync cluster communication and pcsd administrative synchronisation:

# Allow pcsd administrative sync daemon
sudo ufw allow from 192.168.20.0/24 to any port 2224 proto tcp comment 'PCS Daemon'

# Allow Corosync messaging protocol (knet UDP transport)
sudo ufw allow from 192.168.20.0/24 to any port 5404:5405 proto udp comment 'Corosync'
sudo ufw reload

---

Step 2: Installing Pacemaker, Corosync, and PCS

Execute the following package installations on both nodes to deploy the complete clustering toolset:

# Update local repositories and install high-availability packages
sudo apt update && sudo apt install -y pacemaker corosync pcs resource-agents fence-agents

# Enable and start the pcs daemon
sudo systemctl enable --now pcsd
sudo systemctl status pcsd --no-pager

---

Step 3: Configuring Cluster Authentication

The pcs installation automatically creates a system user named hacluster. This user account must have an identical, secure password across all nodes to permit remote configuration exchange.

Execute on both nodes:

# Set password for the hacluster service account
sudo passwd hacluster

From ha-node01 only, authenticate the cluster nodes using the pcs host auth command:

sudo pcs host auth ha-node01 ha-node02 -u hacluster

Enter the password when prompted. Expected output confirms authentication success:

ha-node01: Authorized
ha-node02: Authorized

---

Step 4: Creating and Starting the Cluster

With authentication established, generate the cluster definition from ha-node01. This command creates /etc/corosync/corosync.conf and propagates it across all nodes:

# Setup cluster named 'web_cluster' across ha-node01 and ha-node02
sudo pcs cluster setup web_cluster ha-node01 ha-node02

# Start cluster services across all nodes
sudo pcs cluster start --all

# Enable cluster services to automatically start on boot
sudo pcs cluster enable --all

Verify cluster connectivity and membership status:

sudo pcs cluster status

Expected verification output:

Cluster Status:
 Cluster name: web_cluster
 Nodes:
  ha-node01: Online
  ha-node02: Online
 Corosync: active with 2 nodes
 Pacemaker: active with 2 nodes

---

Step 5: Tuning Cluster Properties for a Two-Node Topology

In standard multi-node deployments, Pacemaker requires a strict mathematical quorum (>50% of nodes online) to take administrative actions. Additionally, Pacemaker enforces STONITH (fencing) by default. In a two-node environment without hardware fencing devices, you must adjust these properties to prevent cluster paralysis:

# Disable STONITH fencing (appropriate for non-storage environments or testing)
sudo pcs property set stonith-enabled=false

# Configure cluster to ignore quorum loss in a two-node topology
sudo pcs property set no-quorum-policy=ignore

# Set default resource stickiness to avoid unnecessary failback when a node recovers
sudo pcs resource defaults update resource-stickiness=100

---

Step 6: Creating the Floating Virtual IP (VIP) Resource

Pacemaker manages services via standardised Open Cluster Framework (OCF) Resource Agents. The ocf:heartbeat:IPaddr2 agent dynamically assigns a secondary IP address to the active node’s physical network interface via ARP poisoning updates.

Execute from ha-node01:

# Create floating Virtual IP resource named 'Virtual_IP'
sudo pcs resource create Virtual_IP ocf:heartbeat:IPaddr2 \
    ip=192.168.20.50 \
    cidr_netmask=24 \
    op monitor interval=10s

Verify that the IP is active on the primary node:

ip addr show

---

Step 7: Configuring Service Resources and Dependency Constraints

Now, configure a managed application service, such as Nginx. The service daemon must be installed on both nodes, but disabled from standard systemd auto-start so Pacemaker maintains exclusive lifecycle control.

Execute on both nodes:

sudo apt install -y nginx
sudo systemctl stop nginx
sudo systemctl disable nginx

Create the Pacemaker service resource from ha-node01:

# Create Nginx service resource managed by systemd agent
sudo pcs resource create WebService systemd:nginx op monitor interval=15s

Enforcing Colocation and Ordering Constraints

In a high-availability architecture, the WebService must run on the exact same physical node as the Virtual_IP, and the IP must be fully bound before Nginx attempts to start.

# Colocation constraint: WebService MUST run on whichever node holds Virtual_IP
sudo pcs constraint colocation add WebService with Virtual_IP INFINITY

# Ordering constraint: Virtual_IP must start BEFORE WebService starts
sudo pcs constraint order Virtual_IP then WebService

---

Step 8: Verifying Cluster Health and Testing Failover

Check the global cluster status and resource placement using pcs status:

sudo pcs status

Expected operational output:

Cluster name: web_cluster
Cluster Summary:
  * Stack: corosync
  * Current DC: ha-node01 (version 2.1.2) - partition with quorum
  * Last updated: Wed Oct 23 20:25:00 2026
  * Last change:  Wed Oct 23 20:24:10 2026 by root via cibadmin on ha-node01
  * 2 nodes configured
  * 2 resource instances configured

Node List:
  * Online: [ ha-node01 ha-node02 ]

Full List of Resources:
  * Virtual_IP  (ocf:heartbeat:IPaddr2):  Started ha-node01
  * WebService  (systemd:nginx):          Started ha-node01

Simulating Node Failure via Standby Mode

To test automated failover non-destructively, place ha-node01 into standby mode:

sudo pcs node standby ha-node01

Re-run pcs status. Pacemaker immediately stops Nginx on ha-node01, migrates Virtual_IP to ha-node02, and restarts Nginx on ha-node02 within seconds.

To restore the node to active cluster service:

sudo pcs node unstandby ha-node01

Because we configured resource-stickiness=100, the services remain gracefully on ha-node02, avoiding unnecessary double-failover disruption.

---

Troubleshooting Common Pacemaker and Corosync Faults

Observed Issue	Root Cause	Diagnostic & Remediation Step
pcs host auth fails with Unable to authenticate	The hacluster password is misaligned or pcsd service is inactive on target	Verify systemctl status pcsd on both machines. Reset password using sudo passwd hacluster to ensure exact matching characters.
Nodes show as UNCLEAN (offline) in pcs status	Corosync UDP packets (ports 5404/5405) blocked by firewall or network partition	Test network connectivity using nc -z -v -u ha-node02 5405. Verify UFW rules on both nodes.
Resources report Blocked or refuse to start on any node	STONITH is enabled without working fencing devices, or quorum was lost	Execute pcs property set stonith-enabled=false and verify no-quorum-policy=ignore for two-node clusters. Inspect /var/log/pacemaker/pacemaker.log.
Service resource enters Failed (unmanaged) state	Resource monitor operation failed repeatedly or underlying binary crashed	Diagnose the root cause in journalctl -u nginx. Once resolved, clear the failure counter via pcs resource cleanup WebService.

---

Operational Governance and Best Practices

• Deploy an External Quorum Witness for 2-Node Clusters: In critical production environments, avoid no-quorum-policy=ignore by deploying corosync-qnet on a lightweight third machine or cloud VM to maintain a true 3-way mathematical voting quorum.
• Implement Real STONITH Devices: Never run shared storage (such as iSCSI, Fibre Channel, or clustered GFS2) without physical fencing (IPMI, iLO, or cloud provider API fencing agents) to eliminate the risk of split-brain data corruption.
• Regularly Audit Cluster Logs: Configure central syslog forwarding for /var/log/corosync/corosync.log and /var/log/pacemaker/pacemaker.log to detect transient network packet loss before full cluster splits occur.