본문 바로가기
리눅스

우분투에서 Pacemaker와 Corosync를 사용해 고가용성 클러스터를 구성하는 방법

변군이글루 2023. 1. 31. 00:11
반응형

우분투에서 Pacemaker와 Corosync을 사용하여 HA(고가용성) 클러스터를 구성하는 방법

Pacemaker와 Corosync을 사용하여 HA (고가용성) 클러스터를 구성하는 것은 복잡한 프로세스일 수 있습니다. Pacemaker를 사용하여 VIP(가상 IP)와 노드 간 HA 클러스터를 설정하는 간략한 개요를 제공합니다.

테스트 환경

호스트 이름 서버 아이피 도메인 운영체제 비고
VIP 192.168.10.110 vip.cluster.local    
node1 192.168.10.111 node1.cluster.local Ubuntu 22.04 LTS  
node2 192.168.10.112 node2.cluster.local Ubuntu 22.04 LTS  
node3 192.168.10.113 node3.cluster.local Ubuntu 22.04 LTS  

시간 동기화

호스트 파일 설정

cat <<EOF | sudo tee -a /etc/hosts

# HA Cluster
192.168.10.110 vip.cluster.local vip
192.168.10.111 node1.cluster.local node1
192.168.10.112 node2.cluster.local node2
192.168.10.113 node3.cluster.local node3
EOF

1. 필요 패키지 설치

모든 노드에 필요한 패키지를 설치합니다.

sudo apt update
sudo apt install -y corosync pacemaker pcs
pacemakerd --version
corosync -v
pcs --version

2. Coresync 구성

/etc/corosync/corosync.conf 파일을 편집하여 클러스터 구성 설정을 정의합니다.

sudo vim /etc/corosync/corosync.conf
cat <<EOF | sudo tee /etc/corosync/corosync.conf
totem {
    version: 2
    secauth: off
    interface {
        ringnumber: 0
        bindnetaddr: 192.168.10.0
        mcastport: 5405
        ttl: 1
    }
}

logging {
    to_syslog: yes
}

nodelist {
    node {
        ring0_addr: 192.168.10.111
        nodeid: 1
    }
    node {
        ring0_addr: 192.168.10.112
        nodeid: 2
    }
    node {
        ring0_addr: 192.168.10.113
        nodeid: 3
    }
}

quorum {
    provider: corosync_votequorum
}

logging {
    to_logfile: yes
    logfile: /var/log/corosync/corosync.log
    to_syslog: yes
}
EOF

Corosync 서비스 재시작

sudo systemctl restart corosync

클러스터 상태 확인

sudo pcs status

3. Pacemaker 클러스터 구성

pcs를 사용하여 클러스터를 설정합니다.

 

pcs 서비스 활성화

sudo systemctl --now enable pcsd

hacluster 사용자 비밀번호 설정

echo -e 'hacluster:hacluster' | sudo chpasswd

노드 인증

sudo pcs host auth -u hacluster -p hacluster \
    192.168.10.111 192.168.10.112 192.168.10.113

클러스터 생성

sudo pcs cluster setup my_cluster \
    192.168.10.111 192.168.10.112 192.168.10.113 --force

클러스터 시작

sudo pcs cluster start --all

클러스터 상태 확인

sudo pcs status
Cluster name: my_cluster

WARNINGS:
No stonith devices and stonith-enabled is not false

Cluster Summary:
  * Stack: corosync
  * Current DC: 192.168.10.111 (version 2.1.2-ada5c3b36e2) - partition with quorum
  * Last updated: Wed Oct 30 22:32:29 2024
  * Last change:  Wed Oct 30 22:31:57 2024 by hacluster via crmd on 192.168.10.111
  * 3 nodes configured
  * 0 resource instances configured

Node List:
  * Online: [ 192.168.10.111 192.168.10.112 192.168.10.113 ]

Full List of Resources:
  * No resources

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled

4. 클러스터 기본 설정

STONITH(Fencing) 비활성화

sudo pcs property set stonith-enabled=false

STONITH(Fencing) 활성화

sudo pcs property set stonith-enabled=true

클러스터 설정 확인

sudo pcs property config
Cluster Properties:
 cluster-infrastructure: corosync
 cluster-name: my_cluster
 dc-version: 2.1.2-ada5c3b36e2
 have-watchdog: false
 stonith-enabled: true

5. VIP 리소스 생성

VIP 리소스를 추가하여 클러스터가 해당 IP를 관리하도록 설정합니다.

sudo pcs resource create VirtualIP ocf:heartbeat:IPaddr2 ip=192.168.10.110 \
    cidr_netmask=24 op monitor interval=30s

클러스터 상태 확인

sudo pcs status
Cluster name: my_cluster

WARNINGS:
No stonith devices and stonith-enabled is not false

Cluster Summary:
  * Stack: corosync
  * Current DC: 192.168.10.113 (version 2.1.2-ada5c3b36e2) - partition with quorum
  * Last updated: Wed Oct 30 22:38:19 2024
  * Last change:  Wed Oct 30 22:38:18 2024 by root via cibadmin on 192.168.10.112
  * 3 nodes configured
  * 1 resource instance configured

Node List:
  * Online: [ 192.168.10.111 192.168.10.112 192.168.10.113 ]

Full List of Resources:
  * VirtualIP	(ocf:heartbeat:IPaddr2):	 Started 192.168.10.111

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled

리소스 상태 확인

sudo pcs resource status
  * VirtualIP	(ocf:heartbeat:IPaddr2):	 Started 192.168.10.111

리소스 구성 검토

sudo pcs resource config
 Resource: VirtualIP (class=ocf provider=heartbeat type=IPaddr2)
  Attributes: cidr_netmask=24 ip=192.168.10.110
  Operations: monitor interval=30s (VirtualIP-monitor-interval-30s)
              start interval=0s timeout=20s (VirtualIP-start-interval-0s)
              stop interval=0s timeout=20s (VirtualIP-stop-interval-0s)

 

더보기

---

node1, node2 노드에서 같이 실행합니다.

1. 호스트 등록(hosts)

/etc/hosts 편집

cat <<EOF > /etc/hosts
# Cluster
192.168.0.60    vip.cluster.local       vip
192.168.0.51    control1.cluster.local  control1
192.168.0.63    node3.cluster.local     node3
EOF

2. Pacemaker와 Corosync 패키지 설치

sudo apt-get update
apt-get install -y pacemaker corosync
pacemakerd --version
$ pacemakerd --version
Pacemaker 2.1.2
Written by Andrew Beekhof
corosync -v
$ corosync -v
Corosync Cluster Engine, version '3.1.6'
Copyright (c) 2006-2021 Red Hat, Inc.

Built-in features: dbus monitoring watchdog augeas systemd xmlconf vqsim nozzle snmp pie relro bindnow
Available crypto models: nss openssl
Available compression models: zlib lz4 lz4hc lzo2 lzma bzip2 zstd
pcs --version
$ pcs --version
0.10.11
$ cat /etc/passwd | grep hacluster
hacluster:x:115:120::/var/lib/pacemaker:/usr/sbin/nologin

pcsd(pacemaker) 활성화 및 시작

systemctl --now enable pcsd

pcsd 서비스 확인

systemctl status pcsd

hacluster 계정의 비밀번호 생성

- hacluster 비밀번호 : hacluster

echo -e 'hacluster:hacluster' | chpasswd

(or)

passwd hacluster
$ passwd hacluster
Changing password for user hacluster.
New password:
BAD PASSWORD: The password contains the user name in some form
Retype new password:
passwd: all authentication tokens updated successfully.

3. pacemaker 클러스터 생성

systemctl restart pcsd
pcs status
$ pcs status
Error: error running crm_mon, is pacemaker running?
  crm_mon: Error: cluster is not available on this node
pcs cluster status
$ pcs cluster status
Error: cluster is not currently running on this node

4. 한쪽 노드에서 클러스터 생성 및 실행하기

pcs host auth control1.cluster.local node3.cluster.local -u hacluster
root@control1:~$ pcs host auth control1.cluster.local node3.cluster.local -u hacluster
Password: 
node3.cluster.local: Authorized
control1.cluster.local: Authorized
pcs cluster setup hacluster control1.cluster.local node3.cluster.local --force
root@control1:~$ pcs cluster setup hacluster control1.cluster.local node3.cluster.local --force
No addresses specified for host 'control1.cluster.local', using 'control1.cluster.local'
No addresses specified for host 'node3.cluster.local', using 'node3.cluster.local'
Destroying cluster on hosts: 'control1.cluster.local', 'node3.cluster.local'...
control1.cluster.local: Successfully destroyed cluster
node3.cluster.local: Successfully destroyed cluster
Requesting remove 'pcsd settings' from 'control1.cluster.local', 'node3.cluster.local'
control1.cluster.local: successful removal of the file 'pcsd settings'
node3.cluster.local: successful removal of the file 'pcsd settings'
Sending 'corosync authkey', 'pacemaker authkey' to 'control1.cluster.local', 'node3.cluster.local'
control1.cluster.local: successful distribution of the file 'corosync authkey'
control1.cluster.local: successful distribution of the file 'pacemaker authkey'
node3.cluster.local: successful distribution of the file 'corosync authkey'
node3.cluster.local: successful distribution of the file 'pacemaker authkey'
Sending 'corosync.conf' to 'control1.cluster.local', 'node3.cluster.local'
control1.cluster.local: successful distribution of the file 'corosync.conf'
node3.cluster.local: successful distribution of the file 'corosync.conf'
Cluster has been successfully set up.
pcs cluster auth -u hacluster -p hacluster
root@control1:~$ pcs cluster auth -u hacluster -p hacluster
control1.cluster.local: Already authorized
node3.cluster.local: Already authorized
Sending cluster config files to the nodes...
pcs cluster start --all
pcs cluster enable --all
control1 node3
$ pcs cluster start --all
node3.cluster.local: Starting Cluster...
control1.cluster.local: Starting Cluster...		 $ pcs cluster start --all
control1.cluster.local: Starting Cluster...
node3.cluster.local: Starting Cluster...
$ pcs cluster enable --all
control1.cluster.local: Cluster Enabled
node3.cluster.local: Cluster Enabled		 $ pcs cluster enable --all
control1.cluster.local: Cluster Enabled
pcs cluster statusnode3.cluster.local: Cluster Enabled
hostname command
control1 $ pcs status
Cluster name: hacluster

WARNINGS:
No stonith devices and stonith-enabled is not false

Cluster Summary:
  * Stack: corosync
  * Current DC: node3.cluster.local (version 2.1.2-ada5c3b36e2) - partition with quorum
  * Last updated: Tue Feb  7 10:25:30 2023
  * Last change:  Tue Feb  7 10:24:56 2023 by hacluster via crmd on node3.cluster.local
  * 2 nodes configured
  * 0 resource instances configured

Node List:
  * Online: [ control1.cluster.local node3.cluster.local ]

Full List of Resources:
  * No resources

Daemon Status:
  corosync: active/enabled
  pacemaker: active/enabled
  pcsd: active/enabled
control1 $ pcs cluster status
Cluster Status:
 Cluster Summary:
   * Stack: corosync
   * Current DC: node3.cluster.local (version 2.1.2-ada5c3b36e2) - partition with quorum
   * Last updated: Tue Feb  7 10:25:33 2023
   * Last change:  Tue Feb  7 10:24:56 2023 by hacluster via crmd on node3.cluster.local
   * 2 nodes configured
   * 0 resource instances configured
 Node List:
   * Online: [ control1.cluster.local node3.cluster.local ]

PCSD Status:
  control1.cluster.local: Online
  node3.cluster.local: Online
node3 $ pcs status
Cluster name: hacluster

WARNINGS:
No stonith devices and stonith-enabled is not false

Cluster Summary:
  * Stack: corosync
  * Current DC: node3.cluster.local (version 2.1.2-ada5c3b36e2) - partition with quorum
  * Last updated: Tue Feb  7 10:25:30 2023
  * Last change:  Tue Feb  7 10:24:56 2023 by hacluster via crmd on node3.cluster.local
  * 2 nodes configured
  * 0 resource instances configured

Node List:
  * Online: [ control1.cluster.local node3.cluster.local ]

Full List of Resources:
  * No resources

Daemon Status:
  corosync: active/enabled
  pacemaker: active/enabled
  pcsd: active/enabled
node3 $ pcs cluster status
Cluster Status:
 Cluster Summary:
   * Stack: corosync
   * Current DC: node3.cluster.local (version 2.1.2-ada5c3b36e2) - partition with quorum
   * Last updated: Tue Feb  7 10:25:33 2023
   * Last change:  Tue Feb  7 10:24:56 2023 by hacluster via crmd on node3.cluster.local
   * 2 nodes configured
   * 0 resource instances configured
 Node List:
   * Online: [ control1.cluster.local node3.cluster.local ]

PCSD Status:
  node3.cluster.local: Online
  control1.cluster.local: Online		 
cat /var/lib/pcsd/known-hosts
cat /etc/corosync/corosync.conf

클러스터 옵션 구성

pcs cluster status
$ pcs cluster status
Cluster Status:
 Cluster Summary:
   * Stack: corosync
   * Current DC: node3.cluster.local (version 2.1.2-ada5c3b36e2) - partition with quorum
   * Last updated: Tue Feb  7 10:27:14 2023
   * Last change:  Tue Feb  7 10:24:56 2023 by hacluster via crmd on node3.cluster.local
   * 2 nodes configured
   * 0 resource instances configured
 Node List:
   * Online: [ control1.cluster.local node3.cluster.local ]

PCSD Status:
  control1.cluster.local: Online
  node3.cluster.local: Online
$ crm_simulate -sL     
[ control1.cluster.local node3.cluster.local ]

No resources
$ pcs constraint config
Location Constraints:
Ordering Constraints:
Colocation Constraints:
Ticket Constraints:

 

정책 변경

- stonith 비활성화 (STONITH = Shoot The Other Node In The Head)

pcs property set stonith-enabled=false

- quorum policy 끄기

pcs property set no-quorum-policy=ignore

 

리소스 표준

$ pcs resource standards
lsb
ocf
service
systemd
$ pcs resource providers
heartbeat
pacemaker
$ pcs resource agents ocf:heartbeat
IPaddr2
iscsi
iSCSILogicalUnit
iSCSITarget
LVM-activate

 

클러스터 삭제

pcs cluster stop --all

known-hosts 파일 삭제

rm -f /var/lib/pcsd/known-hosts

corosync.conf 파일 삭제

rm -f /etc/corosync/corosync.conf

authkey 파일 삭제

rm -f /etc/pacemaker/authkey

클러스터 삭제

pcs cluster destroy

pacemaker corosync pcs 재설치

apt-get reinstall -y pacemaker corosync pcs

---

 

참고URL

- Pacemaker 1.1(Configuration Explained) : Pacemaker-1.1-Pacemaker_Explained-en-US.pdf

 

728x90
반응형
저작자표시 비영리

'리눅스' 카테고리의 다른 글

free 명령어  (0) 2023.01.31
[리눅스] consul 보안(security credentials) 설정  (0) 2023.01.31
[리눅스] envoy를 사용하는 도커 컨테이너 리버스 프록시(docker container reverse proxy)  (0) 2023.01.29
[리눅스] traefik을 사용하는 도커 컨테이너 리버스 프록시(docker container reverse proxy)  (0) 2023.01.27
[리눅스] nomad cluster 구성(install nomad cluster)  (0) 2023.01.27

'리눅스' Related Articles

티스토리툴바