3 Nodes DRBD 8.3 Setup by Ahmad Adnan

Three-way replication via DRBD stacking.

Complied by: Ahmad Adnan

Diagram

Disclaimer:

The following steps are performed on Virtual machines and tested successfully.

It is advised to test first on raw / virtual machines and then in production environment.

 

We will setup for Three Nodes on High Availability Cluster.

Test Scenario:

We have two virtual machines running on Oracle VirtualBox:

OS:       CentOS 5.10

RAM:   512MB

HDD:    2 virtual hard disks (5GB (cos-5.vdi) + 3GB (DRBD.vdi))

===================================================

!!!Tip – Minimal Installation!!!

You can do a minimal install that just requires the first CD by performing the following two steps during the installation:

1. During the category/task selection, deselect all package categories, and choose the Customize now” option at the bottom of screen.

2. During the customized package selection, deselect everything (including the Base group).

Default Swap would be 1GB, you can reduce it by following commands:

!!!Tip – Reducing Swap on an LVM2 Logical Volume!!!

To reduce an LVM2 swap logical volume (assuming /dev/VolGroup00/LogVol01 is the volume you want to extend):

1. Disable swapping for the associated logical volume:

# swapoff -v /dev/VolGroup00/LogVol01

2. Reduce the LVM2 logical volume by 512 MB:

# lvm lvreduce /dev/VolGroup00/LogVol01 -L -512M

3. Format the new swap space:

# mkswap /dev/VolGroup00/LogVol01

4. Enable the extended logical volume:

# swapon -va

5. Test that the logical volume has been reduced properly:

# cat /proc/swaps # free

===================================================

Hard Disk Storage:

cos-Disks

===================================================

Network setup is as follows:

> Three servers: cos-1, cos-1-a, cos-1-b

> cos-1 and cos-1-a are the primary and secondary local nodes

> cos-1-b is the third node which is on a remote network

> Both cos-1 and cos-1-a have interfaces on the 192.168.1.x network (eth0) for external connectivity. (Image – cos-5 Network Adapter 1)

> A crossover link exists on cos-1 and cos-1-a (eth1) for replication using 172.16.6.10 and .20 (Image – cos-5 Network Adapter 2)

> Heartbeat provides a virtual IP of 192.168.5.2 to communicate with the disaster recovery node located in a geographically diverse location

On all three nodes:

 

cos-1                eth0     192.168.1.10

eth1     172.16.6.10/24—|Static IP

cos-1-a                        eth0     192.168.1.11

eth1     172.16.6.20/24—|Static IP

 

cos-1-b                        eth0     192.168.5.3

 

eth0

cos-eth0

eth1

cos-eth1

===================================================

[root@cos-1 ~]# vim /etc/hosts

192.168.1.10 cos-1

192.168.1.11 cos-1-a

save & exit

 

[root@cos-1-a ~]# ping cos-1

PING cos-1 (192.168.1.10) 56(84) bytes of data.

64 bytes from cos-1 (192.168.1.10): icmp_seq=1 ttl=64 time=4.15 ms

64 bytes from cos-1 (192.168.1.10): icmp_seq=2 ttl=64 time=0.126 ms

64 bytes from cos-1 (192.168.1.10): icmp_seq=3 ttl=64 time=1.88 ms

 

[root@cos-1 ~]# ping cos-1-a

PING cos-1-a (192.168.1.11) 56(84) bytes of data.

64 bytes from cos-1-a (192.168.1.11): icmp_seq=1 ttl=64 time=1.32 ms

64 bytes from cos-1-a (192.168.1.11): icmp_seq=2 ttl=64 time=0.523 ms

64 bytes from cos-1-a (192.168.1.11): icmp_seq=3 ttl=64 time=1.79 ms

 

[root@cos-1 ~]# scp /etc/hosts cos-1-a:/etc/hosts

 

!!! Tip !!!

In case scp command fails and displays as:

[root@cos-1 ~]# scp /etc/hosts cos-1-a:/etc/hosts

bash: scp: command not found
lost connection

 

The reason is openssh-clients have not been installed on the remote machine. To copy files over scp , openssh-clients needs to be installed on both machines.

 

[root@cos-1 ~]# rpm -qa | grep openssh-client

openssh-clients-4.3p2-82.el5

 

If not installed following command will show nothing as below:

 

[root@cos-1-a ~]# rpm -qa | grep openssh-client

[root@cos-1-a ~]#

 

Run the following command on each missing node.

[root@cos-1-a ~]# yum install -y openssh-clients

===================================================

For static IP Addresses (eth1) change or add and test as following:

[root@cos-1 ~]# vi /etc/sysconfig/network-scripts/ifcfg-eth1

DEVICE=eth1

BOOTPROTO=static

ONBOOT=yes

HWADDR=

NETWORK=172.16.6.0

NETMASK=255.255.255.0

IPADDR=172.16.6.10

save and exit.

 

[root@cos-1 ~]# service network restart

Shutting down interface eth0:                         [ OK ]

Shutting down interface eth1:                         [ OK ]

Shutting down loopback interface:                 [ OK ]

Bringing up loopback interface:                       [ OK ]

Bringing up interface eth0:

Determining IP information for eth0… done.[ OK ]

Bringing up interface eth1:                               [ OK ]

===================================================

[root@cos-1-a ~]# vi /etc/sysconfig/network-scripts/ifcfg-eth1

 

DEVICE=eth1

BOOTPROTO=static

ONBOOT=yes

HWADDR=

NETWORK=172.16.6.0

NETMASK=255.255.255.0

IPADDR=172.16.6.20

 

save and exit.

 

[root@cos-1a ~]# service network restart

Shutting down interface eth0:                         [ OK ]

Shutting down interface eth1:                         [ OK ]

Shutting down loopback interface:                 [ OK ]

Bringing up loopback interface:                       [ OK ]

Bringing up interface eth0:

Determining IP information for eth0… done.[ OK ]

Bringing up interface eth1:                              [ OK ]

 

[root@cos-1 ~]# ping 172.16.6.20

PING 172.16.6.20 (172.16.6.20) 56(84) bytes of data.

64 bytes from 172.16.6.20: icmp_seq=1 ttl=64 time=0.638 ms

64 bytes from 172.16.6.20: icmp_seq=2 ttl=64 time=1.30 ms

64 bytes from 172.16.6.20: icmp_seq=3 ttl=64 time=0.489 ms

===================================================

[root@cos-1-b ~]# vi /etc/sysconfig/network-scripts/ifcfg-eth0

 

DEVICE=eth0

BOOTPROTO=static

ONBOOT=yes

HWADDR=

DHCP_HOSTNAME=cos-1-b

NETWORK=192.168.5.0

NETMASK=255.255.255.0

IPADDR=192.168.5.3

===================================================

Stop unwanted services on both servers

 

[root@cos-1 ~]# /etc/init/sendmail stop

[root@cos-1 ~]# chkconfig -–level 235 sendmail off

[root@cos-1 ~]# iptables -F

[root@cos-1 ~]# service iptables save

 

[root@cos-1-a ~]# /etc/init/sendmail stop

[root@cos-1-a ~]# chkconfig -–level 235 sendmail off

[root@cos-1-a ~]# iptables -F

[root@cos-1-a ~]# service iptables save

 

[root@cos-1-b ~]# /etc/init/sendmail stop

[root@cos-1-b ~]# chkconfig –level 235 sendmail off

[root@cos-1-b ~]# iptables -F

[root@cos-1-b ~]# service iptables save

=========================================

NTP Configuration

=========================================

[root@cos-1 ~]# yum install ntp -y

[root@cos-1 ~]# vi /etc/ntp.conf

 

Change as following.

 

### Commented By Ahmad Adnan on 14th of June 2014 ###

#server 0.centos.pool.ntp.org

#server 1.centos.pool.ntp.org

#server 2.centos.pool.ntp.org

server 127.127.1.0 # local clock

#fudge 127.127.1.0 stratum 10

# driftfile /var/lib/ntp/drift

keys /etc/ntp/keys

 

save & quit.

 

[root@cos-1 ~]# /etc/init.d/ntpd start

[root@cos-1 ~]# chkconfig –level 235 ntpd on

======================================

[root@cos-1-a ~]# yum install ntp -y

[root@cos-1-a ~]# vi /etc/ntp.conf

 

server 192.168.1.10 ### add this line on second server ###

#server 0.centos.pool.ntp.org

#server 1.centos.pool.ntp.org

#server 2.centos.pool.ntp.org

#server 127.127.1.0 # local clock ##### #####

#fudge 127.127.1.0 stratum 10

driftfile /var/lib/ntp/drift

keys /etc/ntp/keys

 

save & exit

 

[root@cos-1-a ~]# /etc/init.d/ntpd start

[root@cos-1-a ~]# chkconfig –level 235 ntpd on

======================================

[root@cos-1-b ~]# yum install ntp -y

[root@cos-1-b ~]# vi /etc/ntp.conf

 

server 192.168.5.2 ### add this line on second server ###

 

#server 0.centos.pool.ntp.org

#server 1.centos.pool.ntp.org

#server 2.centos.pool.ntp.org

 

#server 127.127.1.0 # local clock ##### #####

#fudge 127.127.1.0 stratum 10

 

driftfile /var/lib/ntp/drift

keys /etc/ntp/keys

save & exit

 

[root@cos-1-b ~]# /etc/init.d/ntpd start

[root@cos-1-b ~]# chkconfig –level 235 ntpd on

====================================

 

[root@cos-1-a ~]# ntpdate -u 192.168.1.10

 

[root@cos-1-a ~]# watch ntpq -p -n

ntp-cos-1-a

[root@cos-1-b ~]# ntpdate -u 192.168.5.2

 

[root@cos-1-b ~]# watch ntpq -p -n

ntp-cos-1-b

[root@cos-1 ~]# watch ntpq -p -n

ntp-cos-1

=========================================

Partition setup on all Servers.

=========================================

Partition setup on all server identical same with fdisk.

We have 3GB disks on both servers and create LVM partition type.

Note: Create same partitions on all servers identical as below:

 

[root@cos-1 ~]# fdisk /dev/sd

sda sda1 sda2 sdb sdb1

 

[root@cos-1 ~]# fdisk /dev/sdb1

 

Command (m for help): p

 

Disk /dev/sdb1: 3221 MB, 3221225472 bytes

255 heads, 63 sectors/track, 391 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

 

Device Boot Start End Blocks Id System

/dev/sdb1 1 522 4192933+ 8e Linux LVM

 

Command (m for help): d

Selected partition 1

 

Command (m for help): n

Command action

e extended

p primary partition (1-4)

p

Partition number (1-4): 1

First cylinder (1-391, default 1):

Using default value 1

Last cylinder or +size or +sizeM or +sizeK (1-391, default 391): 391

 

Command (m for help): p

 

Disk /dev/sdb: 3221 MB, 3221225472 bytes

255 heads, 63 sectors/track, 391 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

 

Device Boot     Start         End     Blocks   Id System

/dev/sdb1               1         366     2939863+ 83 Linux

 

Command (m for help): t

Selected partition 1

Hex code (type L to list codes): 8e

Changed system type of partition 1 to 8e (Linux LVM)

 

Command (m for help): p

 

Disk /dev/sdb: 3221 MB, 3221225472 bytes

255 heads, 63 sectors/track, 391 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

 

Device Boot     Start         End     Blocks   Id System

/dev/sdb1               1         366     2939863+ 8e Linux

 

Command (m for help): w

 

[root@cos-1 ~]# partprobe

=======================================================

Create Physical Volume for LVM this is second step for LVM partition.

Note: Create LVM on all servers identical same as below:

[root@cos-1 ~]# pvcreate /dev/sdb1

 

Create Volume Group with this command

 

[root@cos-1 ~]# vgcreate vgdrbd /dev/sdb1

 

Create Logical volume partition

 

[root@cos-1 ~]# lvcreate -n lvdrbd /dev/mapper/vgdrbd -L +293M

 

Rounding up size to full physical extent 296.00 MB

Logical volume “lvdrbd” created

 

======================================================

DRBD (Distributed Replicated Block Device)

======================================================

Note: Install DRBD on all 3 servers

[root@cos-1 ~]# yum install -y drbd83 kmod-drbd83

 

Installed:

kmod-drbd83.i686 0:8.3.15-3.el5.centos

drbd83.i386 0:8.3.15-2.el5.centos

 

Complete!

[root@cos-1 ~]# chkconfig –level 235 drbd on

[root@cos-1 ~]# modprobe drbd

[root@cos-1 ~]# cat /proc/drbd

version: 8.3.15 (api:88/proto:86-97)

GIT-hash: 0ce4d235fc02b5c53c1c52c53433d11a694eab8c build by

mockbuild@builder17.centos.org, 2013-03-27 16:04:08

 

==========================================

Heartbeat Installation

===========================================

Note: Setting up a third node entails stacking DRBD on top of DRBD. A virtual IP is needed for the third node to connect to, for this we will set up a simple Heartbeat v1 configuration. This section will only be done on cos-1 and cos-1-a.

 

[root@cos-1 ~]# yum install -y heartbeat

 

Installed:

heartbeat.i386 0:2.1.3-3.el5.centos

 

Complete!

 

[root@cos-1 ~]# chkconfig –level 235 heartbeat on

 

==========================================

Heartbeat Configuration

===========================================

[root@cos-1 ~]# vi /etc/ha.d/authkeys

 

auth 1

1 sha1 pa$$

 

save and exit.

 

[root@cos-1 ~]# chmod 600 /etc/ha.d/authkeys

 

[root@cos-1 ~]# scp /etc/ha.d/authkeys cos-1-a:/etc/ha.d/authkeys

 

[root@cos-1 ha.d]# vi /etc/ha.d/ha.cf

 

debugfile /var/log/ha-debug

logfile /var/log/ha-log

logfacility     local0

keepalive 1

deadtime 10

warntime 5

initdead 60

udpport 694

bcast eth1

ucast eth1 172.16.6.20

auto_failback off

node cos-1

node cos-1-a

 

save & exit.

 

[root@cos-1-a ha.d]# vi /etc/ha.d/ha.cf

 

debugfile /var/log/ha-debug

logfile /var/log/ha-log

logfacility     local0

keepalive 1

deadtime 10

warntime 5

initdead 60

udpport 694

bcast eth1

ucast eth1 172.16.6.10

auto_failback off

node cos-1

node cos-1-a

 

save & exit.

 

[root@cos-1 ~]# vi /etc/ha.d/haresources

 

cos-1 IPaddr::192.168.5.2/24/eth0

 

save and exit.

 

[root@cos-1 ~]# scp /etc/ha.d/haresources cos-1-a:/etc/ha.d/haresources

 

Start the heartbeat service on both servers to bring up the virtual IP:

 

[root@cos-1 ~]# /etc/init.d/heartbeat start

logd is already running

Starting High-Availability services:

2014/05/31_22:56:56 INFO: Resource is stopped     [ OK ]

 

[root@cos-1-a ~]# /etc/init.d/heartbeat start

Starting High-Availability services:

2014/05/31_22:57:15 INFO: Resource is stopped     [ OK ]

 

Heartbeat will bring up the new interface (eth0:0).

 

Note: It may take heartbeat up to one minute to bring the interface up.

 

[root@cos-1 ~]# ifconfig

cos-1-eth00-vip

3rd Node is also up now and responding:

[root@cos-1 ~]# ping 192.168.5.3

Ping VIP

====================================================

DRBD Configuration

====================================================

We will first test DRBD connection in Primary and Secondary nodes and once tested we will proceed for 3rd node stacked on Backup. Configuration for DRBD is done via the drbd.conf file. This needs to be the same on all 3 nodes (cos-1, cos-1-a, cos-1-b).

[root@cos-1 ~]# vi /etc/drbd.conf

resource drbd-lower {

protocol C;

net {

shared-secret “PASSWORD”;

}

syncer {

rate 12M;

}

 

on cos-1 {

device     /dev/drbd0;

disk       /dev/vgdrbd/lvdrbd;

address   172.16.6.10:7788;

meta-disk internal;

}

 

on cos-1-a {

device   /dev/drbd0;

disk     /dev/vgdrbd/lvdrbd;

address   172.16.6.20:7788;

meta-disk internal;

}

}

 

save and exit.

[root@cos-1 ~]# scp /etc/drbd.conf cos-1-a:/etc/drbd.conf

[root@cos-1 ~]# drbdadm create-md drbd-lower

you are the 17936th user to install this version

Writing meta data…

initializing activity log

NOT initialized bitmap

New drbd meta data block successfully created.

success

 

[root@cos-1 ~]# /etc/init.d/drbd start

[root@cos-1 ~]# drbdadm — –overwrite-data-of-peer primary drbd-lower

[root@cos-1 ~]# cat /proc/drbd

version: 8.3.15 (api:88/proto:86-97)

0: cs:Connected ro:Primary/Secondary ds:Diskless/UpToDate C r—–

ns:17 nr:35 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0

===================================================

[root@cos-1-a ~]# drbdadm create-md drbd-lower

[root@cos-1-a ~]# /etc/init.d/drbd start

[root@cos-1-a ~]# cat /proc/drbd

version: 8.3.15 (api:88/proto:86-97)

0: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r—–

ns:0 nr:303056 dw:303056 dr:0 al:0 bm:19 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0

==================================================

[root@cos-1 ~]# vi /etc/drbd.conf

 

###add the following ###

 

resource drbd-upper {

protocol A;

syncer {

after drbd-lower;

rate 12M;

al-extents 513;

}

net {

shared-secret “PASSWORD”;

}

stacked-on-top-of drbd-lower {

device /dev/drbd10;

address 192.168.5.2:7788; # IP provided by Heartbeat

}

 

on cos-1-b {

device     /dev/drbd10;

disk       /dev/vgdrbd/lvdrbd;

address   192.168.5.3:7788; # Public IP of the backup node

meta-disk internal;

}

}

 

save & exit.

 

[root@cos-1 ~]# drbdadm –stacked create-md drbd-upper

.

..

You want me to create a v08 style flexible-size internal meta data block.

There appears to be a v08 flexible-size internal meta data block

already in place on /dev/drbd0 at byte offset 310325248

Do you really want to overwrite the existing v08 meta-data?

[need to type ‘yes’ to confirm] yes

 

Writing meta data…

initializing activity log

NOT initialized bitmap

New drbd meta data block successfully created.

[root@cos-1-b ~]# drbdadm create-md drbd-upper

[root@cos-1 ~]# drbdadm –stacked adjust drbd-upper

[root@cos-1-b ~]# drbdadm adjust drbd-upper

[root@cos-1 ~]# drbdadm –stacked — –overwrite-data-of-peer primary drbd-upper

[root@cos-1 ~]# mkfs.ext3 /dev/drbd10

mke2fs 1.39 (29-May-2006)

Filesystem label=

OS type: Linux

Block size=1024 (log=0)

Fragment size=1024 (log=0)

75776 inodes, 303008 blocks

15150 blocks (5.00%) reserved for the super user

First data block=1

Maximum filesystem blocks=67633152

37 block groups

8192 blocks per group, 8192 fragments per group

2048 inodes per group

Superblock backups stored on blocks:

8193, 24577, 40961, 57345, 73729, 204801, 221185

 

Writing inode tables: done

Creating journal (8192 blocks): done

Writing superblocks and filesystem accounting information: done

 

This filesystem will be automatically checked every 39 mounts or

180 days, whichever comes first. Use tune2fs -c or -i to override.

 

[root@cos-1 ~]# mkdir /drbd

[root@cos-1 ~]# mount /dev/drbd10 /drbd/

[root@cos-1 ~]# df -h

Filesystem           Size Used Avail Use% Mounted on

/dev/mapper/VolGroup00-LogVol00              3.8G 790M 2.8G 22% /

/dev/sda1                                                       99M   19M   76M 20% /boot

tmpfs                                                               252M     0 252M   0% /dev/shm

/dev/drbd10                                                    287M   11M 262M   4% /drbd

 

[root@cos-1 ~]# umount /drbd =====================================================

Revising /etc/ha.d/haresources for 3rd node.

=====================================================

[root@cos-1 ~]# vi /etc/ha.d/haresources

 

cos-1 IPaddr::192.168.5.2/24/eth0 drbdupper::drbd-upper Filesystem::/dev/drbd10::/drbd::ext3

 

### Above is 1 line ###

 

[root@cos-2 ~]# /etc/init.d/heartbeat stop

[root@cos-1 ~]# /etc/init.d/heartbeat stop

[root@cos-1 ~]# /etc/init.d/heartbeat start

[root@cos-2 ~]# /etc/init.d/heartbeat start

 

[root@cos-1 ~]# cat /proc/drbd

version: 8.3.15 (api:88/proto:86-97)

0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r—–

ns:303216 nr:0 dw:160 dr:527029 al:1 bm:19 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0

 

10: cs:SyncSource ro:Primary/Secondary ds:UpToDate/Inconsistent A r—n-

ns:222632 nr:0 dw:0 dr:222720 al:0 bm:13 lo:0 pe:7 ua:1 ap:0 ep:1 wo:b oos:81184

[=============>……] sync’ed: 74.4% (81184/303008)K

finish: 0:00:06 speed: 13,048 (13,048) K/sec

 

[root@cos-1 ~]# cat /proc/drbd

version: 8.3.15 (api:88/proto:86-97)

0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r—–

ns:303240 nr:0 dw:184 dr:607317 al:1 bm:19 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0

 

10: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate A r—–

ns:303008 nr:0 dw:0 dr:303008 al:0 bm:19 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0

 

[root@cos-1-b ~]# cat /proc/drbd

version: 8.3.15 (api:88/proto:86-97)

 

10: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate A r—–

ns:0 nr:321296 dw:321296 dr:0 al:0 bm:20 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0

=====================================================

To test the second node cos-1-a =====================================================

[root@cos-1 ~]# service heartbeat stop

 

### wait for some time ###

 

[root@cos-1-a ~]# cat /proc/drbd

version: 8.3.15 (api:88/proto:86-97)

0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r—–

ns:16 nr:321576 dw:321592 dr:47 al:2 bm:19 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0

 

10: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate A r—–

ns:17 nr:0 dw:1 dr:21 al:1 bm:1 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0

 

[root@cos-1-a ~]# ifconfig

eth0     Link encap:Ethernet HWaddr

inet addr:192.168.1.11 Bcast:192.168.1.255 Mask:255.255.255.0

inet6 addr: fe80::a00:27ff:fea4:303d/64 Scope:Link

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

RX packets:4472 errors:0 dropped:0 overruns:0 frame:0

TX packets:1426 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:1000

RX bytes:369227 (360.5 KiB) TX bytes:184362 (180.0 KiB)

 

eth0:0   Link encap:Ethernet HWaddr

inet addr:192.168.5.2 Bcast:192.168.5.255 Mask:255.255.255.0

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

 

eth1     Link encap:Ethernet HWaddr

inet addr:172.16.6.20 Bcast:172.16.6.255 Mask:255.255.255.0

inet6 addr: fe80::a00:27ff:fe8a:fbda/64 Scope:Link

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

RX packets:274528 errors:0 dropped:0 overruns:0 frame:0

TX packets:76318 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:1000

RX bytes:371533180 (354.3 MiB) TX bytes:10374515 (9.8 MiB)

 

[root@cos-1-a ~]# ls /drbd/

123.file copy.file lost+found test.file

 

[root@cos-1 ~]# cat /proc/drbd

version: 8.3.15 (api:88/proto:86-97)

0: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r—–

ns:321576 nr:21 dw:18541 dr:620903 al:39 bm:19 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0

 

10: cs:Unconfigured

 

=======================================================

To access the data on 3rd node if 1st and 2nd both nodes are crashed:

=======================================================

Please note, data/files in /drbd directory will be only available on Primary device. Which means if cos-1 was first active then it will have the files and Virtual IP (eth0:0) but in case cos-1 is crashed then cos-1-a will upgrade from Secondary to Primary and same way data in /drbd and Virtual IP will be available in cos-1-a.

 

About 3rd Disaster Recovery node, it will always remain Secondary, because it is always wait for either cos-1 or cos-1-a to be Active and start replication/A-synchronization. If Data is needed to be extracted from Disaster Recovery i.e. 3rd node then Simply stop the drbd service on 3rd node, and mount the relevant device to access the data as depicted below:

3rdNodeAccess

You can compare the same data is available on cos-1-a which is already present in 3rd Disaster Recovery node. While you can easily see from cat /proc/drbd that DRBD10 node is down:

10: cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown A r—–

ns:1 nr:0 dw:6 dr:9 al:1 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:8

 

Please see the image below:

2ndNodeFiles

Everything is Done, AlhumduLILLAH I RUBB ul Al’amin.

 

***************************************************************************

Troubleshooting: In Case drbdadm –stacked create-md drbd-upper is Refused as following:

***************************************************************************

 

[root@cos-1 ~]# drbdadm –stacked create-md drbd-upper

md_offset 310325248

al_offset 310292480

bm_offset 310280192

 

Found ext3 filesystem

303056 kB data area apparently used

303008 kB left usable by current configuration

 

Device size would be truncated, which

would corrupt data and result in

‘access beyond end of device’ errors.

You need to either

* use external meta data (recommended)

* shrink that filesystem first

* zero out the device (destroy the filesystem)

Operation refused.

 

Command ‘drbdmeta 10 v08 /dev/drbd0 internal create-md’ terminated with exit

code 40

drbdadm create-md drbd-upper: exited with code 40

 

[root@cos-1 ~]# dd if=/dev/zero of=/dev/sdb1 bs=1M count=128

128+0 records in

128+0 records out

134217728 bytes (134 MB) copied, 0.305571 seconds, 439 MB/s

 

[root@cos-1 ~]# drbdadm –stacked create-md drbd-upper

Writing meta data…

initializing activity log

NOT initialized bitmap

New drbd meta data block successfully created.

 

–== Creating metadata ==–

As with nodes, we count the total number of devices mirrored by DRBD at http://usage.drbd.org.

 

The counter works anonymously. It creates a random number to identify

the device and sends that random number, along with the kernel and

DRBD version, to usage.drbd.org.

* If you wish to opt out entirely, simply enter ‘no’.

* To continue, just press [RETURN]

 

success

Recommended For You.

Setup your own Video sharing website like, Youtube.com , Vimeo.com or dailymotion.com using Clip Bucket video hosting / sharing code