DMDSC更换DCR和VOTE磁盘

DMDSC更换DCR和VOTE磁盘

为了提高DMDSC集群运行速度和节点之间通信协调的效率，需要将运行在机械盘上的dcr和vote磁盘替换到SSD高效磁盘上。将原来200M的dcr和vote机械磁盘，换成500M的SSD高效磁盘。

磁盘替换规划信息如下所示：

信息说明	替换前	替换后
dcr和vote磁盘信息	dcr磁盘：/dev/raw/raw1 vote磁盘：/dev/raw/raw2	dcr磁盘：/dev/raw/raw5 vote磁盘：/dev/raw/raw6

1、查询替换前DMDSC磁盘信息

dmdba@dm1:/dev/raw$ pwd

/dev/raw

dmdba@dm1:/dev/raw$ ls -l

total 0

crw-rw---- 1 dmdba dinstall 162, 1 Aug  8 10:04 raw1

crw-rw---- 1 dmdba dinstall 162, 2 Aug  8 13:23 raw2

crw-rw---- 1 dmdba dinstall 162, 3 Aug  8 13:20 raw3

crw-rw---- 1 dmdba dinstall 162, 4 Aug  8 11:20 raw4

crw-rw---- 1 root  disk     162, 0 Aug  8 10:04 rawctl

dmdba@dm1:/dev/raw$

2、查询DMDSC磁盘属性

cd /home/dmdba/dmdbms/bin

dmdba@dm1:/home/dmdba/dmdbms/bin$ ./dmasmcmd

ASM>listdisks '/dev/raw'

[/dev/raw/raw4]: used ASM disk, name:[DMASMDATA0], size:[38510M], group_id:[1], disk_id:[0]

[/dev/raw/raw3]: used ASM disk, name:[DMASMLOG0], size:[2047M], group_id:[0], disk_id:[0]

[/dev/raw/raw2]: used ASM disk, name:[DMASMvote], size:[200M], group_id:[125], disk_id:[0]

[/dev/raw/raw1]: used ASM disk, name:[DMASMdcr], size:[200M], group_id:[126], disk_id:[0]

/dev/raw/raw1和/dev/raw/raw2各200M，是我们需要替换的磁盘。

显示path路径下面所有磁盘的信息，分为三种类型：

（1）normal disk：普通磁盘；

（2）unused asmdisk：初始化未使用的asmdisk；

（3）used asmdisk：已经使用的 asmdisk。

3、校验DCR磁盘状态

cd /home/dmdba/dmdbms/bin

dmdba@dm1:/home/dmdba/dmdbms/bin$ ./dmasmcmd

ASM>check dcrdisk /dev/raw/raw1

ASMCMD check DCRDISK success.

Used time: 6.541(ms).

ASM>

4、导出 DCR 的配置文件

ASM>export dcrdisk '/dev/raw/raw1' to '/tmp/dmdcr_cfg.ini'

ASMCMD export DCRDISK success.

Used time: 7.290(ms).

ASM>

导出的内容就是dmdcr_cfg.ini文件中的内容。

5、查看DCR信息

SQL> select * from v$dcr_info;

LINEID     VERSION     N_GROUP     VTD_PATH      UDP_FLAG    UDP_OGUID            DCR_PATH     

---------- ----------- ----------- ------------- ----------- -------------------- -------------

1          259         3           /dev/raw/raw2 0           63635                /dev/raw/raw1

6、查看DMDSC集群节点信息

SQL> select * from v$dsc_ep_info;

LINEID     EP_NAME EP_SEQNO    EP_GUID              EP_TIMESTAMP         EP_MODE      EP_STATUS

---------- ------- ----------- -------------------- -------------------- ------------ ---------

1          DMOA0   0           408343545            408347646            Control Node OK

2          DMOA1   1           408349893            408353986            Normal Node  OK

7、查看ASM磁盘组信息

SQL> select * from v$asmgroup;

LINEID     GROUP_ID    GROUP_NAME N_DISK      AU_SIZE     EXTENT_SIZE TOTAL_SIZE  FREE_SIZE   TOTAL_FILE_NUM

---------- ----------- ---------- ----------- ----------- ----------- ----------- ----------- --------------

1          0           DMLOG      1           1048576     4           2047        1012        6

2          1           DMDATA     1           1048576     4           38510       38052       21

3          125         VOTE       1           1048576     4           200         176         2

4          126         DCR        1           1048576     4           200         176         2

8、新增加两块磁盘

两块磁盘分别是sdc和sdd。如何给两个节点增加两块共享磁盘，不同服务器和操作系统方法不同，此处不作详细说明。新增磁盘前，建议将DMCSS服务停止，把相关配置文件都备份一下。

1）输入fdisk /dev/sdc

2）依次输入n---p---1---回车---回车---回车，完成第一块磁盘划分

3）输入fdisk /dev/sdd

4）依次输入n---p---1---回车---回车---回车，完成第二块磁盘划分

9、编辑/etc/udev/rules.d/60-raw.rules文件

编辑/etc/udev/rules.d/60-raw.rules文件，两个节点都要添加。

增加以下语句：

--add disk 20210808

ACTION=="add",KERNEL=="sdc1",RUN+="/bin/raw /dev/raw/raw5 %N"

--add disk 20210808

ACTION=="add",KERNEL=="sdd1",RUN+="/bin/raw /dev/raw/raw6 %N"

--ACTION=="add",KERNEL=="raw[1-4]",OWNER="dmdba",GROUP="dinstall",MODE="660"

ACTION=="add",KERNEL=="raw[1-6]",OWNER="dmdba",GROUP="dinstall",MODE="660"

10、执行start_udev

完成裸设备绑定，两个节点都要执行。

#partprobe /dev/sdc

#partprobe /dev/sdd

#/sbin/udevadm control --reload-rules

#udevadm trigger --action=add

#/sbin/udevadm trigger --type=devices --action=change

11、查看裸设备大小

可以通过blockdev --getsize64 /dev/raw/raw5（raw6）命令查看裸设备大小。

12、查看新增加的磁盘

dmdba@dm1:/home/dmdba/dmdbms/bin$ ./dmasmcmd

DMASMCMD V8

ASM>listdisks '/dev/raw/'

[/dev/raw//raw3]: used ASM disk, name:[DMASMLOG0], size:[2047M], group_id:[0], disk_id:[0]

[/dev/raw//raw4]: used ASM disk, name:[DMASMDATA0], size:[38510M], group_id:[1], disk_id:[0]

[/dev/raw//raw6]: normal disk

[/dev/raw//raw5]: normal disk

[/dev/raw//raw2]: used ASM disk, name:[DMASMvote], size:[200M], group_id:[125], disk_id:[0]

[/dev/raw//raw1]: used ASM disk, name:[DMASMdcr], size:[200M], group_id:[126], disk_id:[0]

Used time: 32.893(ms).

ASM>

新的磁盘增加完成后，需要把两个节点的/etc/udev/rules.d/60-raw.rules文件中关于/dev/ra/raw1和/dev/raw/raw2部分注销或删除。在对以上配置文件进行修改前，建议对文件进行备份。

13、修改改dmdcr.ini和dmdcr_cfg.ini文件

两个DSC 节点的配置文件都需要修改。修改DCR_VTD_PATH和DMDCR_PATH参数。

dmdba@dm1:/home/dmdba/dmdbms/data$ head -4 dmdcr_cfg.ini

DCR_N_GRP           = 3

##DCR_VTD_PATH        = /dev/raw/raw2

DCR_VTD_PATH        = /dev/raw/raw6

DCR_OGUID           = 63635

dmdba@dm1:/home/dmdba/dmdbms/data$

dmdba@dm1:/home/dmdba/dmdbms/data$ head -4 dmdcr.ini

DMDCR_PATH=/dev/raw/raw5

DMDCR_MAL_PATH=/home/dmdba/dmdbms/data/dmasvrmal.ini   #dmasmsvr使用的MAL配置文件路径

DMDCR_SEQNO=0

#ASM重启参数，命令行方式启动

dmdba@dm1:/home/dmdba/dmdbms/data$

14、初始化新磁盘

dmdba@dm1:/home/dmdba/dmdbms/bin$ ./dmasmcmd

DMASMCMD V8

ASM>create dcrdisk '/dev/raw/raw5' 'dcr'

[Trace]The ASM initialize dcrdisk /dev/raw/raw5 to name DMASMdcr

Used time: 45.292(ms).

ASM>create votedisk '/dev/raw/raw6' 'vote'

[Trace]The ASM initialize votedisk /dev/raw/raw6 to name DMASMvote

Used time: 28.899(ms).

ASM>

ASM>init dcrdisk '/dev/raw/raw5' from '/home/dmdba/dmdbms/data/dmdcr_cfg.ini' identified by 'root'

[Trace]DG 126 allocate 4 extents for file 0xfe000002.

Used time: 133.976(ms).

ASM>init votedisk '/dev/raw/raw6' from '/home/dmdba/dmdbms/data/dmdcr_cfg.ini'

[Trace]DG 125 allocate 4 extents for file 0xfd000002.

Used time: 46.068(ms).

ASM>

15、启动DMCSS

节点一

dmdba@dm1:/home/dmdba/dmdbms/bin$ ./DmCSSServiceCSS0 start

Starting DmCSSServiceCSS0: [ OK ]

dmdba@dm1:/home/dmdba/dmdbms/bin$

节点二

dmdba@dm2:/home/dmdba/dmdbms/bin$ ./DmCSSServiceCSS1 start

Starting DmCSSServiceCSS1: [ OK ]

dmdba@dm2:/home/dmdba/dmdbms/bin$

DMCSS启动完成后，如果配置了DMASMRV和DMSERVER自动启动，会自动启动着两个服务。如果没有配置。可以通过DMCSSM工具启动着两个服务。

启动DMCSSM工具的命令如下所示：

dmdba@dm1:/home/dmdba/dmdbms/bin$ ./dmcssm INI_PATH=/home/dmdba/dmdbms/data/dmcssm.ini

16、查看DCR信息

SQL> select * from v$dcr_info;

LINEID     VERSION     N_GROUP     VTD_PATH      UDP_FLAG    UDP_OGUID            DCR_PATH     

---------- ----------- ----------- ------------- ----------- -------------------- -------------

1          259         3           /dev/raw/raw6 0           63635                /dev/raw/raw5

used time: 5.598(ms). Execute id is 200.

SQL>

17、查看节点信息

SQL> select * from v$dsc_ep_info;

LINEID     EP_NAME EP_SEQNO    EP_GUID              EP_TIMESTAMP         EP_MODE      EP_STATUS

---------- ------- ----------- -------------------- -------------------- ------------ ---------

1          DMOA0   0           422552337            422552854            Control Node OK

2          DMOA1   1           422557455            422557964            Normal Node  OK

used time: 1.842(ms). Execute id is 201.

SQL>

18、查看ASM磁盘组信息

SQL> select * from v$asmgroup;

LINEID     GROUP_ID    GROUP_NAME N_DISK      AU_SIZE     EXTENT_SIZE TOTAL_SIZE  FREE_SIZE   TOTAL_FILE_NUM

---------- ----------- ---------- ----------- ----------- ----------- ----------- ----------- --------------

1          0           DMLOG      1           1048576     4           2047        1012        6

2          1           DMDATA     1           1048576     4           38510       38036       25

3          125         VOTE       1           1048576     4           511         484         2

4          126         DCR        1           1048576     4           511         484         2

used time: 49.498(ms). Execute id is 202.

SQL>

磁盘已经替换成功，两个磁盘都是500M。

19、查询DMDSC磁盘属性

cd /home/dmdba/dmdbms/bin

dmdba@dm1:/home/dmdba/dmdbms/bin$ ./dmasmcmd

DMASMCMD V8

ASM>listdisks '/dev/raw'

[/dev/raw/raw3]: used ASM disk, name:[DMASMLOG0], size:[2047M], group_id:[0], disk_id:[0]

[/dev/raw/raw6]: used ASM disk, name:[DMASMvote], size:[511M], group_id:[125], disk_id:[0]

[/dev/raw/raw4]: used ASM disk, name:[DMASMDATA0], size:[38510M], group_id:[1], disk_id:[0]

[/dev/raw/raw5]: used ASM disk, name:[DMASMdcr], size:[511M], group_id:[126], disk_id:[0]

Used time: 45.829(ms).

ASM>

20、校验替换后的DCR磁盘

dmdba@dm1:/home/dmdba/dmdbms/bin$ ./dmasmcmd

DMASMCMD V8

ASM>check dcrdisk /dev/raw/raw5

ASMCMD check DCRDISK success.

Used time: 6.665(ms).

ASM>