服务器磁盘卷组缓存cache设置介绍

工具1： storcli

a. 确认软件包是否安装

    [root@localhost ~]#rpm -qa | grep storcli

    storcli-1.21.06-1.noarch

    备注：若检索结果为空，需要安装对应的软件安装包。安装命令如下：

    #rpm -ivh storcli-xx-xx-1.noarch.rpm

b. 查看逻辑卷读/写缓存状态

    备注：请关注标红部分示例为：NRWTD, 参数解释如下：

    NR: No Read Ahead（关闭预读，即读缓存关闭）

    R:Read Ahead(开启预读，即开启读缓存）

    WB: Write Back（开启回写，即开启写缓存）

    AWB: Always Write Back（强制开启回写，强制开启写缓存）

    WT: Write Through(直写模式，关闭回写/写缓存） 

  [root@localhost ~]# /opt/MegaRAID/storcli/storcli64 /c0/v1 show

    Controller = 0

    Status = Success

    Description = None

    Virtual Drives :

    ==============

    --------------------------------------------------------------

    DG/VD TYPE State Access Consist Cache Cac sCC Size Name

    --------------------------------------------------------------

   1/1 RAID10 Optl RW No NRWTD - ON 2.180 TB

   --------------------------------------------------------------

   Cac=CacheCade|Rec=Recovery|OfLn=OffLine|Pdgd=Partially Degraded|Dgrd=Degraded

Optl=Optimal|RO=ReadOnly|RW=ReadWrite|HD=Hidden|TRANS=TransportReady|B=Blocked| Consist=Consistent|R=ReadAheadAlways|NR=NoReadAhead|WB=WriteBack| AWB=AlwaysWriteBack|WT=WriteThrough|C=CachedIO|D=DirectIO|sCC=ScheduledCheck Consistency

c. 读缓存打开/关闭

    语法如下：

    storcli /cx/vx set rdcache=RA|NoRA

    参数说明：

    cx: x为数字，从0开始，代表控制器编号

    vx: x为数字，从0开始，代表逻辑卷编号

    RA: Read Ahead

    NoRA: No Read Ahead

    示例：

    [root@localhost ~]# /opt/MegaRAID/storcli/storcli64 /c0/v1 set rdcache=ra

    Controller = 0

    Status = Success

    Description = None

    Detailed Status :

    ===============

    ---------------------------------------

    VD Property Value Status ErrMsg ErrCd

    ---------------------------------------

    1 rdCache RA Success - 0

    ---------------------------------------

    [root@localhost ~]# /opt/MegaRAID/storcli/storcli64 /c0/v1 set rdcache=nora

    Controller = 0

    Status = Success

    Description = None

    Detailed Status :

    ===============

     ---------------------------------------

    VD Property Value Status ErrMsg ErrCd

    ---------------------------------------

    1 rdCache NoRA Success - 0

    ---------------------------------------

d. 写缓存打开/关闭

    语法如下：

    storcli /cx/vx set wrcache=WT|WB|AWB

    参数说明：

    cx: x为数字，从0开始，代表控制器编号

    vx: x为数字，从0开始，代表逻辑卷编号

    WT: Write Through

    WB: Write Back

    AWB: Always Write Back

    示例：

    [root@localhost ~]# /opt/MegaRAID/storcli/storcli64 /c0/v1 set wrcache=wt

    Controller = 0

    Status = Success

    Description = None

    Detailed Status :

    ===============

    ---------------------------------------

    VD Property Value Status ErrMsg ErrCd

    ---------------------------------------

    1 wrCache WT Success - 0

    ---------------------------------------

    [root@localhost ~]# /opt/MegaRAID/storcli/storcli64 /c0/v1 set wrcache=wb

    Controller = 0

    Status = Success

    Description = None

    Detailed Status :

    ===============

    ---------------------------------------

    VD Property Value Status ErrMsg ErrCd

    ---------------------------------------

    1 wrCache WB Success - 0

    ---------------------------------------

    [root@localhost ~]# /opt/MegaRAID/storcli/storcli64 /c0/v1 set wrcache=awb

    Controller = 0

    Status = Success

    Description = None

    Detailed Status :

    ===============

    ---------------------------------------

    VD Property Value Status ErrMsg ErrCd

    ---------------------------------------

    1 wrCache AWB Success - 0

    ---------------------------------------

工具2：megacli

a.  确认软件包是否安装

    [root@localhost ~]# rpm -qa | grep -i MegaCli

    MegaCli-8.07.10-1.noarch

    备注：若检索结果为空，需要安装对应的软件安装包。安装命令如下：

    #rpm -ivh MegaCli-xx-xx-1.noarch.rpm

b. 查看逻辑卷读/写缓存状态

    请关注标红部分示例为： WriteBack, ReadAheadNone, Direct, Write Cache OK if Bad BBU, 参数解释如下：

    NORA: No Read Ahead（关闭预读，即读缓存关闭）

    RA:Read Ahead(开启预读，即开启读缓存）

    WB: Write Back（开启回写，即开启写缓存）

    ForcedWB: Forced Write Back（强制开启回写，强制开启写缓存）

    WT: Write Through(直写模式，关闭回写/写缓存）

    示例：

    [root@localhost ~]# /opt/MegaRAID/MegaCli/MegaCli64 -ldinfo -l1 -a0

    Adapter 0 -- Virtual Drive Information:

    Virtual Drive: 1 (Target Id: 1)

    Name :

    RAID Level : Primary-1, Secondary-0, RAID Level Qualifier-0

    Size : 2.180 TB

    Sector Size : 512

    Is VD emulated : Yes

    Mirror Data : 2.180 TB

    State : Optimal

    Strip Size : 64 KB

    Number Of Drives per span:2

    Span Depth : 5

    Default Cache Policy: WriteBack, ReadAheadNone, Direct, Write Cache OK if Bad BBU

    Current Cache Policy: WriteBack, ReadAheadNone, Direct, Write Cache OK if Bad BBU

    Default Access Policy: Read/Write

    Current Access Policy: Read/Write

    Disk Cache Policy : Disk's Default

    Ongoing Progresses:

    Check Consistency : Completed 26%, Taken 493 min.

    Encryption Type : None

    Bad Blocks Exist: No

    PI type: No PI

    Is VD Cached: No

    Exit Code: 0x00

c.  读/写缓存打开/关闭

    语法如下：

    MegaCli -LDSetProp {-Name LdNamestring} | -RW|RO|Blocked|RemoveBlocked | WT|WB|ForcedWB [-Immediate] |RA|NORA|ADRA |

    DsblPI

    | Cached|Direct | -EnDskCache|DisDskCache | CachedBadBBU|NoCachedBadBBU

    -Lx|-L0,1,2|-Lall -aN|-a0,1,2|-aALL

    参数说明：

    aN: N为数字，从0开始，代表控制器编号

    Lx: x为数字，从0开始，代表逻辑卷编号

    NORA: No Read Ahead（关闭预读，即读缓存关闭）

    RA:Read Ahead(开启预读，即开启读缓存）

    WB: Write Back（开启回写，即开启写缓存）

    ForcedWB: Forced Write Back（强制开启回写，强制开启写缓存）

    WT: Write Through(直写模式，关闭回写/写缓存）

    读缓存修改示例： 

  [root@localhost ~]# /opt/MegaRAID/MegaCli/MegaCli64 -LdSetProp RA -L1 -a0

  Set Read Policy to ReadAhead on Adapter 0, VD 1 (target id: 1) success

  Exit Code: 0x00

  [root@localhost ~]# /opt/MegaRAID/MegaCli/MegaCli64 -LdSetProp NORA -L1 -a0

  Set Read Policy to NoReadAhead on Adapter 0, VD 1 (target id: 1) success

  Exit Code: 0x00

    读缓存修改示例：

    [root@localhost ~]# /opt/MegaRAID/MegaCli/MegaCli64 -LdSetProp WB -L1 -a0

    Set Write Policy to WriteBack on Adapter 0, VD 1 (target id: 1) success

    Exit Code: 0x00

    [root@localhost ~]#/opt/MegaRAID/MegaCli/MegaCli64 -LdSetProp WT -L1 -a0

    Set Write Policy to WriteThrough on Adapter 0, VD 1 (target id: 1) success

    Exit Code: 0x00

    [root@localhost ~]#/opt/MegaRAID/MegaCli/MegaCli64 -LdSetProp ForcedWB -L1 -a0

    Set Write Policy to Forced WriteBack on Adapter 0, VD 1 (target id: 1) success

    Exit Code: 0x00

工具3：hpssacli/ssacli

说明：

1. hpssacli工具为旧工具，目前大部分线上服务器都安装有对应的安装包，ssacli为新工具，在旧工具无法正常执行的情况下，建议使用新工具，语法完全一致。

2. ssdsmartpath（固态硬盘智能路径）和caching(写缓存）只能打开一个，在一个打开的状态下，另外一个的值必定是关闭的,例如：

    caching：Enable的情况下，ssdsmartpath的值必定为disabled.

a. 确认安装包是否安装

    [root@localhost ~]# rpm -qa | grep hpssacli

    hpssacli-2.20-11.0.x86_64

    备注：若检索结果为空，需要安装对应的软件安装包。安装命令如下：

    #rpm -ivh hpssacli-xx-xx.x86_64.rpm

    #rpm -ivh ssacli-xx-xx.x86_64.rpm

b. 确认硬盘控制器所在slot插槽

    说明：如下所示， 阵列卡型号为Smart Array P840ar, 插槽位置为Slot 0, 务必查询信息，插槽位置后面的命令要用。

    [root@localhost ~]# hpssacli ctrl all show

    Smart Array P840ar in Slot 0 (Embedded) (sn: PVYKH0BRH710CO)

c.  确认逻辑卷写缓存(caching)状态

    参数说明：

    ctrl: Controller 简写

    slot=0: 硬盘控制器所在PCIe插槽位置

    ld 2: ld(logical drive 逻辑卷）,后面的数字为逻辑卷编号，顺序从1开始

    show: 信息展示

    说明：请关注示例中caching的值，示例中为Disabled(即关闭）

    [root@localhost ~]# hpssacli ctrl slot=0 ld 2 show

    Smart Array P840ar in Slot 0 (Embedded)

    array B

    Logical Drive: 2

    Size: 2.2 TB

    Fault Tolerance: 1+0

    Heads: 255

    Sectors Per Track: 32

    Cylinders: 65535

    Strip Size: 256 KB

    Full Stripe Size: 1280 KB

    Status: OK

    MultiDomain Status: OK

    Caching: Disabled

    Unique Identifier: 600508B1001C0D8EB99EB2A7EE5060E4

    Disk Name: /dev/sdb

    Mount Points: /opt/uredis 2.2 TB Partition Number 1

    Logical Drive Label: 06172624PVYKH0BRH710COB70A

    Mirror Group 1:

    physicaldrive 1I:1:5 (port 1I:box 1:bay 5, Solid State SATA, 480.1 GB, OK)

    physicaldrive 1I:1:6 (port 1I:box 1:bay 6, Solid State SATA, 480.1 GB, OK)

    physicaldrive 1I:1:7 (port 1I:box 1:bay 7, Solid State SATA, 480.1 GB, OK)

    physicaldrive 1I:1:8 (port 1I:box 1:bay 8, Solid State SATA, 480.1 GB, OK)

    physicaldrive 1I:1:4 (port 1I:box 1:bay 4, Solid State SATA, 480.1 GB, OK)

    Mirror Group 2:

    physicaldrive 1I:1:3 (port 1I:box 1:bay 3, Solid State SATA, 480.1 GB, OK)

    physicaldrive 2I:2:1 (port 2I:box 2:bay 1, Solid State SATA, 480.1 GB, OK)

    physicaldrive 2I:2:2 (port 2I:box 2:bay 2, Solid State SATA, 480.1 GB, OK)

    physicaldrive 2I:2:3 (port 2I:box 2:bay 3, Solid State SATA, 480.1 GB, OK)

    physicaldrive 2I:2:4 (port 2I:box 2:bay 4, Solid State SATA, 480.1 GB, OK)

    Drive Type: Data

    LD Acceleration Method: HP SSD Smart Path

d. 确认逻辑卷ssdsmartpath状态

    参数说明：

    ctrl: Controller 简写

    slot=0: 硬盘控制器所在PCIe插槽位置

    array b: ld(array, 磁盘阵列，也即逻辑卷）,后面的字母为逻辑卷编号，顺序从a开始

    show: 信息展示

    说明：请关注示例中ssdsmartpath的值，示例中为Enable(即关闭）

    [root@localhost ~]# hpssacli ctrl slot=0 array b show

    Smart Array P840ar in Slot 0 (Embedded)

    Array: B

    Interface Type: Solid State SATA

    Unused Space: 0 MB (0.0%)

    Used Space: 4.4 TB (100.0%)

    Status: OK

    MultiDomain Status: OK

    Array Type: Data

    HP SSD Smart Path: enable

e. 关闭固态硬件智能路径并打开写缓存示例

    [root@localhost ~]# hpssacli ctrl slot=0 array b modify ssdsmartpath=disable

    [root@localhost ~]# hpssacli ctrl slot=0 ld 2 modify caching=enable

f. 关闭写缓存并打开固态硬盘智能路径

    [root@localhost ~]# hpssacli ctrl slot=0 ld 2 modify caching=disable

    [root@localhost ~]# hpssacli ctrl slot=0 array b modify ssdsmartpath=enable

工具4： arcconf

a. 确认软件包是否安装

    [root@localhost ~]# rpm -qa | grep Arcconf

    Arcconf-2.04-22665.x86_64

    备注：若检索结果为空，需要安装对应的软件安装包。安装命令如下：

    #rpm -ivh Arcconf-xx-xx.x86_64.rpm

b. 确认硬盘控制器信息

    说明：控制器编号从1开始，所以示例中控制器ID为1: 

  [root@localhost ~]# /usr/Arcconf/arcconf list

    Controllers found: 1

    ----------------------------------------------------------------------

    Controller information

    ----------------------------------------------------------------------

    Controller ID : Status, Slot, Mode, Name, SerialNumber, WWN

    ----------------------------------------------------------------------

  Controller 1: : Optimal, Slot 2147483647, RAID (Expose RAW), , FB488000, 560DA833FB488000

c. 查看逻辑卷读/写缓存状态

    备注：

    1. 请关注标红部分示例为：参数说明如下：

        Read-cache setting : Enabled(读缓存打开)

        Write-cache setting : Enabled(写缓存打开）

    2. 示例命令行参数说明（/usr/Arcconf/arcconf getconfig 1 ld 1）

        getconfig: prints controller information(打印控制器相关信息）

        第一个1：控制器编号

        ld:logical drive(逻辑卷，即磁盘阵列）

        第二个1：逻辑卷编号，从0开始，示例是第二个逻辑卷，所以为1.

        [root@localhost ~]# /usr/Arcconf/arcconf getconfig 1 ld 1

        Controllers found: 1

        ----------------------------------------------------------------------

        Logical device information

        ----------------------------------------------------------------------

        Logical Device number 1

        Logical Device name : data

        Block Size of member drives : 512 Bytes

        RAID level : 10

        Unique Identifier : 6CDE0BB1

        Status of Logical Device : Optimal

        Additional details : Quick initialized

        Size : 1014 MB

        Parity space : 1024 MB

        Stripe-unit size : 256 KB

        Interface Type : Serial ATA

        Device Type : HDD

        Read-cache setting : Enabled

        Read-cache status : On

        Write-cache setting : Enabled

        Write-cache status : On

        Partitioned : No

        Protected by Hot-Spare : No

        Bootable : No

        Failed stripes : No

        Power settings : Disabled

        --------------------------------------------------------

        Logical Device segment information

        --------------------------------------------------------

        Group 0, Segment 0 : Present (3815447MB, SATA, HDD, Enclosure:0, Slot:2) K4KZMMKL

        Group 0, Segment 1 : Present (3815447MB, SATA, HDD, Enclosure:0, Slot:3) K7G2L3SL

        Group 1, Segment 0 : Present (3815447MB, SATA, HDD, Enclosure:0, Slot:4) K4KYTSLL

        Group 1, Segment 1 : Present (3815447MB, SATA, HDD, Enclosure:0, Slot:5) K7G183TL

d. 读/写缓存设置语法

    语法如下：

    Usage: SETCACHE <Controller#> LOGICALDRIVE <LogicalDrive#> <logical mode> [noprompt] [nologs]

    Usage: SETCACHE <Controller#> DEVICE <Channel# ID#> <physical mode> [nologs]

    Usage: SETCACHE <Controller#> DEVICEALL <policy> [noprompt] [nologs]

    Example: SETCACHE 1 LOGICALDRIVE 0 ron

    Example: SETCACHE 1 DEVICE 0 0 wt

    Example: SETCACHE 1 DEVICEALL drivespecific

    ===================================================================================

    Changes a device's cache mode.

    LOGICALDRIVE parameters

    LogicalDrive# : Number of the logical device whose cache will be altered

    Logical Modes : ron - read cache enabled

    roff - read cache disabled

    wt - write through

    wb - write back

    wbb - write back with battery/ZMM

    con - cache enabled

    coff - cache disabled

    DEVICE parameters

    Channel# ID# : Channel and ID of the device whose cache will be altered

    Physical Modes : wt - write through

    wb - write back

    DEVICEALL parameters

    Policy : disable - write back for all physical drives

    enable - write through for all physical drives

    drivespecific - user can set for an individual physical drive

    部分参数说明：

    rron: read cache enabled(打开读缓存)

    roff: read cache disabled(关闭读缓存)

    wt: write through(只写模式，即关闭写缓存）

    wb:write back(回写模式，即打开写缓存）

    con: cache enabled(打开逻辑卷缓存（打开读/写缓存前提是逻辑卷缓存已打开，在逻辑卷关闭的情况下，无法打开读/写缓存））

    coff: cache disabled(关闭逻辑卷缓存）

    noprompt：no prompt(没有提示，按照默认y执行，部分命令需要和用户交互，使用noprompt参数可按照默认yes答复执行）

e. 读缓存打开/关闭示例

    [root@localhost ~]# /usr/Arcconf/arcconf setcache 1 logicaldrive 1 roff

    Controllers found: 1

    Command completed successfully.

    [root@localhost ~]# /usr/Arcconf/arcconf setcache 1 logicaldrive 1 ron

    Controllers found: 1

    Command completed successfully.

f. 写缓存打开/关闭示例

   [root@localhost ~]# /usr/Arcconf/arcconf setcache 1 logicaldrive 1 wt

   Controllers found: 1

   Command completed successfully.

   You have new mail in /var/spool/mail/root

   [root@localhost ~]# /usr/Arcconf/arcconf setcache 1 logicaldrive 1 wb

   Controllers found: 1

   WARNING: Power failure without battery/ZMM support will lead to data loss.

   Do you wish to continue?

   Press y, then ENTER to continue or press ENTER to abort: y

   Command completed successfully.

   [root@localhost ~]# /usr/Arcconf/arcconf setcache 1 logicaldrive 1 wbb

    Controllers found: 1

    Command completed successfully