The actual my activity is to increase one of the file system size which is part of VXVM mirror volume, when I try to add disks to existing diskgroup I have faced this issue.This issue happened due to DG configuration mismatching and this post may help you when your in the same situation.
Current status of FS / Volume:
[root@ivmprd ~]# df -h /opt/nbuback Filesystem size used avail capacity Mounted on /dev/vx/dsk/NDG/LVOL01 1.2T 1.1T 64G 95% /opt/nbuback [root@ivmprd ~]# vxprint -htg NDG DG NAME NCONFIG NLOG MINORS GROUP-ID ST NAME STATE DM_CNT SPARE_CNT APPVOL_CNT DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE RV NAME RLINK_CNT KSTATE STATE PRIMARY DATAVOLS SRL RL NAME RVG KSTATE STATE REM_HOST REM_DG REM_RLNK CO NAME CACHEVOL KSTATE STATE VT NAME RVG KSTATE STATE NVOLUME V NAME RVG/VSET/CO KSTATE STATE LENGTH READPOL PREFPLEX UTYPE PL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODE SD NAME PLEX DISK DISKOFFS LENGTH [COL/]OFF DEVICE MODE SV NAME PLEX VOLNAME NVOLLAYR LENGTH [COL/]OFF AM/NM MODE SC NAME PLEX CACHE DISKOFFS LENGTH [COL/]OFF DEVICE MODE DC NAME PARENTVOL LOGVOL SP NAME SNAPVOL DCO EX NAME ASSOC VC PERMS MODE STATE SR NAME KSTATE dg NDG default default 20000 1201549779.11.ivmprd dm USDC_01 store1-usp1_01a1 auto 65536 870216448 - dm USDC_02 store1-usp1_01a2 auto 65536 870216448 - dm USDC_03 store1-usp1_01a3 auto 65536 419346688 - dm USDC_04 store1-usp1_01a4 auto 65536 419346688 - dm CADC_01 store2-usp2_02b1 auto 65536 870216448 - dm CADC_02 store2-usp2_02b2 auto 65536 870216448 - dm CADC_03 store2-usp2_02b3 auto 65536 419346688 - dm CADC_04 store2-usp2_02b4 auto 65536 419346688 - v LVOL01 - ENABLED ACTIVE 2537300320 SELECT - fsgen pl LVOL01-03 LVOL01 ENABLED ACTIVE 2537300320 CONCAT - RW sd USDC_01-01 LVOL01-03 USDC_01 0 870216448 0 store1-usp1_01a1 ENA sd USDC_02-01 LVOL01-03 USDC_02 0 828223072 870216448 store1-usp1_01a2 ENA sd USDC_03-01 LVOL01-03 USDC_03 0 419346688 1698439520 store1-usp1_01a3 ENA sd USDC_02-02 LVOL01-03 USDC_02 828223072 83712 2117786208 store1-usp1_01a2 ENA sd USDC_03-02 LVOL01-03 USDC_04 0 419346688 2117869920 store1-usp1_01a4 ENA sd USDC_02-03 LVOL01-03 USDC_02 828306784 83712 2537216608 store1-usp1_01a2 ENA pl LVOL01-04 LVOL01 ENABLED ACTIVE 2537300320 CONCAT - RW sd CADC_01-01 LVOL01-04 CADC_01 0 870216448 0 store2-usp2_02b1 ENA sd CADC_02-01 LVOL01-04 CADC_02 0 828223072 870216448 store2-usp2_02b2 ENA sd CADC_03-01 LVOL01-04 CADC_03 0 419346688 1698439520 store2-usp2_02b3 ENA sd CADC_02-02 LVOL01-04 CADC_02 828223072 83712 2117786208 store2-usp2_02b2 ENA sd CADC_03-02 LVOL01-04 CADC_04 0 419346688 2117869920 store2-usp2_02b4 ENA sd CADC_02-03 LVOL01-04 CADC_02 828306784 83712 2537216608 store2-usp2_02b2 ENA [root@ivmprd ~]# vxdisk -g NDG list DEVICE TYPE DISK GROUP STATUS store1-usp1_01a3 auto:cdsdisk USDC_03 NDG online store1-usp1_01a1 auto:cdsdisk USDC_01 NDG online store1-usp1_01a2 auto:cdsdisk USDC_02 NDG online store1-usp1_01a4 auto:cdsdisk USDC_04 NDG online store2-usp2_02b1 auto:cdsdisk CADC_01 NDG online store2-usp2_02b2 auto:cdsdisk CADC_02 NDG online store2-usp2_02b3 auto:cdsdisk CADC_03 NDG online store2-usp2_02b4 auto:cdsdisk CADC_04 NDG online [root@ivmprd ~]#Now I am planing to increase the file system from 1.2 TB to 1.4TB.
Scan the disk:
[root@ivmprd ~]# devfsadm
[root@ivmprd ~]# echo |format
Searching for disks...done
c2t50060E8006CF9F35d33: configured with capacity of 199.99GB
c2t50060E8006CFE877d32: configured with capacity of 199.99GB
c4t50060E8006CF9F25d33: configured with capacity of 199.99GB
c4t50060E8006CFE867d32: configured with capacity of 199.99GB
We can able to see the disk from OS , Now scan the disk from VXVM level:
[root@ivmprd ~]# vxdctl enable
[root@ivmprd ~]# vxdisk list
DEVICE TYPE DISK GROUP STATUS
disk_0 auto:none - - online invalid
disk_1 auto:none - - online invalid
store1-usp1_01a3 auto:cdsdisk USDC_03 NDG online
store1-usp1_01a1 auto:cdsdisk USDC_01 NDG online
store1-usp1_01a2 auto:cdsdisk USDC_02 NDG online
store1-usp1_01a5 auto - - nolabel
store1-usp1_01a4 auto:cdsdisk USDC_04 NDG online
store2-usp2_02b1 auto:cdsdisk CADC_01 NDG online
store2-usp2_02b2 auto:cdsdisk CADC_02 NDG online
store2-usp2_02b3 auto:cdsdisk CADC_03 NDG online
store2-usp2_02b5 auto - - nolabel
store2-usp2_02b4 auto:cdsdisk CADC_04 NDG online
[root@ivmprd ~]#
Setup the disk to VXVM control:
[root@ivmprd ~]# vxdisksetup -i store1-usp1_01a5
prtvtoc: /dev/vx/rdmp/store1-usp1_01a5: Unable to read Disk geometry errno = 0x6
[root@ivmprd ~]# vxdisksetup -i store2-usp2_02b5
prtvtoc: /dev/vx/rdmp/store2-usp2_02b5: Unable to read Disk geometry errno = 0x6
[root@ivmprd ~]#
We got the error above due to disks not labeled, Label the disks using format command , since my environment multipath is enabled so we need to to label each path of one disk but I have labeled all the four paths.Label the disk using format command:
[root@whms7039 /]# format c2t50060E8006CF9F35d33
c2t50060E8006CF9F35d33: configured with capacity of 199.99GB
selecting c2t50060E8006CF9F35d33
[disk formatted]
FORMAT MENU:
disk - select a disk
type - select (define) a disk type
partition - select (define) a partition table
current - describe the current disk
format - format and analyze the disk
repair - repair a defective sector
label - write label to the disk
analyze - surface analysis
defect - defect list management
backup - search for backup labels
verify - read and display labels
save - save new disk/partition definitions
inquiry - show vendor, product and revision
volname - set 8-character volume name
! - execute , then return
quit
format> p
PARTITION MENU:
0 - change `0' partition
1 - change `1' partition
2 - change `2' partition
3 - change `3' partition
4 - change `4' partition
5 - change `5' partition
6 - change `6' partition
7 - change `7' partition
select - select a predefined table
modify - modify a predefined partition table
name - name the current table
print - display the current table
label - write partition map and label to the disk
! - execute , then return
quit
partition> p
Current partition table (default):
Total disk cylinders available: 54611 + 2 (reserved cylinders)
Part Tag Flag Cylinders Size Blocks
0 root wm 0 - 34 131.25MB (35/0/0) 268800
1 swap wu 35 - 69 131.25MB (35/0/0) 268800
2 backup wu 0 - 54610 199.99GB (54611/0/0) 419412480
3 unassigned wm 0 0 (0/0/0) 0
4 unassigned wm 0 0 (0/0/0) 0
5 unassigned wm 0 0 (0/0/0) 0
6 usr wm 70 - 54610 199.74GB (54541/0/0) 418874880
7 unassigned wm 0 0 (0/0/0) 0
partition> l
Ready to label disk, continue? y
partition> q
FORMAT MENU:
disk - select a disk
type - select (define) a disk type
partition - select (define) a partition table
current - describe the current disk
format - format and analyze the disk
repair - repair a defective sector
label - write label to the disk
analyze - surface analysis
defect - defect list management
backup - search for backup labels
verify - read and display labels
save - save new disk/partition definitions
inquiry - show vendor, product and revision
volname - set 8-character volume name
! - execute , then return
quit
format> q
[root@whms7039 /]# format c4t50060E8006CF9F25d33
selecting c4t50060E8006CF9F25d33
[disk formatted]
FORMAT MENU:
disk - select a disk
type - select (define) a disk type
partition - select (define) a partition table
current - describe the current disk
format - format and analyze the disk
repair - repair a defective sector
label - write label to the disk
analyze - surface analysis
defect - defect list management
backup - search for backup labels
verify - read and display labels
save - save new disk/partition definitions
inquiry - show vendor, product and revision
volname - set 8-character volume name
! - execute , then return
quit
format> p
PARTITION MENU:
0 - change `0' partition
1 - change `1' partition
2 - change `2' partition
3 - change `3' partition
4 - change `4' partition
5 - change `5' partition
6 - change `6' partition
7 - change `7' partition
select - select a predefined table
modify - modify a predefined partition table
name - name the current table
print - display the current table
label - write partition map and label to the disk
! - execute , then return
quit
partition> p
Current partition table (original):
Total disk cylinders available: 54611 + 2 (reserved cylinders)
Part Tag Flag Cylinders Size Blocks
0 root wm 0 - 34 131.25MB (35/0/0) 268800
1 swap wu 35 - 69 131.25MB (35/0/0) 268800
2 backup wu 0 - 54610 199.99GB (54611/0/0) 419412480
3 unassigned wm 0 0 (0/0/0) 0
4 unassigned wm 0 0 (0/0/0) 0
5 unassigned wm 0 0 (0/0/0) 0
6 usr wm 70 - 54610 199.74GB (54541/0/0) 418874880
7 unassigned wm 0 0 (0/0/0) 0
partition> l
Ready to label disk, continue? y
partition> q
FORMAT MENU:
disk - select a disk
type - select (define) a disk type
partition - select (define) a partition table
current - describe the current disk
format - format and analyze the disk
repair - repair a defective sector
label - write label to the disk
analyze - surface analysis
defect - defect list management
backup - search for backup labels
verify - read and display labels
save - save new disk/partition definitions
inquiry - show vendor, product and revision
volname - set 8-character volume name
! - execute , then return
quit
format> q
[root@whms7039 /]# format c2t50060E8006CFE877d32
c2t50060E8006CFE877d32: configured with capacity of 199.99GB
selecting c2t50060E8006CFE877d32
[disk formatted]
FORMAT MENU:
disk - select a disk
type - select (define) a disk type
partition - select (define) a partition table
current - describe the current disk
format - format and analyze the disk
repair - repair a defective sector
label - write label to the disk
analyze - surface analysis
defect - defect list management
backup - search for backup labels
verify - read and display labels
save - save new disk/partition definitions
inquiry - show vendor, product and revision
volname - set 8-character volume name
! - execute , then return
quit
format> p
PARTITION MENU:
0 - change `0' partition
1 - change `1' partition
2 - change `2' partition
3 - change `3' partition
4 - change `4' partition
5 - change `5' partition
6 - change `6' partition
7 - change `7' partition
select - select a predefined table
modify - modify a predefined partition table
name - name the current table
print - display the current table
label - write partition map and label to the disk
! - execute , then return
quit
partition> p
Current partition table (default):
Total disk cylinders available: 54611 + 2 (reserved cylinders)
Part Tag Flag Cylinders Size Blocks
0 root wm 0 - 34 131.25MB (35/0/0) 268800
1 swap wu 35 - 69 131.25MB (35/0/0) 268800
2 backup wu 0 - 54610 199.99GB (54611/0/0) 419412480
3 unassigned wm 0 0 (0/0/0) 0
4 unassigned wm 0 0 (0/0/0) 0
5 unassigned wm 0 0 (0/0/0) 0
6 usr wm 70 - 54610 199.74GB (54541/0/0) 418874880
7 unassigned wm 0 0 (0/0/0) 0
partition> l
Ready to label disk, continue? y
partition> q
FORMAT MENU:
disk - select a disk
type - select (define) a disk type
partition - select (define) a partition table
current - describe the current disk
format - format and analyze the disk
repair - repair a defective sector
label - write label to the disk
analyze - surface analysis
defect - defect list management
backup - search for backup labels
verify - read and display labels
save - save new disk/partition definitions
inquiry - show vendor, product and revision
volname - set 8-character volume name
! - execute , then return
quit
format> q
[root@whms7039 /]# format c4t50060E8006CFE867d32
selecting c4t50060E8006CFE867d32
[disk formatted]
FORMAT MENU:
disk - select a disk
type - select (define) a disk type
partition - select (define) a partition table
current - describe the current disk
format - format and analyze the disk
repair - repair a defective sector
label - write label to the disk
analyze - surface analysis
defect - defect list management
backup - search for backup labels
verify - read and display labels
save - save new disk/partition definitions
inquiry - show vendor, product and revision
volname - set 8-character volume name
! - execute , then return
quit
format> p
PARTITION MENU:
0 - change `0' partition
1 - change `1' partition
2 - change `2' partition
3 - change `3' partition
4 - change `4' partition
5 - change `5' partition
6 - change `6' partition
7 - change `7' partition
select - select a predefined table
modify - modify a predefined partition table
name - name the current table
print - display the current table
label - write partition map and label to the disk
! - execute , then return
quit
partition> p
Current partition table (original):
Total disk cylinders available: 54611 + 2 (reserved cylinders)
Part Tag Flag Cylinders Size Blocks
0 root wm 0 - 34 131.25MB (35/0/0) 268800
1 swap wu 35 - 69 131.25MB (35/0/0) 268800
2 backup wu 0 - 54610 199.99GB (54611/0/0) 419412480
3 unassigned wm 0 0 (0/0/0) 0
4 unassigned wm 0 0 (0/0/0) 0
5 unassigned wm 0 0 (0/0/0) 0
6 usr wm 70 - 54610 199.74GB (54541/0/0) 418874880
7 unassigned wm 0 0 (0/0/0) 0
partition> l
Ready to label disk, continue? y
partition> q
FORMAT MENU:
disk - select a disk
type - select (define) a disk type
partition - select (define) a partition table
current - describe the current disk
format - format and analyze the disk
repair - repair a defective sector
label - write label to the disk
analyze - surface analysis
defect - defect list management
backup - search for backup labels
verify - read and display labels
save - save new disk/partition definitions
inquiry - show vendor, product and revision
volname - set 8-character volume name
! - execute , then return
quit
format> q
[root@whms7039 /]#
Re-read the VXVM configuration again to take changes:
[root@ivmprd ~]# vxdctl enable
[root@ivmprd ~]# vxdisk list
DEVICE TYPE DISK GROUP STATUS
disk_0 auto:none - - online invalid
disk_1 auto:none - - online invalid
store1-usp1_01a3 auto:cdsdisk USDC_03 NDG online
store1-usp1_01a1 auto:cdsdisk USDC_01 NDG online
store1-usp1_01a2 auto:cdsdisk USDC_02 NDG online
store1-usp1_01a5 auto:none - - online invalid
store1-usp1_01a4 auto:cdsdisk USDC_04 NDG online
store2-usp2_02b1 auto:cdsdisk CADC_01 NDG online
store2-usp2_02b2 auto:cdsdisk CADC_02 NDG online
store2-usp2_02b3 auto:cdsdisk CADC_03 NDG online
store2-usp2_02b5 auto:none - - online invalid
store2-usp2_02b4 auto:cdsdisk CADC_04 NDG online
[root@ivmprd ~]# vxdisksetup -i store1-usp1_01a5
[root@ivmprd ~]# vxdisksetup -i store2-usp2_02b5
[root@ivmprd ~]# vxdisk list
DEVICE TYPE DISK GROUP STATUS
disk_0 auto:none - - online invalid
disk_1 auto:none - - online invalid
store1-usp1_01a3 auto:cdsdisk USDC_03 NDG online
store1-usp1_01a1 auto:cdsdisk USDC_01 NDG online
store1-usp1_01a2 auto:cdsdisk USDC_02 NDG online
store1-usp1_01a5 auto:cdsdisk - - online
store1-usp1_01a4 auto:cdsdisk USDC_04 NDG online
store2-usp2_02b1 auto:cdsdisk CADC_01 NDG online
store2-usp2_02b2 auto:cdsdisk CADC_02 NDG online
store2-usp2_02b3 auto:cdsdisk CADC_03 NDG online
store2-usp2_02b5 auto:cdsdisk - - online
store2-usp2_02b4 auto:cdsdisk CADC_04 NDG online
[root@ivmprd ~]#
Set the site tag for those disks. Since each LUN's are coming from different DC:
[root@ivmprd ~]# vxdisk settag site=USDC store1-usp1_01a5
[root@ivmprd ~]# vxdisk settag site=CADC store2-usp2_02b5
[root@ivmprd ~]#
Add the disks to existing DG (NDG):
[root@ivmprd ~]# vxdg -g NDG adddisk USDC_05=store1-usp1_01a5
VxVM vxdg ERROR V-5-1-10128 Configuration daemon error 441
[root@ivmprd ~]#
FLUSH the DG and add the disk to existing DG (NDG):
[root@ivmprd ~]# vxdg flush NDG
[root@ivmprd ~]# vxdg -g NDG adddisk USDC_05=store1-usp1_01a5
[root@ivmprd ~]# vxdg -g NDG adddisk CADC_05=store2-usp2_02b5
[root@ivmprd ~]#
[root@ivmprd ~]# vxdisk -g NDG list
DEVICE TYPE DISK GROUP STATUS
store1-usp1_01a3 auto:cdsdisk USDC_03 NDG online
store1-usp1_01a1 auto:cdsdisk USDC_01 NDG online
store1-usp1_01a2 auto:cdsdisk USDC_02 NDG online
store1-usp1_01a5 auto:cdsdisk USDC_05 NDG online
store1-usp1_01a4 auto:cdsdisk USDC_04 NDG online
store2-usp2_02b1 auto:cdsdisk CADC_01 NDG online
store2-usp2_02b2 auto:cdsdisk CADC_02 NDG online
store2-usp2_02b3 auto:cdsdisk CADC_03 NDG online
store2-usp2_02b5 auto:cdsdisk CADC_05 NDG online
store2-usp2_02b4 auto:cdsdisk CADC_04 NDG online
[root@ivmprd ~]#
Cool... After FLUSH we can able to add those disk to existing DG (NDG).
Just check the available size:
[root@ivmprd ~]# vxassist -g NDG maxsize
Maximum volume size: 922343424 (450363Mb)
[root@ivmprd ~]# vxassist -g NDG maxgrow LVOL01 layout=mirror
Volume LVOL01 can be extended by 461170688 to: 2998471008 (1464097Mb+352 sectors)
[root@ivmprd ~]#
Grow the volume size and check the status:
[root@ivmprd ~]# vxresize -g NDG LVOL01 +200G
[root@ivmprd ~]# vxtask list
TASKID PTID TYPE/STATE PCT PROGRESS
188 RDWRBACK/R 06.87% 2537300320/2956730720/2566121824 RESYNC LVOL01 NDG
[root@ivmprd ~]#
[root@whms7039 /]# df -h /opt/nbuback
Filesystem size used avail capacity Mounted on
/dev/vx/dsk/NDG/LVOL01
1.4T 1.1T 256G 82% /opt/nbuback
[root@ivmprd ~]#
For more info you can refer symantec site, Thanks to symantec for providing this document.
http://www.symantec.com/business/support/index?page=content&id=TECH130463
0 comments:
Post a Comment