This assumes that a vdisk has been created and presented to the host, and that SAN zoning is in place to allow the host to see the storage.
In this run through, the multipath.conf file had the multipath definition commented out before the multipath command was run. This created the mpath0 device. After adding a multipath alias line, the name changes to the vdisk001 alias which is defined.
[root@test01 ~]# multipath remove: mpath0 (dup of vdisk001) create: vdisk001 (36001438005dedc000000300000370000) [size=1 GB][features="0"][hwhandler="0"] \_ round-robin 0 [prio=1] \_ 0:0:0:1 sda 8:0 [active][ready] \_ round-robin 0 [prio=1] \_ 0:0:1:1 sdb 8:16 [active][ready] ...edited...
fdisk shows the underlying eight (Four paths over each SAN fabric) sdX devices and the new /dev/dm-1:-
[root@test01 ~]# fdisk -l Disk /dev/sda: 1073 MB, 1073741824 bytes 34 heads, 61 sectors/track, 1011 cylinders Units = cylinders of 2074 * 512 = 1061888 bytes Disk /dev/sda doesn't contain a valid partition table ...edited... Disk /dev/dm-1: 1073 MB, 1073741824 bytes 34 heads, 61 sectors/track, 1011 cylinders Units = cylinders of 2074 * 512 = 1061888 bytes Disk /dev/dm-1 doesn't contain a valid partition table [root@test01 ~]# ls -l /dev/mapper/vdisk001 brw-rw---- 1 root disk 253, 1 Dec 3 14:26 /dev/mapper/vdisk001
[root@test01 ~]# pvcreate /dev/mapper/vdisk001 Physical volume "/dev/mapper/vdisk001" successfully created
[root@test01 ~]# vgcreate testvg /dev/mapper/vdisk001 Volume group "testvg" successfully created [root@test01 ~]# vgdisplay --- Volume group --- VG Name testvg System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 1 VG Access read/write VG Status resizable MAX LV 0 Cur LV 0 Open LV 0 Max PV 0 Cur PV 1 Act PV 1 VG Size 1020.00 MB PE Size 4.00 MB Total PE 255 Alloc PE / Size 0 / 0 Free PE / Size 255 / 1020.00 MB VG UUID VyUGva-mSgX-2pCC-ZuBI-YdKs-MFQ6-EiNt6c
[root@test01 ~]# lvcreate --size 500M --name test01lv testvg Logical volume "test01lv" created [root@test01 ~]# lvcreate --size 500M --name test02lv testvg Logical volume "test02lv" created [root@test01 ~]# lvdisplay --- Logical volume --- LV Name /dev/testvg/test01lv VG Name testvg LV UUID PffBiq-8zhP-VABE-ga6O-Asnk-Cpig-vhHO0S LV Write Access read/write LV Status available # open 0 LV Size 500.00 MB Current LE 125 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:0 --- Logical volume --- LV Name /dev/testvg/test02lv VG Name testvg LV UUID h2LG5y-w4XB-VHez-BC3Q-pMcf-kQVc-UHV4vH LV Write Access read/write LV Status available # open 0 LV Size 500.00 MB Current LE 125 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:2
[root@test01 ~]# mkfs /dev/testvg/test01lv
mke2fs 1.35 (28-Feb-2004)
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
128016 inodes, 512000 blocks
25600 blocks (5.00%) reserved for the super user
First data block=1
Maximum filesystem blocks=67633152
63 block groups
8192 blocks per group, 8192 fragments per group
2032 inodes per group
Superblock backups stored on blocks:
8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409
Writing inode tables: done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 24 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
[root@test01 ~]# mkfs /dev/testvg/test02lv
mke2fs 1.35 (28-Feb-2004)
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
128016 inodes, 512000 blocks
25600 blocks (5.00%) reserved for the super user
First data block=1
Maximum filesystem blocks=67633152
63 block groups
8192 blocks per group, 8192 fragments per group
2032 inodes per group
Superblock backups stored on blocks:
8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409
Writing inode tables: done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 32 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
[root@test01 ~]# mount /dev/testvg/test01lv /mnt/test01lv [root@test01 ~]# mount /dev/testvg/test02lv /mnt/test02lv [root@test01 ~]# df -h Filesystem Size Used Avail Use% Mounted on /dev/cciss/c0d0p5 27G 18G 8.0G 69% / /dev/cciss/c0d0p1 145M 21M 116M 16% /boot none 7.9G 0 7.9G 0% /dev/shm /dev/cciss/c0d0p3 2.9G 1.7G 1.2G 59% /var /dev/mapper/testvg-test01lv 485M 2.3M 457M 1% /mnt/test01lv /dev/mapper/testvg-test02lv 485M 2.3M 457M 1% /mnt/test02lv [root@test01 ~]#
This assumes that you have a LVM pv with one or more vg's and lv's built on top. You want to move a volume group from one pv to another, for instance to move from a smaller disk to a larger one and then to grow the vg and lv.
Two pv's are present, one of which contains the testvg.
[root@test01 ~]# pvdisplay --- Physical volume --- PV Name /dev/dm-0 VG Name testvg PV Size 2.00 GB / not usable 4.00 MB Allocatable yes PE Size (KByte) 4096 Total PE 511 Free PE 383 Allocated PE 128 PV UUID 2sora7-Utg0-8VIl-kCdM-jdcj-WND9-DqxYn3 "/dev/dm-1" is a new physical volume of "2.00 GB" --- NEW Physical volume --- PV Name /dev/dm-1 VG Name PV Size 2.00 GB Allocatable NO PE Size (KByte) 0 Total PE 0 Free PE 0 Allocated PE 0 PV UUID Ts6Uwd-UYam-RSXq-bsdH-3N2F-7s3N-FfOrDF [root@test01 ~]# pvmove /dev/mapper/vdisk001 /dev/mapper/vdisk002 /dev/mapper/vdisk001: Moved: 79.7% /dev/mapper/vdisk001: Moved: 100.0%
Remove the original pv from vg, this is with the filesystem mounted and active! pvdisplay shows testvg on the second pv.
[root@test01 ~]# vgreduce testvg /dev/dm-0 Removed "/dev/dm-0" from volume group "testvg" [root@test01 ~]# pvdisplay --- Physical volume --- PV Name /dev/dm-1 VG Name testvg PV Size 2.00 GB / not usable 4.00 MB ... edited ... "/dev/dm-0" is a new physical volume of "2.00 GB" --- NEW Physical volume --- PV Name /dev/dm-0 VG Name PV Size 2.00 GB ... edited ... [root@test01 ~]# pvremove /dev/dm-0 Labels on physical volume "/dev/dm-0" successfully wiped
All gone from pv /dev/dm-0.
This will remove the LVM volumes from the server, it does not cover removing the vdisk from your storage array. If you want to move the LUN from one server and remount on another, DO NOT FOLLOW THESE INSTRUCTIONS!!!!!
[root@test01 ~]# mount ...edited... /dev/mapper/testvg-test01lv on /mnt/test01lv type ext2 (rw) /dev/mapper/testvg-test02lv on /mnt/test02lv type ext2 (rw) [root@test01 ~]# [root@test01 ~]# umount /mnt/test01lv [root@test01 ~]# umount /mnt/test02lv [root@test01 ~]# lvdisplay --- Logical volume --- LV Name /dev/testvg/test01lv VG Name testvg LV UUID PffBiq-8zhP-VABE-ga6O-Asnk-Cpig-vhHO0S LV Write Access read/write LV Status available # open 1 LV Size 500.00 MB Current LE 125 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:0 --- Logical volume --- LV Name /dev/testvg/test02lv VG Name testvg LV UUID h2LG5y-w4XB-VHez-BC3Q-pMcf-kQVc-UHV4vH LV Write Access read/write LV Status available # open 0 LV Size 500.00 MB Current LE 125 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:2
The next step will DESTROY your data on test01lv:-
[root@test01 ~]# lvremove /dev/testvg/test01lv Do you really want to remove active logical volume "test01lv"? [y/n]: y Logical volume "test01lv" successfully removed [root@test01 ~]# [root@test01 ~]# lvremove /dev/testvg/test02lv Do you really want to remove active logical volume "test02lv"? [y/n]: y Logical volume "test02lv" successfully removed [root@test01 ~]# [root@test01 ~]# vgdisplay --- Volume group --- VG Name testvg System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 5 VG Access read/write VG Status resizable MAX LV 0 Cur LV 0 Open LV 0 Max PV 0 Cur PV 1 Act PV 1 VG Size 1020.00 MB PE Size 4.00 MB Total PE 255 Alloc PE / Size 0 / 0 Free PE / Size 255 / 1020.00 MB VG UUID VyUGva-mSgX-2pCC-ZuBI-YdKs-MFQ6-EiNt6c [root@test01 ~]# vgremove testvg Volume group "testvg" successfully removed [root@test01 ~]# [root@test01 ~]# pvdisplay "/dev/dm-1" is a new physical volume of "1.00 GB" --- NEW Physical volume --- PV Name /dev/dm-1 VG Name PV Size 1.00 GB Allocatable NO PE Size (KByte) 0 Total PE 0 Free PE 0 Allocated PE 0 PV UUID hJ2jFO-Bn7t-oa73-pqDI-Hetc-X6MY-9G9C5v [root@test01 ~]# pvremove /dev/dm-1 Labels on physical volume "/dev/dm-1" successfully wiped [root@test01 ~]#
Edit /etc/multipath.conf and remove any lines relating to previously configured multipath disks. Restart multipathd with service multipathd restart. This should cause SAN multipath disks to disappear. On RHEL4.x, a restart seems the safest option.