LVM filesystems

Linux SAN disk with LVM based filesystem

This assumes that a vdisk has been created and presented to the host, and that SAN zoning is in place to allow the host to see the storage.

In this run through, the multipath.conf file had the multipath definition commented out before the multipath command was run. This created the mpath0 device. After adding a multipath alias line, the name changes to the vdisk001 alias which is defined.

Edited multipath.conf with existing lines removed for brevity:-

multipaths {
        multipath {
                wwid                    36001438005dedc000000300000510000
                alias                   vdisk001
        }
        multipath {
                wwid                    36001438005dedc000000300000550000
                alias                   vdisk002
        }
}


device
{
vendor                  "HP"
product                 "HSV450"
path_grouping_policy    group_by_prio
getuid_callout          "/sbin/scsi_id -g -u -s /block/%n
path_checker            tur
path_selector           "round-robin 0"
prio_callout            "/sbin/mpath_prio_alua %n"
rr_weight               uniform
rr_min_io               100
failback                immediate
no_path_retry           12
}
[root@gy-test01 ~]# multipath
remove: mpath0 (dup of vdisk001)
create: vdisk001 (36001438005dedc000000300000370000)
[size=1 GB][features="0"][hwhandler="0"]
\_ round-robin 0 [prio=1]
 \_ 0:0:0:1 sda 8:0   [active][ready]
\_ round-robin 0 [prio=1]
 \_ 0:0:1:1 sdb 8:16  [active][ready]

...edited...

fdisk shows the underlying eight (Four paths over each SAN fabric) sdX devices and the new /dev/dm-1:-

[root@gy-test01 ~]# fdisk -l

Disk /dev/sda: 1073 MB, 1073741824 bytes
34 heads, 61 sectors/track, 1011 cylinders
Units = cylinders of 2074 * 512 = 1061888 bytes

Disk /dev/sda doesn't contain a valid partition table

...edited...

Disk /dev/dm-1: 1073 MB, 1073741824 bytes
34 heads, 61 sectors/track, 1011 cylinders
Units = cylinders of 2074 * 512 = 1061888 bytes

Disk /dev/dm-1 doesn't contain a valid partition table


[root@gy-test01 ~]# ls -l /dev/mapper/vdisk001 
brw-rw----  1 root disk 253, 1 Dec  3 14:26 /dev/mapper/vdisk001

LVM

Physical Volume creation

[root@gy-test01 ~]# pvcreate /dev/mapper/vdisk001
  Physical volume "/dev/mapper/vdisk001" successfully created

Volume Group creation

[root@gy-test01 ~]# vgcreate testvg /dev/mapper/vdisk001
  Volume group "testvg" successfully created

[root@gy-test01 ~]# vgdisplay
  --- Volume group ---
  VG Name               testvg
  System ID             
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  1
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                0
  Open LV               0
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               1020.00 MB
  PE Size               4.00 MB
  Total PE              255
  Alloc PE / Size       0 / 0   
  Free  PE / Size       255 / 1020.00 MB
  VG UUID               VyUGva-mSgX-2pCC-ZuBI-YdKs-MFQ6-EiNt6c

Logical Volume creation

 
[root@gy-test01 ~]# lvcreate  --size 500M --name test01lv testvg
  Logical volume "test01lv" created

[root@gy-test01 ~]# lvcreate  --size 500M --name test02lv testvg
  Logical volume "test02lv" created


[root@gy-test01 ~]# lvdisplay 
  --- Logical volume ---
  LV Name                /dev/testvg/test01lv
  VG Name                testvg
  LV UUID                PffBiq-8zhP-VABE-ga6O-Asnk-Cpig-vhHO0S
  LV Write Access        read/write
  LV Status              available
  # open                 0
  LV Size                500.00 MB
  Current LE             125
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:0
   
  --- Logical volume ---
  LV Name                /dev/testvg/test02lv
  VG Name                testvg
  LV UUID                h2LG5y-w4XB-VHez-BC3Q-pMcf-kQVc-UHV4vH
  LV Write Access        read/write
  LV Status              available
  # open                 0
  LV Size                500.00 MB
  Current LE             125
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:2

Extend LV

Extend Logical Volume to 100% of the size available in a Volume Group

# lvextend -l +100%FREE /dev/myvg/testlv

Make Filesystems

[root@gy-test01 ~]# mkfs /dev/testvg/test01lv
mke2fs 1.35 (28-Feb-2004)
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
128016 inodes, 512000 blocks
25600 blocks (5.00%) reserved for the super user
First data block=1
Maximum filesystem blocks=67633152
63 block groups
8192 blocks per group, 8192 fragments per group
2032 inodes per group
Superblock backups stored on blocks: 
        8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409

Writing inode tables: done                            
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 24 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.

[root@gy-test01 ~]# mkfs /dev/testvg/test02lv
mke2fs 1.35 (28-Feb-2004)
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
128016 inodes, 512000 blocks
25600 blocks (5.00%) reserved for the super user
First data block=1
Maximum filesystem blocks=67633152
63 block groups
8192 blocks per group, 8192 fragments per group
2032 inodes per group
Superblock backups stored on blocks: 
        8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409

Writing inode tables: done                            
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 32 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.

Mount filesystems

[root@gy-test01 ~]# mount /dev/testvg/test01lv /mnt/test01lv
[root@gy-test01 ~]# mount /dev/testvg/test02lv /mnt/test02lv

[root@gy-test01 ~]# df -h
Filesystem                    Size  Used Avail Use% Mounted on
/dev/cciss/c0d0p5             27G   18G  8.0G  69% /
/dev/cciss/c0d0p1            145M   21M  116M  16% /boot
none                         7.9G     0  7.9G   0% /dev/shm
/dev/cciss/c0d0p3            2.9G  1.7G  1.2G  59% /var
/dev/mapper/testvg-test01lv  485M  2.3M  457M   1% /mnt/test01lv
/dev/mapper/testvg-test02lv  485M  2.3M  457M   1% /mnt/test02lv
[root@gy-test01 ~]# 

Resize LVM

Idea, resize /home to give more to /data.


/dev/mapper/rootvg-homelv	6.6G  144M  6.2G   3% /home
/dev/mapper/rootvg-varlv	2.9G  176M  2.5G   7% /var
/dev/mapper/datavg-datalv	30G   26G  2.1G  93% /data

reduce /home lv

umount /home
[root@linuxbuild01 ~]# e2fsck -f /dev/rootvg/homelv
e2fsck 1.39 (29-May-2006)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/rootvg/homelv: 11/1786400 files (9.1% non-contiguous), 92677/1785856 blocks


[root@linuxbuild01 ~]# resize2fs -p /dev/rootvg/homelv 2G
resize2fs 1.39 (29-May-2006)
Resizing the filesystem on /dev/rootvg/homelv to 524288 (4k) blocks.
Begin pass 3 (max = 55)
Scanning inode table          XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
The filesystem on /dev/rootvg/homelv is now 524288 blocks long.

[root@linuxbuild01 ~]# 

Moving LVM filesystems

This assumes that you have a LVM pv with one or more vg's and lv's built on top. You want to move a volume group from one pv to another, for instance to move from a smaller disk to a larger one and then to grow the vg and lv.

Two pv's are present, one of which contains the testvg.

[root@gy-test01 ~]# pvdisplay
  --- Physical volume ---
  PV Name               /dev/dm-0
  VG Name               testvg
  PV Size               2.00 GB / not usable 4.00 MB
  Allocatable           yes
  PE Size (KByte)       4096
  Total PE              511
  Free PE               383
  Allocated PE          128
  PV UUID               2sora7-Utg0-8VIl-kCdM-jdcj-WND9-DqxYn3

  "/dev/dm-1" is a new physical volume of "2.00 GB"
  --- NEW Physical volume ---
  PV Name               /dev/dm-1
  VG Name
  PV Size               2.00 GB
  Allocatable           NO
  PE Size (KByte)       0
  Total PE              0
  Free PE               0
  Allocated PE          0
  PV UUID               Ts6Uwd-UYam-RSXq-bsdH-3N2F-7s3N-FfOrDF

[root@gy-test01 ~]# pvmove /dev/mapper/vdisk001 /dev/mapper/vdisk002
  /dev/mapper/vdisk001: Moved: 79.7%
  /dev/mapper/vdisk001: Moved: 100.0%

Remove the original pv from vg, this is with the filesystem mounted and active! pvdisplay shows testvg on the second pv.

[root@gy-test01 ~]# vgreduce testvg /dev/dm-0
  Removed "/dev/dm-0" from volume group "testvg"
 [root@gy-test01 ~]# pvdisplay
  --- Physical volume ---
  PV Name               /dev/dm-1
  VG Name               testvg
  PV Size               2.00 GB / not usable 4.00 MB
... edited ...
  "/dev/dm-0" is a new physical volume of "2.00 GB"
  --- NEW Physical volume ---
  PV Name               /dev/dm-0
  VG Name
  PV Size               2.00 GB
... edited ...

[root@gy-test01 ~]# pvremove  /dev/dm-0
  Labels on physical volume "/dev/dm-0" successfully wiped

All gone from pv /dev/dm-0.

Removing LVM

This will remove the LVM volumes from the server, it does not cover removing the vdisk from EVA8000. If you want to move the LUN from one server and remount on another, DO NOT FOLLOW THESE INSTRUCTIONS!!!!!

[root@gy-test01 ~]# mount
...edited...
/dev/mapper/testvg-test01lv on /mnt/test01lv type ext2 (rw)
/dev/mapper/testvg-test02lv on /mnt/test02lv type ext2 (rw)

[root@gy-test01 ~]# 
[root@gy-test01 ~]# umount /mnt/test01lv
[root@gy-test01 ~]# umount /mnt/test02lv
[root@gy-test01 ~]# lvdisplay
  --- Logical volume ---
  LV Name                /dev/testvg/test01lv
  VG Name                testvg
  LV UUID                PffBiq-8zhP-VABE-ga6O-Asnk-Cpig-vhHO0S
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                500.00 MB
  Current LE             125
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:0
   
  --- Logical volume ---
  LV Name                /dev/testvg/test02lv
  VG Name                testvg
  LV UUID                h2LG5y-w4XB-VHez-BC3Q-pMcf-kQVc-UHV4vH
  LV Write Access        read/write
  LV Status              available
  # open                 0
  LV Size                500.00 MB
  Current LE             125
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:2
   
[root@gy-test01 ~]# lvremove 
  Please enter one or more logical volume paths
  Run `lvremove --help' for more information.
[root@gy-test01 ~]# lvremove test02lv
  Volume group "test02lv" not found

[root@gy-test01 ~]# lvremove /dev/testvg/test01lv
Do you really want to remove active logical volume "test01lv"? [y/n]: y
  Logical volume "test01lv" successfully removed
[root@gy-test01 ~]# 

[root@gy-test01 ~]# lvremove /dev/testvg/test02lv
Do you really want to remove active logical volume "test02lv"? [y/n]: y
  Logical volume "test02lv" successfully removed
[root@gy-test01 ~]# 

[root@gy-test01 ~]# vgdisplay
  --- Volume group ---
  VG Name               testvg
  System ID             
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  5
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                0
  Open LV               0
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               1020.00 MB
  PE Size               4.00 MB
  Total PE              255
  Alloc PE / Size       0 / 0   
  Free  PE / Size       255 / 1020.00 MB
  VG UUID               VyUGva-mSgX-2pCC-ZuBI-YdKs-MFQ6-EiNt6c
   
[root@gy-test01 ~]# vgremove testvg
  Volume group "testvg" successfully removed
[root@gy-test01 ~]# 

[root@gy-test01 ~]# pvdisplay
  "/dev/dm-1" is a new physical volume of "1.00 GB"
  --- NEW Physical volume ---
  PV Name               /dev/dm-1
  VG Name               
  PV Size               1.00 GB
  Allocatable           NO
  PE Size (KByte)       0
  Total PE              0
  Free PE               0
  Allocated PE          0
  PV UUID               hJ2jFO-Bn7t-oa73-pqDI-Hetc-X6MY-9G9C5v
   
[root@gy-test01 ~]# pvremove  /dev/dm-1
  Labels on physical volume "/dev/dm-1" successfully wiped
[root@gy-test01 ~]#

Edit /etc/multipath.conf and remove any lines relating to previously configured multipath disks. Restart multipathd with service multipathd restart. This should cause SAN multipath disks to disappear. On RHEL4.x, a restart seems the safest option.

Creating a new Logical Volume

In this example we will create a new logical volume called wworks, mounted at /wworks which will be part of the vg00 Volume Group at /dev/mapper/vg00-lv_wworks

1. Check to see how much free space you have to allocate the new logical volume:

[root@gi-rgs-rng01 /]# vgdisplay | egrep 'Name|Free'
  VG Name               vg00
  Free  PE / Size       2515 / 78.59 GB

2. Create the new logical volume and assign it to the vg00 volume group:

[root@gi-rgs-rng01 /]# lvcreate -L 30G -n lv_wworks vg00
  Logical volume "lv_wworks" created

3. Format it with a journalled filesystem:

mkfs.ext3 -j /dev/mapper/vg00-lv_wworks

4. Add the new entry to fstab (use the others as a template):

[root@gi-rgs-rng01 /]# cat /etc/fstab | grep wworks
/dev/vg00/lv_wworks     /wwworks                ext3    defaults        1 2

5. Mount the new logical volume and test it:

[root@gi-rgs-rng01 /]# mount -a && df -h | egrep 'File|wworks' && echo "test">/wworks/neil.test && cat /wworks/neil.test
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg00-lv_wworks
                     30G   76M   28G   1% /wworks
test

6. You can also see it as a member of the vg00 Volume Group:

[root@gi-rgs-rng01 /]# lvs
LV        VG   Attr   LSize  Origin Snap%  Move Log Copy%  Convert
lv_home   vg00 -wi-ao 10.00G
lv_log    vg00 -wi-ao 10.00G
lv_root   vg00 -wi-ao 15.00G
lv_swap   vg00 -wi-ao  5.88G
lv_tmp    vg00 -wi-ao  2.00G
lv_var    vg00 -wi-ao 15.00G
lv_wworks vg00 -wi-ao 30.00G

Reducing the size of an LVM

In this example we will reduce the size of /var/log which is an LV at /dev/mapper/vg00-lv_log

[root@gi-rgs-rng01 /]# df -h | egrep 'Filesystem|log'
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg00-lv_log
                     50G   85M   47G   1% /var/log

1. Ensure the filesystem is unmounted

$ umount /var/log

2. If the mount point is in use try identifying what's using it with lsof:

$ lsof | grep "/var/log"

3. Ensure /var/log is no longer mounted.

$ df -h | grep "/var/log"

Ensure the amount of data contained within the LV does not exceed the size you will be reducing it to or this will result in data loss.

4. First the filesystem needs to be checked.

[root@gi-rgs-rng01 /]# e2fsck -f /dev/mapper/vg00-lv_log
e2fsck 1.35 (28-Feb-2004)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/mapper/vg00-lv_log: 56/6553600 files (19.6% non-contiguous), 227290/13107200 blocks

5. Now reduce the filesystem to 10GB.

[root@gi-rgs-rng01 /]# resize2fs /dev/mapper/vg00-lv_log 10G
resize2fs 1.35 (28-Feb-2004)
Resizing the filesystem on /dev/mapper/vg00-lv_log to 2621440 (4k) blocks.
The filesystem on /dev/mapper/vg00-lv_log is now 2621440 blocks long.

6. Now reduce the size of the LV:

[root@gi-rgs-rng01 /]# lvreduce -L 10G /dev/mapper/vg00-lv_log
  WARNING: Reducing active logical volume to 10.00 GB
  THIS MAY DESTROY YOUR DATA (filesystem etc.)
Do you really want to reduce lv_log? [y/n]: y
  Reducing logical volume lv_log to 10.00 GB
  Logical volume lv_log successfully resized

7. Remount the partition and check the new size is correct:

[root@gi-rgs-rng01 /]# mount -a ; df -h | egrep 'Filesystem|log'
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg00-lv_log
                    9.9G   69M  9.3G   1% /var/log
                    
                    

Extending the size of a Logical Volume

In this example we will extend the size of /wworks which is an LV at /dev/mapper/vg00-lv_wworks, to 50GB.

1. Ensure the filesystem is unmounted

$ umount /wworks

2. If the mount point is in use try identifying what's using it with lsof:

$ lsof | grep "/wworks"

3. Ensure /var/log is no longer mounted.

$ df -h | grep "/wworks"

4. Extend the size of the logical volume:

$ lvextend -L 50GB /dev/mapper/vg00-lv_wworks

5. Now extend the filesystem to that new size

$ resizefs /dev/mapper/vg00-lv_wworks 50G

6. Remount the partition and check the new size is correct:

$ mount -a ; df -h | egrep 'Filesystem|wworks'
 

rb/linuxlvm.txt · Last modified: 19/09/2019 15:31 by andrew