====== SAN Storage ====== ===== Overall Sequence ===== - Create Aliases for WWPN's on SAN switch - Create Zone on SAN to allow server to see storage & save config - Create Host on EVA storage and add WWPN's - Create vdisk on EVA storage and present LUN to Host created above. - On server, scan for new storage - Configure /etc/multipathd.conf to combine /dev/sd devices into /dev/mapper devices under multipath control - Partition disk and create filesystem. ===== Find out WWN ===== ==== RHEL4 ==== [root@test01 ~]# cat /proc/scsi/qla2xxx/* | grep adapter-port scsi-qla0-adapter-port=500110a00019ac8c; scsi-qla1-adapter-port=500110a00019ac8e; ==== RHEL5 ==== [root@db03 ~]# cat /sys/class/fc_host/host*/port_name 0x5001438009ac0d2c 0x5001438009ac0e1e ===== Multipath info ====== [root@bi01 ~]# lspci | grep Q 08:00.0 Fibre Channel: QLogic Corp. ISP2432-based 4Gb Fibre Channel to PCI Express HBA (rev 02) 08:00.1 Fibre Channel: QLogic Corp. ISP2432-based 4Gb Fibre Channel to PCI Express HBA (rev 02) All paths here are via one HBA, one fabric has failed (the 1: in the 1st column) [root@bi01 ~]# multipath -ll /dev/sda P_BI_Data_0003 (3600508b4001052920000e00003770000) [size=250 GB][features="1 queue_if_no_path"][hwhandler="0"] \_ round-robin 0 [prio=100][active] \_ 1:0:0:1 sda 8:0 [active][ready] \_ 1:0:3:1 sdm 8:192 [active][ready] \_ round-robin 0 [prio=20][enabled] \_ 1:0:1:1 sde 8:64 [active][ready] \_ 1:0:2:1 sdi 8:128 [active][ready] After enabling the fabric, paths are seen via both HBA's (different server) [root@db01 ~]# multipath -ll /dev/sda ocr1 (3600508b4001052a80000b00001740000) [size=2 GB][features="1 queue_if_no_path"][hwhandler="0"] \_ round-robin 0 [prio=200][active] \_ 0:0:0:1 sda 8:0 [active][ready] \_ 0:0:3:1 sdbu 68:128 [active][ready] \_ 1:0:0:1 sdcs 70:0 [active][ready] \_ 1:0:3:1 sdfm 130:128 [active][ready] \_ round-robin 0 [prio=40][enabled] \_ 0:0:2:1 sdaw 67:0 [active][ready] \_ 1:0:1:1 sddq 71:128 [active][ready] \_ 1:0:2:1 sdeo 129:0 [active][ready] \_ 0:0:1:1 sdy 65:128 [active][ready] [root@bi01 ~]# ls -l /sys/block/*/device lrwxrwxrwx 1 root root 0 Aug 26 12:08 /sys/block/cciss!c0d0/device -> ../../devices/pci0000:40/0000:40:0c.0/0000:41:00.0/0000:42:08.0/disk0 lrwxrwxrwx 1 root root 0 Aug 26 12:08 /sys/block/sda/device -> ../../devices/pci0000:00/0000:00:0e.0/0000:08:00.1/host1/target1:0:0/1:0:0:1 lrwxrwxrwx 1 root root 0 Aug 26 12:08 /sys/block/sdb/device -> ../../devices/pci0000:00/0000:00:0e.0/0000:08:00.1/host1/target1:0:0/1:0:0:2 lrwxrwxrwx 1 root root 0 Aug 26 12:08 /sys/block/sdc/device -> ../../devices/pci0000:00/0000:00:0e.0/0000:08:00.1/host1/target1:0:0/1:0:0:3 lrwxrwxrwx 1 root root 0 Aug 26 12:08 /sys/block/sdd/device -> ../../devices/pci0000:00/0000:00:0e.0/0000:08:00.1/host1/target1:0:0/1:0:0:4 lrwxrwxrwx 1 root root 0 Aug 26 12:08 /sys/block/sde/device -> ../../devices/pci0000:00/0000:00:0e.0/0000:08:00.1/host1/target1:0:1/1:0:1:1 lrwxrwxrwx 1 root root 0 Aug 26 12:08 /sys/block/sdf/device -> ../../devices/pci0000:00/0000:00:0e.0/0000:08:00.1/host1/target1:0:1/1:0:1:2 lrwxrwxrwx 1 root root 0 Aug 26 12:08 /sys/block/sdg/device -> ../../devices/pci0000:00/0000:00:0e.0/0000:08:00.1/host1/target1:0:1/1:0:1:3 lrwxrwxrwx 1 root root 0 Aug 26 12:08 /sys/block/sdh/device -> ../../devices/pci0000:00/0000:00:0e.0/0000:08:00.1/host1/target1:0:1/1:0:1:4 lrwxrwxrwx 1 root root 0 Aug 26 12:08 /sys/block/sdi/device -> ../../devices/pci0000:00/0000:00:0e.0/0000:08:00.1/host1/target1:0:2/1:0:2:1 lrwxrwxrwx 1 root root 0 Aug 26 12:08 /sys/block/sdj/device -> ../../devices/pci0000:00/0000:00:0e.0/0000:08:00.1/host1/target1:0:2/1:0:2:2 lrwxrwxrwx 1 root root 0 Aug 26 12:08 /sys/block/sdk/device -> ../../devices/pci0000:00/0000:00:0e.0/0000:08:00.1/host1/target1:0:2/1:0:2:3 lrwxrwxrwx 1 root root 0 Aug 26 12:08 /sys/block/sdl/device -> ../../devices/pci0000:00/0000:00:0e.0/0000:08:00.1/host1/target1:0:2/1:0:2:4 lrwxrwxrwx 1 root root 0 Aug 26 12:08 /sys/block/sdm/device -> ../../devices/pci0000:00/0000:00:0e.0/0000:08:00.1/host1/target1:0:3/1:0:3:1 lrwxrwxrwx 1 root root 0 Aug 26 12:08 /sys/block/sdn/device -> ../../devices/pci0000:00/0000:00:0e.0/0000:08:00.1/host1/target1:0:3/1:0:3:2 lrwxrwxrwx 1 root root 0 Aug 26 12:08 /sys/block/sdo/device -> ../../devices/pci0000:00/0000:00:0e.0/0000:08:00.1/host1/target1:0:3/1:0:3:3 lrwxrwxrwx 1 root root 0 Aug 26 12:08 /sys/block/sdp/device -> ../../devices/pci0000:00/0000:00:0e.0/0000:08:00.1/host1/target1:0:3/1:0:3:4 ====== Re-Scan Fibre devices ====== ===== RHEL 4 ===== [root@bi01 SANsurfer]# /opt/hp/hp_fibreutils/hp_rescan -a Sending rescan signal to /proc/scsi/qla2xxx/0... Sending rescan signal to /proc/scsi/qla2xxx/1... Adding legacy tape devices to /proc/scsi/device_info Scanning /proc/scsi/qla2xxx/0, target 3, LUN 4 Scanning /proc/scsi/qla2xxx/1, target 3, LUN 4 scsi0 00 00 00 HP HSV210 5110 RAID scsi0 00 00 01 HP HSV210 5110 Direct-Access scsi0 00 00 02 HP HSV210 5110 Direct-Access scsi0 00 00 03 HP HSV210 5110 Direct-Access scsi0 00 00 04 HP HSV210 5110 Direct-Access edited scsi1 00 00 00 HP HSV210 5110 RAID scsi1 00 00 01 HP HSV210 5110 Direct-Access scsi1 00 00 02 HP HSV210 5110 Direct-Access scsi1 00 00 03 HP HSV210 5110 Direct-Access scsi1 00 00 04 HP HSV210 5110 Direct-Access [root@bi01 SANsurfer]# multipath -ll P_BI_Data_0003 (3600508b4001052920000e00003770000) [size=250 GB][features="1 queue_if_no_path"][hwhandler="0"] \_ round-robin 0 [prio=200][active] \_ 1:0:0:1 sda 8:0 [active][ready] \_ 1:0:3:1 sdm 8:192 [active][ready] \_ 0:0:0:1 sdq 65:0 [active][ready] \_ 0:0:1:1 sdu 65:64 [active][ready] \_ round-robin 0 [prio=40][enabled] \_ 1:0:1:1 sde 8:64 [active][ready] \_ 1:0:2:1 sdi 8:128 [active][ready] \_ 0:0:2:1 sdy 65:128 [active][ready] \_ 0:0:3:1 sdac 65:192 [active][ready] Paths now visible via HBA 0 and 1 (0: and 1:) \\ ===== RHEL 5 ===== echo "- - -" > /sys/class/scsi_host//scan Get the from:- \\ [root@r08u6 scsi_host]# ll total 0 drwxr-xr-x 2 root root 0 Aug 4 15:52 host0 drwxr-xr-x 2 root root 0 Aug 4 15:52 host1 drwxr-xr-x 2 root root 0 Aug 4 15:52 host2 ===== Multipath command output ===== Output of multipath command Standard output of multipath command # multipath -ll mydev1 (3600a0b800011a1ee0000040646828cc5) dm-1 IBM,1815 FAStT [size=512M][features=1 queue_if_no_path][hwhandler=1 rdac] \_ round-robin 0 [prio=6][active] \_ 29:0:0:1 sdf 8:80 [active][ready] \_ 28:0:1:1 sdl 8:176 [active][ready] \_ round-robin 0 [prio=0][enabled] \_ 28:0:0:1 sdb 8:16 [active][ghost] \_ 29:0:1:1 sdq 65:0 [active][ghost] Annotated output of multipath command mydev1 (3600a0b800011a1ee0000040646828cc5) dm-1 IBM,1815 FAStT ------ --------------------------------- ---- --- --------------- | | | | |-------> Product | | | |------------------> Vendor | | |-----------------------> sysfs name | |-------------------------------------------------> WWID of the device |------------------------------------------------------ ----------> User defined Alias name [size=512M][features=1 queue_if_no_path][hwhandler=1 rdac] --------- --------------------------- ---------------- | | |--------------------> Hardware Handler, if any | |---------------------------------------------> Features supported |---------------------------------------------------------------> Size of the DM device Path Group 1: \_ round-robin 0 [prio=6][active] -- ------------- ------ ------ | | | |----------------------------------------> Path group state | | |-----------------------------------------------> Path group priority | |--------------------------------------------------------------> Path selector and repeat count |-------------------------------------------------------------------> Path group level First path on Path Group 1: \_ 29:0:0:1 sdf 8:80 [active][ready] -------- --- ---- ------ ----- | | | | |---------------------------------> Physical Path state | | | |----------------------------------------> DM Path state | | |-------------------------------------------------> Major, minor numbers | |-------------------------------------------------------> Linux device name |--------------------------------------------------------------> SCSI information: host, channel, scsi_id and lun Second path on Path Group 1: \_ 28:0:1:1 sdl 8:176 [active][ready] Path Group 2: \_ round-robin 0 [prio=0][enabled] \_ 28:0:0:1 sdb 8:16 [active][ghost] \_ 29:0:1:1 sdq 65:0 [active][ghost]