Rainsbrook

User Tools

Site Tools

Trace: • linuxsan
rb:linuxsan

Table of Contents

SAN Storage

Overall Sequence

  1. Create Aliases for WWPN's on SAN switch
  2. Create Zone on SAN to allow server to see storage & save config
  3. Create Host on EVA storage and add WWPN's
  4. Create vdisk on EVA storage and present LUN to Host created above.
  5. On server, scan for new storage
  6. Configure /etc/multipathd.conf to combine /dev/sd devices into /dev/mapper devices under multipath control
  7. Partition disk and create filesystem.

Find out WWN

RHEL4

[root@test01 ~]# cat /proc/scsi/qla2xxx/* | grep adapter-port
scsi-qla0-adapter-port=500110a00019ac8c;
scsi-qla1-adapter-port=500110a00019ac8e;

RHEL5

[root@db03 ~]# cat /sys/class/fc_host/host*/port_name 
0x5001438009ac0d2c
0x5001438009ac0e1e

Multipath info

[root@bi01 ~]# lspci | grep Q
08:00.0 Fibre Channel: QLogic Corp. ISP2432-based 4Gb Fibre Channel to PCI Express HBA (rev 02)
08:00.1 Fibre Channel: QLogic Corp. ISP2432-based 4Gb Fibre Channel to PCI Express HBA (rev 02)

All paths here are via one HBA, one fabric has failed (the 1: in the 1st column) 

[root@bi01 ~]# multipath -ll /dev/sda
P_BI_Data_0003 (3600508b4001052920000e00003770000)
[size=250 GB][features="1 queue_if_no_path"][hwhandler="0"]
\_ round-robin 0 [prio=100][active]
 \_ 1:0:0:1 sda 8:0   [active][ready]
 \_ 1:0:3:1 sdm 8:192 [active][ready] 
\_ round-robin 0 [prio=20][enabled]
 \_ 1:0:1:1 sde 8:64  [active][ready]
 \_ 1:0:2:1 sdi 8:128 [active][ready]

After enabling the fabric, paths are seen via both HBA's (different server) 

[root@db01 ~]# multipath -ll /dev/sda
ocr1 (3600508b4001052a80000b00001740000)
[size=2 GB][features="1 queue_if_no_path"][hwhandler="0"]
\_ round-robin 0 [prio=200][active]
 \_ 0:0:0:1  sda  8:0     [active][ready]
 \_ 0:0:3:1  sdbu 68:128  [active][ready]
 \_ 1:0:0:1  sdcs 70:0    [active][ready]
 \_ 1:0:3:1  sdfm 130:128 [active][ready]
\_ round-robin 0 [prio=40][enabled]
 \_ 0:0:2:1  sdaw 67:0    [active][ready]
 \_ 1:0:1:1  sddq 71:128  [active][ready]
 \_ 1:0:2:1  sdeo 129:0   [active][ready]
 \_ 0:0:1:1  sdy  65:128  [active][ready]
[root@bi01 ~]# ls -l /sys/block/*/device
lrwxrwxrwx  1 root root 0 Aug 26 12:08 /sys/block/cciss!c0d0/device -> ../../devices/pci0000:40/0000:40:0c.0/0000:41:00.0/0000:42:08.0/disk0
lrwxrwxrwx  1 root root 0 Aug 26 12:08 /sys/block/sda/device -> ../../devices/pci0000:00/0000:00:0e.0/0000:08:00.1/host1/target1:0:0/1:0:0:1
lrwxrwxrwx  1 root root 0 Aug 26 12:08 /sys/block/sdb/device -> ../../devices/pci0000:00/0000:00:0e.0/0000:08:00.1/host1/target1:0:0/1:0:0:2
lrwxrwxrwx  1 root root 0 Aug 26 12:08 /sys/block/sdc/device -> ../../devices/pci0000:00/0000:00:0e.0/0000:08:00.1/host1/target1:0:0/1:0:0:3
lrwxrwxrwx  1 root root 0 Aug 26 12:08 /sys/block/sdd/device -> ../../devices/pci0000:00/0000:00:0e.0/0000:08:00.1/host1/target1:0:0/1:0:0:4
lrwxrwxrwx  1 root root 0 Aug 26 12:08 /sys/block/sde/device -> ../../devices/pci0000:00/0000:00:0e.0/0000:08:00.1/host1/target1:0:1/1:0:1:1
lrwxrwxrwx  1 root root 0 Aug 26 12:08 /sys/block/sdf/device -> ../../devices/pci0000:00/0000:00:0e.0/0000:08:00.1/host1/target1:0:1/1:0:1:2
lrwxrwxrwx  1 root root 0 Aug 26 12:08 /sys/block/sdg/device -> ../../devices/pci0000:00/0000:00:0e.0/0000:08:00.1/host1/target1:0:1/1:0:1:3
lrwxrwxrwx  1 root root 0 Aug 26 12:08 /sys/block/sdh/device -> ../../devices/pci0000:00/0000:00:0e.0/0000:08:00.1/host1/target1:0:1/1:0:1:4
lrwxrwxrwx  1 root root 0 Aug 26 12:08 /sys/block/sdi/device -> ../../devices/pci0000:00/0000:00:0e.0/0000:08:00.1/host1/target1:0:2/1:0:2:1
lrwxrwxrwx  1 root root 0 Aug 26 12:08 /sys/block/sdj/device -> ../../devices/pci0000:00/0000:00:0e.0/0000:08:00.1/host1/target1:0:2/1:0:2:2
lrwxrwxrwx  1 root root 0 Aug 26 12:08 /sys/block/sdk/device -> ../../devices/pci0000:00/0000:00:0e.0/0000:08:00.1/host1/target1:0:2/1:0:2:3
lrwxrwxrwx  1 root root 0 Aug 26 12:08 /sys/block/sdl/device -> ../../devices/pci0000:00/0000:00:0e.0/0000:08:00.1/host1/target1:0:2/1:0:2:4
lrwxrwxrwx  1 root root 0 Aug 26 12:08 /sys/block/sdm/device -> ../../devices/pci0000:00/0000:00:0e.0/0000:08:00.1/host1/target1:0:3/1:0:3:1
lrwxrwxrwx  1 root root 0 Aug 26 12:08 /sys/block/sdn/device -> ../../devices/pci0000:00/0000:00:0e.0/0000:08:00.1/host1/target1:0:3/1:0:3:2
lrwxrwxrwx  1 root root 0 Aug 26 12:08 /sys/block/sdo/device -> ../../devices/pci0000:00/0000:00:0e.0/0000:08:00.1/host1/target1:0:3/1:0:3:3
lrwxrwxrwx  1 root root 0 Aug 26 12:08 /sys/block/sdp/device -> ../../devices/pci0000:00/0000:00:0e.0/0000:08:00.1/host1/target1:0:3/1:0:3:4

Re-Scan Fibre devices

RHEL 4

[root@bi01 SANsurfer]# /opt/hp/hp_fibreutils/hp_rescan -a
Sending rescan signal to /proc/scsi/qla2xxx/0...
Sending rescan signal to /proc/scsi/qla2xxx/1...
Adding legacy tape devices to /proc/scsi/device_info
Scanning /proc/scsi/qla2xxx/0, target 3, LUN 4      
Scanning /proc/scsi/qla2xxx/1, target 3, LUN 4      
                                                      

scsi0  00 00 00 HP         HSV210     5110       RAID
scsi0  00 00 01 HP         HSV210     5110       Direct-Access
scsi0  00 00 02 HP         HSV210     5110       Direct-Access
scsi0  00 00 03 HP         HSV210     5110       Direct-Access
scsi0  00 00 04 HP         HSV210     5110       Direct-Access
edited
scsi1  00 00 00 HP         HSV210     5110       RAID
scsi1  00 00 01 HP         HSV210     5110       Direct-Access
scsi1  00 00 02 HP         HSV210     5110       Direct-Access
scsi1  00 00 03 HP         HSV210     5110       Direct-Access
scsi1  00 00 04 HP         HSV210     5110       Direct-Access

[root@bi01 SANsurfer]# multipath -ll
P_BI_Data_0003 (3600508b4001052920000e00003770000)
[size=250 GB][features="1 queue_if_no_path"][hwhandler="0"]
\_ round-robin 0 [prio=200][active]
 \_ 1:0:0:1 sda  8:0    [active][ready]
 \_ 1:0:3:1 sdm  8:192  [active][ready]
 \_ 0:0:0:1 sdq  65:0   [active][ready]
 \_ 0:0:1:1 sdu  65:64  [active][ready]
\_ round-robin 0 [prio=40][enabled]
 \_ 1:0:1:1 sde  8:64   [active][ready]
 \_ 1:0:2:1 sdi  8:128  [active][ready]
 \_ 0:0:2:1 sdy  65:128 [active][ready]
 \_ 0:0:3:1 sdac 65:192 [active][ready]

Paths now visible via HBA 0 and 1 (0: and 1:)

RHEL 5

echo "- - -" >  /sys/class/scsi_host/<host listings here>/scan

Get the <host listings here> from:- 

[root@r08u6 scsi_host]# ll
total 0
drwxr-xr-x 2 root root 0 Aug  4 15:52 host0
drwxr-xr-x 2 root root 0 Aug  4 15:52 host1
drwxr-xr-x 2 root root 0 Aug  4 15:52 host2

Multipath command output

Output of multipath command

Standard output of multipath command 

# multipath -ll
mydev1 (3600a0b800011a1ee0000040646828cc5) dm-1 IBM,1815      FAStT
[size=512M][features=1 queue_if_no_path][hwhandler=1 rdac]
\_ round-robin 0 [prio=6][active]
 \_ 29:0:0:1 sdf 8:80  [active][ready]
 \_ 28:0:1:1 sdl 8:176 [active][ready]
\_ round-robin 0 [prio=0][enabled]
 \_ 28:0:0:1 sdb 8:16  [active][ghost]
 \_ 29:0:1:1 sdq 65:0  [active][ghost]

Annotated output of multipath command

mydev1 (3600a0b800011a1ee0000040646828cc5) dm-1 IBM,1815      FAStT
------  ---------------------------------  ---- --- ---------------
  |               |                         |    |          |-------> Product  
  |               |                         |    |------------------> Vendor
  |               |                         |-----------------------> sysfs name
  |               |-------------------------------------------------> WWID of the device
  |------------------------------------------------------ ----------> User defined Alias name

 [size=512M][features=1 queue_if_no_path][hwhandler=1 rdac]
 ---------  ---------------------------  ----------------
     |                 |                        |--------------------> Hardware Handler, if any
     |                 |---------------------------------------------> Features supported
     |---------------------------------------------------------------> Size of the DM device

Path Group 1:
\_ round-robin 0 [prio=6][active]
-- -------------  ------  ------
 |    |              |      |----------------------------------------> Path group state
 |    |              |-----------------------------------------------> Path group priority
 |    |--------------------------------------------------------------> Path selector and repeat count
 |-------------------------------------------------------------------> Path group level

First path on Path Group 1:
 \_ 29:0:0:1 sdf 8:80  [active][ready]
    -------- --- ----   ------  -----
      |      |     |        |      |---------------------------------> Physical Path state
      |      |     |        |----------------------------------------> DM Path state
      |      |     |-------------------------------------------------> Major, minor numbers
      |      |-------------------------------------------------------> Linux device name
      |--------------------------------------------------------------> SCSI information: host, channel, scsi_id and lun

Second path on Path Group 1:
 \_ 28:0:1:1 sdl 8:176 [active][ready]

Path Group 2:
\_ round-robin 0 [prio=0][enabled]
 \_ 28:0:0:1 sdb 8:16  [active][ghost]
 \_ 29:0:1:1 sdq 65:0  [active][ghost]
rb/linuxsan.txt · Last modified: by 127.0.0.1
Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki