Table of Contents
SAN Fabric Config
Background
Port Types | |
---|---|
E | Expansion Port, connect two switches together via ISL |
F | Fabric Port, connects to Node |
L | Loop Port, |
U | Universal Port, connects to tape drives etc. |
Fabric IDs | |
---|---|
Domain or Switch ID | All switches in a fabric MUST have different Domain ID number. |
Switch Zoning | |
---|---|
World Wide Name zoning (Soft Zoning) | Zone membership based on WWN |
Port zoning (Hard Zoning) | Relies on physical port for Zone membership |
FabOS hardware support document.
http://community.brocade.com/docs/DOC-1643 Switch Types vs Product name.
Brocade Licencing (sucks)
Storage ports are not always licenced to be used, on the HP blade chassis switches (gy-fcsw1-02 & gy-fcsw2-02 and gy-fcsw1-03 & gy-fcsw2-03), 12 licences are available but 24 ports are available. Of these, ports 1-16 are mapped to server slots, port 17 or 18 are used for uplinks to other FC switches and ports 19 to 23 and port 0 are for sfp modules. These are currently unused.
This means that there are 11 licences to cover 16 slots, so not all ports can be licenced concurrently. If blades are moved within a chassis, it is possible that a FC licence may not be allocated to the new slot, or that a blade not requiring FC connectivity in in a slot with a licence allocated.
If a licence is not allocated, the port will be disabled and a blade in that slot may have trouble booting if it cannot see SAN based storage.
See this page for information on allocating licences:- Brocade Licencing.
Upgrading SAN switch firmware
Brocade SAN Health Diagnostics
Brocade produce a program (sadly running on windows only) which will log in to all the switches and produce a report showing the config and and errors in the fabric. You will need to load a config file which contains the details of the switches to test. A copy is uploaded here:- 20-06-2012.set
The screen capture shows the initial page of SAN Health after the .set config file has been loaded. On the
Report Return
tab, you will need to enter a valid email address to notify once the analysis is complete.
On the SAN Details
tab, click the Test Connectivity
button to ensure all switches are reachable. This has been done on the screen capture, all switches are smiling and green, typical windowsy faux cuteness which is what you get for allowing developers to use windows.
On the Start Audit
tab, click the Preflight Check
button to ensure all conditions are satisfied. If this passes, start the audit.
At the end, you will be invited to submit the results of the data capture to Brocade over https. An email acknowledgement will be sent, following this another email will be sent to indicate that the results are ready for download. This email will contain the URL to collect the report from.
The downloaded zip file will contain a spreadsheet listing all the aliases, zones and configs as well as a drawing in Visio format of the fabric layouts.
Commands
Infrequently used or dangerous commands
(Note that the commands on the link above may be dangerous, particularly cfgenable if you specify the wrong config to load!!)
Change Port name
Note, this is a switch level command not a fabric level command. You cannot change the portname for a port on a switch you are not logged in to.
If you need to view or change the portname for a switch port, here's how:-
fcsw1-01:admin> portshow 2 portName: portHealth: No License Authentication: None portDisableReason: None .... edited ..... fcsw1-01:admin> portname port 0: EVA-B1 port 3: EVA-A3 port 8: asp-db01-pci7 fcsw1-01:admin> portname 2 gy-asp-db03 fcsw1-01:admin> portname port 0: EVA-B1 port 2: gy-asp-db03 port 3: EVA-A3 port 8: asp-db01-pci7
Gotchas!!
Switchshow does not seem to show up a port name, although the java interface does:-
fcsw1-01:admin> switchshow ... edited ... Area Port Media Speed State ============================== 0 0 id N4 Online F-Port 50:00:1f:e1:50:09:10:0c 1 1 id N4 Online F-Port 50:01:43:80:09:ac:0e:32 2 2 id N4 Online F-Port 50:01:43:80:09:ac:0e:1e <------- Named port above not shown. 3 3 id N4 Online F-Port 50:00:1f:e1:50:09:10:0a 16 16 id N4 Online E-Port 10:00:00:05:1e:04:92:ef "gy-fcsw1-02" (downstream) <---- Learnt from Topology, not set.
Aliases
Create new alias with:-
fcsw2-01:admin> alicreate "testalias", "50:01:43:80:09:ac:0d:2c"
Show aliases, use alishow
for all known aliases, or to list a specific one:-
fcsw2-01:admin> alishow testalias alias: testalias 50:01:43:80:09:ac:0d:2c
Later versions of Fabos allow wildcards:-
fcsw2-04:FID128:admin> alishow *msa* alias: gy_msa01 50:05:08:b3:00:94:5f:09
Remove aliases with:-
fcsw2-01:admin> alidelete testalias fcsw2-01:admin>
Rename an alias, this is not straightforward as there is not an alirename command. The process is to duplicate the alias to change with the new alias name and original wwn. Then add this to any zones which refer to this. After it is safe to delete the original alias name from these zones. Remember to cfgsave
.
Zoning
Create a new Zone:-
fcsw2-01:admin> zonecreate "testzone", "testalias"
With multiple aliases in the same zone:-
fcsw2-01:FID128:admin> zonecreate "Z_sma_eva", "EVA_h1fp2;EVA_h1fp4;EVA_h2fp2;EVA_h2fp4;ka_sma01"
Show all zones with zoneshow or just one with:-
fcsw2-01:admin> zoneshow testzone zone: testzone testalias
Delete a zone with:-
fcsw2-01:admin> zonedelete testzone
Check:-
fcsw2-01:admin> zoneshow testzone "testzone" does not exist.
You can show multiple zones at one time with wildcards:-
zoneshow z_wager02_gy_rgs_db0*
Renaming an object
On later versions of FABOS, you can rename an object with the zoneobjectrename
command. An example is given below, the original zone in the config was named in a confusing manner, renaming it clarifies what the zone is for.
Zones and configs are edited for clarity and brevity.
fcsw1-04:FID128:admin> cfgshow cfg2012_07_05_01 cfg: cfg2012_07_05_01 z_wager02_satellite02; Z_EVA_gy_rgs_bi01_A; <-------------------------------- This is the zone name to change z_eva_test01; Z_gy_asp_tape01_gy_rgs_db01_A1; ...edited... z_gy_asp_log01_Wager02
Check the members of the zone, the aliases are the server name and the specific storage device, in this case, Wager02. So it is just the zone name which should be changed.
fcsw1-04:FID128:admin> zoneshow Z_EVA_gy_rgs_bi01_A zone: Z_EVA_gy_rgs_bi01_A gy_rgs_bi01_A; wager02_A1; wager02_A3; wager02_B1; wager02_B3
Rename the object and show the same config but with the new name.
fcsw1-04:FID128:admin> zoneobjectrename "Z_EVA_gy_rgs_bi01_A", "z_gy_rgs_bi01_Wager02" fcsw1-04:FID128:admin> cfgshow cfg2012_07_05_01 cfg: cfg2012_07_05_01 z_wager02_satellite02; z_eva_test01; Z_gy_asp_tape01_gy_rgs_db01_A1; .... edited .... Z_Wager02_gy_test_rac01; Z_Wager02_gy_test_rac02; z_gy_asp_log01_Wager02; z_gy_rgs_bi01_Wager02 <-----------------------------------New Zone name! fcsw1-04:FID128:admin> cfgsave Do you want to save Defined zoning configuration only? (yes, y, no, n): [no] y Updating flash ... fcsw1-04:FID128:admin>
Save just as good practice. cfgsave also sanity checks the defined config to ensure it is valid. In the event of a reboot, the config will be loaded automatically, if there is an error, the switch will fail to load a valid config and will not pass traffic. cfgsave will report an error and refuse to save if it is not happy.
Config saving
CLI way
Create a new config with cfgcreate
with the new zone just created, add existing zones in to config with cfgadd
. You can see the existing zones which must be added with the zoneshow
command, look for the line which shows Effective configuration:
, the next line will show the current cfg, eg. cfg: cfg2011_05_06_1
. Higher up in the output from zoneshow
command there will be a block of zones listed under the effective config name. All these must be copied in to the new config, if they are not, when you load the new config all your existing servers will cease to see any disk. This will be very bad. There is a zoneobjectcopy command in later fabos releases which copies a config to a new config so you can just add the new zones in.
Save to flash with cfgsave
.
You can add multiple zones to cfgcreate
.
fscw2-03:admin> cfgcreate "cfg2011_09_20_1", "z_wager02_satellite02" fscw2-03:admin> cfgadd "cfg2011_09_20_1", "APP01_Zone; APP02_Zone; APP03_Zone; LOG01_Zone; > Z_eva_gy_asp_db01; Z_eva_gy_asp_db02; z_eva_gy_asp_db03; > z_eva_gy_asp_db04; Z_EVA_gy_asp_sma01_B; Z_EVA_gy_ops01_B; > Z_EVA_gy_rgs_bi01_B; Z_EVA_gy_rgs_db01_B; Z_EVA_gy_rgs_db02_B; > Z_EVA_gy_rgs_db03_B; Z_EVA_gy_rgs_db04_B; Z_EVA_gy_test_rac01; > Z_EVA_gy_test_rac02; z_eva_test01; Z_gy_rgs_bi01_tape_B; > z_tape_bkup03; z_wager02_gy_asp_db03; z_wager02_gy_asp_db04; > Z_wager02_gy_asp_sma01_B; Z_Wager02_gyops01_B"
Maybe better way, but could only work on later firmware (6.x).
Create some new zones:-
fcsw2-01:FID128:admin> zonecreate "dbholdingzone", "ka_rgs_db01;ka_rgs_db02;ka_rgs_db03;ka_rgs_db04" fcsw2-01:FID128:admin> zonecreate "HoldingZone", "ka_bkup01;ka_spare01;ka_log01;ka_spare_db02;ka_tapedrive03;ka_tapedrive04"
Copy the effective config to a new one with zoneobjectcopy
, get the effective config name from zoneshow
.
Add the new zones into the new object created, check all is present:-
fcsw2-01:FID128:admin> zoneobjectcopy "cfg2012_02_28_01", "cfg2012_02_29_01" fcsw2-01:FID128:admin> cfgadd "cfg2012_02_29_01", "HoldinfZone;dbholdingzone" fcsw2-01:FID128:admin> cfgshow cfg2012_02_29_01 cfg: cfg2012_02_29_01 Z_sma_eva; HoldinfZone; dbholdingzone
Save the config to prevent loss, note that this is still not enabled.
fcsw2-01:FID128:admin> cfgsave You are about to save the Defined zoning configuration. This action will only save the changes on Defined configuration. Any changes made on the Effective configuration will not take effect until it is re-enabled. Do you want to save Defined zoning configuration only? (yes, y, no, n): [no] y Updating flash ... fcsw2-01:FID128:admin>
Enabling a Config
Until a config is activated or “enabled”, the old zones will be in use. To enable a new config (following adding or modifying a zone for example), use the following:-
fscw2-03:admin> cfgenable "cfg2011_09_20_1" You are about to enable a new zoning configuration. This action will replace the old zoning configuration with the current configuration selected. If the update includes changes to one or more traffic isolation zones, the update may result in localized disruption to traffic on ports associated with the traffic isolation zone changes Do you want to enable 'cfg2011_09_20_01' configuration (yes, y, no, n): [no] y zone config "cfg2011_09_20_01" is in effect Updating flash ... fcsw2-01:FID128:admin>
Heed the warning, saying “y” at this point will overwrite the old config with the new one. Note <fc #FF0000>Overwrite</fc>, not <fc #FF00FF>Add to</fc>. If you have not included all the old zones you wish to keep, you will risk servers losing disks.
Deleting a Config
Too many old configs can be confusing, so they should be deleted. To see all the available configs on the switch, use this command. Note that in later versions of FABOS, pipes (|)
and grep
are available as well as ?
& *
wildcards.
fcsw2-04:FID128:admin> cfgshow cfg* | grep cfg cfg: cfg2010_12_01_1 cfg: cfg2010_12_02_1 cfg: cfg2011_02_03_1 cfg: cfg2011_02_04_1 cfg: cfg2011_05_06_1 cfg: cfg2011_09_20_1 cfg: cfg2012_03_27_01 cfg: cfg2012_04_18_01 cfg: cfg2012_05_16_01 cfg: cfg2012_05_24_01 cfg: cfg2012_06_22_01 cfg: cfg2012_06_28_01 fcsw2-04:FID128:admin>
To see in more detail, give cfgshow the full name of the config to see:-
fcsw2-04:FID128:admin> cfgdelete "cfg2010_12_01_1" fcsw2-04:FID128:admin>
Remember to cfgsave
after deleting to update the flash.