====== SAN Fabric Config ======
===== Background =====
^ Port Types ^ ^
| E | Expansion Port, connect two switches together via ISL |
| F | Fabric Port, connects to Node |
| L | Loop Port, |
| U | Universal Port, connects to tape drives etc. |
^ Fabric IDs ^ ^
| Domain or Switch ID | All switches in a fabric MUST have different Domain ID number.|
^ Switch Zoning ^ ^
| World Wide Name zoning (Soft Zoning) | Zone membership based on WWN |
| Port zoning (Hard Zoning)| Relies on physical port for Zone membership |
[[http://community.brocade.com/docs/DOC-1662|FabOS hardware support document.]] \\
[[http://community.brocade.com/docs/DOC-1643 Switch Types vs Product name.]]
===== Brocade Licencing (sucks) =====
Storage ports are not always licenced to be used, on the HP blade chassis switches (gy-fcsw1-02 & gy-fcsw2-02 and gy-fcsw1-03 & gy-fcsw2-03), 12 licences are available but 24 ports are available. Of these, ports 1-16 are mapped to server slots, port 17 or 18 are used for uplinks to other FC switches and ports 19 to 23 and port 0 are for sfp modules. These are currently unused.
This means that there are 11 licences to cover 16 slots, so not all ports can be licenced concurrently. If blades are moved within a chassis, it is possible that a FC licence may not be allocated to the new slot, or that a blade not requiring FC connectivity in in a slot with a licence allocated.
If a licence is not allocated, the port will be disabled and a blade in that slot may have trouble booting if it cannot see SAN based storage.
See this page for information on allocating licences:- [[rb:brocade-licencing|Brocade Licencing]].
===== Upgrading SAN switch firmware =====
[[rb:sanconfig-upgrade|Upgrade SAN firmware]]
===== Brocade SAN Health Diagnostics =====
Brocade produce a program (sadly running on windows only) which will log in to all the switches and produce a report showing the config and and errors in the fabric.
You will need to load a config file which contains the details of the switches to test. A copy is uploaded here:- {{:tech:20-06-2012.set|}}
{{:rb:san-health1.png?200 |SAN Health screen capture}}
The screen capture shows the initial page of SAN Health after the .set config file has been loaded. On the ''Report Return'' tab, you will need to enter a valid email address to notify once the analysis is complete.
On the ''SAN Details'' tab, click the ''Test Connectivity'' button to ensure all switches are reachable. This has been done on the screen capture, all switches are smiling and green, typical windowsy faux cuteness which is what you get for allowing developers to use windows.
On the ''Start Audit'' tab, click the ''Preflight Check'' button to ensure all conditions are satisfied. If this passes, start the audit.
At the end, you will be invited to submit the results of the data capture to Brocade over https. An email acknowledgement will be sent, following this another email will be sent to indicate that the results are ready for download. This email will contain the URL to collect the report from.
The downloaded zip file will contain a spreadsheet listing all the aliases, zones and configs as well as a drawing in Visio format of the fabric layouts.
===== Commands =====
[[rb:sanconfig-infrequent|Infrequently used or dangerous commands]]
(Note that the commands on the link above may be dangerous, particularly //cfgenable// if you specify the wrong config to load!!)
==== Change Port name ====
Note, this is a //switch level// command not a //fabric level// command. You cannot change the portname for a port on a switch you are not logged in to.
If you need to view or change the portname for a switch port, here's how:-
fcsw1-01:admin> portshow 2
portName:
portHealth: No License
Authentication: None
portDisableReason: None
.... edited .....
fcsw1-01:admin> portname
port 0: EVA-B1
port 3: EVA-A3
port 8: asp-db01-pci7
fcsw1-01:admin> portname 2 gy-asp-db03
fcsw1-01:admin> portname
port 0: EVA-B1
port 2: gy-asp-db03
port 3: EVA-A3
port 8: asp-db01-pci7
Gotchas!!\\
Switchshow does not seem to show up a port name, although the java interface does:-
fcsw1-01:admin> switchshow
... edited ...
Area Port Media Speed State
==============================
0 0 id N4 Online F-Port 50:00:1f:e1:50:09:10:0c
1 1 id N4 Online F-Port 50:01:43:80:09:ac:0e:32
2 2 id N4 Online F-Port 50:01:43:80:09:ac:0e:1e <------- Named port above not shown.
3 3 id N4 Online F-Port 50:00:1f:e1:50:09:10:0a
16 16 id N4 Online E-Port 10:00:00:05:1e:04:92:ef "gy-fcsw1-02" (downstream) <---- Learnt from Topology, not set.
==== Aliases ====
**Create** new alias with:-
fcsw2-01:admin> alicreate "testalias", "50:01:43:80:09:ac:0d:2c"
**Show** aliases, use ''alishow'' for all known aliases, or to list a specific one:-
fcsw2-01:admin> alishow testalias
alias: testalias
50:01:43:80:09:ac:0d:2c
Later versions of Fabos allow wildcards:-
fcsw2-04:FID128:admin> alishow *msa*
alias: gy_msa01
50:05:08:b3:00:94:5f:09
**Remove** aliases with:-
fcsw2-01:admin> alidelete testalias
fcsw2-01:admin>
**Rename** an alias, this is not straightforward as there is not an alirename command. The process is to duplicate the alias to change with the new alias name and original wwn. Then add this to any zones which refer to this. After it is safe to delete the original alias name from these zones. Remember to ''cfgsave''.
==== Zoning ====
**Create** a new Zone:-
fcsw2-01:admin> zonecreate "testzone", "testalias"
With multiple aliases in the same zone:-
fcsw2-01:FID128:admin> zonecreate "Z_sma_eva", "EVA_h1fp2;EVA_h1fp4;EVA_h2fp2;EVA_h2fp4;ka_sma01"
**Show** all zones with zoneshow or just one with:-
fcsw2-01:admin> zoneshow testzone
zone: testzone
testalias
**Delete** a zone with:-
fcsw2-01:admin> zonedelete testzone
Check:-
fcsw2-01:admin> zoneshow testzone
"testzone" does not exist.
You can show multiple zones at one time with wildcards:-
zoneshow z_wager02_gy_rgs_db0*
==== Renaming an object ====
On later versions of FABOS, you can rename an object with the ''zoneobjectrename'' command. An example is given below, the original zone in the config was named in a confusing manner, renaming it clarifies what the zone is for.
Zones and configs are edited for clarity and brevity.
fcsw1-04:FID128:admin> cfgshow cfg2012_07_05_01
cfg: cfg2012_07_05_01
z_wager02_satellite02;
Z_EVA_gy_rgs_bi01_A; <-------------------------------- This is the zone name to change
z_eva_test01;
Z_gy_asp_tape01_gy_rgs_db01_A1;
...edited...
z_gy_asp_log01_Wager02
Check the members of the zone, the aliases are the server name and the specific storage device, in this case, Wager02. So it is just the zone name which should be changed.
fcsw1-04:FID128:admin> zoneshow Z_EVA_gy_rgs_bi01_A
zone: Z_EVA_gy_rgs_bi01_A
gy_rgs_bi01_A; wager02_A1; wager02_A3; wager02_B1; wager02_B3
Rename the object and show the same config but with the new name.
fcsw1-04:FID128:admin> zoneobjectrename "Z_EVA_gy_rgs_bi01_A", "z_gy_rgs_bi01_Wager02"
fcsw1-04:FID128:admin> cfgshow cfg2012_07_05_01
cfg: cfg2012_07_05_01
z_wager02_satellite02; z_eva_test01;
Z_gy_asp_tape01_gy_rgs_db01_A1;
.... edited ....
Z_Wager02_gy_test_rac01; Z_Wager02_gy_test_rac02;
z_gy_asp_log01_Wager02;
z_gy_rgs_bi01_Wager02 <-----------------------------------New Zone name!
fcsw1-04:FID128:admin> cfgsave
Do you want to save Defined zoning configuration only? (yes, y, no, n): [no] y
Updating flash ...
fcsw1-04:FID128:admin>
Save just as good practice. cfgsave also sanity checks the defined config to ensure it is valid. In the event of a reboot, the config will be loaded automatically, if there is an error, the switch will fail to load a valid config and will not pass traffic. cfgsave will report an error and refuse to save if it is not happy.
==== Config saving ====
CLI way \\
Create a new config with ''cfgcreate'' with the new zone just created, add existing zones in to config with ''cfgadd''. You can see the existing zones which must be added with the ''zoneshow'' command, look for the line which shows ''Effective configuration:'', the next line will show the current cfg, eg. ''cfg: cfg2011_05_06_1''. Higher up in the output from ''zoneshow'' command there will be a block of zones listed under the effective config name. All these must be copied in to the new config, if they are not, when you load the new config all your existing servers will cease to see any disk. This will be very bad. There is a zoneobjectcopy command in later fabos releases which copies a config to a new config so you can just add the new zones in.
Save to flash with ''cfgsave''.
You can add multiple zones to ''cfgcreate''.
fscw2-03:admin> cfgcreate "cfg2011_09_20_1", "z_wager02_satellite02"
fscw2-03:admin> cfgadd "cfg2011_09_20_1", "APP01_Zone; APP02_Zone; APP03_Zone; LOG01_Zone;
> Z_eva_gy_asp_db01; Z_eva_gy_asp_db02; z_eva_gy_asp_db03;
> z_eva_gy_asp_db04; Z_EVA_gy_asp_sma01_B; Z_EVA_gy_ops01_B;
> Z_EVA_gy_rgs_bi01_B; Z_EVA_gy_rgs_db01_B; Z_EVA_gy_rgs_db02_B;
> Z_EVA_gy_rgs_db03_B; Z_EVA_gy_rgs_db04_B; Z_EVA_gy_test_rac01;
> Z_EVA_gy_test_rac02; z_eva_test01; Z_gy_rgs_bi01_tape_B;
> z_tape_bkup03; z_wager02_gy_asp_db03; z_wager02_gy_asp_db04;
> Z_wager02_gy_asp_sma01_B; Z_Wager02_gyops01_B"
Maybe better way, but could only work on later firmware (6.x).
Create some new zones:-
fcsw2-01:FID128:admin> zonecreate "dbholdingzone", "ka_rgs_db01;ka_rgs_db02;ka_rgs_db03;ka_rgs_db04"
fcsw2-01:FID128:admin> zonecreate "HoldingZone", "ka_bkup01;ka_spare01;ka_log01;ka_spare_db02;ka_tapedrive03;ka_tapedrive04"
Copy the effective config to a new one with ''zoneobjectcopy'', get the effective config name from ''zoneshow''. \\
Add the new zones into the new object created, check all is present:-
fcsw2-01:FID128:admin> zoneobjectcopy "cfg2012_02_28_01", "cfg2012_02_29_01"
fcsw2-01:FID128:admin> cfgadd "cfg2012_02_29_01", "HoldinfZone;dbholdingzone"
fcsw2-01:FID128:admin> cfgshow cfg2012_02_29_01
cfg: cfg2012_02_29_01
Z_sma_eva; HoldinfZone; dbholdingzone
Save the config to prevent loss, note that this is still not enabled.
fcsw2-01:FID128:admin> cfgsave
You are about to save the Defined zoning configuration. This
action will only save the changes on Defined configuration.
Any changes made on the Effective configuration will not
take effect until it is re-enabled.
Do you want to save Defined zoning configuration only? (yes, y, no, n): [no] y
Updating flash ...
fcsw2-01:FID128:admin>
==== Enabling a Config ====
Until a config is activated or "enabled", the old zones will be in use. To enable a new config (following adding or modifying a zone for example), use the following:-
fscw2-03:admin> cfgenable "cfg2011_09_20_1"
You are about to enable a new zoning configuration.
This action will replace the old zoning configuration with the
current configuration selected. If the update includes changes
to one or more traffic isolation zones, the update may result in
localized disruption to traffic on ports associated with
the traffic isolation zone changes
Do you want to enable 'cfg2011_09_20_01' configuration (yes, y, no, n): [no] y
zone config "cfg2011_09_20_01" is in effect
Updating flash ...
fcsw2-01:FID128:admin>
Heed the warning, saying "y" at this point will overwrite the old config with the new one. Note **Overwrite**, not **Add to**. If you have not included all the old zones you wish to keep, you will risk servers losing disks.
==== Deleting a Config ====
Too many old configs can be confusing, so they should be deleted. To see all the available configs on the switch, use this command. Note that in later versions of FABOS, ''pipes (|)'' and ''grep'' are available as well as ''?'' & ''*'' wildcards.
fcsw2-04:FID128:admin> cfgshow cfg* | grep cfg
cfg: cfg2010_12_01_1
cfg: cfg2010_12_02_1
cfg: cfg2011_02_03_1
cfg: cfg2011_02_04_1
cfg: cfg2011_05_06_1
cfg: cfg2011_09_20_1
cfg: cfg2012_03_27_01
cfg: cfg2012_04_18_01
cfg: cfg2012_05_16_01
cfg: cfg2012_05_24_01
cfg: cfg2012_06_22_01
cfg: cfg2012_06_28_01
fcsw2-04:FID128:admin>
To see in more detail, give cfgshow the full name of the config to see:-
fcsw2-04:FID128:admin> cfgdelete "cfg2010_12_01_1"
fcsw2-04:FID128:admin>
Remember to ''cfgsave'' after deleting to update the flash.