SAN Fabric Config

Background

Port Types
E Expansion Port, connect two switches together via ISL
F Fabric Port, connects to Node
L Loop Port,
U Universal Port, connects to tape drives etc.
Fabric IDs
Domain or Switch ID All switches in a fabric MUST have different Domain ID number.
Switch Zoning
World Wide Name zoning (Soft Zoning) Zone membership based on WWN
Port zoning (Hard Zoning) Relies on physical port for Zone membership

FabOS hardware support document.
http://community.brocade.com/docs/DOC-1643 Switch Types vs Product name.

Brocade Licencing (sucks)

Storage ports are not always licenced to be used, on the HP blade chassis switches (gy-fcsw1-02 & gy-fcsw2-02 and gy-fcsw1-03 & gy-fcsw2-03), 12 licences are available but 24 ports are available. Of these, ports 1-16 are mapped to server slots, port 17 or 18 are used for uplinks to other FC switches and ports 19 to 23 and port 0 are for sfp modules. These are currently unused.

This means that there are 11 licences to cover 16 slots, so not all ports can be licenced concurrently. If blades are moved within a chassis, it is possible that a FC licence may not be allocated to the new slot, or that a blade not requiring FC connectivity in in a slot with a licence allocated.

If a licence is not allocated, the port will be disabled and a blade in that slot may have trouble booting if it cannot see SAN based storage.

See this page for information on allocating licences:- Brocade Licencing.

Upgrading SAN switch firmware

Brocade SAN Health Diagnostics

Brocade produce a program (sadly running on windows only) which will log in to all the switches and produce a report showing the config and and errors in the fabric. You will need to load a config file which contains the details of the switches to test. A copy is uploaded here:- 20-06-2012.set

SAN Health screen capture The screen capture shows the initial page of SAN Health after the .set config file has been loaded. On the Report Return tab, you will need to enter a valid email address to notify once the analysis is complete.

On the SAN Details tab, click the Test Connectivity button to ensure all switches are reachable. This has been done on the screen capture, all switches are smiling and green, typical windowsy faux cuteness which is what you get for allowing developers to use windows.

On the Start Audit tab, click the Preflight Check button to ensure all conditions are satisfied. If this passes, start the audit.

At the end, you will be invited to submit the results of the data capture to Brocade over https. An email acknowledgement will be sent, following this another email will be sent to indicate that the results are ready for download. This email will contain the URL to collect the report from.

The downloaded zip file will contain a spreadsheet listing all the aliases, zones and configs as well as a drawing in Visio format of the fabric layouts.

Commands

Infrequently used or dangerous commands

(Note that the commands on the link above may be dangerous, particularly cfgenable if you specify the wrong config to load!!)

Change Port name

Note, this is a switch level command not a fabric level command. You cannot change the portname for a port on a switch you are not logged in to.

If you need to view or change the portname for a switch port, here's how:-

fcsw1-01:admin> portshow 2
portName: 
portHealth: No License

Authentication: None
portDisableReason: None
.... edited .....

fcsw1-01:admin> portname
port  0: EVA-B1
port  3: EVA-A3
port  8: asp-db01-pci7

fcsw1-01:admin> portname 2 gy-asp-db03

fcsw1-01:admin> portname              
port  0: EVA-B1
port  2: gy-asp-db03
port  3: EVA-A3
port  8: asp-db01-pci7

Gotchas!!
Switchshow does not seem to show up a port name, although the java interface does:-

fcsw1-01:admin> switchshow
... edited ...
Area Port Media Speed State 
==============================
  0   0   id    N4   Online    F-Port  50:00:1f:e1:50:09:10:0c
  1   1   id    N4   Online    F-Port  50:01:43:80:09:ac:0e:32
  2   2   id    N4   Online    F-Port  50:01:43:80:09:ac:0e:1e   <------- Named port above not shown.
  3   3   id    N4   Online    F-Port  50:00:1f:e1:50:09:10:0a
 16  16   id    N4   Online    E-Port  10:00:00:05:1e:04:92:ef "gy-fcsw1-02" (downstream)  <---- Learnt from Topology, not set.

Aliases

Create new alias with:-

fcsw2-01:admin> alicreate "testalias", "50:01:43:80:09:ac:0d:2c"

Show aliases, use alishow for all known aliases, or to list a specific one:-

fcsw2-01:admin> alishow testalias
 alias: testalias       
                50:01:43:80:09:ac:0d:2c

Later versions of Fabos allow wildcards:-

fcsw2-04:FID128:admin> alishow *msa*
 alias: gy_msa01
                50:05:08:b3:00:94:5f:09

Remove aliases with:-

fcsw2-01:admin> alidelete testalias
fcsw2-01:admin>

Rename an alias, this is not straightforward as there is not an alirename command. The process is to duplicate the alias to change with the new alias name and original wwn. Then add this to any zones which refer to this. After it is safe to delete the original alias name from these zones. Remember to cfgsave.

Zoning

Create a new Zone:-

fcsw2-01:admin> zonecreate "testzone", "testalias"

With multiple aliases in the same zone:-

fcsw2-01:FID128:admin> zonecreate "Z_sma_eva", "EVA_h1fp2;EVA_h1fp4;EVA_h2fp2;EVA_h2fp4;ka_sma01"

Show all zones with zoneshow or just one with:-

fcsw2-01:admin> zoneshow testzone
  zone:  testzone        
                testalias

Delete a zone with:-

fcsw2-01:admin> zonedelete testzone

Check:-

fcsw2-01:admin> zoneshow testzone

"testzone" does not exist.

You can show multiple zones at one time with wildcards:-

 zoneshow z_wager02_gy_rgs_db0*

Renaming an object

On later versions of FABOS, you can rename an object with the zoneobjectrename command. An example is given below, the original zone in the config was named in a confusing manner, renaming it clarifies what the zone is for.

Zones and configs are edited for clarity and brevity.

fcsw1-04:FID128:admin> cfgshow cfg2012_07_05_01
 cfg:   cfg2012_07_05_01
                z_wager02_satellite02; 
                Z_EVA_gy_rgs_bi01_A;   <-------------------------------- This is the zone name to change
                z_eva_test01;
                Z_gy_asp_tape01_gy_rgs_db01_A1;
                ...edited...
                z_gy_asp_log01_Wager02

Check the members of the zone, the aliases are the server name and the specific storage device, in this case, Wager02. So it is just the zone name which should be changed.

fcsw1-04:FID128:admin> zoneshow Z_EVA_gy_rgs_bi01_A
 zone:  Z_EVA_gy_rgs_bi01_A
                gy_rgs_bi01_A; wager02_A1; wager02_A3; wager02_B1; wager02_B3

Rename the object and show the same config but with the new name.

fcsw1-04:FID128:admin> zoneobjectrename "Z_EVA_gy_rgs_bi01_A", "z_gy_rgs_bi01_Wager02"
fcsw1-04:FID128:admin> cfgshow cfg2012_07_05_01
 cfg:   cfg2012_07_05_01
                z_wager02_satellite02; z_eva_test01;
                Z_gy_asp_tape01_gy_rgs_db01_A1;
                .... edited ....
                Z_Wager02_gy_test_rac01; Z_Wager02_gy_test_rac02;
                z_gy_asp_log01_Wager02; 
                z_gy_rgs_bi01_Wager02  <-----------------------------------New Zone name!

fcsw1-04:FID128:admin> cfgsave
Do you want to save Defined zoning configuration only?  (yes, y, no, n): [no] y
Updating flash ...
fcsw1-04:FID128:admin>

Save just as good practice. cfgsave also sanity checks the defined config to ensure it is valid. In the event of a reboot, the config will be loaded automatically, if there is an error, the switch will fail to load a valid config and will not pass traffic. cfgsave will report an error and refuse to save if it is not happy.

Config saving

CLI way
Create a new config with cfgcreate with the new zone just created, add existing zones in to config with cfgadd. You can see the existing zones which must be added with the zoneshow command, look for the line which shows Effective configuration:, the next line will show the current cfg, eg. cfg: cfg2011_05_06_1. Higher up in the output from zoneshow command there will be a block of zones listed under the effective config name. All these must be copied in to the new config, if they are not, when you load the new config all your existing servers will cease to see any disk. This will be very bad. There is a zoneobjectcopy command in later fabos releases which copies a config to a new config so you can just add the new zones in.

Save to flash with cfgsave. You can add multiple zones to cfgcreate.

fscw2-03:admin> cfgcreate "cfg2011_09_20_1", "z_wager02_satellite02"
fscw2-03:admin> cfgadd "cfg2011_09_20_1", "APP01_Zone; APP02_Zone; APP03_Zone; LOG01_Zone; 
>                 Z_eva_gy_asp_db01; Z_eva_gy_asp_db02; z_eva_gy_asp_db03; 
>                 z_eva_gy_asp_db04; Z_EVA_gy_asp_sma01_B; Z_EVA_gy_ops01_B; 
>                 Z_EVA_gy_rgs_bi01_B; Z_EVA_gy_rgs_db01_B; Z_EVA_gy_rgs_db02_B; 
>                 Z_EVA_gy_rgs_db03_B; Z_EVA_gy_rgs_db04_B; Z_EVA_gy_test_rac01; 
>                 Z_EVA_gy_test_rac02; z_eva_test01; Z_gy_rgs_bi01_tape_B; 
>                 z_tape_bkup03; z_wager02_gy_asp_db03; z_wager02_gy_asp_db04; 
>                 Z_wager02_gy_asp_sma01_B; Z_Wager02_gyops01_B"

Maybe better way, but could only work on later firmware (6.x).

Create some new zones:-

fcsw2-01:FID128:admin> zonecreate "dbholdingzone", "ka_rgs_db01;ka_rgs_db02;ka_rgs_db03;ka_rgs_db04"
fcsw2-01:FID128:admin> zonecreate "HoldingZone", "ka_bkup01;ka_spare01;ka_log01;ka_spare_db02;ka_tapedrive03;ka_tapedrive04"

Copy the effective config to a new one with zoneobjectcopy, get the effective config name from zoneshow.
Add the new zones into the new object created, check all is present:-

fcsw2-01:FID128:admin> zoneobjectcopy "cfg2012_02_28_01", "cfg2012_02_29_01"
fcsw2-01:FID128:admin> cfgadd "cfg2012_02_29_01", "HoldinfZone;dbholdingzone"
fcsw2-01:FID128:admin> cfgshow cfg2012_02_29_01
 cfg:   cfg2012_02_29_01
                Z_sma_eva; HoldinfZone; dbholdingzone

Save the config to prevent loss, note that this is still not enabled.

fcsw2-01:FID128:admin> cfgsave
You are about to save the Defined zoning configuration. This
action will only save the changes on Defined configuration.
Any changes made on the Effective configuration will not
take effect until it is re-enabled.
Do you want to save Defined zoning configuration only?  (yes, y, no, n): [no] y
Updating flash ...
fcsw2-01:FID128:admin>

Enabling a Config

Until a config is activated or “enabled”, the old zones will be in use. To enable a new config (following adding or modifying a zone for example), use the following:-

fscw2-03:admin> cfgenable "cfg2011_09_20_1"
You are about to enable a new zoning configuration.
This action will replace the old zoning configuration with the
current configuration selected. If the update includes changes
to one or more traffic isolation zones, the update may result in
localized disruption to traffic on ports associated with
the traffic isolation zone changes
Do you want to enable 'cfg2011_09_20_01' configuration  (yes, y, no, n): [no] y
zone config "cfg2011_09_20_01" is in effect
Updating flash ...
fcsw2-01:FID128:admin>

Heed the warning, saying “y” at this point will overwrite the old config with the new one. Note <fc #FF0000>Overwrite</fc>, not <fc #FF00FF>Add to</fc>. If you have not included all the old zones you wish to keep, you will risk servers losing disks.

Deleting a Config

Too many old configs can be confusing, so they should be deleted. To see all the available configs on the switch, use this command. Note that in later versions of FABOS, pipes (|) and grep are available as well as ? & * wildcards.

fcsw2-04:FID128:admin> cfgshow cfg* | grep cfg
 cfg:   cfg2010_12_01_1
 cfg:   cfg2010_12_02_1
 cfg:   cfg2011_02_03_1
 cfg:   cfg2011_02_04_1
 cfg:   cfg2011_05_06_1
 cfg:   cfg2011_09_20_1
 cfg:   cfg2012_03_27_01
 cfg:   cfg2012_04_18_01
 cfg:   cfg2012_05_16_01
 cfg:   cfg2012_05_24_01
 cfg:   cfg2012_06_22_01
 cfg:   cfg2012_06_28_01
fcsw2-04:FID128:admin>

To see in more detail, give cfgshow the full name of the config to see:-

fcsw2-04:FID128:admin> cfgdelete "cfg2010_12_01_1"
fcsw2-04:FID128:admin>

Remember to cfgsave after deleting to update the flash.

HOW-TO configure syslog servers in brocade switches

fcsw3-01:admin> syslogdIpAdd "192.168.190.35" 
Syslog IP address 192.168.190.35 added

fcsw3-01:admin> syslogdIpShow
syslog.1        192.168.190.35

fcsw3-01:admin>

How-to Enable https on Brocade switches

Generate private key on the switch:

fcsw4-01:admin> seccertutil genkey -keysize 1024
Generating a new key pair will automatically do the following:
1. Delete all existing CSRs.
2. Delete all existing certificates.
3. Reset the certificate filename to none.
4. Disable secure protocols.

Continue (yes, y, no, n): [no] y
Generating new rsa public/private key pair
Done.

Create a request (CSR)

fcsw3-01:admin> seccertutil gencsr -country GI -state GI -locality Gibraltar -org "IGT Interactive" 
 -orgunit Interactive -cn fcsw4-01.company.com
Generating CSR, file name is: 192.168.191.63.csr

Export it to CA server (or just copy the content of csr file to CA server)

Option 1.
seccertutil export [-ldapcacert [-certname <certificate name>] | -fcapswcert | -fcapswcsr | -fcapcacert]
                   [-protocol <FTP | SCP>] [-ipaddr <IP address>] [-remotedir <remote directiory>]
                   [-login <login name>] [-password <password>]
Option 2.
fcsw3-01:admin> seccertutil showcsr
...
Copy the content directly to a file in CA Authority 

Sign CSR using CA

[[how to sign a csr using CA]]

Import the .pem key into the switch and enable https:

fcsw3-01:admin> seccertutil import -config swcert -enable https -protocol SCP -ipaddr 192.168.191.9 -remotedir /root/certs 
-certname fcsw3-01.company.com.pem -login root
Password:
Success: imported certificate [fcsw3-01.company.com.pem].
Certificate file in configuration has been updated.
Secure http has been enabled.
   

rb/brocade-commands.txt · Last modified: 08/07/2019 12:24 by andrew