Fibre Channel switching demystified at a basic level

Fiber Channel has been compared to Ethernet and for good reasons. Both are OSI L2 protocols, and both use globaly unique physical addresses, but not in the same way! While MAC addresses are always present in ethernet traffic, FC WWNs are quite a bit different.  They never appear in the FC headers.

Let's understand the underlying FC switching fabric first to put things in perspective. When multiple FC switches are connected together using E_ports, they form a switch fabric. The domain ID is unique to each switch in the fabric and there lies the first problem with scalability: only 239 are available. The lowest [PS_priority+WWWN] becomes the PS (principal switch). The PS selection process occurs when the E_ports are first connected, and the BF (the mundane sounding "build fabric") frames are exchanged. If two switches contain the same domain ID then the link between the two is "isolated". The domain id can be chosen at random, unless it is administratively set. This is required to generate the initial discovery traffic such as BF, EFP, SW_ACC frames. If you are really curious, the S_id and D_id (FCIDs) in these frames are always to set to 0xFFFFFD which is the fabric controller address.

So, given the 239 domain IDs in any given fabric, how to we break through the limit to scale up? Enter NPV or N-port virtualization. An NPV enabled switch does not take up a domain ID, but instead, relays the FLOGIs comming received on F_ports (from N_ports on hosts) up to the core switch via NP_ports. In other words, ports on the NPV edge switch that connect to the F_ports on the core are always set to type "NPV". NPV mode and Fabric mode are mutually exclusive, and a reboot is necessary when selecting either mode.

What is NPIV? N_port id virtualization is a feature that allows F_ports to accept multiple FLOGI requests from the same N_port and assign FCIDs accordingly.  Ask: are NPIV and NPV mutually exclusive on a any given device? Hint: NPIV is a server feature, while NPV is a switch feature, typically. Whether it is NPV enabled switch or a NPIV enabled host, the F_port essentially behaves the same and accepts multiple FLOGIs. So, an NPV enabled switch looks exactly like an NPIV enabled host to a F_port. Phew! Enough already, right?

So, how does FLOGI work? What is the initial FCID in the frame that comes out of an N_port going toward the F_port? Ans: 0x000000 - this is the initial value of the FCID, and this frame is sent to 0xFFFFFE which represents the FLOGI server. The payload contains the WWNN and service parameters. The FLOGI servers assigns the FCID (N_port ID) and BB_Credit.

Next, the host can use its newly acquired N-port ID to continue with the PLOGI process where it sends its WWPN to FCID map. The destination FCID is that of the FCNS: 0xFFFFFC. The FCNS now registers this information it its database and exposes this to other devices according to zoning that has been configured.

Try the command: "show fcns database" if you SAN uses a Cisco switch.

Troubleshooting CRC and input errors with Nexus 5000 in transit path

When you encounter CRC (input) errors on an upstream switch interface, and you have a Nexus 5000 in the transit path (downstream), MTU Stomping by the latter is likely a factor.

On a Nexus 5000 switch, when a frame is received on a 10 Gb/s interface, it is considered to be in the cut-through path.  If there is no "network qos" type policy with jumbo frame support, and the hosts connected this switch generate jumbo frames, we will encounter the following situation:

There are logical and physical causes for the Nexus 5000 to drop a frame. There are also situations when a frame cannot be dropped because of the cut-through nature of the switch architecture. If a drop is necessary, but the frame is being switched in a cut-through path, then the only option is to stomp the Ethernet frame check sequence (FCS). Stomping a frame involves setting the FCS to a known value that does not pass a CRC check. This causes subsequent CRC checks to fail later in the path for this frame. A downstream store-and-forward device, or a host, will be able to drop this frame.

"show queueing interface ethernet x/y" will reveal the hardware MTU.  This setting is enforced by the application of a "network qos" type policy-map under "system qos" section of running configuration.

IP subnetting unraveled

Consider the following address in CIDR notation:

172.19.235.75 /13

1) What is the network address?

2) What is the broadcast address?

First, determine the subnet mask in dotted decimal notation: 255.248.0.0

Next, calculate: IP_Address AND Mask.  This reveals the network address as 172.16.0.0 (NET_address)

Then, determine the wildcard mask (WC_mask) by inverting 1's and 0's in the subnet mask: 0.7.255.255

Finally, calculate: NET_address OR WC_mask, and this reveals the broadcast address: 172.23.255.255

Now, practice this method on 10.0.0.100 /28 and if you are feeling adventurous, try 192.168.255.254 /22