Tuesday, November 11, 2014

Fibre Channel switching demystified at a basic level

Fiber Channel has been compared to Ethernet and for good reasons. Both are OSI L2 protocols, and both use globaly unique physical addresses, but not in the same way! While MAC addresses are always present in ethernet traffic, FC WWNs are quite a bit different.  They never appear in the FC headers.

Let's understand the underlying FC switching fabric first to put things in perspective. When multiple FC switches are connected together using E_ports, they form a switch fabric. The domain ID is unique to each switch in the fabric and there lies the first problem with scalability: only 239 are available. The lowest [PS_priority+WWWN] becomes the PS (principal switch). The PS selection process occurs when the E_ports are first connected, and the BF (the mundane sounding "build fabric") frames are exchanged. If two switches contain the same domain ID then the link between the two is "isolated". The domain id can be chosen at random, unless it is administratively set. This is required to generate the initial discovery traffic such as BF, EFP, SW_ACC frames. If you are really curious, the S_id and D_id (FCIDs) in these frames are always to set to 0xFFFFFD which is the fabric controller address.

So, given the 239 domain IDs in any given fabric, how to we break through the limit to scale up? Enter NPV or N-port virtualization. An NPV enabled switch does not take up a domain ID, but instead, relays the FLOGIs comming received on F_ports (from N_ports on hosts) up to the core switch via NP_ports. In other words, ports on the NPV edge switch that connect to the F_ports on the core are always set to type "NPV". NPV mode and Fabric mode are mutually exclusive, and a reboot is necessary when selecting either mode.

What is NPIV? N_port id virtualization is a feature that allows F_ports to accept multiple FLOGI requests from the same N_port and assign FCIDs accordingly.  Ask: are NPIV and NPV mutually exclusive on a any given device? Hint: NPIV is a server feature, while NPV is a switch feature, typically. Whether it is NPV enabled switch or a NPIV enabled host, the F_port essentially behaves the same and accepts multiple FLOGIs. So, an NPV enabled switch looks exactly like an NPIV enabled host to a F_port. Phew! Enough already, right?

So, how does FLOGI work? What is the initial FCID in the frame that comes out of an N_port going toward the F_port? Ans: 0x000000 - this is the initial value of the FCID, and this frame is sent to 0xFFFFFE which represents the FLOGI server. The payload contains the WWNN and service parameters. The FLOGI servers assigns the FCID (N_port ID) and BB_Credit.

Next, the host can use its newly acquired N-port ID to continue with the PLOGI process where it sends its WWPN to FCID map. The destination FCID is that of the FCNS: 0xFFFFFC. The FCNS now registers this information it its database and exposes this to other devices according to zoning that has been configured.

Try the command: "show fcns database" if you SAN uses a Cisco switch.