Thursday, July 26, 2012

Cisco UCS: Tracking layer-2 paths


Basic tenets of network design:
Dual Nexus 7018s in the access layer
Dual UCS 6100 fabric interconnnects
ESXi 5.0 - hosts are dual attached to A and B side of fabric

We are tracking down an IP address in VLAN 301.

Please note the MAC address tables on the Nexus 7018s, the UCS 6100 (A & B), and Nexus 1000V… the following steps illustrate the well-known method used:

Nexus-7018-01# show ip arp vlan 301
IP ARP Table
Total number of entries: 2
Address         Age       MAC Address     Interface
192.168.247.66   00:06:57  0026.980c.76c1  Vlan301
192.168.247.70   00:01:18  0050.8888.6238  Vlan301


Nexus-7018-01# show mac address-table vlan 301
Legend:
        * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
        age - seconds since last seen,+ - primary entry using vPC Peer-Link
   VLAN     MAC Address      Type      age     Secure NTFY Ports/SWID.SSID.LID
---------+-----------------+--------+---------+------+----+------------------
G 301      0026.980c.9bc1    static       -       F    F  sup-eth1(R)
* 301      0010.18a5.d9e0    dynamic   120        F    F  Po1
* 301      0026.980c.76c1    dynamic   450        F    F  Eth12/3
* 301      0050.8888.6237    dynamic   90         F    F  Eth12/3
* 301      0050.8888.6238    dynamic   90         F    F  Eth12/3 -> trunk to Nexus-7018-02
* 301      0050.8888.6239    dynamic   120        F    F  Eth12/7
* 301      547f.ee33.eb55    dynamic   120        F    F  Eth12/3
* 301      547f.ee35.cd95    dynamic   120        F    F  Eth12/7

Here we check the L2 adjacency table on Nexus-7018-02:

Nexus-7018-02# show mac address-table vlan 301
Legend:
        * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
        age - seconds since last seen,+ - primary entry using vPC Peer-Link
   VLAN     MAC Address      Type      age     Secure NTFY Ports/SWID.SSID.LID
---------+-----------------+--------+---------+------+----+------------------
...
* 301      0050.8888.6238    dynamic   90         F    F  Eth12/7 -> UCS6100 - UCS-6100-01 -A 
...

SSH to UCS 6100s first, then "connect nxos" to enter Nexus like CLI directly on the fabric.

UCS-6100-01-B(nxos)# show mac address-table vlan 301
Legend:
        * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
        age - seconds since last seen,+ - primary entry using vPC Peer-Link
   VLAN     MAC Address      Type      age     Secure NTFY    Ports
---------+-----------------+--------+---------+------+----+------------------
* 301      0050.8888.6239    dynamic   20         F    F  Veth934

(the MAC address does not appear on Fab B), we could have bypassed and gone directly to Fab A based on CDP information on Nexus-7018-02.)

UCS-6100-01-A(nxos)# show mac address-table vlan 301
Legend:
        * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
        age - seconds since last seen,+ - primary entry using vPC Peer-Link
   VLAN     MAC Address      Type      age     Secure NTFY    Ports
---------+-----------------+--------+---------+------+----+------------------
* 301      0050.8888.6237    dynamic   0          F    F  Veth957
* 301      0050.8888.6238    dynamic   0          F    F  Veth961 -> MAC address learnt here -> note the association on CDP neighbor on the 1000V down a few steps  below…

We now check the mac-address table on the Nexus 1000V:

n1k-lab01# show mac address-table vlan 301
VLAN      MAC Address       Type    Age       Port                           Mod
---------+-----------------+-------+---------+------------------------------+---
...
301       0050.8888.6238    static  0         Veth20                         -> this reveals MAC address 
on VM guest
...


This leads us to Veth20 - we check the Nexus 1000V again:

n1k-lab01# show interface virtual pinning module 6

------------------------------------------------------
Veth      Pinned        Associated PO List of
          Sub Group id  interface     Eth interface(s)
------------------------------------------------------
Veth17    0             Po4           Eth6/1
Veth19    1             Po4           Eth6/2
Veth20    2             Po4           Eth6/3 -> this indicates VEM 4 connected over Po4

How to manually pin traffic:

n1k-lab01(config-if)# where
  conf; interface Vethernet20      admin@n1k-lab01
n1k-lab01(config-if)# pinning ?
  id  Configure sub-group ID for pinning

n1k-lab01(config-if)# pinning id ?
    Enter sub-group ID

n1k-lab01(config-if)# pinning id 1

No comments:

Post a Comment