Cisco UCS Configuration Guide for vSphere – Part 3 – LAN

This is part 3 of the Cisco UCS Configuration Guide.  This post is broken into two components: configuring the physical Server/FC/LAN links and the creation of UCS LAN policies.

Eight part series, describing how to configure a Cisco UCS from start to finish for a vSphere deployment:

• Cisco UCS Configuration Guide – Part 1 – Overview
• Cisco UCS Configuration Guide – Part 2 – Equipment
• Cisco UCS Configuration Guide – Part 3 – LAN
• Cisco UCS Configuration Guide – Part 4 – SAN
• Cisco UCS Configuration Guide – Part 5 – Server
• Cisco UCS Configuration Guide – Part 6 – Admin
• Cisco UCS Configuration Guide – Part 7 – Firmware Upgrade
• Cisco UCS Configuration Guide – Part 8 – ESXi Driver Upgrade

The UCS Manager LAN configuration has five areas: LAN Cloud, Appliances, Internal LAN, Pools and Policies.  The diagram below shows the breakdown of those areas.

Configure the Physical Links

Before you begin, you will need the following:

• Installed, connected and powered-on Cisco UCS system as described in Part 1 and Part 2.
• Be connected to the Cisco UCS Manager GUI as described in Part 2.

The first step is to configure the physical links to the UCS Chassis I/O Modules, Core LAN Switches and SAN Switches:

1. Select the “Equipment” tab and expand the “Equipment” tree to “Fabric Interconnects”.

2. Select “Fabric Interconnect A” and check that the “Status” “Ethernet Mode” and “FC Mode” are “End Host”.  If either is set to “Switch Mode”, simply select the “Set Ethernet End-Host Mode” or “Set FC End-Host Mode” links.  Refer to Screenshot below.

Example UCS Manager Fabric Interconnect “General” Tab Screenshot:

3. Check that your Server, LAN and SAN SFP+ modules appear in the “Ethernet” and “FC” ports list.

4. Right mouse click on each port and select “Configure as Uplink Port” for connections to the LAN core, “Configure as FC Uplink Port” for “FC Port” connections to the SAN fabric and “Configure as Server Port” for connections to the UCS Chassis FEX.  Refer to Screenshot below.

Example UCS Manager Fabric Interconnect “Port Configuration” Screenshot:

5. Follow steps 2 to 4 for “Fabric Interconnect B”.

6. With the Server Ports being configured, the UCS Chassis will be discovered and appear in the UCS Manager.  Each Server will have to be “Acknowledged”, just browse to each chassis and select each server highlighted in RED and click the “Acknowledge” button.

7. Once the LAN “Uplink Ports” are enabled, launch the “LAN Uplinks Manager” and select the “Create PortChannel” button to configure PortChannel to the Core LAN Switches.  Refer to Screenshot below.  Each PortChannel configuration will have a total of 2 (1+1), 4 (2+2) or 8 (4+4) uplinks from Fabric Interconnect A and Fabric Interconnect B.

Example UCS Manager Fabric Interconnect “LAN Uplinks Manager” Screenshot:

8. You should now have a UCS Manager screen with “Green” activated, error free links from the Fabric Interconnect to the LAN Core, SAN Fabric and UCS Chassis FEX modules.

9. The UCS Chassis should also be displayed under the “Equipment” tab in the “Chassis” list.  You will have a number of alarms present, ignore them for the time being, they will be resolved as you progress through the configuration process.

Configure LAN Policies

Before you begin, you will need the following:

• QoS System Class – for this exercise Best-Effort and Fiber-Channel will be used.
• Block of IP Addresses in the same subnet as UCS Management, enough for blade server KVM – for this exercise the range 10.1.1.10 – 10.1.1.20 will be used.
• Two Blocks of MAC Addresses for the virtual NICs, one for Fabric A and one for Fabric B – for this exercise the ranges 00:25:B5:32:00:00 – 00:25:B5:32:FF:FF & 00:25:B5:42:00:00 –  00:25:B5:42:FF:FF will be used.
• VLANs that will be tagged over the PortChannel interface – for this exercise VLANs 101, 102 and 103 will be used.

To configure the Cisco UCS LAN Policies, you need to:

1. Access the “LAN” tab of the UCS Manager.  Refer to Screenshot below.

Example UCS Manager Fabric Interconnect “LAN” Tab Screenshot:

2. Configure the “QoS System Class”, by selecting the “LAN Cloud” tree and then the “QoS System Class” object.  Refer to Screenshot below.

Example UCS Manager Fabric Interconnect “LAN – QoS System Class” Screenshot:

3. Configure the “QoS System Class” settings as per the table below:

4. If using the Cisco Nexus family of switches, make sure the following matching QoS policy is configured if you are using Jumbo frames.  You will not have consistent end-to-end MTU without this setting.  End-to-End MTU consistency can be tested from ESXi with the “vmkping -d -s 9000 <IP Address>” command.  Ignore this step if Standard MTU sizes (1500) are used.

policy-map type network-qos jumbo

  class type network-qos class-default

    mtu 9216

system qos

  service-policy type network-qos jumbo

5. Configure the IP and MAC pools, by selecting the “Pools” tree and then the “IP Pools” object.  Refer to Screenshot below.

Example UCS Manager Fabric Interconnect “LAN – Pools” Screenshot:

6. Modify the “IP Pool ext-mgmt” and “iscsi-initiator-pool” with the following settings.  iSCSI is not required, but this configuration will clear the iSCSI alarm that appears.

7. Create the “MAC Pools” “UCSFAB-A-MAC” and “UCSFAB-B-MAC” with the following settings.

8. Configure the “QoS Policies”, by selecting the “Policies” tree and then the “QoS Policies” object. Refer to Screenshot below.

Example UCS Manager Fabric Interconnect “LAN – QoS Policy” Screenshot:

9. Create the “Fiber-Channel”, “VM-Traffic” and “vSphere-Management” QoS Policies with the following settings.  The “Host Control” “none” setting means that vSphere Virtual Distributed Switch Network I/O Control “QoS tags” will not be used.

10. Configure the “VLANs”, by selecting the “LAN Cloud” tree and then the “VLANs” object. Refer to Screenshot below.

Example UCS Manager Fabric Interconnect “LAN – VLANs” Screenshot:

11. Create the “VLANs” “VLAN101”, “VLAN102” and “VLAN103” with the following settings:

12. Configure the “Network Control Policy”, by selecting the “Policies” tree and then the “Network Control Policy” object. Refer to Screenshot below.

Example UCS Manager Fabric Interconnect “LAN – Network Control Policy” Screenshot:

13. Create the “Network Control Policy” “ESXi”  with the following settings.  The “Action on Uplink Fail” with “Link Down” allows the vSphere Virtual Distributed Switch Portgroup Teaming and Failover “Network Failover Detection” to be set to “Link Status Only”.  The “CDP” “Enabled” setting allows the vSphere Virtual Distributed Switch “Discovery Protocol” to set set to “CDP”.

14. Configure the “vNIC Templates” by selecting the “Policies” tree and then the “vNIC Templates” object. Refer to Screenshot below.

Example UCS Manager Fabric Interconnect “LAN – vNIC Templates” Screenshot:

 15. Create the “vNIC Templates” “vmnic0-MGT-A”, “vmnic1-MGT-B”, “vmnic2-VM-A” & “vmnic3-VM-B”  with the following settings:

16. Assign the following VLANs to each “vNIC Template”:

17.This completes the LAN Policy configuration, proceed to Part 4 to configure the SAN policies.