Cisco UCS Configuration Guide for vSphere – Part 3 – LAN

vcdx133 This is part 3 of the Cisco UCS Configuration Guide.  This post is broken into two components: configuring the physical Server/FC/LAN links and the creation of UCS LAN policies.

Seven part series, describing how to configure a Cisco UCS from start to finish for a vSphere deployment:

The UCS Manager LAN configuration has five areas: LAN Cloud, Appliances, Internal LAN, Pools and Policies.  The diagram below shows the breakdown of those areas.

UCS_LAN_Overview

Configure the Physical Links

Before you begin, you will need the following:

  • Installed, connected and powered-on Cisco UCS system as described in Part 1 and Part 2.
  • Be connected to the Cisco UCS Manager GUI as described in Part 2.

The first step is to configure the physical links to the UCS Chassis I/O Modules, Core LAN Switches and SAN Switches:

1. Select the “Equipment” tab and expand the “Equipment” tree to “Fabric Interconnects”.

2. Select “Fabric Interconnect A” and check that the “Status” “Ethernet Mode” and “FC Mode” are “End Host”.  If either is set to “Switch Mode”, simply select the “Set Ethernet End-Host Mode” or “Set FC End-Host Mode” links.  Refer to Screenshot below.

Example UCS Manager Fabric Interconnect “General” Tab Screenshot:

UCS_Fabric_Interconnect

3. Check that your Server, LAN and SAN SFP+ modules appear in the “Ethernet” and “FC” ports list.

4. Right mouse click on each port and select “Configure as Uplink Port” for connections to the LAN core, “Configure as FC Uplink Port” for “FC Port” connections to the SAN fabric and “Configure as Server Port” for connections to the UCS Chassis FEX.  Refer to Screenshot below.

Example UCS Manager Fabric Interconnect “Port Configuration” Screenshot:

UCS_Fabric_Interconnect_Port_Configuration
5. Follow steps 2 to 4 for “Fabric Interconnect B”.

6. With the Server Ports being configured, the UCS Chassis will be discovered and appear in the UCS Manager.  Each Server will have to be “Acknowledged”, just browse to each chassis and select each server highlighted in RED and click the “Acknowledge” button.

7. Once the LAN “Uplink Ports” are enabled, launch the “LAN Uplinks Manager” and select the “Create PortChannel” button to configure PortChannel to the Core LAN Switches.  Refer to Screenshot below.  Each PortChannel configuration will have a total of 2 (1+1), 4 (2+2) or 8 (4+4) uplinks from Fabric Interconnect A and Fabric Interconnect B.

Example UCS Manager Fabric Interconnect “LAN Uplinks Manager” Screenshot:

UCS_LAN_Uplinks_Manager
8. You should now have a UCS Manager screen with “Green” activated, error free links from the Fabric Interconnect to the LAN Core, SAN Fabric and UCS Chassis FEX modules.

9. The UCS Chassis should also be displayed under the “Equipment” tab in the “Chassis” list.  You will have a number of alarms present, ignore them for the time being, they will be resolved as you progress through the configuration process.

Configure LAN Policies

Before you begin, you will need the following:

  • QoS System Class – for this exercise Best-Effort and Fiber-Channel will be used.
  • Block of IP Addresses in the same subnet as UCS Management, enough for blade server KVM – for this exercise the range 10.1.1.10 – 10.1.1.20 will be used.
  • Two Blocks of MAC Addresses for the virtual NICs, one for Fabric A and one for Fabric B – for this exercise the ranges 00:25:B5:32:00:00 – 00:25:B5:32:FF:FF & 00:25:B5:42:00:00 –  00:25:B5:42:FF:FF will be used.
  • VLANs that will be tagged over the PortChannel interface – for this exercise VLANs 101, 102 and 103 will be used.

To configure the Cisco UCS LAN Policies, you need to:

1. Access the “LAN” tab of the UCS Manager.  Refer to Screenshot below.

Example UCS Manager Fabric Interconnect “LAN” Tab Screenshot:

UCS_LAN_Tab

2. Configure the “QoS System Class”, by selecting the “LAN Cloud” tree and then the “QoS System Class” object.  Refer to Screenshot below.

Example UCS Manager Fabric Interconnect “LAN – QoS System Class” Screenshot:

UCS_LAN_QoS_System_Class

3. Configure the “QoS System Class” settings as per the table below:

UCS_QoS_System_Class_Config

4. If using the Cisco Nexus family of switches, make sure the following matching QoS policy is configured if you are using Jumbo frames.  You will not have consistent end-to-end MTU without this setting.  End-to-End MTU consistency can be tested from ESXi with the “vmkping -d -s 9000 <IP Address>” command.  Ignore this step if Standard MTU sizes (1500) are used.

policy-map type network-qos jumbo

  class type network-qos class-default

    mtu 9216

system qos

  service-policy type network-qos jumbo

5. Configure the IP and MAC pools, by selecting the “Pools” tree and then the “IP Pools” object.  Refer to Screenshot below.

Example UCS Manager Fabric Interconnect “LAN – Pools” Screenshot:

UCS_LAN_Pools

6. Modify the “IP Pool ext-mgmt” and “iscsi-initiator-pool” with the following settings.  iSCSI is not required, but this configuration will clear the iSCSI alarm that appears.

UCS_LAN_IP_Pool_ext-mgmt

7. Create the “MAC Pools” “UCSFAB-A-MAC” and “UCSFAB-B-MAC” with the following settings.

UCS_LAN_MAC_Pools

8. Configure the “QoS Policies”, by selecting the “Policies” tree and then the “QoS Policies” object. Refer to Screenshot below.

Example UCS Manager Fabric Interconnect “LAN – QoS Policy” Screenshot:

UCS_LAN_QoS_Policy

9. Create the “Fiber-Channel”, “VM-Traffic” and “vSphere-Management” QoS Policies with the following settings.  The “Host Control” “none” setting means that vSphere Virtual Distributed Switch Network I/O Control “QoS tags” will not be used.

UCS_LAN_QoS_Policy_Config

10. Configure the “VLANs”, by selecting the “LAN Cloud” tree and then the “VLANs” object. Refer to Screenshot below.

Example UCS Manager Fabric Interconnect “LAN – VLANs” Screenshot:

UCS_LAN_VLANs

11. Create the “VLANs” “VLAN101”, “VLAN102” and “VLAN103” with the following settings:

UCS_LAN_VLANs_Config

12. Configure the “Network Control Policy”, by selecting the “Policies” tree and then the “Network Control Policy” object. Refer to Screenshot below.

Example UCS Manager Fabric Interconnect “LAN – Network Control Policy” Screenshot:

UCS_LAN_Network_Control_Policy

13. Create the “Network Control Policy” “ESXi”  with the following settings.  The “Action on Uplink Fail” with “Link Down” allows the vSphere Virtual Distributed Switch Portgroup Teaming and Failover “Network Failover Detection” to be set to “Link Status Only”.  The “CDP” “Enabled” setting allows the vSphere Virtual Distributed Switch “Discovery Protocol” to set set to “CDP”.

UCS_LAN_Network_Control_Policy_Config

14. Configure the “vNIC Templates” by selecting the “Policies” tree and then the “vNIC Templates” object. Refer to Screenshot below.

Example UCS Manager Fabric Interconnect “LAN – vNIC Templates” Screenshot:

UCS_LAN_vNIC_Templates

 15. Create the “vNIC Templates” “vmnic0-MGT-A”, “vmnic1-MGT-B”, “vmnic2-VM-A” & “vmnic3-VM-B”  with the following settings:

UCS_LAN_vNIC_Templates_Config

16. Assign the following VLANs to each “vNIC Template”:

UCS_LAN_vNIC_Templates_VLANs_Config

17.This completes the LAN Policy configuration, proceed to Part 4 to configure the SAN policies.

3 thoughts on “Cisco UCS Configuration Guide for vSphere – Part 3 – LAN

  1. Pingback: Cisco UCS Configuration Guide for vSphere – Part 3 – LAN «

  2. Hello,

    I have 3 servers with ESXi 5.5 and I want to create 2 vlans for storage propose but I don’t have blade servers.

    Do you know if its possible configure only vlans for use UCS Fabric Interconnects 6100 40XP to connect different vnics for use vMotion and vSan without blades? My networks cards are Intel Adapter X710-DA4

    In case that I can, can you explain me how to do it, please?

    Thanks

    • Your statement: “I do not have blade servers”, means you have C-series rack mounted servers? If yes, it is not possible unless you have the PCI-e VIC Adapters for the C series rack mounted servers, these adapters contain the ASIC functionality for vHBAs and vNICs.

      What you are describing, sounds like stand-alone UCS C-series servers that cannot be connected to the UCS Fabric Interconnects (need VICs for that with the 10GbE Coax cables to connect the C-series to the Fabric Interconnect).

      My advice, get Cisco Support on the line to assist you in understanding the capabilities of the hardware you have and how to configure it.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s