NSX-v – Design Deep Dive

This is the VMware NSX for vSphere Design Deep Dive.  I have aggregated all of the design considerations I could find that need to be assessed in a VMware NSX-v architecture design.  Brevity and bullet-points are used to keep the information concise and readable.  If you want more information on a concept use the Additional Resources section at the end.

This post will be updated with additional information as part of the NSX Link-O-Rama.  If you have content to contribute, post a comment below.

I have separated the design decisions into the areas specified by the VCDX-NV blueprint.

Requirements/Constraints

  • Physical network has the design qualities of simplicity, performance, scalability and reliability
  • Physical network must be configured for 1600 MTU or larger (for additional VXLAN header size)
  • Physical network should support Layer 2 or Layer 3 Multi-Cast (for Hybrid or Multi-Cast VXLAN Replication)
  • VMware vCenter Server 5.5 or later
  • VMware vSphere ESXi 5.1 or later (5.1 requires Multi-Cast VXLAN Replication)
  • VMware Tools in all supported VMs for Endpoint and Data Security
  • Limitation of Scale – 1 vCenter to 1 NSX Manager, Multi vCenter/NSX Manager not supported at this time
  • Limitation of Scale – Configuration Maximums: 12 Clusters per NSX Manager
  • NSX Manager does not support High Availability – have to rely upon vSphere HA
  • Dedicated Layer 2 VLAN for NSX Management (single broadcast domain between NSX Management components)
  • Single Logical Interface (LIF) per Logical Switch or VLAN
  • Cannot route on Layer 2 bridged interfaces
  • VLAN LIFs can only be on one VDS

Assumptions

  • You have a set of business requirements that has led to the selection of VMware NSX-v (Conceptual Model and Logical Design) and now you are ready for the Physical Design.

Physical Design Decisions

A. NSX Infrastructure Components

  1. NSX-v version? Understand the feature disparity between 6.0 and 6.1, eg. ECMP, L2VPN.
  2. Cluster Design: Separate or combined Compute, Management, Edge Clusters?
  3. Management Cluster: Dedicated or shared vCenter (with NSX Manager)?
  4. Logical Routing Deployment: Physical Router as Next Hop (1 Tier) or ESG as Next Hop (2 Tier)?
  5. NSX Controllers: How many deployed? For Availability or Performance?
  6. vSphere DRS VM Anti-affinity rules for NSX Controllers: yes or no? (ESG anti-affinity rules created automatically)
  7. vSphere HA VM Restart priority for NSX Manager, NSX Controllers, ESGs, DLR Controllers: which priority?
  8. Datastores for NSX Manager, Controllers, DLR Control VMs and ESGs?
  9. NSX Edge Form Factors: Compact to X-Large
  10. NSX Edge Availability: Standalone, Active/Standby or ECMP?
  11. Understanding of Traffic Flows: North/South and East/West
  12. NTP Master?
  13. Impact of NSX-v Configuration Maximums
  14. Naming Standard of NSX components?
  15. Capacity Planning: how will your design handle future expansion with respect to configuration maximums and current limitations?
  16. Future Proofing: how will your design adapt to new features such as NSX Federation?
  17. Management/Monitoring/Troubleshooting: SNMP, RSPAN, Syslog, NetFlow, API integration (including vROps NSX Plugin and LogInsight NSX Plugin)
  18. Backup/Recovery of NSX infrastructure
  19. Multi-site BC/DR considerations

B. Virtual Networking

  1. NSX Controller Address Assignment: Dedicated IP Pool?
  2. VTEP Address Assignment: Manual (after DHCP timeout), IP Pools or DHCP (required if you have Per Rack VLANs – with IP Helper per ToR switch to fix the Layer 2 boundary issue)
  3. Routing of vMotion and IP Storage: Yes or no?
  4. IP Addressing Schema: NSX Manager, NSX Controllers, DLR, ESG, VMkernel (Storage, vMotion, Management, VTEP)
  5. Logical Switches: quantity and function?
  6. Interior Routing Protocol: Static, BGP or OSPF
  7. Exterior Routing Protocol: Static, BGP, OSPF or IS-IS
  8. VXLAN Transport Zones: Single Global or multiple?
  9. VXLAN Replication mode: Unicast, Hybrid or MultiCast? (Ease of deployment versus performance and scale)
  10. VXLAN: Segment ID/VNI Pool Range?
  11. VTEPs per Host: Single or Multiple? (this depends upon the VDS Teaming and Failover design)
  12. Distributed Logical Router: VXLAN-VLAN Bridging?
  13. Number of VDS: One or more?
  14. VDS Uplink Teaming & Failover: Originating Port, Source MAC Hash, LACP, IP Hash or Explicit Failover? (LBT not supported)
  15. VDS Network I/O Control: Configured?
  16. VDS Traffic Shaping: Configured?
  17. ESG Load Balancing: One-armed or Inline?

C. Physical Networking

  1. Physical Network: Traditional 3 tier (Core, Aggregation, Access), Collapsed Core or Leaf and Spine?
  2. Traditional 3 Tier: Spanning Tree, LAG?
  3. QoS: end-to-end or edge?  Integrated with Network I/O Control?
  4. Multi Rack redundancy: Span Management and Edge Clusters across 3 racks?
  5. Top of Rack Switch redundancy: One or two ToR switches?
  6. Host Design: Specific hardware specification for Management and Edge Clusters?
  7. Per Rack VLANs: Yes or no? (VLANs not extended between racks; different subnets per rack with leaf and spine switched network, requires VMware Request for Qualification)
  8. Physical network MTU size: 1600 MTU or larger?
  9. Physical network Multicast: Layer 2 or Layer 3 Multi-Cast? (not required for Unicast)
  10. External Network/Transport VLANs: How many?

D. NSX Security

  1. Endpoint and Data Security: AntiVirus and File Scanning polices? (Windows OS supported, Linux is not)
  2. Edge Services being used: Firewall, NAT, DHCP Server/Relay, Routing Protocols, IPSec/SSL VPNs and L2 VPN?
  3. L3-L7 features: NSX Native or 3rd party?
  4. NSX Policy Repository: Via NSX API or local?
  5. Distributed Firewall Micro-segmentation: Yes or no?
  6. ESG Failover states: Impact to Firewall & NAT, DHCP Server, Routing Protocols, IPSec/SSL VPNs and L2 VPN?
  7. Advanced Security: Insertion, Chaining and Steering?
  8. RBAC?

E. NSX – CMP Integration

  1. Cloud Management Platform integration: vRA or something else?
  2. Multi-Tenant: Single ESG per tenant with how many Logical Switches?
  3. Network topologies for Blueprints: External, Routed, Private, NAT
  4. Security policies for Blueprints
  5. L4-L7 services for Blueprints

Additional Resources

4 thoughts on “NSX-v – Design Deep Dive

  1. Pingback: Designing for VSAN & NSX Integration – The Wifi-Cable

    • Certainly is – you will have to read the release notes for 6.3.3 to address the new features that have appeared since this was written, eg. Cross-vCenter NSX.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s