Menu
The dark side of server virtualisation

The dark side of server virtualisation

Cisco's answer

Cisco provides its own soft switch solution to replace vSwitch called the 1000V. There are two components to the 1000V. The VSM is the Virtual Switch Module and replaces the vSwitch software running inside the hypervisor. The Virtual Element Manager (VEM) is where the network policies for the VSM are configured and stored.

The graphic below shows how the process works. The VM's VLAN and policies are first configured based on the VM's UUID or vMAC addresses in the VEM. VCenter starts a new VM or moves a VM in Step 2. The Hypervisor informs the VSM in Step 3. The VSM then retrieves the policy information from the VEM in Step 4. If the network switch is from the Nexus product line, it also retrieves the necessary VLAN and policy information from the VEM. At this point both the switch in the hypervisor and the Nexus switch have all the correct information on how to handle VM2. When VM2 starts sending traffic, all the correct policies are applied in the 1000V switch in the hypervisor (blue dot.)

The benefits of Cisco's approach are the same as with the first approach. It blocks any communication between the two VM if it is not allowed and applies the appropriate policies the first time traffic hits a switch. If the 1000V is used with the Nexus switch that is virtualization-ready, it will solve the VLAN problem in the network switch. It has the additional benefit of moving the switch in the hypervisor under the control of the network management software, returning the clear accountability to the network group. There are downsides. Currently Cisco only has a solution for VMware and not for Xen and HyperV.

The fourth way

The fourth approach takes a network equipment centric view; see Figure 3. In Step 1, the VM is defined in the network management software by its virtual NIC. In Step 2, vCenter directs the hypervisor to start up VM. The hypervisor sends out an advertisement packet announcing it is starting VM2, in Step 3. The advertisement has VM2's vNIC and its UUID. In Step 4, the switch sees the advertisement and sends a request for its VLAN and other policy information. The switch then applies the policies to any traffic entering the network.

The key point is the switch only applies policy in the network switch, shown by the blue dot, and not at the vSwitch. The switch also monitors for messages from the hypervisor that indicate the VM has moved and then removes the VLAN and policy information associated with the vNIC. Vendors employing this solution include Arista Networks, Blade and Enterasys, along with HP and Juniper through their orchestration approach. Other vendors such as Brocade planning to offer this solution. Extreme Networks uses this technique for QoS and policy but not for VLANs.

This approach tries to remove the need to involve the virtualization group in enforcing network policies. However, there are still two problems. The first is that the server virtualization and networking groups must still coordinate VLAN numbers. Currently Enterasys has the ability to automatically provide the vSwitch with the VLAN number, with Arista planning to add it shortly.

The biggest problem with this approach is that it does not apply policies at the vSwitch, thus allowing traffic between VMs on the server to bypass ACLs and other security policies. Enterasys and Arista plan to overcome this problem by adding the ability to push the policies down to the vSwitch, shown by A in the diagram.

A future way to overcome this problem is by allowing "hairpin turns" in the switch. With hairpin turns the vSwitch is configured to send all traffic, even VM1 to VM2 traffic, directly to the network switch. The network switch then applies the policies and assigns QoS. The VM1 to VM2 traffic would be sent back to the vSwitch which then delivers it to VM2.

This would make vSwitch a "dumb" switch with only a forwarding role. The problem is that 802.1D, the bridging standard that all Layer 2 switches are based on, does not allow traffic that came from a port to be sent back down the same port the traffic came from. Thus under the current rules the network switch could not return the packet from VM1 addressed to VM2 since it would break this rule. This was added to prevent loops from being formed. The IEEE is working a revision to 802.1D that allows the switch to perform hairpin turns and additional work is underway to standardize the dump switch in the hypervisor. When this becomes widespread it addresses the problem and has the benefit of removing most of the coordination between the network and server group.

Enterasys currently has a workaround that directs the vSwitch to put each VM in a separate VLAN. They select VLAN numbers that are not currently being used to prevent any potential problems. Since the VMs are in different VLANs they cannot communicate with each other. When the packet arrives at the network switch, the switch replaces the VLAN numbers with the real one assigned to the VM making it appear to the network and their destination that they have always been in the correct VLAN.

Conclusion

There is another VLAN configuration problem that the techniques outlined above do not address. When a new VLAN number is assigned to a port, it needs to be connected to all the other ports with that VLAN number. This requires that all the aggregation switches in the path need to have the VLAN defined on them.

For example, let's say VLAN 5 supports an accounts payable application. All the VMs that support the application are located on one top-of-rack switch which has been configured for VLAN 5. For workload reasons, one of the VM's running the accounts payable application is moved to another rack with its own switches in another part of the data center that is reached by way of several aggregation switches.

Preserving the VLAN means that all the intermediate switches need to be configured with VLAN5; if they are not then the VLAN is broken. Currently none of the solutions outlined deal with how to automatically assign the VLAN in the aggregation switch. This means that if a VM can move across a data center, the VLAN number must be preconfigured in all the aggregation and core switches. Additionally it is not likely that this problem will be solved in the immediate future.

The industry has worked out a set of adequate solutions for the port VLAN and policy problems. There is no magic, most require the networking and virtualization groups to coordinate their activities in the short term to make it work smoothly. The hairpin turn is the best long-term solution and the industry is moving toward it. It is important that network managers understand how the various solutions work with the different virtualization vendors they support, as the full range of solutions are not available for all the virtualization solutions currently in the market.

Layland is head of Layland Consulting. He can be reached at robin@layland.com.

Cisco's answer

Cisco provides its own soft switch solution to replace vSwitch called the 1000V. There are two components to the 1000V. The VSM is the Virtual Switch Module and replaces the vSwitch software running inside the hypervisor. The Virtual Element Manager (VEM) is where the network policies for the VSM are configured and stored.

The graphic below shows how the process works. The VM's VLAN and policies are first configured based on the VM's UUID or vMAC addresses in the VEM. VCenter starts a new VM or moves a VM in Step 2. The Hypervisor informs the VSM in Step 3. The VSM then retrieves the policy information from the VEM in Step 4. If the network switch is from the Nexus product line, it also retrieves the necessary VLAN and policy information from the VEM. At this point both the switch in the hypervisor and the Nexus switch have all the correct information on how to handle VM2. When VM2 starts sending traffic, all the correct policies are applied in the 1000V switch in the hypervisor (blue dot.)

The benefits of Cisco's approach are the same as with the first approach. It blocks any communication between the two VM if it is not allowed and applies the appropriate policies the first time traffic hits a switch. If the 1000V is used with the Nexus switch that is virtualization-ready, it will solve the VLAN problem in the network switch. It has the additional benefit of moving the switch in the hypervisor under the control of the network management software, returning the clear accountability to the network group. There are downsides. Currently Cisco only has a solution for VMware and not for Xen and HyperV.

The fourth way

The fourth approach takes a network equipment centric view; see Figure 3. In Step 1, the VM is defined in the network management software by its virtual NIC. In Step 2, vCenter directs the hypervisor to start up VM. The hypervisor sends out an advertisement packet announcing it is starting VM2, in Step 3. The advertisement has VM2's vNIC and its UUID. In Step 4, the switch sees the advertisement and sends a request for its VLAN and other policy information. The switch then applies the policies to any traffic entering the network.

The key point is the switch only applies policy in the network switch, shown by the blue dot, and not at the vSwitch. The switch also monitors for messages from the hypervisor that indicate the VM has moved and then removes the VLAN and policy information associated with the vNIC. Vendors employing this solution include Arista Networks, Blade and Enterasys, along with HP and Juniper through their orchestration approach. Other vendors such as Brocade planning to offer this solution. Extreme Networks uses this technique for QoS and policy but not for VLANs.

This approach tries to remove the need to involve the virtualization group in enforcing network policies. However, there are still two problems. The first is that the server virtualization and networking groups must still coordinate VLAN numbers. Currently Enterasys has the ability to automatically provide the vSwitch with the VLAN number, with Arista planning to add it shortly.

The biggest problem with this approach is that it does not apply policies at the vSwitch, thus allowing traffic between VMs on the server to bypass ACLs and other security policies. Enterasys and Arista plan to overcome this problem by adding the ability to push the policies down to the vSwitch, shown by A in the diagram.

A future way to overcome this problem is by allowing "hairpin turns" in the switch. With hairpin turns the vSwitch is configured to send all traffic, even VM1 to VM2 traffic, directly to the network switch. The network switch then applies the policies and assigns QoS. The VM1 to VM2 traffic would be sent back to the vSwitch which then delivers it to VM2.

This would make vSwitch a "dumb" switch with only a forwarding role. The problem is that 802.1D, the bridging standard that all Layer 2 switches are based on, does not allow traffic that came from a port to be sent back down the same port the traffic came from. Thus under the current rules the network switch could not return the packet from VM1 addressed to VM2 since it would break this rule. This was added to prevent loops from being formed. The IEEE is working a revision to 802.1D that allows the switch to perform hairpin turns and additional work is underway to standardize the dump switch in the hypervisor. When this becomes widespread it addresses the problem and has the benefit of removing most of the coordination between the network and server group.

Enterasys currently has a workaround that directs the vSwitch to put each VM in a separate VLAN. They select VLAN numbers that are not currently being used to prevent any potential problems. Since the VMs are in different VLANs they cannot communicate with each other. When the packet arrives at the network switch, the switch replaces the VLAN numbers with the real one assigned to the VM making it appear to the network and their destination that they have always been in the correct VLAN.

Conclusion

There is another VLAN configuration problem that the techniques outlined above do not address. When a new VLAN number is assigned to a port, it needs to be connected to all the other ports with that VLAN number. This requires that all the aggregation switches in the path need to have the VLAN defined on them.

For example, let's say VLAN 5 supports an accounts payable application. All the VMs that support the application are located on one top-of-rack switch which has been configured for VLAN 5. For workload reasons, one of the VM's running the accounts payable application is moved to another rack with its own switches in another part of the data center that is reached by way of several aggregation switches.

Preserving the VLAN means that all the intermediate switches need to be configured with VLAN5; if they are not then the VLAN is broken. Currently none of the solutions outlined deal with how to automatically assign the VLAN in the aggregation switch. This means that if a VM can move across a data center, the VLAN number must be preconfigured in all the aggregation and core switches. Additionally it is not likely that this problem will be solved in the immediate future.

The industry has worked out a set of adequate solutions for the port VLAN and policy problems. There is no magic, most require the networking and virtualization groups to coordinate their activities in the short term to make it work smoothly. The hairpin turn is the best long-term solution and the industry is moving toward it. It is important that network managers understand how the various solutions work with the different virtualization vendors they support, as the full range of solutions are not available for all the virtualization solutions currently in the market.

Layland is head of Layland Consulting. He can be reached at robin@layland.com.


Follow Us

Join the newsletter!

Or

Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.

Featured

Slideshows

The Kiwi channel gathers for the 2020 Reseller News Women in ICT Awards

The Kiwi channel gathers for the 2020 Reseller News Women in ICT Awards

Hundreds of leaders from the New Zealand IT industry gathered at the Hilton in Auckland on 17 November to celebrate the finest female talent in the Kiwi channel and recognise the winners of the Reseller News Women in ICT Awards (WIICTA) 2020.

The Kiwi channel gathers for the 2020 Reseller News Women in ICT Awards
Leading female front runners honoured at the 2020 Reseller News Women in ICT Awards

Leading female front runners honoured at the 2020 Reseller News Women in ICT Awards

The leading female front runners of the New Zealand ICT industry joined together for the annual Reseller News Women in ICT Awards event at the Hilton in Auckland, during which hundreds of guests celebrated 13 outstanding individuals who won awards, chosen from more than 50 finalists representing over 30 organisations.

Leading female front runners honoured at the 2020 Reseller News Women in ICT Awards
Channel gathers to celebrate the Reseller News Innovation Awards 2020 winners

Channel gathers to celebrate the Reseller News Innovation Awards 2020 winners

More than 500 channel leaders gathered in Auckland on 21 October at the ​Reseller News Innovation Awards ​2020 to celebrate the achievements of the New Zealand technology industry's top partners, start-ups, vendors, distributors and individuals.

Channel gathers to celebrate the Reseller News Innovation Awards 2020 winners
Show Comments