10GE (10 Gigabit Ethernet) networks are commonplace in service provider, cloud and datacentre environments. Whilst there is some adoption in the enterprise, many of the solutions can be cost prohibitive or they tend to be all-or-nothing solutions. The purpose of this post is to highlight some ways you can increase network throughput within your organisation. A 10GE network will increase the performance of your storage environment (e.g. ISCSI), your virtualisation platform (e.g. Hyper-V, VMWare, Citrix) and improve the performance of your overall enterprise network.
Steps to higher performance
#1 – Bonded 1Gbps Links (LAG)
- Switch : 1GE switch. Huawei S5700.
- Redundancy : 2 x Switches in a stack.
If your current server environment uses 1Gbps network cards then the first logical step to improve performance is to bond these links together. Both Intel and Broadcom server-class network cards support some form of interface aggregation, also known as a Link Aggregation Group (LAG). This can increase performance by combining multiple 1Gbps links into bundles. This does not necessarily translate to 2 x 1Gbps or 3 x 1Gbps. More about this later.
#2 – Servers/SANs have 10Gbps Links
- Switch : 1GE switch with 10GE ports. Huawei S5700
- Redundancy : 2 x Switches in a stack
If you require a low density of 10GE ports, then the simplest solution is a Gigabit switch with 10GE uplinks (typically 2 or 4 per switch). If you combined these into a stack (consistent of 2 switches), then this can provide 4 or 8 10GE ports – sufficient for an SME. Two ports can be connected to your SAN, and the remaining ports to your VMWare/Hyper-V/Citrix host servers.
#3 – Multiple servers, 10Gbps Links
- Switch : 10GE switch. Huawei S6700
- Redundancy : 2 x Switches or 1GE fallback
For denser environments with several 10GE servers or SAN controllers, a larger number of 10GE ports are required. In these cases you should consider a native 10GE switch such as the Huawei S6700. Similarly to the previous examples, you should consider two switches in a stack for redundancy.
What about the servers?
If you servers have 1Gbps cards, then it is easy to upgrade them to 10Gbps. Both Intel and Broadcom sell 10Gbps NICs, additional server vendors such as Dell/HP/IBM sell their own branded models. It is a straight forward installation and straight away your server has 10 x throughput.
The downsides of LAGs?
The issue with LAGs is more to do with the way Ethernet networks operate. Take the Broadcom NetXtreme NICs as an example, they can operate in LAGs (or Teams) in the following modes:
When MTG are used to install a new network, replace an existing network or to troubleshoot a problematic network, the most common things we see are:
- Smart Load Balancing (SLB). This is often the most common configuration. This can actually cause something called “MAC Flapping” in the network where switches see constant L2 address changes and begin to log errors. The important thing about SLB is it does not simply translate to a multiple of 1GE (2 x 1GE != 2GE). SLB works by configuring it on the server but not configuration on the switch is require, hence the inconsistent results.
- SLB – Failover mode. The NICs simply operate in Active:Passive mode. If one fails, the other takes over. This assumes you have multiple access switches connected to the server, and not just to the same switch.
- Link Aggregation (802.3ad) – This is the most graceful and intelligent mode of operation. The NIC and switches are configured to use LACP/802.3ad which provides a mutual method of load-sharing traffic to known algorithms. This is standards based and the best solution. *Recommended*
- Generic Trunking – Similar to 802.3ad this uses a draft derivative, we would not use this mode.
Switch to 10GE Today
MTG can design, implement and support a variety of enterprise network environments. Working with vendors such as Huawei, Cisco, Juniper and Brocade, we can optimise existing environments or replace them with a more robust, modern solution. Servers, SANs and virtualised environments are moving towards 10GE.
- Increase the performance and IOPs of your existing Storage Environment (SAN). A 10GE network should be a standard purchase alongside any modern SAN.
- Optimise the VDI experience with a high performance 10GE network core.
- Increase capacity in your network core; aggregate your 1GE access switches back to a redundant 10GE core network.
- Provide an ultra-high capacity backplane for your server and storage environments. Virtualised server loads can operate congestion free on a high speed 10Gbps network.