Balancing the end-user experience of Leaseweb hosted services and network economics
At Leaseweb Network, we manage and support the networks and traffic of the various Leaseweb services companies around the world. We also support the advanced networking needs of their customers and protect them from security threats.
To ensure an optimized end-user experience for Leaseweb hosted services, we run our own global Internet backbone – establishing direct connectivity (peering) with eyeball networks in 58 PoPs in Europe, the US and Asia – and we have connectivity (IP Transit) to virtually all major Tier 1 Internet backbones in any of the Leaseweb data centers. Our total network capacity is 5 Tbps – compared to actual traffic levels of about 2.5 Tbps in the first quarter of 2015. This combination of peering and multiple Transit providers gives us a lot of flexibility and peak capacity to reach eyeball networks. So how do we manage this global network?
Cutthroat competition and its impact
Over the years the economics of our Internet traffic have changed significantly, as well as the challenges to find the most attractive route to reach a certain eyeball network. Whereas 10 years ago global networks and peering ports operated at 50-70% of their capacity (leaving sufficient capacity for growth and peaks), we experience today that many large networks run “red hot” and that private peering ports – especially those between global networks and the large eyeball networks – are often congested, resulting in packet loss, high latency and subsequently in a lousy end-user experience, especially for video-centric services and business critical real-time applications.
Price levels and market structures have also changed quite a bit: When I started as a Sales Director for a large US based Internet backbone about 12 years ago, procurement decisions for IP Transit services were taken by the CTO or MD of a company, prices would be well above 20 Euro per Mbps and total traffic costs reflected a big chunk of their operating costs. Today decisions are typically taken by the networking engineering team and the total cost for that traffic is only a fraction of the cost to run an eyeball network.
The shifting power balance and emergence of the Net Neutrality debate
Where both eyeball networks and content providers used to buy Transit from the large global backbones, many large eyeball networks now feel that they are in a position to charge global networks or content providers for receiving traffic requested by their end-customers, effectively turning themselves from a “customer” into a “seller” for wholesale Internet traffic. Not only in Europe and the US does this create new challenges to reach those networks, it also applies to some of the large networks in Asia – where especially reaching Chinese networks has a completely different cost base than some of the other countries. In the western world regulators increasingly step up to protect consumer interests and ensure the Internet remains an open network where all services can be reached by customers under the same conditions (Net Neutrality). For those who want to better understand trends in Net Neutrality, here’s an entertaining explanation:
Balancing the use of peering and IP Transit
In addition to the IP Transit price decreases and shift in power balance, the growth in peering relations and the number of Internet exchanges is a development that has a major impact on the way we route our traffic. It has driven the build-out of our global peering network – supported by a cost model where the cost of peering (Internet exchange ports, colocation) and the capacity between the Leaseweb data centers and those peering locations (typically multiple 10Gbps wavelengths) are lower than procuring IP Transit. The balance however has shifted in the past 5 years, with costs for IP Transit services coming down faster than the cost of international wavelengths and other peering costs, shifting the value of a global peering network from “cost savings” to “better quality”. In this context it’s interesting to see the continued growth in the number of cities where Internet exchanges are being established (especially in the US) and the “battle of the giants” (AMS-IX, DE-CIX and LINX) who continue to open new exchanges in new territories (US, Asia, Caribbean, France, Africa….) and even launch April’s Fools Day jokes about opening new exchanges in each other’s backyard (did you see the AMS-IX and LINX press release on April 1st in which they announce their new Allied-IX in Frankfurt?).
Hardware and cross connects
Some last elements worth mentioning in our cost base are the investment (and associated depreciation) in network equipment and costs for colocation and cross connects. In 2003 we saw an average cost of $20,000 per Gigabit port capacity in our routing equipment. Since then we’ve seen those costs reduce by a factor of ten every six to seven, with similar economics applying to our switching and optical transmission equipment. This facilitates the further development of our global network, since equipment costs are not a main cost driver in our network. The only cost elements which have hardly come down are the cost for Colocation services (and power…) and for cross-connects; in many cases these have only gone up. Especially in the major US data centers the costs of cross-connects are outrageous compared to European cost levels (10x more expensive…). And in Hong Kong, this could even go up to a factor of 15-20…
For us at Leaseweb, these developments are both good and bad news. The good news is that our overall traffic costs keep going down and that we have more and more options to reach eyeball networks with good quality and at competitive rates. The downside obviously is that it’s increasingly challenging to find the best way to reach an eyeball network. To properly serve the various applications supported by the Leaseweb platforms (from cloud storage to HD video), we have created separate bandwidth options for premium and high volume traffic – managed by our Network Design & Operations team and using a set of sophisticated tools to monitor and manage traffic. More about this in my next blog – keep watching this space. And if you’d like to talk about the latest developments around the Leaseweb network, I’ll be at the Leaseweb Tech Summit on 4 June.