The Utility and Pain of Internet Peering

In the early 1990s TWICS, a commercial bulletin board service provider in Tokyo, jumped on the Internet. Access was very poor based on modern Internet speeds, however at the time 128kbps over frame relay (provided by Sprint international) was unique, and in fact represented the first truly commercial Internet access point in Japan.

The good old boys of the Japanese academic community were appalled, and did everything in their power to intimidate TWICS into disconnecting their connection, to the point of sending envelopes filled with razor blades to TWICS staff and the late Roger Boisvert (*), who through Intercon International KK acted as their project manager. The traditional academic community did not believe anybody outside of the academic community should ever have the right to access the Internet, and were determined to never let that happen in Japan.

Since the beginning, the Internet has been a dichotomy of those who wish to control or profit from the Internet, and those who envision potential and future of the Internet. Internet “peering” originally came about when academic networks needed to interconnect their own “Internets” to allow interchange of traffic and information between separately operated and managed networks. In the Internet academic “stone age” of the NSFNet, peering was a normal and required method of participating in the community. But,… if you were planning to send any level of public or commercial traffic through the network you would violate the NSFNET’s “acceptable use policy/AUP” preventing use of publically-funded networks for non-academic or government use.

Commercial internet Exchange Points such as the CIX, and eventually the NSF supported network access points/NAPs popped up to accommodate the growing interest in public access and commercial Internet. Face it, if you went through university or the military with access to the Internet or Milnet, and then jumped into the commercial world, it would be pretty difficult to give up the obvious power of interconnected networks bringing you close to nearly every point on the globe.

The Tier 1 Subsidy

To help privatize the untenable growth of the NSFNet (due to “utility” academic network access), the US Government helped pump up American telecom carriers such as Sprint, AT&T, and MCI by handing out contracts to take over control and management of the world’s largest Internet networks, which included the NSFNet and the NSF’s international Connection Managers bringing the international community into the NSFNet backbone.

This allowed Sprint, AT&T, and MCI to gain visibility into the entire Internet community of the day, as well as take advantage of their own national fiber/transmission networks to continue building up the NSFNet community on long term contracts. With that infrastructure in place, those networks were clear leaders in the development of large commercial internet networks. The Tier 1 Internet provider community is born.

Interconnection and Peering in the Rest of the World

In the Internet world Tier1 networks are required (today…), as they “see” and connect with all other available routes to individual networks and content providers scattered around the world. Millions and millions of them. The Tier 1 networks are also generally facility-based network providers (they own and operate metro and long distance fiber optic infrastructure) which in addition to offering a global directory for users and content to find each other, but also allows traffic to transit their network on a global or continental scale.

Thus a web hosting company based in San Diego can eventually provide content to a user located in Jakarta, with a larger network maintaining the Internet “directory” and long distance transmission capacity to make the connection either directly or with another interconnected network located in the “distant end” country.

Of course, if you are a content provider, local internet access provider, regional network, or global second tier network, this makes you somewhat dependant on one or more “Tier 1s” to make the connection. That, as in all supply/demand relationships, may get expensive depending on the nature of your business relationship with the “transit” network provider.

Thus, content providers and smaller networks (something less than a Tier 1 network) try to find places to interconnect that will allow them to “peer” with other networks and content providers, and wherever possible avoid the expense of relying on a larger network to make the connection. Internet “Peering.”

Peering Defined (Wikipedia)

Peering is a voluntary interconnection of administratively separate Internet
networks for the purpose of exchanging traffic between the customers of each network. The pure definition of peering is settlement-free or “sender keeps all,” meaning that neither party pays the other for the exchanged traffic; instead, each derives revenue from its own customers. Marketing and commercial pressures have led to the word peering routinely being used when there is some settlement involved, even though that is not the accurate technical use of the word. The phrase “settlement-free peering” is sometimes used to reflect this reality and unambiguously describe the pure cost-free peering situation.

That is a very “friendly” definition of peering. In reality, peering has become a very complicated process, with a constant struggle between the need to increase efficiency and performance on networks, to gaining business advantage over competition.

Bill Norton, long time Internet personality and evangelist has a new web site called “DR Peering,” which is dedicated to helping Internet engineers and managers sift through the maze of relationships and complications surrounding Internet peering. Not only the business of peering, but also in many cases the psychology of peering.

Peering Realities

In a perfect world peering allows networks to interconnect, reducing the number of transit “hops” along the route from points “A” to “B,” where either side may represent users, networks, applications, content, telephony, or anything else that can be chopped up into packets, 1s and 0s, and sent over a network, giving those end points the best possible performance.

Dr Peering provides an “Intro to Peering 101~204,” reference materials, blogs, and even advice columns on the topic of peering. Bill helps “newbies” understand the best ways to peer, the finances and business of peering, and the difficulties newbies will encounter on the route to a better environment for their customers.

And once you have navigated the peering scene, you realize we are back to the world of who wants to control, and who wants to provide vision. While on one level peering is determined by which vendor provides the best booze and most exciting party at a NANOG “Beer and Gear” or after party, there is another level you have to deal with as the Tier 1s, Tier 1 “wanna-be networks,” and global content providers jockey for dominance in their defined environment.

At that point it becomes a game, where personalities often take precedence over business requirements, and the ultimate loser will be the end user.

Another reality. Large networks would like to eliminate smaller networks wherever possible, as well as control content within their networks. Understandable, it is a natural business objective to gain advantage in your market and increase profits by rubbing out your competition. In the Internet world that means a small access network, or content provider, will budget their cost of global “eyeball or content” access based on the availability of peering within their community.

The greater the peering opportunity, the greater the potential of reducing operational expenses. Less peering, more power to the larger Tier 1 or regional networks, and eventually the law of supply and demand will result in the big networks increasing their pricing, diluting the supply of peers, and increasing operational expenses. Today transit pricing for small networks and content providers is on a downswing, but only because competition is fierce in the network and peering community supported by exchanges such as PAIX, LINX, AMS-IX, Equinix, DE-CIX, and Any2.

At the most basic level, eyeballs (users) need content, and content has no value without users. As the Internet becomes an essential component of everybody on the planet’s life, and in fact becomes (as the US Government has stated) a “basic right of every citizen,” then the existing struggle for internet control and dominance among individual players becomes a hindrance or roadblock in the development of network access and compute/storage capacity as a utility.

The large networks want to act as a value-added service, rather than a basic utility, forcing network-enabled content into a tiered, premium, or controlled commodity. Thus the network neutrality debates and controversy surrounding freedom of access to applications and content.

This Does Not Help the Right to Broadband and Content

There are analogies provided for just about everything. Carr builds a great analogy between cloud computing and the electrical grid in his book the “Big Switch.” The Internet itself is often referred to as the “Information Highway.” The marriage of cloud computing and broadband access can be referred to as the “4th Utility.”

Internet protocols and technologies have become, and will continue to be reinforced as a part of the future every person on our planet will engage over the next generations. This is the time we should be laying serious infrastructure pipe, and not worrying about whose content should be preferred, settlements between networks, and who gives the best beer head at a NANOG party.

At this point in the global development of Internet infrastructure, much of the debate surrounding peering – paid or unpaid, amounts to noise. It is simply retarding the development of global Internet infrastructure, and may eventually prevent the velocity of innovation in all things Internet the world craves to bring us into a new generation of many-to-many and individual communications.

The Road Ahead

All is not lost. There are visionaries such as Hunter Newby aggressively pushing development of infrastructure to “address America’s need to eliminate obstacles for broadband access, wireless backhaul and lower latency through new, next generation long haul dark fiber construction with sound principles and an open access philosophy.”

Oddly, as a lifelong “anti-establishment” evangelist, I tend to think we need better controls by government over the future of Internet and Internet vision. Not by the extreme right wing nuts who want to ensure the Internet is monitored, regulated, and restricted to those who meet their niche religions or political cults, but rather on the level of pushing an agenda to build infrastructure as a utility with sufficient capacity to meet all future needs.

The government should subsidize research and development, and push deployment of infrastructure much as the Interstate Highway System and electrical and water utilities. You will have to pay for the utility, but you will – as a user – not be held hostage to the utility. And have competition on utility access.

In the Internet world, we will only meet our objectives if peering is made a necessary requirement, and is a planned utility at each potential geographic or logical interconnection point. In some countries such as Mongolia, an ISP must connect to the Mongolia Internet Exchange as a requirement of receiving an ISP license. Why? Mongolia needs both high performance access to the global Internet – as well as high performance access to national resources. It makes a lot of sense. Why give an American, Chinese, or Singaporean money to send an email from one Mongolian user to another Mongolian user (while in the same country)? Peering is an essential component of a healthy Internet.

The same applies to Los Angeles, Chicago, Omaha, or any other location where there is proximity between the content and user, or user and user. And peering as close to the end users as technically possible supports all the performance and economic benefits needed to support a schoolhouse in Baudette (Minn), without placing an undue financial burden on the local access provider based on predatory network or peering policies mandated by regional or Tier 1 networks.

We’ve come a long way, but are still taking baby steps in the evolution of the Internet. Let’s move ahead with a passion and vision.

(*)  Roger Boisvert was a friend for many years, both during my tensure as  US Air Force officer and telecom manager with Sprint based in Tokyo (I met him while he was still with McKinsey and a leader in the Tokyo PC User’s Group), and afterwards through different companies, groups, functions, and conferences in Japan and the US.  Roger was murdered in Los Angeles nine years ago, and is a true loss to the internet community, not only in Japan but throughout the world.

Future Visions of Global Telecoms with Bjarni Thorvardarson and Hibernia Atlantic

Bjarni Thorvardarson is a rare telecom visionary. He thinks on a level of telecoms at an intercontinental level, rather than a national or local level. Hibernia Atlantic is his current project, and with recent news the submarine and terrestrial cable system is now in the global media distribution business, he is shaking up the telecom community. An Icelandic native, he has brought his knowledge and skills to the United States, basing Hibernia Atlantic in Summit, New Jersey.

Pacific-Tier Communications series on Entrepreneurs and Thought Leaders continues with Bjarni Thorvardarson, CEO of Hibernia Atlantic (www.hiberniaatlantic.com)

Pacific-Tier: Bjarni, what’s been happening with Hibernia Atlantic in the past few years, since I had my last opportunity to visit with you in Summit?

Bjarni Thorvardarson CEO Hibernia AtlanticBjarni Thorvardarson:
We’ve actually had a busy couple years – a very busy couple of years.

As you may recall, we started this business by buying a submarine asset that was formerly owned by 360 Networks. We’ve been busy trying to build our terrestrial network to try and connect this submarine cable to anywhere. We no longer refer to ourselves as simply a submarine cable, but rather a capacity provider in the eastern North America region and in Europe. Less than half of our business is now in trans-Atlantic capacity.

Even though that remains our core competence, and the core of our business, in terms of our business it is less than half of our revenue.

So that is part of the transformation that has happened over the past few years.

In terms of revenue, in 2005 we generated about $2 million in revenue, then $7 million, the $18 million the following year, then $28 million, and last year we generated about $38 million dollars.

And now the target for 2010, with our addition of MediaXstream, is about $75 million dollars in revenue. So, we’ve been growing about 40~50&, up to 80 or 90% a year. So you see it’s a very rapid growth. We are riding on a couple things. One is that we are operating in the biggest capacity market in the world, which is the Northeastern part of North America and Europe. And we are focused on a niche sector, which is the big bandwidth market – which is by itself growing about 40% in volume terms a year

And now, since we still have a relatively small market share, we are growing even faster than the other (traditional) markets (players). That’s how we’ve been successfully growing our revenues.

Last year, in 2009, we were confident for the first time, and we were profitable. We are very happy with the way things are going.

Pacific-Tier: That’s excellent! Can you tell me a little about yourself, and how you got into the business of both submarine and terrestrial bandwidth and capacity?

Bjarni Thorvardarson:
Sure. I’ve been in the IT and telco business for about 17 years since I finished my business studies. By education I have a degree in engineering from the University of Wisconsin (At Madison). Later on I added a science degree from the London Business School.

I went into the telecommunications business, and from there into investment banking (around 1998). Then shortly thereafter I started a fund that was investing in telecommunications and IT. That was 1999 into 2001. From there Ken Peterson actually got interested, Ken Peterson was the owner of Columbia Ventures (CVC), got into an investment in telecommunications. He brought a co-investor in with me, and bought the fund eventually. That’s how it came about that I started working for CVC.

And I’ve been investing on their behalf in telecommunications since 2002.

Now one of the investments that we had made was indeed Hibernia Atlantic. That was 2003. Then in 2004 we sold an investment that I was responsible for here in Ireland. Then I took over the responsibility of Hibernia Atlantic. Since 2005 I have been with Hibernia Atlantic, running it for CVC.

That’s how it came about that I’ve been investing and spending my time in telecommunications.

When it came that I was to take on the Hibernia Atlantic project, it was meant to be a 6 month project. We’d see what we could do, fix a couple things, and recruit the right people. It’s one of those things that a 6 month window turned into a sliding 6 month window. And during the next 6 months we ended doing something exciting. That’s what happens when you get interested in what you are doing.

You see the potentiality, you see what can be done, and you kind of stick around – and so its 5 years now. I no longer refer to it as my 6 month project, I now refer to it as my passion. It’s what I do. I’ve been commuting between Ireland and the US now for the better part of 5 years, and relocated to New Jersey where we have the Summit headquarters, or US headquarters a few years ago. So I am pretty heavily enrolled in the Hibernia project.

Pacific-Tier: That’s good. You mentioned earlier the topic of moving from telephony to broadband. Where does Hibernia and your plans fit into what I would call the “globalization of communications,” or the “flattening of the communications architecture…” How do you fit into that model?

Bjarni Thorvardarson:
Hibernia is very much a long-haul provider. We started in the long-haul wholesale capacity business, so provided the infrastructure provider to other service providers – the likes of BT and France Telecom, Cogent and the like… We did a very good business connecting the biggest consumer markets to places like New York, London, Amsterdam, Chicago, connecting those markets and enabling service providers in those markets to connect to different parts of the world.

Our business is really predicated, it is built on the globalization of business, or the globalization and international movement of information. That was the core part of our business model.

Now since then we have moved on to going up the value chain (if you like) to become the service provider to enterprises ourselves, and begin focusing on the finance vertical, which is a very demanding market. They (financial markets) are demanding and expecting low latency circuits between different trading markets and centers. That was a big first step into the enterprise world.

The next step we took was to the media sector, which we did first when we were acquainted with or partnered up with MediaXtreme, investing in MediaXstream a couple years ago. Then finally culminated in the acquisition of MediaXstream last month. So that’s our big step into the media market.

So now Hibernia’s approach to the market is threefold:

  1. We are still very much the legacy we started, which is the wholesale provider to other service providers and telcos around the world
  2. Second is the finance sector
  3. Third is the media sector

But they all are very much relying on the globalization of business and people’s general view of the world. So we have to look and depend on it.

Pacific-Tier: So in a traditional sense submarine and terrestrial long-haul networks relied on SONET or SDH technologies as the basic (communications) protocol. Is Ethernet playing a stronger role in anything Hibernia is doing now?

Bjarni Thorvardarson:
We started providing waves (2.5 or 10 Gigabit) via SDH and SONET as a product to the wholesale sector. For technical reasons including that was the technology Hibernia was built on. And also that was the product the wholesale providers relied on. They need to connect their different POPs (Point of Presence) equipment. A POP in New York, to a POP in London, that equipment relied on and called for SDH/SONET to connect the POPs.

Now as we grow into the enterprise sector, then the guys, the traders, or whoever we are doing business with – they don’t have SONET or SDH equipment. They have Ethernet equipment or equipment that calls for Ethernet protocols. So it is incumbent on us to be able to provide that without cumbersome translation from one protocol to another.

So we have since built Ethernet at the core of our protocols. Now we can offer Ethernet over SONET, which is dedicated Ether net point-to-point. And we also built, using H3C equipment, a product that we can connect customer to and point-to-point to multipoint capacity.

So moving from the SONET/SDH world to the Ethernet world, or switched Ethernet is very much what we are doing. I am right with you there that we are phasing out one world and moving to another one. Even the telco providers are increasingly moving into the Ethernet world. Especially when it comes to building out their ISP or Internet networks.

Pacific-Tier: When you see organizations like the Carrier Ethernet Neutral Exchange (www.cenx.com) and things like that popping up that are basically designing their product on the old bilateral telecommunication company design,… Do you believe that bilateral Ethernet, or that bilateral carrier relationships still have a role, or will companies like Hibernia make many of those old relationships irrelevant?

Bjarni Thorvardarson:
Hibernia, in its traditional sense, is not going to replace bilateral agreements. But bilateral agreements are going to be phased out when it comes to the exchange of Internet traffic, because exchanges are going to replace them. It is like the minutes (telephone settlement) business extremely cumbersome. If you want to build a bilateral relationship with other telecommunications providers you want to exchange traffic with through some of the voice exchanges you can do business in a matter of days.

And that is the same with the exchange of Internet traffic. If you want to do peering on a bilateral basis with companies it takes you years to build up. If you want to do it through an intermediary (such as a public internet exchange Point), clearly it is moving from the bilateral agreements to the exchanges.

Now how does that related to the price a carrier has to pay when going through the exchanges? To transit pricing? Or what have you?

And we can see where these intermediaries are actually charging less and less for the service of being in-between the delivery of the data and the content origination. You can see that in transit pricing, and how transit pricing is continuing to plummet. So I think that we are becoming less reliant on the bilateral agreement. And I firmly believe the opportunity and the necessity of getting more exchanges up and running is important.

And I think the same transition, you can see the same transition when it comes to not only Internet peering, Internet traffic is also the interconnection of Ethernet circuits, the same transition occurred that we saw 50, 60, 70 years ago when it came to voice traffic. If you wanted to make a call from London to New York you had to call an operator in London, and he made a physical cross connect to a long-distance line that terminated in New York.

The operator in New York then made the physical transition to the local tail line to the customer in New York.

That’s very much the same as when you are setting up an Ethernet circuit today. You have to build up a physical cross-connect in New York between the local tail provider and the Hibernia facility, and then in London to the tail provider over there.

With INNs and with proper Ethernet virtual cross-connects which are relying on a virtual exchange, or like exchanges that you are referring to, it’s going to revolutionize the provisioning and setup time of these Ethernet circuits. We can see a leap in that direction over the next couple years.

Pacific-Tier: One other thing I’d like to ask about Hibernia’s role in the Internet and international community in particular. I’ve been spending a lot of time in developing countries myself over the past year or so. So does Hibernia play a larger role, more than just an economic role,.. Do you also have a larger role in supporting the global community to provide a product that will bring global communications, education, entertainment, media, – can you bring that type of thing to another level?

Bjarni Thorvardarson:
I think that when you have a major company, a large company on the global economic scale, then you have to sit back an think about what your global social responsibility is to the world, and what you can contribute to the world. Hibernia is light years away from being at that size, and we can best fulfill our role now by looking for what is our economical role in this world.

Today that role is to provide large scale, high interconnectivity in all the market we operate in at a very attractive price. And by doing that we can contribute to the successful globalization commerce that will facilitate the business which will break down barriers that might prevent doing business, or from offering access to multinational companies.

That’s really what I think is our role in the world, to enable companies and people around the world be operating seamlessly as if they were sitting at two desks right next to each other (companies), and thus taking away the physical barriers of being located thousands of miles apart.

Pacific-Tier: If you look at the ideas, of say Carr’s concept of the “Big Switch” (Nicholas Carr), where telecom companies, and computing companies, and storage companies actually become nothing more than a huge, ubiquitous utility that people expect. Do you agree with that idea, or do you believe companies like Hibernia should be able to offer much higher value than the idea of a utility “big fat pipe?”

Bjarni Thorvardarson:
Well I think everyone has to know which business they are in, and that people can be in more than one business at a time. I say that from experience, because CVC (his prior venture capital company) was in the manufacturing of aluminum, and the manufacture of advanced products that were made from aluminum.

Making or smelting aluminum is very much a commodity business. The success of the business is predicated, or based on you operating a business efficiently. It’s about cost, cost, and cost. If the price per pound of aluminum that you smelt is higher than your competitors, you are out of the market. So that’s how we operated in the aluminum market.

But we also had exclusion companies. That is changing the aluminum ingot to bars that can be used by manufacturers, and be converted to door frames, and window frames, and converted into materials that could be sold to the end users or consumers.

So we were very much aware of the different needs of the value-added service market and the commodity market.

And I think the telecom business is very much the same. You have to know whether you are in the commodity market, the utility market, and there is a fair amount of utility market in the telecommunications world. I think the core of what Hibernia does is just that. It is a utility capacity between the POPs. It is providing 10G (Gigabit) capacity between London and New York, or London and Amsterdam, connecting all these high capacity markets, and it is a utility market.

You have to be very efficient in terms of how you operate your market.

Then, when you go up to the media market, or to the finance market, it is no longer a commodity market. The trader that is trading between London and New York, he does care about the price he is paying, but even more concerned with no having a second-rate service.

So you have to know which market you are operating in, and telecommunications will remain within the two markets.

Now Nicholas Carr’s concept or theory of the “Big Switch” where the world is going to cloud computing as a utility, where you plug into a socket in the wall and you are connected to a network of computing power is a noble one, and a very interesting one, and I think it certainly is going in that direction, but the difference between the bits and bytes, and the electrons that flow on the wires of the utility companies or the electric companies – it is bits and bytes of sensitive information that you do not want leaving the company or be flowing on the wires outside of the company.

So there are many challenges the “Big Switch” theory or concept. But there are a number of companies that are building up a very successful business model. Amazon being one, and a number of other companies offering cloud computing and growing extremely fast.

I am fascinated by the concept and the model of business, but I don’t think there is quite the pure cut between computing and the traditional utilities.

Pacific-Tier: Any other vision or looks into the future Hibernia may be able to share as you peer into the next 3 to 5 years?

Bjarni Thorvardarson:
I wish I could pretend to have a crystal ball, and say what our visions are, but our vision, really for the near to mid-term future is to continue our growth into the different enterprise verticals. We need to continue to service the market we comfortably define as our core markets, being North America and Europe. That’s where we will continue to focus our attention.

But we will to some extent continue to introduce new products that will leverage our network, and continue focusing on different verticals that we can also continue leveraging the network. The game for Hibernia over the next couple years is leveraging the asset. Those assets are not only our network, but also the experience of our company (employees) – the people we have, the processes, and the systems we have. Our competence and the network – that is what Hibernia is going to be not only for the next few years, but beyond.

Pacific-Tier: Do you see any potential partnerships or expansion across other parts of North America or into Asia at some time in the future?

Bjarni Thorvardarson:
Without any doubt, I am sure we will find partnerships that will benefit both parties, but it is nothing I can speak about or speculate about right now.

Pacific-Tier: Fair enough! Any final words on Hibernia, yourself personally, or what you see as interesting things happening in the market?

Bjarni Thorvardarson:
Not really, I am just enjoying working in this space, and I’m looking at a number of exciting opportunities. M&A opportunities, growth opportunities, and I am just excited to be here.

Pacific-Tier: As we all are, and thank you very much for your thoughts – it has been a great discussion.

Bjarni Thorvardarson:
Thank you! It is my pleasure!

Mr. Bjarni Thorvardarson is the CEO of Hibernia Atlantic since January 2005. Mr. Thorvardarson joined CVC, Hibernia’s parent company, in 2002 from ISB bank where he launched and managed the listed Talenta-Technology fund which focused on emerging communication and IT opportunities. Prior work experience includes investment banking at FBA bank, management of an MIS department and European Sales Director of an IT company. Mr. Thorvardarson holds an M.Sc. degree in Engineering from UW-Madison, an MBA from ISG in Paris and an M.Sc. in Finance from London Business School. Mr. Thorvardarson serves on the board of One Communications, Magnet Networks.

Read the Pacific-Tier series on Entrepreneurs and Innovators

Internet Network Peering – Discussion with Peter Cohen, Industry Expert

Why does one house with a cable connection to the Internet have great performance, and the house next to them with a different Internet provider have marginal or potentially poor performance? The answer may lie in the sometimes dark art of Internet peering.

Peter Cohen

Peter Cohen stopped by to discuss the topic, and try and shed some light on peering recently at the North American Network Operator’s Group (NANOG) conference in Philadelphia. Peter has worked in the peering community for more than 12 years, with experience at some of the largest Internet Service Provider networks in the United States, as well as managing peering for Telia-Sonera, the national telecom network of Sweden.

“Peering is a voluntary interconnection of administratively separate Internet networks for the purpose of exchanging traffic between the customers of each network.” Wikipedia

Savageau: Peter, why is peering important to Internet networks?

Cohen: Peering cuts out the middle man, allowing networks to connect directly to each other without a transit or intermediate network (such as Verizon, AT&T, or Sprint). In addition, peering helps networks to improve data throughput, reduce some operating costs, and provide some protection against high transit pricing (by Tier 1 and transit network providers).

Savageau: Do all networks peer?

Cohen: Peering is not for everybody. Smaller networks and enterprise networks generally do not have enough traffic (to make peering worth it). Also, a lot depends on a network’s location. Peering normally occurs in large markets, such as the NFL cities. A network needs to have other networks or content providers who want to peer with them. You also need to have an ASN number (autonomous system network), which is reserved for those networks who have a need (traffic volume) to peer with multiple networks and content delivery networks.

Peering is not a free connection to the rest of the Internet. When a networks peers with another network, they only receive routes available directly on that network’s, or customers of that network (if it is a larger regional network which provides Internet services to both access networks and smaller content networks). In addition to having peering relationships with one or more networks, unless a network is among the largest (which we call Tier 1 Internet networks), they will still need a “default route” for routing network traffic to the rest of the global Internet.

A default route, also known as the gateway of last resort, is the network route used by a router when no other known route exists for a given IP packet’s destination address. All the packets for destinations not known by the router’s routing table are sent to the default route. Wikipedia

Savageau: Where do networks peer?

Cohen: Networks peer in larger cities where there is a higher density of other networks and content providers. In Los Angeles the Wilshire Corridor in downtown LA has several locations, such as One Wilshire where networks can connect at either public Internet Exchange Points/IXPs, or through private network interconnections (PNIs).

Los Angeles supports several large public exchange points, including those managed by Equinix, Switch and Data – as well as the (CoreSite) Any2 Exchange. Other cities such as New York, San Francisco, and Miami are also major points of peering in the Internet community.

Savageau: How is peering done differently in the United States, versus Europe or ASIA?

Cohen: A lot of the US peering model was developed early in the development of public Internet. There were very few commercial networks present in the Internet, and those networks peered with each other. As the Internet of the 1980s and early 1990s was “hubbed” in the United States, those early American networks were built from the beginning with global views of Internet routing – thus becoming the first Tier 1 networks.

Peter Cohen Tablet Engineering

The Europeans and Asians were a little later to the game. Many of the early non-US networks were operated by national carriers with very little reason to peer with or support competitive networks within their countries (no reason to make your competition stronger and more competitive in your national market). Thus the national carriers only peered at points within the United States, and since those networks did not have global Internet routing visibility, they had to buy transit from the American Tier 1 Internets.

Even today, in some countries national carriers will not peer within their own market, but will peer with competitors at peering points and exchanges outside of their home country.

On the other hand, the European networking community did learn from the American experience, and today the largest pubic exchange points exist outside of the United States, allowing smaller networks and content companies to gain all the advantages of local peering, without the need to “hairpin” their traffic through a Tier 1.

Savageau: Is the current model of peering in the US good enough? Change needed?

Cohen: Traditionally networks which peer come in two types. Those who only peer with private network interconnections/PNIs, and those who peer at public exchanges, and also support PNIs. PNIs are most useful when traffic on a public exchange hits a certain threshold, and then it makes more sense for the networks to interconnect with much higher capacity individual circuits.

A new model emerging in the US is with remote exchange point connections (Remote IXP Access). Remote IXP access allows a network geographically separated from a major public peering point to lease a long distance circuit, usually around a Gigabit, and “test the waters” by connecting to the IXP. If the peering point effectively serves the needs of the remote IXP access network, then they may at some point establish a direct point of presence the facility supporting the IXP.

The only real drawback of remote IXP access is that without a physical router present at the data center or carrier hotel housing the IXP, they are limited to the single exchange point connection, and cannot establish a PNI with another network at the facility.

Savageau: Final question. Most difficult topic for last! Give me your feelings on the topic of net neutrality, how it is currently affecting the Internet community, and how you think it will evolve.

Cohen: Many of the current policies in effect between networks and content providers are purely based on internal politics. That’s why you may go into a neighborhood and find an Internet Service Provider giving outstanding performance to one customer, and across the street you will get poor performance. Most local Internet providers are called MSOs, or Multi-Service Operators. MSOs include cable TV companies, phone companies, and satellite TV companies.

One MSO, such as AT&T, may determine the content providers should pay a higher rate to subsidize the cost of network build outs. A cable company MSO may determine that peering with content networks is the best policy.

There is no regulation at this level, and the ultimate victims are end users in the home. End users who believe they are paying a flat rate to their MSO to receive the best possible performance on their home Internet connection – but at the end of the day are receiving service performance based in a large part n the personalities of persons making decisions on peering models, not necessarily on business rules or objectives.

Savageau: Peter, keep looking out for us

============

Peter Cohen has been involved in the Internet since the 1990’s starting out at CAIS Internet.  He has held a wide range of positions and has worked extensively with peering, transit, interconnections and network design/colocation for ISPs, Carriers and other Internet related companies around the world.  His work and travel have brought him to more than 35 countries and 5 continents.  He is a frequent speaker at NANOG, Ripe, and other industry events worldwide.

Presently Peter works at Switch & Data where he facilitates interconnections among customers and the PAIX product.  He lives with his wife and two children in McLean, Virginia.  He enjoys cooking, reading, golf and tennis in his spare time.

%d bloggers like this: