Solutions in the Architecture

Currently, nearly all developing countries suffer from Internet connectivity that is expensive and slow, in comparison to developed countries. To a large extent, this is the result of the fact that virtually all developing country Internet networks and service providers rely - directly or indirectly - on international satellite links to larger foreign upstream providers. As a result, nearly all Internet traffic in nearly every developing country must travel across multiple satellite hops to get routed and exchanged via a backbone in another country before it reaches its destination. In other words, nearly all Internet traffic in developing countries - even traffic from one Internet service provider (ISP) to another ISP in the same country - is routed overseas, most often via the United States or Europe. As a result, developing country Internet connections are significantly slower, less reliable, and more expensive than in developed countries.

One of the most effective mechanisms to enable local exchange of local Internet traffic - thereby producing both cost and service improvement - is the Internet Exchange Point (IXP). An IXP is a shared switching facility that allows ISPs to exchange Internet traffic with each other.

There are perhaps 150 IXPs throughout the developed world, but only a handful in developing countries. Currently, there are only two IXPs on the African continent outside South Africa (in Kenya and Mozambique). As a result, nearly every African Internet Service Provider (ISP) must rely on satellite connectivity, which is more expensive and entails vastly greater network latency than the use of fiber optic cable. An IXP allows ISPs to interconnect easily; through the IXP, ISPs can exchange locally-bound Internet traffic locally without having to send those packets across multiple international satellite hops to reach their destinations.

In developed countries, ISPs connect to their national or local competitors so that customers' traffic is passed on directly to its destination network, without monetary payments between the networks. These common inter-ISP traffic-exchange arrangements are known as "peering" relationships. By contrast, nearly all developing country ISPs are limited to one or two "transit" relationships, in which the developing country ISP hands all of its traffic to a foreign upstream provider, for which the ISP must pay the full cost of all inbound and outbound (usually expensive and slow satellite) bandwidth. An IXP provides a neutral switching facility that allows ISPs to interconnect locally, and enables the easy establishment and maintenance of peering relationships.

The case for interconnection among African Internet service providers -- and the case for neutral exchange points to enable it -- is powerful, yet only a handful of IXP facilities currently exist in developing countries. Among the central inhibitors are legal restrictions (such as prohibitions on non-regulated telecommunications facilities, enforced monopolies on international connectivity, restrictive licensing regimes, and burdensome tax treatments), telecommunications regulatory agencies (who seek to extend their statutory authority over telephony to Internet infrastructure), and monopoly telecom operators and dominant ISPs (who seek to prevent effective competition).

Think of an Internet Exchange Point (IXP) as simply a room with a switch.[Note 9] Multiple ISPs are able to run their wires into the room, install a router[Note 10] next to the switch, make a physical connection between the router and the switch, and then use the switch to exchange traffic with one or more of the other ISPs that are similarly connected to it. An IXP can make it easy and efficient for a group of ISPs to interconnect with one another.

To be a bit more technically precise, an IXP is a physical network facility operated by a single entity to facilitate the exchange of Internet traffic between three or more ISPs. An IXP is characterized by neutrality among all user/subscriber ISPs; often, it will be administered by a non-profit ISP association, a university or institute, or a for-profit company. In the case of a for-profit IXP operator, the need for trust and neutrality dictates that the administrator should not be one of the competing interconnected ISPs.

Typically, the IXP operator owns and maintains the switching platforms used to interconnect the various users/subscribers. The exchange point consists of a shared switch fabric, where users arrange peering via bi-lateral agreements and then establish BGP-4 sessions between their routers to exchange routes and traffic.[Note 11]

It is important to note that an IXP enables its member ISPs to interconnect with each other, but does not mandate that every member exchange traffic with every other. Rather, it is up to each ISP connected to the IXP to determine whether, with whom, and on what basis (paid or unpaid) it will interconnect with another ISP to exchange traffic. Careful use of BGP-4 allows an ISP to control whether its network will accept traffic from a given ISP.

Suppose that Andrew has traveled to join Ethan in Ghana. The two are typing away on their laptops in the same building. Ethan is using Geekcorps's ISP; Andrew is using a different ISP's dial-up service, using his modem over a telephone line. When Ethan sends Andrew an email, that email will likely travel from Ethan's ISP up to a satellite, down to the United States or Europe, over at least one backbone carrier, up to another satellite, and back down to Ghana, where it ends up with Andrew's ISP. And remember: Andrew and Ethan are in the same building! Their ISPs do not interconnect, meaning that each sends its traffic abroad, just to be routed toward its destination. The result: high costs and slow speeds.

Internet exchange facilities are among the most critical elements in the infrastructure of the Internet. By definition, the Internet is a network of voluntarily interconnected networks; IXPs are the points at which multiple networks interconnect. Without IXPs, there would be no Internet, as we have come to know it. Let's look closely at two consequences of the lack of interconnection: cost and quality of service.

Cost. International links entail both upstream and downstream packet traffic, the costs of which must be borne by either the sending or the receiving ISP. Here, we observe a troubling imbalance: Unlike in the telephony world, where ITU-mandated rules require that the costs of international calls be shared between telecom operators, international Internet connectivity operates according to the peering/transit dichotomy. ISPs are not subject to the ITU's cost-sharing rules; rather, connectivity costs are allocated according to bilateral contracts, which can generally be classified as either peering or transit agreements. (It should be noted that this dichotomy is a vast oversimplification: ISPs have developed a vast range of varying interconnection agreements, involving often highly sophisticated settlement regimes; however, for purposes of analyzing developing country connectivity costs and options, the basic models cover most situations.)

A peering agreement is a bilateral business and technical arrangement in which two connectivity providers agree to accept traffic from one another (and from one another's customers, and their customers' customers). In a peering agreement, there is no obligation for the peer to carry traffic to third parties. There are no cash payments involved - rather, it is more like barter, with each ISP trading direct connectivity to its customers in exchange for connectivity to the ISP's customers.
A transit agreement is also a bilateral business and technical arrangement, but one in which the transit provider agrees to carry traffic from the customer to third parties, and from third parties to the customer. The customer ISP is thus regarded as an end point for the traffic; the transit provider serves as a conduit to the global Internet. Generally, the transit provider will undertake to carry traffic not only to/from its other customers but to/from every destination on the Internet. Transit agreements typically involve a defined price for access to the entire Internet.

For virtually all developing country ISPs, the only option for connectivity to the global Internet is a transit agreement. That is, a developing country ISP has such a small customer base that the international Tier-1 and Tier-2 providers have no business incentive to enter a shared-cost peering agreement with it. [Note 12] Instead, the developing country ISP must sign a transit agreement with its upstream provider.

For example, see Worldcom’s Peering Policy. Many of the criteria would be difficult for developing country ISPs to satisfy, e.g., a Traffic Exchange Ratio not exceeding 1.5:1.

The result (to oversimplify slightly) is that developing country ISPs must pay 100% of both outbound and inbound traffic; under the terms of the transit agreement, the ISP on the other end of the international link does not share the cost of exchanged traffic. This means that the developing country ISP must pay 100% of the international transit costs for all packet traffic (email, web pages, file transfers, etc.) that originates with its customers and that terminates with its customers. In other words, if the customer of a Nigerian ISP sends an email to a friend in the United States, the Nigerian ISP bears the full cost of the packets' outbound transmission over its international link. Neither the recipient's ISP nor intermediate upstream carriers bear any of the overseas transit cost. If the friend in the United States sends an email reply back to Nigeria, the Nigerian ISP must again bear the full cost of inbound transmission over its international link.

For Africa, then, the result is a massive outflow of capital, amounting to perhaps hundreds of millions of dollars per year -- the amount paid by African ISPs to send domestic traffic over international connections. In other words, the perverse situation is that African Internet service providers -- small companies struggling to provide network services to the poorest populations in the world -- are effectively subsidizing the largest, richest ISPs in Europe and the United States.

Quality of Service. Due to the lack of fiber optic links, most developing country ISPs use VSAT satellite circuits for international connectivity to upstream ISPs. Satellite connections introduce significant latency (delay) in the network. More problematic is the reality that, without an IXP, even domestic traffic must be exchanged internationally, entailing at least two satellite hops. (Indeed, even if fiber connections were widely available, the length of transatlantic cables introduces needless, though much smaller, latency in the connection.)

Significant network latency translates into achingly slow connections for users, putting a tremendous range of Internet services out of practical reach. Local Internet enterprises find themselves at an inherent disadvantage if they attempt to serve international customers. Ironically, they find themselves at a double disadvantage in serving domestic customers, whose queries must traverse at least two satellite hops to reach them, and another two satellite hops to receive the response. Forcing local ISPs to interconnect in another country thus places a major obstacle to the development of domestic Internet-based business. Indeed, many and perhaps most developing country Internet services are hosted on servers in the United States or Europe, to eliminate at least one satellite hop out of each transaction (including domestic).

Making the problem worse, nearly every developing country is experiencing rapidly growing demand for Internet connectivity, with ISPs offering faster local connections and users requiring greater volumes and more bandwidth-intensive types of Internet services. The growth in demand places ever-increasing burdens on the transmission capabilities of ISPs, which must struggle to secure adequate bandwidth to keep pace. In many cases, ISPs use their transmission lines at 100% of capacity, resulting in dropped transmission of packets of data, re-transmissions of dropped packets (thanks to TCP!), and a resulting compounded latency for completing Internet transactions.

An IXP slashes network latency by eliminating the need for any satellite hops in the routing of domestic-bound traffic. The result is that more customers use domestic Internet services, increasing local demand for bandwidth and prompting a cycle in which ever more bandwidth is dedicated to local interconnection. Since domestic bandwidth is always cheaper than international bandwidth, the business cases for domestic Internet enterprises improve dramatically - not just for ISPs, but for online banking, e-commerce sites, online government, enterprise VPNs, content hosting, web services, etc.

Regardless of the medium, then, a closer connection will be cheaper, faster, and more efficient. Put another way, the localization of packet traffic - keeping the physical path traversed by packets as short as possible - produces measurable improvements in service cost, performance, and efficiency.

The experience of the Kenyan ISPs in attempting to organize and launch an IXP provides an excellent illustrative example of the practical barriers that confront the deployment of IXPs in Africa.

Prior to Kenya's, there was no IXP on the African continent outside South Africa. In early 2000, TESPOK, the association of Kenya's competitive ISPs (i.e., those other than Telkom Kenya, the state-owned monopoly telecom), undertook to organize a neutral, non-profit IXP for its members. After nearly a year of preparatory work, including the design and implementation of a capable technical operation, funding model, and legal framework, the KIXP was launched in late November 2000, located in Nairobi. Almost immediately, Telkom Kenya filed a complaint with the Communications Commission of Kenya (CCK) arguing that the KIXP violated Telkom Kenya's exclusive monopoly on the carriage of international traffic. Within two weeks, the CCK concluded that the KIXP required a license, and ordered that it be shut down as an illegal telecommunications facility.

Telkom Kenya's legal monopoly does, in fact, extend to all fixed network infrastructure, including local, national, international, and leased lines. In Kenya, ISP services are open to competition, but ISPs rely on Telkom Kenya (through its Jambonet subsidiary) for underlying infrastructure. In addition, Telkom Kenya has the exclusive right to operate a national backbone for purposes of carrying international traffic.

Until KIXP, all Internet traffic in Kenya was exchanged internationally. According to TESPOK, roughly 30% of upstream traffic was to a domestic destination. During the two weeks of KIXP's operation, measurements indicated that latency was reduced from an average fo 1200-2000 milliseconds (via satellite) to 60-80 milliseconds (via KIXP). Likewise, monthly bandwidth costs for a 64 kbit/s circuit dropped from US$ 3375 to US$200, and for a 512 kbit/s circuit from US$9546 to US$650.

In response to the CCK's closure order, the Kenyan ISPs argued that the KIXP was a closed user group, and therefore would be legal under the Kenyan Telecommunications Act. Also, they noted that the local exchange of domestic Internet traffic does not contravene Telkom Kenya's international monopoly, as all international traffic would continue to flow over its international links. Telkom Kenya's opposition to KIXP was fierce, fed by the fear of losing a significant portion of its international leased line revenues.

After nearly a year of intensive efforts, including public pressure, threats of litigation, and private diplomacy, TESPOK finally received the approval of CCK in the form of a license, granted in November 2001. The commission's licensing order represented a fairly dramatic turn-around in the CCK's thinking, stating: "An IXP is not an international gateway but a peering facility that enables ISPs to exchange local traffic. The Internet is expanding very fast and since Telkom Kenya has demonstrated that it has some apparently insurmountable difficulty in rolling out Internet facilities, it would be in the best interest of the market to allow other companies to offer IXP services in the country." Nevertheless, the CCK requested TESPOK to partner with Telkom Kenya, and the ISPs accordingly approached the company with a proposal to cooperate. By February 2002, however, TESPOK had received no response and elected to re-launch KIXP on their own. Since its facilities went live in early 2002, KIXP has interconnected 5 Kenyan ISPs, with 8 others in process.

Interconnection makes such perfect sense, right? Why isn't it happening all over the developing world? We can identify some common themes:

First, we see strong resistance by the current providers of international leased-line, submarine cable, or regulated VSAT connectivity. In most cases, this means a state-owned monopoly telecom operator. A monopoly telecom can be expected to seek monopoly rents, and leverage its legal exclusivity over international links. In addition to the fear of effective competition, the telecom will generally fail to appreciate that reducing the cost of Internet connectivity for domestic consumers will generate vastly greater investment, more users, and actually greater international leased line revenues. Indeed, a strong case can be made that greater domestic use of the Internet generates a better-connected populace in the broad sense, leading to even greater use of international direct-dial telephony to foster commercial and personal international relationships.

Second, government regulators often side with the telecom, and their alarm is understandable. The governments of developing countries are often heavily dependent on revenues from the monopoly telecom operator; facing massive budget pressures already, they are reluctant to sanction activities which might squeeze those revenues. For a variety of reasons (ranging from close personal relationships to outright corruption), the telecom operator's views often carry great weight with regulatory authorities. Often, statutory or other licensing requirements exist which can, arguably, be applied to IXPs. In most cases, the regulatory authority is, at least initially, quite unfamiliar with the technical and economic aspects of Internet facilities and ISP traffic exchange.

Third, we regularly see resistance from the competitive ISPs themselves. Those that feel secure in their market position fear the effects of making connectivity cheaper for their competitors. Moreover, an IXP essentially allows any interested domestic ISP in a developing country to peer with its domestic competitors. This requires a degree of trust among competing ISPs that is quite common in the developing world, but fairly unusual in Africa. Anecdotal experience indicates that even small competitors are reluctant to band together, reflecting a powerful sense of competitiveness.

Achieving cooperation among competitors is a profound challenge. In the United States, ISPs have their roots in the cooperative academic networks that came together to form the Internet; in other words, the cooperative technical operations and the techies that ran them were later joined by business managers who fought for advantage in the competitive marketplace. In the US, then, it has proven relatively easy for rival ISPs to remain cooperative at the level of network operations. In countries that are new to the Internet, however, the business-side competitive imperatives have come first, giving little support to the necessary culture of technical cooperation among peers.

As a result, nearly all developing country ISPs behave like resellers: they buy connectivity from foreign suppliers, and resell to their domestic markets. They do not behave like elements of a national (or regional or continental) Internet backbone.

Optional assignment: Listen to Andrew’s audio summary of interconnection, IXPs and the Kenya case study - Part I and Part II.

BOLD 2003: Development and the Internet