The mysteries of Internet routing
25 May 2007 | Published in Internet, Luxembourg | 4 Comments
I blogged last year about the willingness of the Luxembourg authorities to enhance the country’s connectivity. I appended a traceroute showing the how a packet moving between two sites geographically distant of less than a kilometer from each other would actually travel through most of Europe. Not only is there no improvement to date, it is actually getting worse. Here is today’s traceroute. It goes is the other direction than last year’s.
traceroute to www.isoc.lu (158.64.14.86), 64 hops max, 40 byte packets
(5 internal hops deleted)
6 147.67.224.2 (147.67.224.2) 8.301 ms 9.866 ms 14.375 ms
7 uu194-7-242-205.unknown.uunet.be (194.7.242.205) 11.214 ms 7.786 ms 8.543 ms
8 uu194-7-242-201.unknown.uunet.be (194.7.242.201) 10.411 ms 8.841 ms 9.023 ms
9 so-3-0-0.xr2.bru2.alter.net (146.188.2.121) 8.792 ms 9.223 ms 9.005 ms
10 so-1-0-0.tr2.bru2.alter.net (146.188.9.153) 11.849 ms 10.144 ms 11.786 ms
11 so-7-1-0.tr1.ams2.alter.net (146.188.15.225) 13.349 ms 12.818 ms 12.397 ms
12 pos1-0.br1.ams3.alter.net (146.188.3.214) 12.849 ms 13.775 ms 12.857 ms
13 pos4-0.core2.amsterdam.level3.net (146.188.67.202) 15.719 ms 12.937 ms 12.696 ms
14 so-4-0-0.mp2.amsterdam1.level3.net (4.68.113.82) 13.680 ms 13.982 ms 12.900 ms
15 ae-1-0.bbr1.frankfurt1.level3.net (212.187.128.30) 23.811 ms as-1-0.bbr2.frankfurt1.level3.net (212.187.128.97) 23.178 ms ae-1-0.bbr1.frankfurt1.level3.net (212.187.128.30) 23.395 ms
16 ae-31-53.ebr1.frankfurt1.level3.net (4.68.118.94) 33.517 ms ae-32-56.ebr2.frankfurt1.level3.net (4.68.118.190) 23.688 ms ae-31-55.ebr1.frankfurt1.level3.net (4.68.118.158) 23.166 ms
17 ae-4-4.car2.milan1.level3.net (4.69.133.137) 32.141 ms 31.910 ms ae-1-100.ebr2.frankfurt1.level3.net (4.69.132.126) 29.827 ms
18 dante.car2.milan1.level3.net (213.242.65.26) 32.716 ms ae-4-4.car2.milan1.level3.net (4.69.133.137) 32.112 ms dante.car2.milan1.level3.net (213.242.65.26) 32.570 ms
19 dante.car2.milan1.level3.net (213.242.65.142) 33.288 ms so-6-3-0.rt1.gen.ch.geant2.net (62.40.112.33) 51.075 ms dante.car2.milan1.level3.net (213.242.65.142) 42.808 ms
20 so-6-3-0.rt1.gen.ch.geant2.net (62.40.112.33) 38.728 ms so-7-2-0.rt1.fra.de.geant2.net (62.40.112.22) 37.665 ms so-6-3-0.rt1.gen.ch.geant2.net (62.40.112.33) 37.290 ms
21 so-1-3-0.rt1.lux.lu.geant2.net (62.40.112.54) 41.656 ms 43.051 ms so-7-2-0.rt1.fra.de.geant2.net (62.40.112.22) 37.016 ms
22 restena-gw.rt1.lux.lu.geant2.net (62.40.124.150) 43.677 ms so-1-3-0.rt1.lux.lu.geant2.net (62.40.112.54) 42.767 ms 42.155 ms
23 gate-2-v33.bce.restena.lu (158.64.16.38) 42.558 ms restena-gw.rt1.lux.lu.geant2.net (62.40.124.150) 48.978 ms gate-2-v33.bce.restena.lu (158.64.16.38) 51.384 ms
24 gate-1-v26.rest.restena.lu (158.64.16.218) 47.132 ms gate-2-v33.bce.restena.lu (158.64.16.38) 44.242 ms gate-1-v26.rest.restena.lu (158.64.16.218) 43.795 ms
25 gate-1-v26.rest.restena.lu (158.64.16.218) 44.614 ms 44.706 ms boaz (158.64.14.86) 43.837 ms !<10>
26 www.isoc.lu (158.64.14.86) 42.866 ms !<10> 42.585 ms !<10> 41.996 ms !<10>
This time we go from Luxembourg to Brussels, Amsterdam, Frankfurt, Milan and Geneva and finally back at home. 15 hops last year. 21 this year.
Much of the blame goes to Verizon, who consistenly declined to peer at the local internet Exchange, citing the fact that the peering vould be very uneven.
Share on Twitter
27 May 2007 at 17:38 (#)
We should name the ISPs who don’t peer at the LIX on the ISOC wiki.
Can you explain what they mean by a very uneven peering, and tell us what you think of their explanation?
27 May 2007 at 19:28 (#)
Peering between ISPs means they exchange traffic without any settlement. One of the conditions is that the amount of traffic should be roughly similar in both directions. Verizon is the most important Internet carrier in the world. Most of the other ISPs peering at the LIX are small in comparision. For those smaller ISPs wishing to peer with a large one, large carriers like Verizon usually ask for a financial settlement, based on the amount of traffic.
See Wikipedia for a longer explanation.
27 January 2008 at 14:36 (#)
I don’t know how you get these strange results, but they are wrong. There are direct IP connectivities within Luxembourg.
All major ISP’s in Luxembourg are connected to the LIX (Luxembourg Internet Exchange).
Look under:
: http://www.restena.lu/restena/en/EN-LIXMembers.html
You have to use the cmd command line to get correct answers.
For exemple: If you enter in the command line:
tracert http://www.isoc.lu,
you get a direct answer without passing via other countries.
Luxembourg is also directly connected to the largest Internet Exchange in the world, like AMS-IX, DE-CIX, LINX, etc.
See also
http://en.wikipedia.org/wiki/List_of_Internet_Exchange_Points_by_size
27 January 2008 at 16:49 (#)
Although this is an old post, it is still very much true. From the list you mention, several major ISPs are not peering in Luxembourg, altough they have local customers: Verizon, France Télécom, British Telecom, Telenet and Altice/Coditel. I certainly miss some others.
The fact there are fibers from Luxembourg to major IXPs in Frankfurt and Amsterdam does not mean that the Luxembourg ISPs sign peering agreements with other ISPs. See the lists at https://www.ams-ix.net/connected/ and http://www.decix.net/info/connected.html. The only Luxembourg ISP peering in Frankfurt is P&T.
This year’s traceroute is a bit better. From Luxembourg-Kirchberg to another host at Luxembourg-Kirchberg:
We go from Luxembourg, to Frankfurt, Dusseldorf, Amsterdam, Brussels and finally Luxembourg.
traceroute to europa.eu (147.67.243.25), 30 hops max, 40 byte packets
1 gate-isoc.rest.restena.lu (158.64.14.85) 0.964 ms 1.107 ms 1.380 ms
2 gate-1-v34.rest.restena.lu (158.64.16.57) 0.509 ms 0.585 ms 0.625 ms
3 gate-2-v26.bce.restena.lu (158.64.16.217) 2.688 ms 2.968 ms 3.143 ms
4 gate-1-v33.bce.restena.lu (158.64.16.37) 0.529 ms 0.576 ms 0.742 ms
5 restena.rt1.lux.lu.geant2.net (62.40.124.149) 0.514 ms 0.496 ms 0.669 ms
6 so-1-3-0.rt1.fra.de.geant2.net (62.40.112.53) 5.285 ms 5.441 ms 5.366 ms
7 ffm-b2-pos10-2.telia.net (213.248.77.89) 5.650 ms 5.773 ms 5.652 ms
8 ffm-bb2-link.telia.net (80.91.249.85) 5.703 ms 5.755 ms ffm-bb1-link.telia.net (80.91.251.150) 5.731 ms
9 ffm-b3-link.telia.net (80.91.254.41) 5.854 ms ffm-b3-link.telia.net (80.91.249.137) 5.839 ms ffm-b3-link.telia.net (80.91.249.141) 6.012 ms
10 ixp2.de-fra.eu.ignite.net (80.81.193.84) 7.410 ms 7.373 ms 7.367 ms
11 t2c1-ge12-0-0.de-fra.eu.bt.net (166.49.172.97) 6.841 ms 6.818 ms 6.778 ms
12 t2c1-p2-0.de-dus.eu.bt.net (166.49.195.153) 10.659 ms 10.638 ms 10.606 ms
13 t2c1-p4-0.nl-ams2.eu.bt.net (166.49.195.45) 23.848 ms 23.125 ms 22.886 ms
14 t2c2-p9-1.be-bru.eu.bt.net (166.49.164.189) 24.377 ms 24.325 ms 24.642 ms
15 t2a2-ge0-0.be-bru.eu.bt.net (166.49.190.18) 25.320 ms 25.408 ms 25.394 ms
16 166-49-133-114.eu.bt.net (166.49.133.114) 22.483 ms 22.438 ms 22.434 ms
17 212.8.176.126 (212.8.176.126) 29.483 ms 29.113 ms *