Skip to main content

Double IP's in traceroutes, bad routing setup?

  • 28 April 2024
  • 12 reacties
  • 234 Bekeken

Dear Odido, 

 

I hope someone can have a look at this, as this is something within the Odido network and it’s peering agreements. I have asked another Odido member to do a test as well, and they see the same issue. Within 300-450 packets (not ms) we see on the same hop, 2 different internal IP’s show up. In certain cases, hop 5 and 7 for instance will have the exact same IP address (there are often public ones). Now the latter case should not be happening and that points to an unhealthy routing table, or some same weight routing algorithm being applied. The former case can cause out of order packets, which aren’t fun if you are playing an FPS.

 

The fact that it’s happening so frequently also points to something being a bit odd. 

Here is an example taken from another user: hop 3 and 4 being odd

 

  143.177.x.x  7.371 ms  7.297 ms  7.332 ms
 2  10.10.12.5  8.170 ms  8.050 ms  7.984 ms
 3  80.249.208.1  8.162 ms  7.981 ms
    80.249.209.1  8.623 ms
 4  80.249.208.1  7.721 ms  7.739 ms  7.858 ms
 5  91.200.16.11  8.097 ms  8.114 ms  8.042 ms
 6  185.55.136.60  8.661 ms  8.576 ms  8.558 ms

 

MTR to Zscaler and AMS-IX
Double IP’s and routing changes

 

12 reacties

Pieter_B
Rank
Reputatie 7
Badge +4

To be honest, i could not figure out the question or problems that you experience on your connection to the network. Other than what i call a ‘deep dive’ into some routing, which might not even be owned by Odido.

So do you problems with you connections, that can be clearly identified by the kind of routing you discovered?

LET OP gewoon Medeklant. Glasvezel sinds 2016 (1Gb) - Linux gebruiker sinds 2006 ( Current: Kubuntu noble 24.04 )- Router/Firewall (AMD GX-424CC SOC) pfSense 2.7.2 & 4 port INTEL NIC (i226/2.5G) & Switch Unifi 8 poort 60w POE & Switch Unifi 8 poort & Wifi Unifi AP-AC Pro
C
Reputatie 1

To be honest, i could not figure out the question or problems that you experience on your connection to the network. Other than what i call a ‘deep dive’ into some routing, which might not even be owned by Odido.

So do you problems with you connections, that can be clearly identified by the kind of routing you discovered?

As I said it’s not just my connection that sees this. The issue that this causes is the out of order packets and packet bursts, that is especially noticeable with FPS gaming. I often also see random spikes as well. 

But regardless, the same IP address should not show up twice in a trace route. That is not normal from what I have understood. 

Pieter_B
Rank
Reputatie 7
Badge +4

I look at some IP’s that i was noticed before having some issues … they are owned by TWELVE99.NET .. a gaming company. So they are routing internally things around.

 

LET OP gewoon Medeklant. Glasvezel sinds 2016 (1Gb) - Linux gebruiker sinds 2006 ( Current: Kubuntu noble 24.04 )- Router/Firewall (AMD GX-424CC SOC) pfSense 2.7.2 & 4 port INTEL NIC (i226/2.5G) & Switch Unifi 8 poort 60w POE & Switch Unifi 8 poort & Wifi Unifi AP-AC Pro
C
Reputatie 1

I look at some IP’s that i was noticed before having some issues … they are owned by TWELVE99.NET .. a gaming company. So they are routing internally things around.

 

Sure they might be a owned by a gaming company, but that doesn’t explain the route switching in the early hops. Which are private IPv4 addresses. Or the duplicated routing addresses. 

My main concern is the route flip flop. That causes missed packets in UDP. 

C
Reputatie 1

Okay so I had been monitoring a bit more. 

I am even getting this route flip flop to DNS server and other IP addresses. I setup a small test and what that flip flop happens I get packet loss. Sometimes the route can change from being (from the router directly), 7 hops, to 9-10. All caused by private IP addresses. 

C
Reputatie 1

It seems that I get 57 route changes in an hour. So that means roughly every minute there is a route change that happens. This is just to something like cloudflare. which is 7 hops from me at 5.868ms. 

This seems a bit excessive? 

A
Reputatie 1

Route change might be due to load balancing, this is ok, as soon as for single tcp connection route is not flipping(although even that might be ok). yes, for UDP this might be uneasy, but, in principle, in properly configured system “flip-flop” should not result in packet loss. (Although might lead to out of order, but this is UDP, so, nothing is really granted anyway :) )

but route looping  -  that is really worrisome, and combined with route jumps, this is really looks like asking for troubles :)

C
Reputatie 1

 

this is what I see happen no matter the IP address 

C
Reputatie 1

and of course can’t be completed without seeing the same IP address twice :D 

A
Reputatie 1

Yep, for me it is also always double pass through some IP address. But that IP address is always different (depending on target)

 

router.lan (31.187.248.54) -> 20.50.2.37 (20.50.2.37)  2024-07-07T23:02:01+0200
Keys:  Help   Display mode   Restart statistics   Order of fields   quit
                                       Packets               Pings
 Host                                Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. 31.187.248.1                      0.0%     6    3.5   3.6   3.3   4.3   0.3
 2. 10.10.80.106                     33.3%     6    6.1   6.0   5.8   6.1   0.2
 3. 10.226.4.6                       60.0%     5    5.9   6.0   5.9   6.0   0.1
 4. 104.44.235.234                   20.0%     5    7.8  13.4   7.2  30.2  11.2
 5. 104.44.44.43                      0.0%     5    6.5   5.9   5.3   6.5   0.4
 6. 104.44.235.234                    0.0%     5    6.0  11.2   6.0  22.7   7.2
 7. (waiting for reply)
 8. (waiting for reply)
 9. (waiting for reply)
10. (waiting for reply)
11. (waiting for reply)
12. 20.50.2.37                        0.0%     5    4.9   4.9   4.7   5.2   0.2

 

router.lan (31.187.248.54) -> 37.244.16.136 (37.244.16.2024-07-07T23:02:42+0200
Keys:  Help   Display mode   Restart statistics   Order of fields   quit
                                       Packets               Pings
 Host                                Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. 31.187.248.1                      0.0%     5    3.7   3.8   3.1   5.2   0.8
 2. 10.10.80.106                      0.0%     5    6.3   6.2   5.8   6.4   0.2
 3. 10.226.4.6                       25.0%     5    6.4   6.2   6.0   6.4   0.2
 4. 137.221.78.33                    40.0%     5   40.1  18.1   6.7  40.1  19.1
 5. 80.249.208.83                     0.0%     5    6.6   6.8   6.4   7.2   0.3
 6. 137.221.78.33                     0.0%     5   28.0  29.2   6.9  62.7  21.7
 7. (waiting for reply)
 8. 137.221.78.51                     0.0%     4    6.1   6.4   6.1   6.8   0.3
 9. 137.221.66.45                     0.0%     4    6.6   6.6   6.3   6.8   0.2
10. 10.105.1.209                      0.0%     4    7.0   6.6   6.3   7.0   0.3
11. 37.244.16.136                     0.0%     4    6.7   6.5   6.3   6.7   0.2

 

A
Reputatie 1

Look at packet loss around that IP loop.

It is always huge.

This is anything but normal :)

It is clear when we have granted 100% packet loss from host which are firewalled. 

But those we have 20-60% loss - are not. it just something awkward there with routing.

A
Reputatie 1

I tried all 3: ICMP echo, UDP, TCP - same result, huge packet loss on those hops. Up to 90%. 

No surprise burst tests results in what we see. Bufferbloat and packet loss go hand in hand normally :)

Reageer


ForumvoorwaardenCookie instellingen