Image showing a patch rack with red ethernet cables and black accents.

Troubleshooting a flaky internet connection

Debug a flaky internet connection and figure out where the problem lies using Linux…

Lately while playing some online video games I noticed some irritating rubber banding, not great when playing Chivalry II. Luckily some other games implemented some clear icons showing what is wrong with your connection. It seems my rubber banding is due to a high ping, so time to troubleshoot.

First order of business, time to see if it is not something on the game server or the fault of my gaming device. Using Linux I ping a public internet address, a good target are the DNS servers of Cloudflare or Google. For Linux the command is ping 1.1.1.1 (or 8.8.8.8 for Google), we will see an output like this:

PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
64 bytes from 1.1.1.1: icmp_seq=1 ttl=57 time=5.75 ms
64 bytes from 1.1.1.1: icmp_seq=2 ttl=57 time=4.40 ms
64 bytes from 1.1.1.1: icmp_seq=3 ttl=57 time=5.40 ms
64 bytes from 1.1.1.1: icmp_seq=4 ttl=57 time=3.80 ms
64 bytes from 1.1.1.1: icmp_seq=5 ttl=57 time=5.07 ms
64 bytes from 1.1.1.1: icmp_seq=6 ttl=57 time=5.95 ms
^C
--- 1.1.1.1 ping statistics ---
6 packets transmitted, 6 received, 0% packet loss, time 5007ms
rtt min/avg/max/mdev = 3.798/5.060/5.948/0.752 ms

To stop the command press Ctrl + C.

Now keep this running while doing your thing, if you notice an interruption have a look at the terminal output. If you happen to see a big spike or a dropped packet you now know the problem is probably somewhere between you and Cloudflare. You can also keep it running and stopping it after a while and inspecting the statistics. If the value for max is a lot larger compared to your mdev (standard deviation) you also have small hiccups in your internet connection.

Where does it go sideways?

Now we narrowed down our problem a bit, but we still don’t know exactly where the problem is. So my idea is now to combine multiple simultaneous ping commands to see at which point the connection breaks down.

Now looking online on Stack Exchange the consensus is to install a more powerful ping utility like Nping. There is nothing wrong with this solution, however I would like to just use ping, and I wanted to brush up on basic bash operators anyway.

So using the & we can send the execute command to the background. This is nice, now we can run multiple ping commands. But I also want to help out the ISP, so I want to pipe it into a file using >. Now we have a problem, piping all commands individually overwrites parts of the file. So we need the (...) to combine the command as a single output, we can then pipe the complete output into a file.

So now we can do the following (ping 1.1.1.1 & ping <isp default gateway> & ping <local default gateway>) > ping-outputs.txt, now all three commands run at the same time. If you want to add more IPs you can just add another ampersand and ping command, be sure to put it inside the parenthesis.

We will get an output like this:

PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=117 time=6.85 ms
64 bytes from 1.1.1.1: icmp_seq=1 ttl=57 time=7.12 ms
64 bytes from 1.1.1.1: icmp_seq=2 ttl=57 time=9.81 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=117 time=10.2 ms
64 bytes from 1.1.1.1: icmp_seq=3 ttl=57 time=3.67 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=117 time=3.66 ms
64 bytes from 8.8.8.8: icmp_seq=4 ttl=117 time=3.65 ms
64 bytes from 1.1.1.1: icmp_seq=4 ttl=57 time=3.82 ms
^C
--- 8.8.8.8 ping statistics ---

4 packets transmitted, 4 received, 0% packet loss, time 3005ms
--- 1.1.1.1 ping statistics ---
rtt min/avg/max/mdev = 3.653/6.095/10.214/2.711 ms
4 packets transmitted, 4 received, 0% packet loss, time 3005ms
rtt min/avg/max/mdev = 3.665/6.105/9.814/2.548 ms

If you don’t want to pipe it into the file this command will do: ping 1.1.1.1 & ping <isp domain> & ping <default gateway>.

Now we can see whether the problem already starts at our own router (the default gateway) or is at the ISP side. If the ISP default gateway has large ping spikes it might be time to contact them, which was the case for me. Now the hope is that you get a proper resolution.

Summary

So in summary we can easily debug basic latency and jitter problems using the combined ping command:

(ping 1.1.1.1 & ping <isp domain> & ping <default gateway>) > ping-outputs.txt

Comments