I recently installed conky (system monitoring application) on my Linux Mint 17.3 PC, and I have since noticed some intermittent odd network behaviour.
At random times during the day, I see a sustained spike in upload & download traffic on my eth0 interface (normal wired NIC). It lasts 10-30 minutes, then goes away on its own.
Normally, the conky output for network traffic looks like this:
10KB/s down; 11KB/s up. I'm listening to music on YouTube; so this seems pretty reasonable.
The white "BAR" is the up/send side of my NIC max'ing out this chart. Yes, it's only 630 KB/s, but that seems pretty dang high, considering that I'm not initiating it.
So, I'm naturally wondering "what is using all of that bandwidth?"
I've dug in a bit, and (using a combination of nethogs, netstat, tcpdump, and Wireshark) I have identified that the culprit in all situations is a single connection to a remote host.
The connection is either on :80 or :443; and is simply a sustained loop of ACK's (coming from their side) and RST's (coming from me).
Here's the actual tcpdump output for the extra-geeky:
Anyhow, my (limited) understanding of what's going on with the RST's & ACK's is this:
They say: "ACK", meaning "OK, I got what you said"
I say: "RST", meaning "Right, let's end this conversation"
But even after I 'reset' the convo; they keep ACK'ing me!
This goes on for several minutes; then the traffic just dies off after awhile. (I haven't timed it; that might be interesting info too).
A few other pieces of info:
So, all of that to lead up to my questions:
asked 05 Jan '17, 13:36
This is one of the coolest RST loops that I have ever seen. Here is what's going on:
Your Linux box is trying to reset a connection. You figured that out correctly. For some reason this packet doesn't make it to the destination address. Instead you get these retransmitted ACKs all the time.
From a receivers point of view it is perfectly normal, if data arrives after a Reset has been send. That's usually data in transit that was transmitted by the remote system before it has received the RST. This happens usually within a few milliseconds (1 TCP roundtrip time to be precise).
In your tracefile we see TCP packets without data arriving for more than 8 seconds. That's odd. Let's take a closer look.
The ACK packets with source address 126.96.36.199 have a TTL of 64 and an IP ID of 0. Both are odd values. The TTL has an initial value of 255, 128 or 64 (depending on your operating system) and is reduced by one when a router forwards a packet. A TTL of 64 indicates that the very packet originates from a host on your local network. My guess is, that your firewall is sending this packet for what ever reason.
Next the IP ID. This is something like a 16 bit serial number that is used to process fragmented IP packets. Depending on your OS the IP ID could be incremented by one for each packet in a TCP connection or each packet send by the host (independent of the TCP connection). Or it could be a just a random number. In your tracefile all ACK packets have the same IP ID of zero. This is another indicator, that the packets are not coming from the remote host. If 188.8.131.52 had send these packets as real retransmissions we would see different IP IDs.
I assume, that your firewall is doing something strange. I am pretty sure that neither the RST packets nor the ACK packets are visible on the external interface of the firewall. It would be greatly appreciated if you could tell us what manufacturer / software / configuration is responsible for this behaviour.
Oh, by the way: Capturing directly on the firewall may or may not give you a correct representation of the traffic leaving the physical interface. The result depends on the order of operation on the firewall. The tracefile will be different, depending on the fact if the filter engine kicks in before or after Wireshark / tcpdump have access to the packet.
Anyway, thank you for one of the most remarkable TCP behavior that I have seen in a while. Thumbs up.
answered 07 Jan '17, 02:42