First, I apologize for the cropped picture... I tried to get everything relevent. Len=0 on all Red->Blue packets. So yeah, Red is a workstation downloading a file over SSL from distant server Blue. I have no access or control over distant server blue, nor any of the infrastructure beyond the first two hops. There's some NAT and IOS based firewalls on my end, but other than that, it's all "cloud". What's breaking my brain is that transfer goes along just fine, from anywhere to 200 to 1800 seconds, and then Blue goes into what appears to be retransmission backoff... "you never ACK'd, so I send again 1 second later, again 2 seconds, 4 seconds, 8... it goes all the way up to 120s and then just keeps sending one packet at a time every 120s. The only two things I can think of that would cause that behavior would be either windowing issues, or dropped packets. I can't see any reason to suspect windowing because none of the packets reflect that ("Win=66000" from RED, "12574" from blue, both are consistent throughout the transfer), so that's out. I also can't say it's dropped packets, because there are literally NO retransmissions in the entire transfer. 300MB transferred over half an hour, and the only thing wireshark flagged on was a zero-window during the first 11 seconds, probably related to me deciding where I wanted to save the stupid thing. So by all accounts... it's moving along flawlessly. What's more, even after the backoff begins, it's sending new packets with new, correct and expected sequence numbers. So it sends Seq#x... I ack Seq#x... it waits 120 seconds and then sends Seq#x+1320. So if it's sending new data, it would have to indicate that it's recieving my ACKs, correct? So no dropped packets, right? I just want to get another opinion on this before I officially give up and blame it on some wonky proxy or web server on the distant end (no proxies on my end at all). And I suppose if anyone happens to have any idea WHY this would happen, I could try to contact the other end and help them out... But mostly I just want to know I'm not insane here.
Kurt: Enjoy. Looking at that, it looks rather information dense. Mind explaining what about that is meaningful? asked 31 Dec '12, 17:02  W-George edited 02 Jan '13, 15:33  Kurt Knochner ♦ |
One Answer:
It's a little hard to say for sure, since we can't see the sequence numbers used by the server, but when I compare the length of the server packets they seem to relate to the increasing ACK numbers coming from the client. So in that regard I'd agree with you that it seems that there is no packet loss. The interesting thing is that the "bad" time delays start at 1 second and double up each time. This is a typical "uh, there's somehting going wrong here" backoff mechanism, but it is impossible to tell without looking at the server side (which you said you can't). If I'd have to guess I'd suspect a bandwidth managing device throttling down the conversation close to the server side. Whenever a communication seems fine end-to-end (with matching sequence/ack numbers and no packet loss) I'd almost bet on a device in the middle fooling things up. Only way to tell is to capture at client and server simultaneously, but this doesn't seem to be possible in this case. answered 01 Jan '13, 02:36  Jasper ♦♦ So yes, the ACK numbers you see do coincide with the sequence numbers of the server's packets, and they're all 1320 apart, which coincides with the data portion of the packet. One potential issue is that I've got two routers using IP Inspect, but neither of them claims to be dropping packets. However, to my knowledge, this behavior has only been observed from this one distant end. I was just able to successfully download the file at my house, but the transfer this time was from a completely seperate IP address space (different Class A network, even). So that doesn't help really. (02 Jan '13, 04:58) W-George |
can you please add the following output for the time interval when the 'effect' is visible.
Sure. I'll give that a shot when I'm back in the office in the morning.
did you have a chance to generate the mentioned output?
@W-George: the output just lists the relative time, the IP Identification field, length of the TCP playload, sequence number/acknowledge number, and the TCP flags. With that we can see a bit more of what the stacks do.
It's interesting to see that the IP-ID is increased by 2 for each packet coming from the server when it gets slow. While it may be because the server is talking to someone else in the meantime I suspect that there is in fact one packet belonging to the conversation that doesn't get through to the client. Because when it is still fast, we see sequential IP-IDs.