I am troubleshooting why doing a FTP GET is much slower than doing a FTP PUT. I've noticed in Wireshark traces, on the FTP-DATA packets, that the TCP window size scaling factor is consistently larger when doing a PUT (32) versus a GET (128). I suspect this is affecting throughput, but am not certain. I also observed similar behavior with iPerf; using the -R (reverse) flag, I consistently see much worse performance. When setting the window size larger, it makes a difference on the send but not on the receive. Any ideas? asked 23 Nov '16, 07:22  PhoenixGrad |
2 Answers:
I think you have a bottle neck in the path. If your Window Size is bigger more data can be send at once and might fill the bottleneck. http://smallvoid.com/article/tcpip-rwin-size.html But if you send data you got a full window event - not a zero window. And if you receive data you deal with retransmissions and packet loss. answered 23 Nov '16, 11:30  Christian_R edited 23 Nov '16, 12:57 Although I agree with your conclusion that there is a bottleneck in the path, I don't know how to identify it. One of the mysteries about this particular issue is that when doing a "send" we consistently get two to four times the throughput of a "receive." (28 Nov '16, 06:10) PhoenixGrad Maybe it is because the sender in the case of "receive" is able to send data much faster then in the other case. Easily said maybe it is because the sender in the case of receive has 10Gbit/s interfaces? (28 Nov '16, 06:20) Christian_R |
I see that the 10.x.x.x system has a IP TTL of 64 and the 192.168.x.x system seems to have an IP TTL of 128 (and is passing 10 routing hops). From this I assume that the 10.x.x.x is a linux or osx host and that the 192.168.x.x is a Windows host. Is that correct? In the TCP streamgraph (tcptrace) of tcp.stream 7 I see that the 192.168.x.x host is throttling it's bandwidth. I assume this is done by keeping a small congestion window because of the packet loss. CAn you try a Linux-to-Linux iPerf in the same locations and see what the throughput is? What should be the bandwidth between those two nodes (in each direction) by the way? answered 24 Nov '16, 03:16  SYN-bit ♦♦ Thank you for the analysis and response. For this particular capture, both servers were Windows hosts; 10.92.x.x is Windows 2012 and 192.168.x.x is Windows 2008. I will set up a Linux to Linux iPerf test in the same locations and let you know the results. (28 Nov '16, 06:06) PhoenixGrad |
Could you share us a trace? Either ftp or iperf.
Capture done with iPerf and WireShark.
0 - 10: Send, no window size specified. 24 - 35: Receive, no window size specified. 52 - 63: Send, 1M window size specified. 83 - 93: Receive, 1M window size specified.
https://drive.google.com/file/d/0BzBe-IJh8phKUWJleHVydzBvY1U/view?usp=sharing
Bear in mind that both the window size itself and the window size scaling factor is chosen by each participant of the TCP session independently, depending on that participant's capabilities.
For the throughput, the window size on the receiving side is important: for PUT, the one of the remote side, for GET, the one of the local side.
You cannot affect window size at the remote side, but as you say you have issues with GET, your chances might be higher.
Window size scaling factor is a secondary thing; the primary thing is that the recipient can use a window so big that it needs to use a scaling factor to be able to express that window size using the 16-bit field available. So the best choice of a scaling factor is one which allows to express the maximum possible size of the window using a number between 32768 and 65535, as in this case, the finest possible granularity is achieved.
Thank you, Sindy, for the insights. Since setting the window size with the iPerf flag does not seem to have any effect, my guess is that the OS (Windows 2012) could be limiting the window size and/or scaling factor. Is there any way to conclusively determine that?
According to this link, there seems to be no direct link between what window size you ask the iperf to use and what window the OS kernel actually uses. So you can replicate the tests done there, i.e. to observe a difference in measured throughput and see no difference in the maximum window size actually indicated in TCP, and you should see the window to shrink in TCP more often when a smaller "iperf window" is set. The TCP window reacts to behaviour of the receiving application - if the size of the receive buffer is fixed and the application doesn't pick up the data from the buffer, the window size must decrease.