Hello Can someone please help me with the following question? I understand (from what I have read so far), the Window Size plays an important role in the overall potential throughput of a systems and thereby application layer performance (assuming the application in question can get the data out of the receive buffer fast enough) I read that in a network with a 1 ms delay and with a window size of 64k (where window scaling is not in effect) the maximum throughput is 524 Mbps. Whereas if Window Scaling option is available (which I be believe was introduced in RFC 1323?) the throughput on the same network can go up to 8590 Mbps. The reason I ask is we have a layer two (switched) network using Cisco Nexus switches, not sure which model. Attached to these switches are HP Servers running Windows 2008 R2 and NetApp Cluster Mode filers (with 4 LIFs). We have 10GB NICs in the Servers and NetApp controllers. However I have been told users are saying performance of their applications (sitting on top of this environment) is slow, e.g. slower than it was when it was on the older NetApp 7 mode filers. We have boiled it down a bit and believe the delay as at the network level as the delay to write data is between 170 and 270 ms (when measured from Windows Port driver to the network attached NetApp cluster mode filer). NetApp team are saying write to disk is very fast once processed by network layer, so issue appears to be at network delay. Network team are saying nothing odd going on at the nexus switches. Therefore I am thinking about window size as I see a maximum of 64k and the Window Scaling option is set to -1 (unknown) and I also see NOP NOP (no option) in the TCP header information. Therefore I am thinking either the sender (windows server) or receiver (NetApp cluster mode filer) does not support the Window Scaling option (or it is disabled by some kind of configuration option), or the network switches are stripping it out (but I am informed the switches are only working at layer 2 so not touching the TCP headers, but I have to bear in mind this information may be incomplete). Question: Taking the above into account, in general (without packet trace at this point, may be able to post one later) is it true to say if I see a window size of 64k and Window Scaling option set to -1, this would have detrimental effect on the achievable throughput between two hosts on a 10GB Ethernet network with a 1ms (or less) delay, as I referred to above? Thanks very much in advance Ernie asked 13 Nov '16, 02:49  EBrant edited 13 Nov '16, 04:03  sindy |
2 Answers:
Please seee a related problem here: https://ask.wireshark.org/questions/55972/slow-writes-even-slower-reads-spanning-wan-to-netapp and a discussion about that question here: https://blog.packet-foo.com/2016/10/trace-file-case-files-smb2-performance/#more-1393 But for detailed WindowScaling analysis you MUST capture the 3-way handshake, because the WS factor is advertised only there. And a WS of -1 means that you haven´t captured the 3-WayHandshake. answered 13 Nov '16, 03:22  Christian_R |
Firstly, to confirm what you have read and understood. Yes, TCP Window Size does play an important part in throughput. A receiver advertises a Receive Window - which represents the size of the receive buffer that it has allocated for that connection. To us, in English, it means the amount of data that a sender can transmit before needing to wait to receive acknowledgements (ACKs). Once the sender has transmitted that amount of data, it must stop transmitting and wait for ACKs to come back. Once data is ACKed, the sender can transmit more data to again fill the Receive Window. The effect is to specify the maximum data volume that can be transmitted per round trip. With Window Scaling disabled, the throughput can only be 64 KB per round trip time. The RTT being the time it takes your 64KB of data to get to the receiver and then the receiver's ACKs to get back to you. For a RTT of 1ms, this roughly means 64KB/ms - or 64MB/sec. A straight multiplication by 8 gives us 512Mb/s (close to the 524 Mbps you mention). Overheads will likely mean an actual throughput lower than that. The terms "packets in flight" or "bytes in flight" are used to indicate the "sent but not yet ACKed" data. With no TCP Window Scaling, you can never have more than 64KB "in flight". Clearly then, the RTT between your client and server and Window Scaling matters a lot in terms of throughput. For maximum throughput, you want the Receive Window to be large enough so that the sender can continuously transmit without having to stop and wait for ACKs. Hence, longer RTTs and larger bandwidths will need larger Receive Windows. You'll see the term "Bandwidth Delay Product" used as a measure of the window size needed to achieve maximum throughput for any given bandwidth and RTT. Your Wireshark observation about the 64KB Window Size can't be relied on because we (and Wireshark) didn't get to see what the actual Window Scale was as specified in the 3-way handshake. That 64KB is an assumption made by Wireshark. However, you can display the "Bytes in Flight" value to show how much data was actually on the wire before ACKs were observed. With one of your data packets highlighted in Wireshark, look in the "SEQ/ACK Analysis" portion of the "TCP Protocol" section of the Packet Details pane. Right-click on the "Bytes in flight" value and choose "Apply as Column". You will then be able to sort the column to find the largest value and get a hint as to your actual Transmit Window size. If you never see a value greater than 64KB, then perhaps TCP Window Scaling is not enabled for your connections. Windows 2008 (and clients from Windows 7 onwards) should have Window Scaling enabled by default. Naturally, the most definitive method is to capture the SYN/SYN-ACKs. Note that, due to any number of reasons, the sender may not always fill the Receive Window. answered 20 Feb '17, 22:35  Philst |
Just to state explicitly what @Christian_R only implies: displaying a window scaling factor value of -1 is how the TCP dissector of Wireshark announces that the actual window size cannot be determined from the capture. If any of the two TCP session participants does not support window scaling but the SYN, SYN+ACK exchange is available in the capture, the dissector shows a window scaling factor of 1 and uses it to calculate the actual window size.
Hello Christian and Sindy, thanks to you both for taking the time to reply to my question.
This is extremely useful information e.g. WS of -1 means that you haven´t captured the 3-WayHandshake and that a Window size of 1 (as further explained by sindy) SYN, SYN+ACK exchange is available in the capture, the dissector shows a window scaling factor of 1 and uses it to calculate the actual window size.
The above is very useful information to me for someone who has started to learn WireShark :)
I will start Wireshark, then disable and re-enable the network card, this should hopefully capture the three way handshake then I can check again
Thanks again Ernie
Or you can just disconnect and reconnect the remote drive.