Hello, My question does not really pertain directly to Wireshark, but rather to TCP. I hope that is OK. If not, let me know... This question was prompted by Laura Chappell's Tip #57, that I just received, and it involves out-of-order packets. She asks the question: "Does your application depend on packets arriving in order before the data is presented or do a few out-of-order packets go unnoticed...?" This puzzled me because I thought that TCP would NOT pass out-of-order packets to the application layer, but rather would wait until the packets were ordered before placing them in the application buffer. In other words, if the packets come in like this: 5, 6, 8, 9, 10, 7, 11, ..., then TCP might pass packets 5 & 6 to the application buffer, but would NOT place 8 in the buffer until 7 is received, when it could then place packets 7 thru 10 in the buffer, IN ORDER. In this way, the application might be "delayed" by out of order packets, but would never "see" out of order packets. Is my understanding incorrect? thx, Feenyman99 asked 28 Oct '10, 15:05  feenyman99 |
3 Answers:
You are ringht, TCP segments can arrive at the NIC out-of-order, but they are always presented to the Application in-order. Of course if packets arrive at the NIC out-of-order, there will be a delay in presenting the data to the Application. Some applications will not have a problem with the delay (like Laura's example of showing a webpage in a browser) but other applications might have a problem with the delay (for instance when streaming audio over TCP). IMHO, that is what Laura was pointing out. answered 28 Oct '10, 15:22  SYN-bit ♦♦ Awesome - thx for the quick response. Out-of-order packets and resultant app delays now make perfect sense, thx to Sake and Laura. Love those tips, Laura - keep 'em comin :-) (29 Oct '10, 05:26) feenyman99 |
One rule of thumb you can use is by looking at the PSH bit. Often (very often, actually) when applications try to control the flow of the data, it will send chunks of data to TCP. This is often referred to as "buffer tearing." So if you see some amount of data that's being transferred followed by one packet with the PSH bit set, this is your problem. Regardless of what your TCP window size may be (send or receive window), you can't send more than what the application buffer allows you to send. So you can count the number of application bytes being transferred between the PSH bits. Typically, this will be on a boundary number like 2048, 16348 or 65536. So to add insult to inury, if one of the packets go missing, the entire chunk (after the missing piece) will need to be resent. Again, this can have a huge impact on performance as the new transfer may have to follow tcp slowstart rules. TCP enhancements like Selective ACK can help greatly in cases like this. If you look at some of the Sharkfest sessions (Advanced TCP) by me (Hansang Bae), you'll see several examples of this "bound by PSH bit" scenario. Good luck. answered 01 Nov '10, 14:40  hansangb edited 01 Nov '10, 14:41 |
If you're seeing "bursts" of out-of-order packets, you also want to check the elapsed time of the "burst." I've seen situation where a high-speed, low-latency network will actually create what appears to be a horrendous flood of out-of-orders, but a time check shows that the mess unscrambled itself within a second or so...don't let the out-of-order flag send you down a rabbit hole in your troubleshooting. answered 03 Nov '10, 20:20  wesmorgan1 |
Thanks to Sake for clearing that up... yes - you're right that TCP won't pass out-of-order packets up, but it's the buffering delay caused by out-of-order packets data that slows things down. Whether we notice that all depends on the application behavior. Clear as mud?