I have a slightly nuanced question on the recovery of TCP from packet loss. For sake of setting context, SACK is turned OFF. Additionally, we're dealing with a Linux host using CUBIC and a NetApp Filer using New Reno. Under Wikipedia's summarization of SACK (and why SACK is a good idea), I will base my question. "Relying purely on the cumulative acknowledgment scheme employed by the original TCP protocol can lead to inefficiencies when packets are lost. For example, suppose 10,000 bytes are sent in 10 different TCP packets, and the first packet is lost during transmission. In a pure cumulative acknowledgment protocol, the receiver cannot say that it received bytes 1,000 to 9,999 successfully, but failed to receive the first packet, containing bytes 0 to 999. Thus the sender may then have to resend all 10,000 bytes." The last sentence says may. This implies that it is not definitive and is probably conditional. Meaning, if SACK is off, the sender may need to resend all 10,000 bytes. My recollection of this from school (a long time ago) was that if SACK was disabled we would see a full re-transmission of the TCP window from the point of the loss by the sender. However, in practice, I am not seeing that in a data stream I have been analyzing between a Linux client and a NetApp Filer over TCP NFS. To illustrate the behavior I see, let's use Wikipedia's example, but add in a little more complexity. Say we have 10,000 bytes transmitted from the client to the Filer, and the filer receives bytes 2,000-5,000 and 7,000-10,000. In this case, because SACK is turned off, the Filer issues DUP ACKs when it notices it is missing data, only acknowledging up to the last contiguous byte of payload. Additionally, let us say that there are four missing packets in each missing gap. The Linux client then begins by sending the first missing packet after the last acknowledged byte, and then waits for the Filer to ACK it. Once ACKed, it sends the second missing packet, noticing only the last packets' payload was acknowledged and not the full window. The filer then sends an ACK back acknowledging up to the end of the second TCP packet, implying to the sender that it is missing more packets. We then start seeing the client exponentially increase RTO with each subsequent missing packet re-transmit. The nuance is that when we get to bytes 2,000-5,000, those are not re-transmitted by the client, and rather the sender is acknowledging up to byte 5,000 once data contiguous to byte 1,999 was successfully re-transmitted. So under what circumstances will the sender retransmit bytes 2,000-5,000 (and 7,000-10,000) and when won't it, assuming SACK is off? Additionally, are there similar nuances with SACK enabled? I did read of some issues with TCP sequence randomization when using SACK, for example. Thank you! asked 13 Nov '14, 09:29  swass |
One Answer:
Modern stacks do not discard incoming packets after they have determined packet loss; they report the last sequence number right before the gap in their ACK number, but keep everything after that. There's only one condition: the incoming frames need to fit into the advertised window size, which is the buffer reserved for incoming bytes anyway. So unless the sender sends more than the window size, everything is kept and ACKed in one single ACK as soon as the retransmission(s) came in, which totally makes sense - dropping packets after packet loss doesn't make any sense unless there is no buffer space. Of course, TCP implementation vary and there may be some that still do, but those are exotic stacks in most cases (e.g. embedded hardware where the developers don't care about getting it done right as long as it works in basic setups) answered 13 Nov '14, 10:11  Jasper ♦♦ |
Thank you! This seems to address my question and makes sense. It also explains why I was told (back in the 90's) in school that the sender would retransmit the full window. The network stack has clearly evolved a lot since then.