TCP: correct calculation of RTO

Question

I'm having here an issue with an Application Delivery Controller (ADC) were the vendor and I can't agree on the reason for the issue.

Setup:

ADC is running a virtual server on 172.31.90.25:443
Round Trip Time (RTT) between client (172.20.144.129) and ADC is approx 220-230 ms
There is a "tcpprofile" configured on the ADC with a option "TCP minimum Restransmission Timeout(RTO) in millisec" with value 200
The initial congestion window (cwnd) on the ADC is 4MSS
Slow start increment is configured with value 2

Problem:

Client establishes a TCP connection to ADC and send first data (TLS Client Hello)
ADC sends first data (TLS Server Hello; Seq 1 + Seq 108)
ADC spurious retransmits these two packets after ~205 ms
Client acknowledges first data packets of server after approx 240 ms
ADC retransmits all data packets after ~205 ms (the entire TCP session)
ADC never increments cwnd
Throughput is low

Notes:

With a RTT smaller than 200 ms everything is fine (cwnd increments)
According to vendor support this behaviour is normal and minRTO should be set to 2-3 x max RTT
My understanding is that RTO should be dynamically calculated per TCP session.
Although there is a Riverbed Steelhead option in the SYN packet. Connection is not optimized.
Trace captured on the ADC.
Capture file can be found here

Questions:

If the vendor support is right it would be necessary to configure a minRTO value greater than the worst RTT, otherwise cnwd would not increment for a client with a high RTT. Am I right?
If I configure such a high minRTO clients with a small RTT (e.g. 20 ms) would have to wait minRTO to get a retransmit. This makes no sense for me. Opinions?

Side note:

For me ADC is buggy. I expect that RTO is calculated (at least) after the first ACK from the client.
minRTO should not affect calculation of RTO. It would only define the minimum value.

Answer 1

If the vendor support is right it would be necessary to configure a minRTO value greater than the worst RTT, otherwise cnwd would not increment for a client with a high RTT. Am I right?

Yes in your case I would tune the minRTO to 300ms, as 200ms RTT is not uncommon but nowadays it is no low RTT and we must avoid to go into congestion avoidanvce mode when there is no real packet loss.

If I configure such a high minRTO clients with a small RTT (e.g. 20 ms) would have to wait minRTO to get a retransmit. This makes no sense for me. Opinions?

How often do you see a RTO? The most common retransmission is a Fast Retransmission, which stays untouched.

Answer 2

0

From a brief look at the Time Sequence Graph in your capture file this looks very much like a Nagle Algorithm + Delayed ACK issue, which is described nicely here. You might want to try turning off these options at least on the client and see if that helps.

answered 02 May '17, 08:51

djdawson
46●3
accept rate: 25%

Thanks for taking a look.

It's not a Nagle/Delayed Ack issue. RTT is ~240 ms. ADC (server) retransmits data already after 205 ms. The configured TCP profile (Server side) has Delayed ACK and Nagle disabled.

The client ACKs data after two packets (or after a PSH).

(02 May '17, 12:05) Uli

Thanks again for taking another look.

The client was located in Bangkok/Thailand and the server in Bavaria/Germany. The latency between these two location is approx. 100-120 ms (=> Round Trip Time ~220-240 ms).

The capture was taken at the server site.

In the screenshot @49.88 the server transmits data, @50.08 the server retransmits the data. The ACK from the client arrives @50.10. Due to the round trip time this is the ACK for the inital data transmit.

Therefore my inital question about calculating RTO at server site.

(02 May '17, 22:35) Uli

1

You're right - I was confused about where the capture was taken, and agree that this would be a server issue. Sorry for any confusion I may have caused.

Getting back to your original questions, I agree that it probably shouldn't be necessary to configure a minimum RTO, since TCP will compute it's own, and I also agree that it could cause problems for clients with small RTO. I also think it seems reasonable that setting a minRTO shouldn't affect the RTO calculation, but it also wouldn't surprise me if it did, either because of a bug or some other issue in the particular TCP stack. It would seem useful to see how things perform with TCP using its default settings, since the RTO behavior of the server certainly doesn't look right.

(03 May '17, 08:05) djdawson

Your "answer" has been converted to a comment as that's how this site works. Please read the FAQ for more information.

I'm also trying to move the other "answer" to a comment on this answer but the Ask site seems a bit broken in this respect at the moment.

(03 May '17, 08:22) grahamb ♦

When I configure a greater minRTO (e.g. 400 ms) the connection with a 205 ms RTT works smooth. cwnd increments normal.

Configuring no minRTO is not possible on the ADC. There have to be one.

(04 May '17, 08:26) Uli

Answer 3

I suspect the Nagle/Delayed ACK issue on the client, not the server. If you look at the following screenshots of the Time Sequence graph of the Server-to-Client traffic you can clearly see the client waiting for each packet for just over 200 msec before sending the ACK for the retransmitted packet, which was probably sent as a result of the RTO at the server. Since Nagel and Delayed ACK are not negotiated between TCP endpoints, it would be necessary to disable these options on the client in order to verify if that's the issue.

alt text