We have a latency sensitive app wherein the receiver is waiting for two messages from the sender. Each Message size is ~500bytes and the application sends it within 1msec interval. The problem is that the receiver application sees these two messages 10msec apart which results in delays in our application. Digging further into the wireshark traces we see that on the sender side, the 2nd send message is not actually sent until the tcp.ack for the 1st message arrives (10msec later) which seems to be the root cause of the problem. Question is why would tcp wait for this ack and not send out the 2nd message immediately? We have set the “TCPNoDelay” option on the socket (and hence Nagle turned off) which seems like an obvious culprit, but it did not help. This is the pattern we are trying to understand: Send, <10msec>, Ack, Send I would think that with “TCPNoDelay” option set (and hence Nagle turned off) on the socket, assuming 10msec delayed Ack timer, the behavior should be: Send, Send, <10msec>, Ack Any ideas whats going on? Platform: Windows Server 2012 R2 asked 26 May '16, 10:50  shahankur11 edited 26 May '16, 11:00 |
3 Answers:
In every packet the sender sends the Push bit is set, so the Application is saying I'm finished, I have no more data to send the receiver ACKs the packet and the sequence starts over. the result of this is that there is never more than 309 bytes in flight I.E one packets worth of data, if you deploy this application over a WAN then performance will be dismal to say the least. If the Application had more data to send the Push bit would not/should not be set. This is not a case of the sender waiting for the ACK to the first packet, it's the cause of the Push bit being sent in every packet indicating the application has flushed the buffer and has no more data to send at that point. answered 30 May '16, 03:40  Mike_F |
I'm not sure I agree with the answer of @Mike_F. Setting the PUSH flag when sending shouldn't make the senders' application or tcp stack wait for an ack before sending any more data, that depends on the window size. RFC 793 and the amplification in RFC 1122 state that the PUSH flag informs a senders' tcp stack to send all buffered data immediately. At the receiver, the PSH bit forces the TCP stack to immediately send data to the application and not wait for the application buffer to be filled before doing so. answered 30 May '16, 07:37  grahamb ♦ edited 30 May '16, 14:02 |
Is it due to slow start for TCP connection? answered 31 May '16, 01:25  srinu_bel |
+1 to grahamb. The application is doing 2 sends, back-to-back without waiting for even the completions to fire. Each after each send the application calls flush buffer which probably sets the PSH flag. TCP Stack should not be waiting for Ack to come for the first send before sending the 2nd send out.
Well the ACK of the data is delayed by 10 ms. But the next DATA packet after the ACK is received are delayed sometimes by around 80ms at client side. So this 80ms are part of your problem, too.
So it could be application related, too. Seems to be an interesting apllication sends data all the time but never gets back an ack at application level. So why do you wait between two data packets?
But nevertheless, here I found some links around nagle: http://www.speedguide.net/articles/windows-8-10-2012-server-tcpip-tweaks-5077
http://kb.globalscape.com/KnowledgebaseArticle10438.aspx
Can you confirm that you don't run the server application on a virtual server?