Hi I have a user who periodically has poor performance when using an on-line database, he connects over the internet using HTTPS. The client connects directly through the firewall missing out the proxy and is using IE8. I've taken a packet trace and I've noticed that the TCP streams often aren't shut down gracefully. The client will ACK the last packet received from the server then there will be a 60 second wait before the client sends a FIN and the connection closes. This is the IE time out because when I changed HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\InternetSetting\KeepAliveTimeout value to 30 seconds the delay changed to 30 seconds. This is what a trace of the closure looks like If the previous packet to the first FIN is data from the database then it closes without any delay I think the user experiences performance issues when all 6 available TCP streams are timing out. Is that a reasonable explanation for what is going on? Can anyone suggest why it might be behaving like this? Thanks in advance for any help. Alex asked 18 Jul '13, 03:43 troubleshootist |
2 Answers:
In the old days, it was netiquette to open not more than two connections to the same server:
These days, the amount of persistent connections per server has increased to improve performance. You can now download 6 objects in parallel (IE8+) instead of 2. Whenever a connection is not in use, the inactivity timer starts to count down. When it hits 0, the connection is closed. However, if the user needs a new object from the server (because it clicked on a new link on the site), the browser will look if there are any open idle connections to the server and use it instead of opening a new connection. This saves TCP (and in your case also SSL) connection setup time. So the closure of the sessions is normal as the browser waits for the user to click on a link to get new objects over the current connection. If you see the connection closed straight away for some sessions, this can be caused by several things: - The server issued a "Connection: Close" in the last response in the session - The request limit for the connection has been reached (most web servers keep a request count per connection and have a fixed limit on each connection In your colored packet list, the client requested some web page objects over 3 connections, then there was user-time (reading the page etc). The connections idled out and almost 32 seconds after the last session idled out, the user requested a new http object, so a new connection had to be created. All in all very normal behavior. So what does the user report as "Slow performance" and do you have a tracefile of a good session and one for which he reported "slow performance"? answered 18 Jul '13, 15:02 SYN-bit ♦♦ Although the user might be using HTTP/1.0 with "Connection: Keep-Alive" headers to keep the connection open. In HTTP/1.0 persistent connections were optional, in HTTP/1.1 they are enabled by default. Either way, closing the connections or keeping them open may indeed give different performance, but IMHO not in such an amount that a user might start complaining about performance. Especially since his performance is variable, while the use of persistent connections by the particular user is most probably not changing. You might want to focus on response times of individual HTTP objects. Either by doing SSL decryption or by looking at the direction of the "SSL ApplicationData" frames. That should give you an idea on the performance... (19 Jul '13, 03:41) SYN-bit ♦♦ |
IE or in general a browser kepping HTTP sessions open for a certain keepalive period is absolutely normal and the snippets you provided show a perfectly fine graceful TCP connection teardown with both sides having sent and ACKed their FINs. Why do you suppose a performance problem resulting from that regular behaviour? Is your client in any way limited to having only 6 TCP connections open e.g. on the firewall? answered 18 Jul '13, 05:11 Landi E8 only supports a maximum of 6 concurrent HTTP connections to a single server (http://weblogs.asp.net/mschwarz/archive/2008/07/21/internet-explorer-8-and-maximum-concurrent-connections.aspx) If all these connections are being used wouldn't the user experience the browser freezing? So here is an example where three separate TCP streams (I've colourised them for easier reading) Send ACK's then wait 30 seconds and it's not until they've closed that the new (yellow) TCP stream is connected. I have seen this happening with all 6 connections at one time. Is there a graph I can produce in Wireshark that will trace the number of open connections to the database at any given time? You'll have to forgive me I'm not really a comms guy but shouldn't something ideally be initiating a TCP close rather than rely on the IE timeout? (18 Jul '13, 05:45) troubleshootist You're right in expecting a TCP close, but that "timeout" is rather a feature afaik based on HTTP1.1 because it enables you to re-request additional information using the very same TCP session already established after a certain period of time. What really wonders to me is the fact that whatever application there is - is using IE and its limited 6 connections to apparently handle whatever it's doing instead of requesting all the needed info inside a single HTTP session once established. For what I see - and there might be coming additions/correction from the other users here - I'd EITHER expect the application to use HTTP1.0 without keepalives and thus closing each TCP session when the requests have been answered OR use HTTP1.1 with keepalives but then REUSE the open connections instead of spawning parralel ones esp. to the exact same destination server. (18 Jul '13, 10:52) Landi |
Hi
Thanks for all the replies, it's great to get feedback and input from experts.
Unfortunately the fault is intermittent, and I don't have a logon to the database so I can't replicate the issue.
I've set up the users PC to start dumpcap on logon and capture over port 80 and 443. I've placed a shortcut on his quick link bar to press every time he experiences issue and that logs the time and allows him to add comments. Unfortunately this gives me large logs and not very accurate information.
When I first started the capturing data from him we sent him via the proxy which had SSL decryption set. What I initially thought was problematic was it seemed to be communicating in HTTP 1.0
When we used Fiddler to look inside the tunnel it was using 1.1 to communicate. Other browsers used 1.1 though. Could this be a problem for us