This is a static archive of our old Q&A Site. Please post any new questions and answers at ask.wireshark.org.

SSL request hangs intermittently with Amazon LB SSL (Capture Included)

0

Greetings,

I've been trying very desperately to track down the source of an issue I'm having with a large SSL request to my Apache server (75K-100K). Every 20 requests or so, when issuing the request in FF, the request hangs. The server is Apache running in Amazon EC2 behind a LB that handles the SSL. (us-east-1)

I'm able to capture the wireshark trace (see http://cloudshark.org/captures/b9425eb191fc ) and I believe what I'm seeing is that the LB becomes unresponsive possibly under load? I see slowness in other browsers but in Firefox, the request often hangs indefinitely.

Notes on the trace file.

  • I'm issuing the request about once every 16 seconds, starting at 6891,7601, 8150, 9110
  • 9110 is the start of a failed (or hung) request. TCP stream 155
  • Looking at the ACK in 9188 or 9351 you can see delays in the ACK receipt from RTT.
  • As far as decoding, attempts to add the private key fail with "using server decoder" -> "no decoder available".
  • Keep-alive is enabled on the server.

Any input on this would be greatly appreciated. Original ideas were issues with the cipher or encryption issues, but I don't see errors in the trace other than normal terminations.

Chris

asked 03 Jan '14, 06:42

Chris%20in%20Ottawa's gravatar image

Chris in Ottawa
11114
accept rate: 0%

edited 03 Jan '14, 10:34


2 Answers:

2

I've been looking at a hung session (tcp.port==64544) in the trace and I think the problem is due to your windows client not retransmitting "in time". In the example below, its retransmission timer increases from 300ms all the way up to 48 secs.

A few observations.

  • The client is using Segmentation offload
  • Both client and server agreed upon SACK
  • The server never sends the SACK option reporting a gap

So this behaviour might be a combination of SACK and LSO not working together well. You could try truning it off as described here http://www.peerwisdom.org/2013/04/25/disabling-large-send-offload-windows/

alt text

answered 04 Jan '14, 06:41

mrEEde's gravatar image

mrEEde
3.9k152270
accept rate: 20%

Thank you for spending the time to go through this. I'll read up on these protocols, try your suggestion and run more tests from alternate clients, networks, etc. I'll post back. - Chris

(06 Jan '14, 05:54) Chris in Ottawa

0

I couldn't get the capture from the link. In any case did you try to run capture on both side (server and client)? what about fiddler?

answered 03 Jan '14, 10:31

Boaz%20Galil's gravatar image

Boaz Galil
56116
accept rate: 0%

Fixed up the link to the capture above. Not really sure how to go about getting the trace from the Load Balancer side of things (I have requests in to Amazon) Good idea about Fiddler, it might reveal something as well. Chris

(03 Jan '14, 10:37) Chris in Ottawa