This is our old Q&A Site. Please post any new questions and answers at ask.wireshark.org.

Hello people, I have a strange issue. I have an apche mod_proxying an application server. It works, but at any time randomly I get a connection refused from the server resulting a 503 to the user. It doesn't matter the traffic size, it can happen with 1 user too with just a request to a page. It happens like one request every 10000.

Wiresharking from the client I see the following, I send a SYN, then a TCP Retransmission after 3 seconds, and suddenly RST,ACK.

694839  88.387936   10.10.1.1   10.10.65.65 TCP 74  50854 > http-alt [SYN] Seq=0 Win=5840 Len=0 MSS=1460 SACK_PERM=1 TSval=2254806118 TSecr=0 WS=128
717344  91.387571   10.10.1.1   10.10.65.65 TCP 74  [TCP Retransmission] 50854 > http-alt [SYN] Seq=0 Win=5840 Len=0 MSS=1460 SACK_PERM=1 TSval=2254809118 TSecr=0 WS=128
744094  95.079609   10.10.65.65 10.10.1.1   TCP 60  http-alt > 50854 [RST, ACK] Seq=1 Ack=1 Win=5840 Len=0

From the server, there is no connection seen. There is a firewall in the middle separating dmz from private network. Supposedly that firewall doesn't filter packets, just let it through. The application server is working fine. OS doesn't show transmission error. It doesn't occur on full GC. Can anybody give me a hint what else can I look? Thank you.

asked 17 Sep '13, 15:25

Meli's gravatar image

Meli
11112
accept rate: 0%

Can you provide the actual packet capture including the full TCP headers of these messages (http://www.cloudshark.org/)? Also can you capture between the server and the firewall? Firewall in between always makes me suspicious, especially with a client-side capture (where the RSTs are not necessarily originated from the server). It's possible some kind of logic on the firewall (application-layer rules, or even session limiting) could be causing this. It is odd to see one not responded to and an RST to the other, though.

(17 Sep '13, 18:49) Quadratic

I'll try to cut/filter and post it, the file is huge. We capture the server tcp too, in that moment we should see an incoming connection with port 50854 but we couldn't find it. As I read, a retransmission would continue 3, 6, etc, seconds, and we don't see a pattern either because of the reset. Me mitigate this issue by decreasing the retry=0 in the mod_proxy, but it's not the solution. However, once in a while a 503 still occurs.

(18 Sep '13, 08:46) Meli

If someone have this problem, be sure to check the ethernet, not only ram, cpu, ulimit, tcp values. In our case we missed ifconfig, as networking guys said they didn't see anything strange. However, surprise! we found RX error in frames which are CRC failure due to something wrong with the network interface. Thank you.

permanent link

answered 18 Sep '13, 16:03

Meli's gravatar image

Meli
11112
accept rate: 0%

Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×104
×17

question asked: 17 Sep '13, 15:25

question was seen: 6,162 times

last updated: 18 Sep '13, 16:03

p​o​w​e​r​e​d by O​S​Q​A