This is a static archive of our old Q&A Site. Please post any new questions and answers at ask.wireshark.org.

Packet capture and Apache logs do not agree - data discrepencies

0

I recently started to see a burst of HTTP '400' response codes in our Apache logs running on RHEL5 servers. According to our logs we are serving up a '400' every few seconds. When I manually cut/paste the URL receiving a '400' in a browser it works without any problems. What's odd is that Apache is showing those responses taking 1 second to complete.

To debug I ran tshark on one of the RHEL5 web servers. I didn't enter in any abnormal packet capture. Here was my syntax: tshark -i bond0 -f "host port 80" -w ~/test.cap

I am unable to find the '400' errors that I see in the Apache logs in the capture. What is odd is there are '400' errors for queries that I am unable to find in the Apache logs, for another URI entirely. The number of '400' errors I see in the capture for URI X and the number I see in the Apache logs for URI Y do not line up directly. We also do not have any sort of rewrite rules or proxies in place that would account for a URL being altered.

I ran a wide open packet capture (i.e. not using 'host port 80') on our border router and see the same behavior. What is odd is that I do not see the web server's '400' error ever leaving our network.

I have run subsequent captures wide open, capturing all traffic just in case something was out of whack with the filter. I also ran captures against every interface on the servers in case something might be wrong or misconfigured with the bonded interface but that isn't the case.

I have always been able to directly correlate entries in wireshark to log entries. This is a first for me. Has anyone run into this problem before?

All servers are 64 bit running RHEL5, wireshark version wireshark-1.0.15-1.el5_5.1. I also analyzed the capture on a 64 bit windows laptop running the 64 bit 1.4.1 wireshark version.

asked 10 Nov '10, 08:46

devrick0's gravatar image

devrick0
1333
accept rate: 0%


3 Answers:

0

That capture filter isn't valide - "host port 80." You should have seen an error message though.

Can you open the "wide open" traces you captured and apply a display filter for http.response.code == 400 - there is a chance you might not be able to 'see' the 400 errors in the packet list if the TCP preferences are set to "allow subdissector to reassemble TCP streams." I usually set that off on standard HTTP communications so I can see the response codes clearly in the list.

If the 400 errors are being sent and Wireshark can see them, so should you.

answered 11 Nov '10, 17:10

lchappell's gravatar image

lchappell ♦
1.2k2730
accept rate: 8%

Yes, my mistake, it was "tcp port 80" when I tried the filter.

I was able to have the inbound requests that were generating the '400' errors corrected so those are gone. I checked the load balancer and confirmed the errors are not leaving the server. I checked all the interfaces again and I don't see the '400' errors. My next thought is to look at wireshark and see if I can track down inbound requests that didn't receive a response from the web server.

I realize that I am lacking in wireshark skills so I ordered "Wireshark Network Analysis" yesterday on amazon.

(12 Nov '10, 07:25) devrick0

One nice feature of Wireshark is Statistics > HTTP > Packet Counter that will list the response types by number group (1xx, 2xx, 3xx, etc.). Check that out as well.

(12 Nov '10, 11:06) lchappell ♦

0

You can use justniffer for logging http traffic

answered 15 Sep '11, 05:19

Augustyn's gravatar image

Augustyn
1
accept rate: 0%

0

We had similar symptoms and the problem was that 2 HTTP requests seemed to be intermingled within the same TCP communication left open because of the keep-alive directive. In our case a proxy is talking to the apache, not the browsers directly. Because the queries are intermingled our proxy only detects the 400 response on one of the requests, I assume the other is considered to have timed out. Apache actually doesn't log anything at all in the access_log.

answered 13 Sep '12, 10:57

adamloopkin's gravatar image

adamloopkin
1
accept rate: 0%