This is our old Q&A Site. Please post any new questions and answers at ask.wireshark.org.

I am troubleshooting a problem with lost syslog messages. The messages are sent via UDP. I run the current version of Wireshark 1.12.3 on W2K3 and can see 100% of the incoming messages. But the syslog receiver (in this case syslog4j) does only write about 80% of the messages to disk. Sidenote: there is not much traffic on the wire only around 100 messages in 5 minutes.

Is the fact that I can see all messages in Wireshark a clue that all is sent to syslog4j and that one dropping packets ? Or might there be a firewall messing with the data ? How can I find out if the order is "npf -> firewall -> syslog4j" or "firewall -> npf -> syslog4j" ?

asked 04 Feb '15, 13:36

marged's gravatar image

marged
6224
accept rate: 0%


It plugs into NDIS. There is no reason why syslog4j, or any other implementation of the syslog protocol, would need to use NPF or any other packet capture mechanism - it merely needs to have a UDP or TCP socket sending to (client) or receiving from (server) the appropriate port - so I would never expect to see either "npf -> firewall -> syslog4j" or "firewall -> npf -> syslog4j" or anything else containing both "npf" and "syslog4j".

Both NPF and a UDP socket will drop packets if they arrive faster than the consumer can process them.

If the steady-state arrival rate of packets is greater than the rate at which dumpcap (in the case of Wireshark) or syslog4j can write them to disk, the only thing you can do is probably to change the hardware or software so that packets can be written faster to disk.

At a rate of 1 message every 3 seconds, that's probably not what's happening. However, if that's just the steady-state rate, and the traffic is bursty, so that you get a burst with a lot of packets arriving in a short period of time and then no packets at all, averaging to 1 message every 3 seconds, if the program processing the messages can't write out the burst of packets fast enough, the buffer for NPF (in the case of dumpcap) or the UDP socket (in the case of syslog4j) could be overrun. Wireshark, and thus dumpcap, chooses a 2MB buffer by default; the UDP socket, however, probably has a smaller by default, although syslog4j might make the buffer bigger.

This all assumes that syslog4j isn't discarding the messages itself for some reason.

permanent link

answered 04 Feb '15, 16:16

Guy%20Harris's gravatar image

Guy Harris ♦♦
17.4k335196
accept rate: 19%

Thanks for the detailed answer and the link provided. I had a look at the PDF, especially chapter 4.2 but I still have one question.

I did not mean to say that syslog4j needs the npf. I suppose that both npf and the firewall are hooking into NDIS. I used Wireshark/npf to check if the data is "on the wire". But what if npf comes before the firewall ? And the firewall before syslog4j ? Then I would see the packets but the firewall could remove/drop them afterwards and they would never reach syslog4j I don't know if hooking into NDIS means chaining or if all hooks get the data in parallel

And a sidenote: yes, the messages come in in bulk, the sender sends them every five minutes. But the message should perfectly fit into the 2MB buffer

(04 Feb '15, 22:02) marged

NPF does not come before the firewall, so that's not an issue.

NDIS can have multiple other protocols plugged into it, and, for each of the protocols, if a packet arrives, then, if the filter (not to be confused with, for example, a libpcap/WinPcap capture filter) the protocol specified accepts the packet, delivers a copy of the packet to that protocol. There's no chaining. I couldn't find a good discussion of this on the MSDN Web site, but here's a picture that sort of shows it.

And the 2MB buffer is for NPF, i.e. for dumpcap and thus Wireshark, as well as tcpdump etc.. I don't know what the default socket buffer size for a UDP socket is on Windows Server 2K3, but I wouldn't be surprised if it were less than 2MB.

(04 Feb '15, 23:06) Guy Harris ♦♦
Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×23
×19
×2

question asked: 04 Feb '15, 13:36

question was seen: 1,302 times

last updated: 04 Feb '15, 23:06

p​o​w​e​r​e​d by O​S​Q​A