I know about editcap which removes duplicate packets from capture files. However, I want to remove duplicate packets from libpcap itself. Is there any facility in libpcap ? If not then, Is there any other way to achieve it ? I mean by analyzing editcap source code, taking logic of removing duplicates from it and adding that into libpcap. Is it the proper way ? asked 16 Nov '16, 02:13  Mehul28 |
One Answer:
The proper "unix" way would be to write a filter which would read the output of dumpcap from a pipe, store the frames in a history window of a depth stated as a command line parameter, and forward the frames to its output pipe on which tshark or Wireshark would be listening. This way, you'd not risk breaking anything in the libpcap. But it doesn't actually matter whether you implement the algorithm into libpcap or as a separate filter. You would compare each new frame with all those in the buffer, ignoring the timestamp while comparing, and only forward it to the output if it would not match any of them. It is actually what editcap does, except it seems not to be able to act as a filter, i.e. to read its input and write its output from/to a pipe. You just have to bear in mind that the first chunk of data you receive is the pcap or pcapng header, and that you must copy it to the output unchanged, and that by interpreting it you recognize how the individual frames are formatted. Another thing to bear in mind is that the window must be controlled both by depth (number of frames) and time elapsed between the original and the suspected duplicate. Some frames carrying low level protocols which really exist in the network may be undistinguishable from one another, so only those really close by timestamp are duplicates caused by e.g. port mirroring on a VLAN, which may cause each packet to be captured twice (or even more times if a broadcast packet comes in through one port and gets out through ten others), depending on how the mirroring is implemented on the switch. answered 16 Nov '16, 02:24  sindy edited 16 Nov '16, 04:50 |
Ok. Your suggestion is very good.
But my requirement is to make some changes in libpcap code to filter duplicates.
Do you have any idea on that ?
As stated above, the filtering algorithm is the same regardless whether you place it into libpcap or into a userspace executable. libpcap has no additional information which you could use to identify the duplicates easier than that - it sends out all the information it has itself.
ok. Thanks @sindy