This is our old Q&A Site. Please post any new questions and answers at ask.wireshark.org.

I know about editcap which removes duplicate packets from capture files.

However, I want to remove duplicate packets from libpcap itself. Is there any facility in libpcap ?

If not then, Is there any other way to achieve it ?

I mean by analyzing editcap source code, taking logic of removing duplicates from it and adding that into libpcap. Is it the proper way ?

asked 16 Nov '16, 02:13

Mehul28's gravatar image

Mehul28
0458
accept rate: 0%


The proper "unix" way would be to write a filter which would read the output of dumpcap from a pipe, store the frames in a history window of a depth stated as a command line parameter, and forward the frames to its output pipe on which tshark or Wireshark would be listening. This way, you'd not risk breaking anything in the libpcap. But it doesn't actually matter whether you implement the algorithm into libpcap or as a separate filter.

You would compare each new frame with all those in the buffer, ignoring the timestamp while comparing, and only forward it to the output if it would not match any of them. It is actually what editcap does, except it seems not to be able to act as a filter, i.e. to read its input and write its output from/to a pipe.

You just have to bear in mind that the first chunk of data you receive is the pcap or pcapng header, and that you must copy it to the output unchanged, and that by interpreting it you recognize how the individual frames are formatted.

Another thing to bear in mind is that the window must be controlled both by depth (number of frames) and time elapsed between the original and the suspected duplicate. Some frames carrying low level protocols which really exist in the network may be undistinguishable from one another, so only those really close by timestamp are duplicates caused by e.g. port mirroring on a VLAN, which may cause each packet to be captured twice (or even more times if a broadcast packet comes in through one port and gets out through ten others), depending on how the mirroring is implemented on the switch.

permanent link

answered 16 Nov '16, 02:24

sindy's gravatar image

sindy
6.0k4851
accept rate: 24%

edited 16 Nov '16, 04:50

Ok. Your suggestion is very good.

But my requirement is to make some changes in libpcap code to filter duplicates.

Do you have any idea on that ?

(16 Nov '16, 03:27) Mehul28

As stated above, the filtering algorithm is the same regardless whether you place it into libpcap or into a userspace executable. libpcap has no additional information which you could use to identify the duplicates easier than that - it sends out all the information it has itself.

(16 Nov '16, 03:35) sindy

ok. Thanks @sindy

(16 Nov '16, 03:49) Mehul28
Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×41
×37

question asked: 16 Nov '16, 02:13

question was seen: 1,059 times

last updated: 16 Nov '16, 04:50

p​o​w​e​r​e​d by O​S​Q​A