How to split captured traffic into multiple files depending on the delay between packets using tshark

Question

Dear All,

I want to split captured traffic based on the delay between packet. so that if the the delay between two packet is more than the threshold, save the trace in new file.

How can I do it using tshark commands?

Edit1:

here is the code I use for splitting traffic based on Sindy's answer:

    end=true
    id=1
    while $end
    do 
        tshark -r $trace -o mate.config:/home/zaha/Documents/mate.config -Y "mate.burst == $id" -w capture/flows/${nbase}/mainflow/${base}_$id.pcapng
        if [ -s capture/flows/${nbase}/mainflow/${base}_$id.pcapng ]; then
            id=$[$id +1]
        else
            end=false
        fi
    done

And use Sindy's MATE configuration with just changing the delay time.

But while loop didn't stop. should I use other option to check whether .pcapng file is empty or not?

Accepted Answer

1

If I get you right and you want to create a new file each time the pause between packets is longer than the threshold, it cannot be done while capturing. MATE could be a way to mark all frames belonging to the same "burst" with a unique numeric ID (mate.burst in the example code below), allowing you to filter the capture file on that ID and write the result for each ID into its own file, maybe using a script incrementing the ID and calling tshark -r my_capture.pcap -Y "mate.burst == $id" -w my_capture_$id.pcap in a loop until the output file becomes empty.

An example of MATE configuration with a gap threshold of 0.01 second follows:

 Transform make_start {
   Match (delay>0.01) Insert (start_flag);
   Match (number=1) Insert (start_flag);
 };
Transform set_all_to_0 {
Match (number) Replace (number=0);
};
Pdu any_frame Proto frame Transport mate {
Extract delay From frame.time_delta;
Extract number From frame.number;
Transform make_start, set_all_to_0;
};
Gop burst On any_frame Match (number) {
Start (start_flag);
};
Done;

answered 03 Sep ‘16, 02:35

sindy
6.0k●4●8●51
accept rate: 24%

edited 03 Sep ‘16, 23:54

Thanks for your reply, I want to do it after capturing, while reading traffic using -r and other filtering options in my processing phase. Is there other solution?

(03 Sep ‘16, 03:37) Zahra

I want to do it after capturing

Well, I’ve already described the suggestion as only applicable in the post-processing mode, so I’m not sure whether I get your reaction properly.

If you want to split the capture into parts up to the gaps between frames only after applying your display filter, I’m afraid you would have to use two instances of tshark in a chain. Tshark does support an idea of a separate “read filter” and “display filter” which use the same syntax but serve a different purpose, yet the two-pass mode of tshark which is currently mandatory for use of read filter has some trouble with MATE. But you may give it a try:

tshark -r my_capture.pcap -2 -R "your_filter_expression" -Y "mate.burst == N" -w my_capture_N.pcap

If it does not work, you have to revert to the chain method:

tshark -r my_capture.pcap -Y "your_filter_expression" -w - | tshark -r - -Y "mate.burst == N" -w my_capture_N.pcap

But it may not be possible on Windows.

(03 Sep ‘16, 03:51) sindy

Sorry I didn’t try MATE before, how should I add MATE configuration to my bash script. In https://wiki.wireshark.org/Mate/GettingStarted wasn’t any explanation for do it in ubuntu and also in bash script. Could you plz help me in this?

(03 Sep ‘16, 09:02) Zahra

1

tshark shares a common preferences file with Wireshark, so you can use Wireshark’s GUI to set the MATE configuration file. But you can use -o preference:value to override any of the values stored in the preferences file, so for our case, it would be -o mate.config:/full/path/to/your/mate/config/file

(03 Sep ‘16, 11:47) sindy

thanks, now tshark works fine, but there is a problem about check file is empty or not?

(03 Sep ‘16, 13:09) Zahra

1

The thing is that for a pcap or pcapng file, “empty” means “contains no frames”, not “has zero size”, so -s filename returns true even for empty files. The reason is that tshark creates the file and writes the header into it right at start, not as late as when writing the first frame. For pcap, the size of an empty file is 24 bytes; for pcapng, it is ~~128 bytes~~ variable depending on the environment.

Google of “file size bash” returns links to several Q&A sites with sophisticated answers; I would myself use wc -c < filename. So the whole replacement of if [ -s capture/flows/${nbase}/mainflow/${base}$id.pcapng ] ; in your script would be if (($(wc -c < capture/flows/${nbase}/mainflow/${base}$id.pcapng)>128)) ;

(03 Sep ‘16, 23:34) sindy

in my case empty pcapng file is 380, is there any differences by case?

(03 Sep ‘16, 23:55) Zahra

1

I haven’t analysed it deeply, but it depends on what tshark writes to the file at start. If the interface description(s) are written, the resulting size depends on the interface name(s) which are strings whose size depends on the environment. Just for the fun of it, can you post your empty pcapng file?

It seems that tshark has no option allowing to easily evaluate emptiness of a file, nor the manual says anything about a return code.

So to make the script portable, you may have to use a never matching display filter like -Y “eth and !eth” to generate an output file which is surely empty and use its size as a reference for the comparison.

(04 Sep ‘16, 00:16) sindy

thanks, here is my empty .pcapng file. http://s000.tinyupload.com/?file_id=03339871074200476632

(04 Sep ‘16, 00:31) Zahra

1

OK, so the size of an “empty” pcapng really depends on a lot of factors, not just on the environment where tshark runs.

in my case, there is just the signature of the application which has created it, which was tshark because the input file was a pcap one, not a pcapng one.
in your case, the original creator of the pcapng was dumpcap, so on top of its signature, there is also an interface description and capture filter because tshark has copied all this information from the input file to the output one.

(04 Sep ‘16, 01:04) sindy

showing 5 of 10 show 5 more comments