This is our old Q&A Site. Please post any new questions and answers at ask.wireshark.org.

Hi I'm trying to baseline SYN rates on our network. I have a 145GB pcap and I'm trying to use tshark to extract them.

tshark -Y tcp[13]==02 -r "reallybigpcap.cap" -w syns.pcap

This eventually fails with an failed to allocate memory error. I think it's trying to load the whole file into ram and fails.

Can anyone suggest another tool that can do what I want ?

asked 17 Mar '15, 07:30

20fathoms's gravatar image

20fathoms
1111
accept rate: 0%

thanks all!

(18 Mar '15, 13:05) 20fathoms

If an answer has solved your issue, please accept the answer for the benefit of other users by clicking the checkmark icon next to the answer. Please read the FAQ for more information.

(18 Mar '15, 16:03) grahamb ♦

You might first try truncating the file to only the first 66 bytes (IPv4) to limit the amount of additional protocol decoding that tshark needs to do. Use editcap to create a new file.

editcap -s 66 reallybigcap.cap newsmallercap.cap

This will only grab the bytes up through the IPv4 header in each packet. Then, re-try your tshark command with the "newsmallercap.cap" file. Tshark will still use up memory, but hopefully less than before.

If there's a better way, please let me know too!

permanent link

answered 17 Mar '15, 11:08

zachad's gravatar image

zachad
331149
accept rate: 21%

You can use tcpdump or windump for this purpose as it does not keep (as much) state (as tshark). If the file is still too big to process, you can use editcap to split it into chunks and then process each chunk and then merge the filtered parts back to one file with mergecap.

Editcap and mergecap came with wireshark and tcpdump/windump are separate programs.

permanent link

answered 17 Mar '15, 12:05

SYN-bit's gravatar image

SYN-bit ♦♦
17.1k957245
accept rate: 20%

I wonder if this is a good use case for enhancing dumpcap to be able to read capture files?

By the way, I'll just mention another possible method for getting information from a big file. It involves the use of tail, rawshark and grep. Admittedly, it's sort of a hack, and you can't save the resulting packets to a new file, but it might be useful to someone:

tail -c +25 file.pcap | rawshark -d encap:EN10MB -F field1 -F field2 ... -F fieldn -R "tcp.flags.syn == 1" -r - | grep "1 \-$"

(19 Mar '15, 14:44) cmaynard ♦♦

I would rather see editcap extended with capture filter capabilities, seems like a more logical place :-)

(19 Mar '15, 15:42) SYN-bit ♦♦
Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×832
×26
×4

question asked: 17 Mar '15, 07:30

question was seen: 2,373 times

last updated: 19 Mar '15, 15:42

p​o​w​e​r​e​d by O​S​Q​A