Dump raw packet ‘data’ field only?

Question

Here's the problem:

I have some network traffic pcap files. I need the raw data layer packets from these files, which I can get (in one file) by right-clicking the 'data' layer, and 'Export selected packet bytes...', but I then have to combine these raw files for all packets in the capture.

I can print just the data, in ASCII format, using tshark:

tshark -r infile -T fields -e data

But, when I try to do the same thing for the raw data:

tshark -r infile -T fields -e data -w outfile.raw

I'm not sure that outfile.raw is what I want. A file that I've converted manually can't be opened in Wireshark, as it can't understand the format. The one generated using the above command (outfile.raw) can be, so I'm assuming it's still outputting the headers.

Is there any way to either convert the hex/ascii back to raw packet data, or to output JUST the data payload in raw format?

I have many files to convert in this fashion, and being able to script the process would greatly speed things up...

Accepted Answer

To answer my question for future googlers: I used @Kurt's suggestion, and converted the ascii to binary.

I had to remove the newlines that tshark adds in between the packets, so:

tshark -r infile -T fields -e data | tr -d '\n' > tempfile

I then used the following short python script to convert from tempfile to binary: import binascii import sys string = open(sys.argv[1],'r').read() sys.stdout.write(binascii.unhexlify(string)) # needs to be stdout.write to avoid trailing newline

You can then redirect the output of the python script to a file, and you get exactly what I need.

Also, it turns out this is equivalent to "follow tcp stream" in the Wireshark gui, and exporting the data as raw. EDIT: the reason 'follow tcp stream' wouldn't have worked in this situation is that I had two streams I needed in one file, in the order they were sent.

Answer 2

I think -w forces tshark to write the packets out again in pcap format, which you can easily verify by running the capinfos tool, e.g. "capinfos outfile.raw". It will tell you what File Type it is.

Maybe you can try to redirect the console output into a file by using the ">" operator. I haven't tried it, but maybe something like this works (or gives you an idea):

tshark -r infile -T fields -e data >outfile.raw