Scripting Follow TCP Stream -> Save As [Raw]

Question

I captured HTTP traffic using tcpdump. For each TCP stream I want to extract the RAW TCP contents, ideally all streams to the same file.

Manually, I am currently doing the following:

for each $i:
  select tcp.stream eq $i
  Save As [Raw] to file$i
concatenate files

Is there any way to script this using thark? I was trying for quite some time, but did not succeed.

Accepted Answer

1

Have a look at -z follow,tcp,raw option of tshark. Still needs some post processing, but should get you started.

answered 30 Jun '16, 05:53

Jaap ♦
11.7k●16●101
accept rate: 14%

I'h tried that before (tshark -r in.pcap -z follow,tcp,raw,0 -w f). As it seems, Ethernet/IP/TCP headers are still saved to the file. This is exactly what I wanted to avoid.

(30 Jun '16, 06:30) fk18

It is not a filter, it's a statistical tap, which generates statistical output on the console. In this case it also produces records of the data you seek. That is where the post processing comes in; having to pick up this console output and rework it into a form you can use further down in your toolchain.

(30 Jun '16, 07:58) Jaap ♦

1

What do you mean the Ethernet/IP/TCP headers are saved? If I use -z follow,tcp,ascii,0 on a capture file with HTTP traffic the actual followed data contains only the HTTP (switching to raw is similar but is harder for me to read :-)).

There are some brief headers telling you what the tool is doing (which can easily be grep'd out) as well as the frame list (which can be suppressed by adding the -q option) but there aren't any lower-level headers in there.

OHHHH, I see... The -z follow option sends its output to the standard output. If you're putting -w f and looking at the resulting file f then, yes, you're going to see the full headers because f is going to be a PCAPNG file. That's not the output of the -z follow option...

(01 Jul '16, 07:13) JeffMorriss ♦

Thanks to your answers and this post, the following script does exactly what I wanted:

infile=in.pcap
outfile=out
ext=txt
for stream in $(tshark -nlr $infile -Y tcp.flags.syn==1 -T fields -e tcp.stream | sort -n | uniq | sed 's/\r//')
do
    echo "Processing stream $stream: ${outfile}_${stream}.${ext}"
    tshark -nlr $infile -qz "follow,tcp,raw,$stream" | tail -n +7 | sed 's/^\s\+//g' | xxd -r -p > ${outfile}_${stream}.${ext}
done

(19 Jul '16, 23:11) fk18