This is our old Q&A Site. Please post any new questions and answers at ask.wireshark.org.

I captured HTTP traffic using tcpdump. For each TCP stream I want to extract the RAW TCP contents, ideally all streams to the same file.

Manually, I am currently doing the following:

for each $i:
  select tcp.stream eq $i
  Save As [Raw] to file$i
concatenate files

Is there any way to script this using thark? I was trying for quite some time, but did not succeed.

asked 30 Jun '16, 04:38

fk18's gravatar image

fk18
6114
accept rate: 0%


Have a look at -z follow,tcp,raw option of tshark. Still needs some post processing, but should get you started.

permanent link

answered 30 Jun '16, 05:53

Jaap's gravatar image

Jaap ♦
11.7k16101
accept rate: 14%

I'h tried that before (tshark -r in.pcap -z follow,tcp,raw,0 -w f). As it seems, Ethernet/IP/TCP headers are still saved to the file. This is exactly what I wanted to avoid.

(30 Jun '16, 06:30) fk18

It is not a filter, it's a statistical tap, which generates statistical output on the console. In this case it also produces records of the data you seek. That is where the post processing comes in; having to pick up this console output and rework it into a form you can use further down in your toolchain.

(30 Jun '16, 07:58) Jaap ♦
1

What do you mean the Ethernet/IP/TCP headers are saved? If I use -z follow,tcp,ascii,0 on a capture file with HTTP traffic the actual followed data contains only the HTTP (switching to raw is similar but is harder for me to read :-)).

There are some brief headers telling you what the tool is doing (which can easily be grep'd out) as well as the frame list (which can be suppressed by adding the -q option) but there aren't any lower-level headers in there.

OHHHH, I see... The -z follow option sends its output to the standard output. If you're putting -w f and looking at the resulting file f then, yes, you're going to see the full headers because f is going to be a PCAPNG file. That's not the output of the -z follow option...

(01 Jul '16, 07:13) JeffMorriss ♦

Thanks to your answers and this post, the following script does exactly what I wanted:

infile=in.pcap
outfile=out
ext=txt
for stream in $(tshark -nlr $infile -Y tcp.flags.syn==1 -T fields -e tcp.stream | sort -n | uniq | sed 's/\r//')
do
    echo "Processing stream $stream: ${outfile}_${stream}.${ext}"
    tshark -nlr $infile -qz "follow,tcp,raw,$stream" | tail -n +7 | sed 's/^\s\+//g' | xxd -r -p > ${outfile}_${stream}.${ext}
done
(19 Jul '16, 23:11) fk18
Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×752
×293
×91
×20

question asked: 30 Jun '16, 04:38

question was seen: 4,871 times

last updated: 20 Jul '16, 01:43

p​o​w​e​r​e​d by O​S​Q​A