tshark output data in not completly correct

Question

Hello,

I am using tshark to retreive a lot of data from a captured file.

I am using the command:

tshark.exe -r C:\Temp\input.pcap -z "follow,tcp,hex,0" -w C:\Temp\output.ps -Y "tcp.stream eq 0"

The output contains extra header and trailer (unwanted) data in front of the requested data

I expect that the comandline is not correct, but I am unable to find the problem

Header data

X M<+ ÿÿÿÿÿÿÿÿ 3 TShark 1.12.8 (v1.12.8-0-g5b6e543 from master-1.12) X l 2 |Y€J J ÝåjÄ„<Š°+ä E <Õ[email protected] 9vN°daÀ¨ ò¾f#Œj’;A 9¥± ¬ ýXî l 2 I[€> > <Š°+ä ÝåjÄ„ E 0M ÿ¬_À¨ ò°da#Œ¾f mýbj’;Bp û\ ´ \ 2 E”€< < ÝåjÄ„<Š°+ä E (Õ[email protected] 9va°daÀ¨ ò¾f#Œj’;B mýcP9û \ \ 2 B˜€< < <Š°+ä ÝåjÄ„ E (N ÿ¬fÀ¨ ò°da#Œ¾f mýcj’;BP` Ô ´\ 2 ±Ç½â â ÝåjÄ„<Š°+ä E ÔÕ[email protected] 9p´°daÀ¨ ò¾f#Œj’;B mýcP9þ

Trailer data

     \       2 \RÁ<   <   <Š°+ä ÝåjÄ„ E  (a  ÿ¬SÀ¨ ò°da#Œ¾f mýcj“œ²P>ý“²  ´\      \       2 YSÁ<   <   <Š°+ä ÝåjÄ„ E  (b  ÿ¬RÀ¨ ò°da#Œ¾f mýcj“œ²P>ý“±  ´\      \       2 FˆÁ<   <    ÝåjÄ„<Š°+ä E  (Õ”@ 9v °daÀ¨ ò¾f#Œj“œ² mýdP9™¦        \

X M<+ ÿÿÿÿÿÿÿÿ 3 TShark 1.12.8 (v1.12.8-0-g5b6e543 from master-1.12) X l 2 |Y€J J ÝåjÄ„<Š°+ä E <Õ[email protected] 9vN°daÀ¨ ò¾f#Œj’;A 9¥± ¬ ýXî l 2 I[€> > <Š°+ä ÝåjÄ„ E 0M ÿ¬_À¨ ò°da#Œ¾f mýbj’;Bp û\ ´ \ 2 E”€< < ÝåjÄ„<Š°+ä E (Õ[email protected] 9va°daÀ¨ ò¾f#Œj’;B mýcP9û \ \ 2 B˜€< < <Š°+ä ÝåjÄ„ E (N ÿ¬fÀ¨ ò°da#Œ¾f mýcj’;BP` Ô ´\ 2 ±Ç½â â ÝåjÄ„<Š°+ä E ÔÕ[email protected] 9p´°daÀ¨ ò¾f#Œj’;B mýcP9þ

Answer 1

You are writing a pcap file with the option -w ..... That's certainly not what you want if you are using -z follow.

Please try this, to direct the output of tshark into a file.

tshark.exe -r C:\Temp\input.pcap -z "follow,tcp,hex,0" -q > C:\Temp\output.txt

BTW: You don't need the option "-Y ...." as you specified the TCP Stream already in the follow option.

The output would be hex data and you need to parse that with a script to get the payload.

===================================================================
Follow: tcp,hex
Filter: tcp.stream eq 2
Node 0: 192.168.90.57:41170
Node 1: 104.25.10.6:443
00000000  17 03 03 03 52 00 00 00  00 00 00 00 4e f1 aa 31  ....R... ....N..1
00000010  43 37 e6 74 ed cc cf fa  ec fb 18 0f 72 65 9d 57  C7.t.... ....re.W
00000020  df f9 d9 0c 9f 17 3a 3c  3a 42 11 92 11 4a 87 76  ......:< :B...J.v
00000030  30 9c e0 68 e0 e4 d1 8f  ce af 7a a5 bf 24 cd 06  0..h.... ..z..$..
00000040  30 47 cd 60 00 85 44 f8  00 3f 12 c1 5a 1b 16 cd  0G.`..D. .?..Z...
00000050  64 5f a7 df d5 75 1f b1  fe b7 5d b6 4a cf 76 71  d_...u.. ..].J.vq
00000060  dd 60 50 d1 30 4a e6 a1  4d 4f 3c 2e b4 3b bf 55  .`P.0J.. MO<..;.U
00000070  1b 37 d1 c1 2e 88 9b a3  04 9a ce 6e 8b 5b 1e 86  .7...... ...n.[..
00000080  f2 47 ad fe 45 25 e3 7f  03 e1 71 af 38 c2 c4 e3  .G..E%.. ..q.8...
00000090  5e bb 0b 0e 99 d5 7a c9  01 f1 d5 9d 49 51 5c 21  ^.....z. ....IQ\!
000000A0  0b 1e dd 57 fc 71 2d b2  53 01 78 bb 01 75 13 66  ...W.q-. S.x..u.f
000000B0  20 3e 94 04 6c 19 f8 b4  e9 92 45 0d 59 a5 e6 14   >..l... ..E.Y...

If you need raw data, please run this:

tshark.exe -r C:\Temp\input.pcap -z "follow,tcp,raw,0" -q > C:\Temp\output.raw

This will give slightly different output.

===================================================================
Follow: tcp,raw
Filter: tcp.stream eq 2
Node 0: 192.168.90.57:41170
Node 1: 104.25.10.6:443
1703030352000000000000004ef1aa314337e674edcccffaecfb180f72659d57dff9
bf24cd063047cd60008544f8003f12c15a1b16cd645fa7dfd5751fb1feb75db64acf
049ace6e8b5b1e86f247adfe4525e37f03e171af38c2c4e35ebb0b0e99d57ac901f1
6c19f8b4e992450d59a5e614b3b4bddff9f70cd6e485d6744d4157a7df44edad3cd7
5b651c7bae4ede2cc5fe85fbf626eb6ed75492256ffd7573bd4a779fe03705f84b32

If you need 'text' data, please run this:

tshark.exe -r C:\Temp\input.pcap -z "follow,tcp,ascii,0" -q > C:\Temp\output.text

++UPDATE++

based on your comment, I'd like to add this:

Now I understand what you want. It's the binary data, as saved in the of TCP Follow Stream pop-up in Wireshark.

Unfortunately, there is no such functionality in tshark. So, what you can do is this:

let tshark output hex data (-z follow,tcp,hex,0)
use a script (perl, python, whatever) or an editor (notepad++) to convert the output to something that xxd (included in gvim), understands (see below)
run xxd to convert the hex data to binary data

This is what tshark generates:

===================================================================
Follow: tcp,hex
Filter: tcp.stream eq 36
Node 0: 176.31.100.97:56001
Node 1: 192.168.32.242:9100
00000000  1b 25 2d 31 32 33 34 35  58 40 50 4a 4c 0a 40 50  .%-12345 X@PJL.@P
00000010  4a 4c 20 4a 4f 42 20 4e  41 4d 45 20 3d 20 22 36  JL JOB N AME = "6
00000020  31 39 30 38 33 2d 35 35  64 64 64 63 30 37 34 64  19083-55 dddc074d
00000030  65 38 36 2e 70 64 66 22  20 44 49 53 50 4c 41 59  e86.pdf"  DISPLAY
00000040  20 3d 20 22 33 30 37 35  33 37 20 61 70 61 63 68   = "3075 37 apach

this is what xxd needs to be able to convert it to binary

00000000  1b 25 2d 31 32 33 34 35 58 40 50 4a 4c 0a 40 50  .%-12345 X@PJL.@P
00000010  4a 4c 20 4a 4f 42 20 4e 41 4d 45 20 3d 20 22 36  JL JOB N AME = "6
00000020  31 39 30 38 33 2d 35 35 64 64 64 63 30 37 34 64  19083-55 dddc074d
00000030  65 38 36 2e 70 64 66 22 20 44 49 53 50 4c 41 59  e86.pdf"  DISPLAY
00000040  20 3d 20 22 33 30 37 35 33 37 20 61 70 61 63 68   = "3075 37 apach
00000050  65 20 36 31 39 30 38 33 2d 35 35 64 64 64 63 30  e 619083 -55dddc0
00000060  37 34 64 65 22 0a 40 50 4a 4c 20 45 4e 54 45 52  74de".@P JL ENTER
00000070  20 4c 41 4e 47 55 41 47 45 20 3d 20 50 4f 53 54   LANGUAG E = POST

So, basically, the process is:

run tshark: tshark -nr input.pcap -q -z follow,tcp,hex,0 > tshark.out.hex
open tshark.out.hex with an editor or use a script to do the following:
remove the 'comment' lines at the beginning
remove the double space in the middle of the output, so '35__58' becomes '35_58', etc.
save tshark.out.hex
run xxd: xxd.exe -r -g 1 tshark.out.hex tshark.out.bin

Result: tshark.out.bin is identical to your file 'correct_data.raw' in the ZIP file.

You will find xxd.exe in the installation directory of vim

Regards
Kurt