TSHARK trying to capture image URLs to text file

Question

I am trying to capture to text file the url's of images downloaded during an interval of a minute via wi-fi on Windows 10 home. I have installed Wireshark 2.0.5 and it works. However, although I can capture traffic to file, for the life of me I cannot get a suitable display or capture filter to work. Can someone please tell me where I am going wrong?

Here are my failures:

C:\Program Files\Wireshark>tshark -a duration:60 -w c:\temp\shout.txt http.content_type =="image" 
Capturing on 'Wi-Fi' 
tshark: Invalid capture filter "http.contenttype ==image" for interface 'Wi-Fi'.
That string looks like a valid display filter; however, it isn't a valid capture filter (syntax error).
Note that display filters and capture filters don't have the same syntax, so you can't use most display filter expressions as capture filters.
See the User's Guide for a description of the capture filter syntax.
C:\Program Files\Wireshark>tshark -a duration:60 http.content_type =="image"
Capturing on 'Wi-Fi'
tshark: Invalid capture filter "http.content_type ==image" for interface 'Wi-Fi'.
That string looks like a valid display filter; however, it isn't a valid capture filter (syntax error).
Note that display filters and capture filters don't have the same syntax, so you can't use most display filter expressions as capture filters.
See the User's Guide for a description of the capture filter syntax. 0 packets captured
C:\Program Files\Wireshark>tshark -a duration:60 -w c:\temp\shout.txt -f http.content_type =="image"
tshark: A default capture filter was specified both with "-f" and with additional command-line arguments.
C:\Program Files\Wireshark>tshark -a duration:60 -f http.content_type =="image"
tshark: A default capture filter was specified both with "-f" and with additional command-line arguments.
C:\Program Files\Wireshark>tshark -a duration:60 http.content_type =="image"
Capturing on 'Wi-Fi'
tshark: Invalid capture filter "http.content_type ==image" for interface 'Wi-Fi'.
That string looks like a valid display filter; however, it isn't a valid capture filter (syntax error).
Note that display filters and capture filters don't have the same syntax, so you can't use most display filter expressions as capture filters.
See the User's Guide for a description of the capture filter syntax. 0 packets captured
C:\Program Files\Wireshark>tshark -f http.content_type =="image"
tshark: A default capture filter was specified both with "-f" and with additional command-line arguments.
C:\Program Files\Wireshark>tshark -f 'http.content_type =="image"'
tshark: A default capture filter was specified both with "-f" and with additional command-line arguments.
C:\Program Files\Wireshark>tshark -f 'http.content_type ==image
tshark: A default capture filter was specified both with "-f" and with additional command-line arguments.
C:\Program Files\Wireshark>tshark http.content_type ==image
Capturing on 'Wi-Fi'
tshark: Invalid capture filter "http.content_type ==image" for interface 'Wi-Fi'.
That string looks like a valid display filter; however, it isn't a valid capture filter (syntax error).
Note that display filters and capture filters don't have the same syntax, so you can't use most display filter expressions as capture filters.
See the User's Guide for a description of the capture filter syntax. 0 packets captured
C:\Program Files\Wireshark>tshark http.content_type == image
Capturing on 'Wi-Fi' tshark: Invalid capture filter "http.content_type == image" for interface 'Wi-Fi'.
That string looks like a valid display filter; however, it isn't a valid capture filter (syntax error).
Note that display filters and capture filters don't have the same syntax, so you can't use most display filter expressions as capture filters.
See the User's Guide for a description of the capture filter syntax. 0 packets captured
C:\Program Files\Wireshark>tshark http.content_type == image -w c:\temp\shout.txt
Capturing on 'Wi-Fi'
tshark: Invalid capture filter "http.content_type == image -w c:\temp\shout.txt" for interface 'Wi-Fi'.
That string isn't a valid capture filter (syntax error). See the User's Guide for a description of the capture filter syntax. 0 packets captured
C:\Program Files\Wireshark>tshark http.content_type == "image" -w c:\temp\shout.txt
Capturing on 'Wi-Fi'
tshark: Invalid capture filter "http.content_type == image -w c:\temp\shout.txt" for interface 'Wi-Fi'.
That string isn't a valid capture filter (syntax error). See the User's Guide for a description of the capture filter syntax. 0 packets captured
C:\Program Files\Wireshark>tshark 'http.content_type == "image"' -w c:\temp\shout.txt
Capturing on 'Wi-Fi'
tshark: Invalid capture filter "'http.content_type == image' -w c:\temp\shout.txt" for interface 'Wi-Fi'.
That string isn't a valid capture filter (illegal token). See the User's Guide for a description of the capture filter syntax. 0 packets captured
C:\Program Files\Wireshark>tshark -f 'http.content_type == "image"' -w c:\temp\shout.txt
tshark: A default capture filter was specified both with "-f" and with additional command-line arguments.
C:\Program Files\Wireshark>tshark -v TShark (Wireshark) 2.0.5 (v2.0.5-0-ga3be9c6 from master-2.0)
Copyright 1998-2016 Gerald Combs and contributors. License GPLv2+: GNU GPL version 2 or later This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Compiled (32-bit) with WinPcap (413), with libz 1.2.8, with GLib 2.38.0, with SMI 0.4.8, with c-ares 1.11.0, with Lua 5.2, with GnuTLS 3.2.15, with Gcrypt 1.6.2, with MIT Kerberos, with GeoIP.
Running on 32-bit Windows 10, build 10586, with locale EnglishUnited Kingdom.1252, with WinPcap version 4.1.3 (packet.dll version 4.1.0.2980), based on libpcap version 1.0 branch 10_rel0b (20091008), with GnuTLS 3.2.15, with Gcrypt 1.6.2. Intel(R) Atom(TM) CPU N550 @ 1.50GHz, with 2038MB of physical memory.
Built using Microsoft Visual C++ 12.0 build 40629
C:</code>

Answer 1

0

The responses of the form:

That string looks like a valid display filter; however, it isn't a valid capture filter (syntax error).

give the answer. http.content_type=="image" is a display filter so you should use it with the -Y flag, e.g. tshark -a duration:60 -Y http.content_type=="image".

answered 09 Sep '16, 02:22

grahamb ♦
19.8k●3●30●206
accept rate: 22%

edited 09 Sep '16, 06:12

Wow - fast response, thanks - BUT I still get the error:

C:\Program Files\Wireshark>tshark -a duration:60 -Y http.content_type =="image" -w c:\temp\shout.txt Capturing on 'Wi-Fi' tshark: Invalid capture filter "==image -w c:\temp\shout.txt" for interface 'Wi-Fi'.

That string isn't a valid capture filter (syntax error). See the User's Guide for a description of the capture filter syntax. 0 packets captured

(09 Sep '16, 02:34) MikeWa

The == in the display filter syntax confuses the shell, but although that can be fixed by closing the whole display filter expression into a pair of quotes as below rather than just the "image" part of it,

tshark.exe -a duration:60 -Y "http.content_type == image" -w c:\temp\shout.txt

you'll get another error:

tshark: Display filters aren't supported when capturing and saving the captured packets.

So bad luck here, you'll have to capture with just a capture filter (if you can define one narrow enough to make sense for your case), and use display filter later to further limit the result.

(09 Sep '16, 02:53) sindy

The == in the display filter syntax confuses the shell

Sorry, I had used one of the earlier attempts without spaces around the == and that does work on both PowerShell and Cmd shells. I've fixed my answer.

That doesn't help with the desire to capture and write to a file with a display filter though.

(09 Sep '16, 06:11) grahamb ♦

After a while writing powershell regular expressions to filter the "raw" capture without the image filter I read your second reply.

The command

tshark -a duration:60 -Y "http.content_type == image"

does indeed run, but it does not capture any image packet URLs.

I smell rats in the documentation, or worse, the software.

Mike

(09 Sep '16, 14:01) MikeWa

Of course, while I am sampling, I am running web queries to generate the traffic I need to catch, and this works well using:

tshark - duration:60 -w c:\temp\shout.txt tcp port 80

followed by :

tshark -r c:\temp\shout.txt -z http_req,tree >c:\temp\shouta.txt

and further treatment with regular expressions in Powershell

but is a far cry from what the Wireshark documentation led me to believe was possible.

(09 Sep '16, 14:19) MikeWa

Can you publish a sample capture file together with an example of what would be your desired tshark output so that I could check (using the latest available tshark, 2.2.0) whether there is no misunderstanding between what you want and what tshark can do? It is not possible to upload a capture file to this site directly, you have to use Cloudshark or any plain file sharing service and place a login-free link to the capture here.

(09 Sep '16, 14:47) sindy

Hello - Thanks Sindy for this helpful suggestion.

The capture file I have contains the complete set of records captured by the call " tshark -a duration:60 -w <some path=""> tcp port 80 "

What I am trying to do is pretty simple - I would like to use a Powershell script to launch tshark to capture all the URLs containing images (ideally by type jpg, gif) loaded by a browser on the local machine during a short defined period, and to save these URLs to a text file.

I am using the display filter "-z http_req,tree " and then filtering in Powershell.

In fact just saving the HTTP requests by HTTP host as text would almost get me there.

Mike

(12 Sep '16, 06:15) MikeWa

showing 5 of 7 show 2 more comments

Answer 2

Unfortunately the info you require is contained in two different packets, the "Content-Type:" header that indicates an image is in the response, and the URI is in the associated request. Wireshark filters, (display or capture) only operate on a single packet at a time to include or exclude that packet and do not allow the selection of 1 packet based on the content of another.

I can think of two ways to proceed:

Do two runs over the capture, the first identifies the frame number of request packets and the second run builds a filter for those packets, in both cases use the -T fields -e xxx option to limit the output to the items required, e.g.

$frames = tshark -r file -T fields -e http.request.in http.content_type contains image
$filter = ""
foreach($frame in $frames) {
    if ($filter -eq "") {
        $filter = "frame.number == $frame"
    }
    else {
        $filter += " or frame.number == $frame"
    }
}
tshark -r file -T fields -e http.request.full_uri $filter

or

Use MATE to build an association between the packets and add the URL to the response.