Is it possible to use a blacklist.txt file as an input address list for a wireshark or tshark display filter? My current blacklist of 4000 entries errors out in the display filter box ( syntax is ok , i wonder what max number of entries is ;-) ). I'd be very happy if something exists along the lines of:
asked 25 Aug '15, 07:40 Marc |
4 Answers:
One option would be:
answered 26 Aug '15, 17:24 Roland Thanks , that will do for the tshark part of my Q ..although i guess this will take 1 ip address at a time, filter the pcap file and then hop to the next , for 4000 ip adresses X 2183 tracefiles of 262 MB it'll be a long process... (28 Aug '15, 23:56) Marc In your question there was no mention of 2k trace files. The command in your last comment should be:
(30 Aug '15, 02:51) Roland |
You could write a Lua plugin script to do it, running inside tshark or Wireshark. You can pass the filename(s) of your blacklist to the Lua plugin using the command line, and have the Lua plugin open the blacklist file(s) and build a match table, and have the plugin create either a post-dissector or Listener tap, which will get invoked for every packet letting you check the source and dest IP addresses against the match table; and if it matches then have the plugin either (1) add a field called "blacklisted" which you can then use as your actual display filter for tshark/wireshark, or (2) have the plugin save the blacklisted packet to a new pcap file directly. answered 26 Aug '15, 19:02 Hadriel Ok, sweet , have to learn how to do that then ;-) But the extra possibilities do appeal to me! , Thanks! (28 Aug '15, 23:58) Marc |
Based on the information provided so far, I conclude, that you want to write the packets to/from blacklisted IP addresses to a new pcap file. If so, here is my simple Perl example. Save the code to a file and run it like this:
This will create a new pcap file called: capturefile1.pcap_blacklist.pcap (I did not invest any time to create a 'pretty' new name. I leave that up to you). If you want to process several files, run it in a loop on Unix/BSD like systems with a bash
Blacklist file: one IP per line!
Perl code:
HINT: The Perl script will be able to read pcap-ng files only if the libpcap version on your system is able to read pcap-ng, otherwise the script will throw an error! If that happens, either install the latest libpcap or convert the capture files to pcap style with editcap. Regards answered 29 Aug ‘15, 07:52 Kurt Knochner ♦ edited 29 Aug ‘15, 08:08 Hi Kurt, cool , i’ll give your Perl script a try too! Sofar i’ve come up with:
although only the first part works at the moment .. :-) (30 Aug ‘15, 01:51) Marc btw i know this should be a comment but have no clue on how to post code in a comment .. (30 Aug ‘15, 01:52) Marc I converted your answer to a comment. What I do, if I need formatting in a comment: I write the comment as an answer and if it looks O.K. in the preview I copy the text in the edit window and paste it into a commment. (30 Aug ‘15, 02:54) Kurt Knochner ♦ Hi Kurt, yep, that works! Thanks for the script. I will try it out in due time , for now i stuck with the path of getting the info from the t-shark summary command and then processing it afterwards, like so ( thanks to Roland for cleaning up the second line too)
only reason being speed at the moment ;-) (01 Sep '15, 23:54) Marc |
Here's a quick Python script for you. It outputs any IP addresses in the .pcap which match those in the blacklist file. Requires Python 3. Save this as a python file (.py) and run it (./filename.py). Change these in the code to the files you're analysing: [.pcap file = packets.pcap] [blacklist file = blklstfile]
Not quite what you're asking for but it will flag any blacklisted IP addresses if they appear in the PCAP file. answered 27 Aug '15, 13:52 tbm edited 27 Aug '15, 14:43 Right! Same here , looking like exactly what i want and all of these are different paths to get to the same answers: do i have any blacklisted ip's in a load of pcap files? It will be a bit of a learning curve do it either with Lua or Python but i will give it a whirl! However i do like the idea of letting the shark create a 'ip summary' first then automate some unix search power on those lists .. .. Something along the lines of "tshark -r <pcapfile> -q -z conv,ip > pcapfile_ip_summary.csv" then search those "for ip in $(cat blacklist.txt); do ... . (29 Aug '15, 00:03) Marc |
Right, the limit of the Wireshark display filter field seems to be 64K
which leaves room for about 2290 ip addresses in one filter pass ... so for now i guess i'll filter all files twice .. :-(
So much for trying that in cygwin, argument list is too long ..
Do you need only a list of non-blacklisted IP addresses in the capture files, or do you want to remove frames from/to the blacklisted IPs in all capture files?
The correlation i need is in which files do what blacklisted ip adresses occur?
my perl script does that. It does not output a simple list of files, it writes all frames from/to blacklisted IPs to a new file called *_blacklist.pcap. So, actually you get what you want (a list of files that contain blacklisted IPs), but it also gives you the whole communication from/to blacklisted IPs.