Select Ip source addresses with 100 packets or more and save to another file

Question

I have a large pcap file that I split in several smaller files using editcap. I sorted each file by IP source address, (using |sort -k 3 command), but now I need to select only those IP source addresses with 100 packets or more, and then write only those addresses to another file for further filtering. I need to do that from command line because I have to write a bash script to do that for all smaller files resulting from the split (about 800 files). Help is greatly appreciated.

Accepted Answer

You could script something like this to create a list of all IP addresses with at least 100 packets in a single small file:

for file in `ls -1 <all-small-files>`
do
    tshark -r $file -T fields -e ip.src | sort | uniq -c | awk '$1 >= 100 {print $2}' 
done | sort -u > ip-with-100-packets.txt

Or you could use tcpdump on the large file (as it does not keep state and therefor can handle the big file without running out of memory). :

tcpdump -n -r netflix.pcapng  | cut -d ' ' -f 3 | sed -Ee 's/\.[0-9]+$//' | sort | uniq -c | awk '$1 >= 100 {print $2}' > ip-with-100-packets.txt

This will give you all ip's with at least 100 packets in the original large file.