This is our old Q&A Site. Please post any new questions and answers at ask.wireshark.org.

This is not exactly a Wireshark question, but I wasn't sure where to ask. If you have an idea for a community where to turn to, let me know. In the mean time...

I'm setting up a server/gateway machine (a linux desktop) that is supposed to capture all traffic coming through it via LAN or VPN. And by that I mean write to disk every single bit of network traffic content. The purpose is security and debugging review and analysis.

However, I don't want to just create a gigantic dump file of packets. For it to be useful, it has to:

1) Filter out some white-listed content. So not write to disk stuff like YouTube videos or BitTorrent downloads. (Ergo: white-listing rules by protocol, IPs/domains, header/content keywords, DPI. Also, some rules could alert me in real-time when strange things happen.)

2) Let me analyze that traffic in a useful manner. That is, I'd like to be able to easily query for things like:

  • all the files (html, js, png) that my Android Firefox loaded this morning when it crashed; (Assuming I know the time more or less, and the URL I tried to open.)
  • all the communications that an Android music app sent home this week, so that I can figure out how it knows my location;
  • the update my Ubuntu laptop downloaded yesterday, since from that moment it behaves weirdly.

etc, etc.

How do I go about this? Are there open-source tools that do this?

asked 27 Dec '13, 16:05

Ricky%20Moore's gravatar image

Ricky Moore
11113
accept rate: 0%

edited 27 Dec '13, 16:07


I don't think there is any open source tool that is able to fulfill all your requirements. However, there are interesting projects, that might be inspiring to you.

http://www.xplico.org/
https://github.com/aol/moloch (Article about Moloch)
https://labs.ripe.net/Members/wnagele/large-scale-pcap-data-analysis-using-apache-hadoop

So, it's up to you to grab one of those and 'tweak' it to meet your requirements.

Regards
Kurt

permanent link

answered 30 Dec '13, 12:52

Kurt%20Knochner's gravatar image

Kurt Knochner ♦
24.8k1039237
accept rate: 15%

Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×21

question asked: 27 Dec '13, 16:05

question was seen: 1,431 times

last updated: 30 Dec '13, 12:52

p​o​w​e​r​e​d by O​S​Q​A