This is a static archive of our old Q&A Site. Please post any new questions and answers at ask.wireshark.org.

TShark Inconsistencies - display filters and reassembled packets

0

Hi guys Related to: https://ask.wireshark.org/questions/47999/mate-with-tshark-on-wireshark-20

I've been trying to work out what's missing with our batch processing of files with TShark, and it looks like even when we use two pass filtering, I'm still not picking up all the data that I would expect to see.

I've got a batch file (details inline) where we can pass in a basic filter and get all the GIOP traffic back out again, but it drops all reassembled packets. Here's a screenshot of the raw capture filtered directly in Wireshark alt text

Typically I would want to batch filter the traffic to take only the CORBA traffic using a script like this: It doesn't actually matter what I pass in at this point, as a lot of the expected data gets dropped. alt text

However, I've just noticed that after doing so, my traffic looks like this - we are dropping any reassembled packets which I (possibly erroneously) would have expected to be there:

alt text

Is there anything you think I can attempt to batch filter out non-corba traffic to reduce the size of the sample files that I'm looking at?

Cheers Scott

Sample file is still relevant, but I've uploaded new, fresh sample files now, this one is filtered by TShark just on GIOP traffic - the best sensible examples are at 12.00.09

https://dl.dropboxusercontent.com/u/7916275/ISA_SBS_CAR2_00230_20160427115926_filtered.zip

This file is the unfiltered traffic. https://dl.dropboxusercontent.com/u/7916275/ISA_SBS_CAR2_00230_20160427115926.zip

asked 01 May '16, 17:37

Scott%20Harman's gravatar image

Scott Harman
46131319
accept rate: 50%

edited 01 May '16, 18:39


2 Answers:

2

@Scott Harman, although -R and -Y use the same "display filter" syntax, their roles are different. So what you have to do is to activate the two-pass mode but still use -Y.

What I did on your last "raw" file:

  • first, I've used tshark to let through only first 50000 frames, because the complete file kills Wireshark 2.0.3: while Wireshark dies silently, tshark says:

** (tshark.exe:1916): WARNING **: Dissector bug, protocol COSNAMING, in packet 51973: STATUS_ACCESS_VIOLATION: dissector accessed an invalid memory address

  • then, I've ran the following:

tshark.exe" -Y "giop.len == 131088" -r c:\Users\MyName\Downloads\ISA_SBS_modified.pcapng -w c:\Users\MyName\Downloads\isa_sbs_test.pcapng

The resulting file contained only the last TCP packet of each giop PDU matching the condition.

  • and then, I've added the -2 to the above:

tshark.exe" -Y "giop.len == 131088" -r c:\Users\MyName\Downloads\ISA_SBS_modified.pcapng -2 -w c:\Users\MyName\Downloads\isa_sbs_test_2pass.pcapng

...et voilà, all contributing TCP packets are there.

But beware of Bug 12161 - if you would attempt to filter by mate fields, things would go much worse (unless someone has silently fixed it in 2.0.3 without updating the bug information). I don't know, however, whether the MATE config may be used, and it is only the use of mate fields in filter expressions which causes the issue, or whether mere MATE processing of the frames is enough for things to go wrong.

answered 02 May '16, 01:10

sindy's gravatar image

sindy
6.0k4851
accept rate: 24%

edited 03 May '16, 03:05

Thanks @sindy- on 2.1.0 (based on a build from about a week ago) using both -Y and -2 works successfully with no additional filtering, as our Q_Quentin plugin successfully parses the data. Can you convert your comment to an answer so I can accept it. I'm going to roll my own 2.0.3 (having some Qt silliness at present) and can supply our plugin to see if you get the same results as me if that sounds sensible? Once processed, I'm able to use MATE in the gui so I can do silly things like: (mate.giop_req.giop_request_op contains "Fragments" && !mate.giop_req.giop_request_op contains "SubType") || mate.giop_req.giop_request_op contains "Clip" so I can track all media movement from a single filter

(02 May '16, 16:23) Scott Harman

@cmaynard was faster than me in converting my Comment into an Answer (thank you, Christopher). I have an issue with the following sentence:

I'm going to roll my own 2.0.3 (having some Qt silliness at present) and can supply our plugin to see if you get the same results as me if that sounds sensible?

English is not my first language so I didn't get what exactly you mean by "rolling" the 2.0.3. But if you want me to test your plugin with an "off-the-shelf" 2.0.3, there is no problem on my side. Just bear in mind the COSNAMIG dissector issue mentioned in the Answer, i.e. I can only do any tests on the trimmed capture.

(03 May '16, 03:19) sindy

Hi @sindy I built a version of 2.0.3 with our own custom giop plugin, so the COSNAMING error does not occur with our build, so the larger reassembled packets are correctly interpreted as Q_QUENTIN rather than COSNAMING or anything else. As a result, filtering on just GIOP with both -Y and -2 does work entirely successfully. Our build is available here if you want to verify my tests? https://dl.dropboxusercontent.com/u/7916275/Wireshark-win32-2.0.3.exe

(03 May '16, 18:06) Scott Harman

Put this way I'm not sure I understand the purpose of such a test - after all, what matters is that it works for you :-) Given that I know nothing about Q_QUENTIN, and that you haven't posted your MATE config (which could be another thing to verify), what additional information do you expect from my repetition of the test using your capture file and your customized version of Wireshark, i.e. the only difference would be the hardware used?

To check whether MATE still behaves the same in tshark as reported in Bug 12161, it is actually enough that I disable the COSNAMING dissector (and that I have you MATE config, of course).

What bothers me is whether I may use a representative part of your capture to file a bug regarding the COSNAMING (whatever it is) dissector, as I principally don't like that any contents of a capture file would be enough to crash Wireshark.

(04 May '16, 13:11) sindy

Mate config is here: https://dl.dropboxusercontent.com/u/7916275/quantel.mate and you can use my sample capture to file the bug. I believe that the MATE issue I've seen are the same as what I was previously enquiring about.

(09 May '16, 18:36) Scott Harman

1

Did you try replacing -2R %DFILTER% with -Y %DFILTER%?

Also, you might want to enclose %DFILTER% in quotes; otherwise more complicated filters will likely fail.

NOTE: For some strange reason I originally kept getting an error when attempting to post this answer with DFILTER enclosed in %'s, so I changed the format, but it seems to be working now after switching from Firefox (44.0.2) to IE (11.0.9600.18204). Here was the weird error I was getting:

Error %SYSTEM-E-NOTWATCH, not on my watch.

answered 01 May '16, 17:55

cmaynard's gravatar image

cmaynard ♦♦
9.4k1038142
accept rate: 20%

edited 01 May '16, 17:58

Hi @cmaynard - makes no difference - the resulting files are identical and still missing any reassembled packets. I added the heuristic dissector, but suspecting i'm missing something fairly fundamental.

(01 May '16, 18:04) Scott Harman

Is the same sample capture file posted to the other question relevant here as well? If not, can you post one?

(01 May '16, 18:11) cmaynard ♦♦

A couple of more questions:

  • What version of Wireshark are you using?
  • What are your preferences set to, in particular IPv4, TCP and GIOP and especially with respect to reassembly?
(01 May '16, 18:18) cmaynard ♦♦

Is there only a single TCP port for the giop traffic? Perhaps a filter such as "tcp.port eq <port>" would work?

(01 May '16, 18:35) cmaynard ♦♦

Reassembly is set on all protocols Currently running 2.0.1 but also testing the bleeding edge 2.1.0 development build I've updated the question with links to fresh sample files

(01 May '16, 18:38) Scott Harman

I can give it a shot based on the large port range that we have defined - will try that now

(01 May '16, 18:44) Scott Harman

Yeah - sadly the range for traffic for our app is about 400 ports, so it's a little unwieldy - but will have another stab shortly.

(01 May '16, 20:27) Scott Harman
showing 5 of 7 show 2 more comments