This is our old Q&A Site. Please post any new questions and answers at ask.wireshark.org.

I would like rework the dissector for Google Protobuf found under the name "protobuf-wireshark-runtime-0.1.tar.gz" here:

I cannot promise that I can finish the work for upstream, but I will try :-)

Google Protobuf is not a protocol itself, it's more a generator for different protocols (message types in protobuf). Protobuf uses text files called protofiles which contain the definition for one or more message types. I think this may be comparable to Corba IDL or SOAP.

Although encoded protobuf messages are somehow self contained (tags, length, wire type), they contain absolutely no information about the top level message type. So the message type must be known by wireshark (e.g. user defined mapping between TCP/UDP port numbers and the message type).

As as first step I've defined preferences for the protobuf protocol so that the user can specify multiple (UAT) entries containing protofiles and the name of the messages types.

Main question:

How shall I implement the dissection of the different message types in wireshark?

Alternative 1:

Instantiate as much dissectors as there are different message types. Let wireshark dispatch the frame to the corresponding dissector (e.g. by TCP/UDP port number / decode as... dialog).

Questions regarding alternative 1:

  1. Can I create multiple dissectors "inside" one protocol?
  2. Can I pass individual this pointers (e.g. void *) to the dissectors dissect function?
  3. dissector_add_for_decode_as() seems not to allow registering multiple dissectors for the same protocol. Am I correct?

Alternative 2:

Create only one dissector and differentiate between the different message type inside the dissector function.

Questions regarding alternative 2:

  1. In the dissect function, I need information about the lower layers (e.g. TCP/UDP port number) in order dissect the message correctly (choose the right message type). How can I get these information?
  2. Can I also get information about the lower layers if the message has not been received by other protocols than TCP or UDP (e.g. HDLC)?

asked 13 Dec '16, 22:25

Christian%20Eggers's gravatar image

Christian Eg...
6112
accept rate: 0%

Hi, Disclaimer I don't know much about googles protocol buffers.

One way of looking at it is that Google protocol buffer provides you with the means of easily design a protocol which means that you actually have different protocols using the same frame work. E.G you need a dissector per protocol. It might be possible to use one dissector with configuration files or data structures for the protocol in question in which case there should be preference settings saying decode port xx as protocol yy or something along those lines or have a script creating a dissector from input files like asn1 based dissectors do.

(14 Dec '16, 03:45) Anders ♦

Wireshark Generic Dissector (WSGD) uses a couple of text files to indicate what to dissect and how to format the dissection, a similar approach could be used for Google Protocol Buffers.

permanent link

answered 14 Dec '16, 05:38

grahamb's gravatar image

grahamb ♦
19.8k330206
accept rate: 22%

Thank you for pointing to something which is similar to my purpose. It looks like WSGD registers exactly one protocol and one dissector per WSDG file.

So lets think about alternative 3: Register one fixed protocol as a placeholder where the use can configure preferences for protobuf. I will check whether dynamic registration and deregistration of protocols is possible or the user has to restart wireshark after altering the list of protobuf message types.

(14 Dec '16, 11:06) Christian Eg...

Similar (I think) to Anders' suggestion, I think the right answer is probably close to Alternative 2 but slightly different (call it Alternative 3): one dissector per protocol where a protocol is effectively a collection of related messages.

To answer your Alternative 2 questions: if your dissector is called based on a matching uint field (e.g., the TCP port number) then pinfo->match_uint will be set to the uint value. This works for dissectors registered for specific ports but I'm not sure if this will include Decode-as cases.

permanent link

answered 14 Dec '16, 11:24

JeffMorriss's gravatar image

JeffMorriss ♦
6.2k572
accept rate: 27%

Most protocols have either a fixed (iana assigned) port number or one particular TCP OR UDP port number can be configured in the protocols preferences.

In the case of Google Protocol Buffers I don't like the scheme because ... ... protocol buffers are not a protocol itself, so they have no standard port numbers. ... protobuf messages may be sent via TCP, UDP or something completely different (e.g. HDLC or raw ethernet frames). ... I don't see a way to let the user configure this various possibilities via the protocols preferences page

I would prefer when exclusively the "Decode As..." dialog can used to select protobuf as the dissector for data using a particular transport. Unfortunately it seems that I'll have to call dissector_add_for_decode_as() for each dissector table individually.

In case my dissector is called this way, pinfo->match_uint may contain something like the port or protocol number, but this is not useful for me because if don't let the user configure the port number via the protocol preferences page, I also don't have any relation between pinfo->match_uint and the assigned protobuf message type (did anybody understand my problem?).

Conclusion: I think that alternative 2 is not an ideal option. If multiple dissectors per protocol are not possible, I will have to register multiple protocols.

(14 Dec '16, 12:16) Christian Eg...

It's true that protocol buffers aren't a protocol themselves but generally people are implementing a protocol using protocol buffers (i.e., they make a protocol with N message types whose formatting happens to be the protocol buffers format). When they are implementing a protocol they often (in my experience) do things like choose a transport and a (probably not IANA registered) port. IOW I don't think just knowing the port number will tell you the message type. But it could tell you what group of message types the message will be from.

If your dissector had a configuration file or UAT which allowed the user to specify the name of the protocol, the transport, the "port" (or equivalent), and the path to the proto file then you could establish a (dynamically configured) dissector for that protocol(**). If the port wasn't specified (or was 0) then the "protocol" would only be available via Decode-As.

(**) Actually I think I'm starting to remember that the .proto file doesn't give you enough information. You need the "compiled" output to match the names to the numbers. Or something like that...

(14 Dec '16, 12:39) JeffMorriss ♦

Perhaps I think too complicated, but how can I let the user choose a transport via UAT? Do I have to use UAT_VS_DEF() and code my own enum with transports which I think may be suitable for protobuf together with an additional uint field?

Option 1: Offer a list of transports I personally use for protobuf --> Users who use another transport than me, will have to change the code.

Option 2: Make protobuf available for ALL transports. No idea how to do this. In order to be available in "Decode As...", I would have to call dissector_add_for_decode_as() for ALL dissector tables. Not sure whether this makes sense.

Both options depend on that my preferred/all dissector tables use an uint for matching the higher layer protocol.

Decoding protobuf itself (by giving a textual proto file) is already solved in the original protobuf-runtime implementation. The protobuf definition is "compiled" on the fly inside wireshark and dissection is done by a protobuf feature called "reflection".

(14 Dec '16, 13:28) Christian Eg...

So as an example if you "told" the Tcp dissector to decode port x as protocol buffer and had a preference in the protocol buffer dissector to decode to port x as protocol type Y and loaded the protocol description for that port. That would work? In that case use a unit dissector table in the protocol buffer dissector per transport type and dynamically register a dissector with the Def there.

(14 Dec '16, 14:32) Anders ♦

I'm not too familiar with UATs but I would think there would be a way to do it with an enum.

Personally I don't think it's realistic to support ALL transports. Maybe there is a way to do it gracefully but I think trying to register for all dissector tables isn't very nice.

You'll probably cover 99% of use cases by doing TCP and UDP and another 0.9% by adding SCTP. And anyway it's Open Source software: if someone actually wants to run protobufs over, say, Bluetooth, they can always add support when they find the need. And Wireshark doesn't get bloated with a bunch of stuff that nobody actually uses.

(15 Dec '16, 05:59) JeffMorriss ♦

Thanks to all who gave information about this topic. I will try to summarize:

1. Similar work

Wireshark Generic Dissector does something similar. At startup it scans particular directories for configuration files and instantiates as much protocols+dissectors as there are configuration files.

2. Multiple dissectors per protocol?

As nobody wrote something else, I assume that it is only possible to register one dissector per protocol. If I want to register more (e.g. dynamically) dissectors, I'll have to register them together with an individual protocol.

3. One or multiple dissectors at all?

One protofile usually contains a "tree" of message type definitions. My intention for "alternative 1" was to have one dissector per "tree" of message types. So I think I agree to what JeffMorriss answered.

It is most likely that I will register one "static" protocol+dissector for protobuf, which is able to perform "raw decoding" which means that no protofile is needed and only "raw" tag numbers + values can be shown in the proto tree.

This "static" protocol will have a UAT in its preferences page, which allows the user to define further "dynamic" protocols (in contrast to let the user create configuration files as WSGD does).

4. Registration in dissector tables?

Nobody wrote that it is usual/acceptable to register a dissector for all transport protocols. So the "normal" way is to add a enum preference with a selection of pre-defined transport names and a range preference for the (uint) port/protocol number.

Remaining issues

1. Dynamic deregistration is not possible.

At least proto_deregister_protocol(), prefs_deregister_protocol(), uat_destroy() and deregister_dissector() are not exported to wireshark plugins. So there seems to be no possibility to remove "dynamically" instantiated protocols after the user removed them from the preferences UAT.

Because of this, I would also register "dynamic" protocol only on startup, so the user has to restart Wireshark after altering the list of dynamic protocols.

2. Missing this pointer in dissector_t

Most C functions which use a pointer to a user defined callback let also define a void * pointer which will usually be passed as first/last parameter when to callback is executed. When call_dissector_through_handle() runs a dissector, nothing like this is provided (only pinfo->current_proto seems to contain something useful, the last parameter (void *) is used for something different by call_dissector_with_data()).

For dynamically created dissectors this can be a difficult challenge because usually they all share the same dissector callback. WSGD creates a fixed "pool" of callback functions so there is an arbitrary limit on the number of dynamic dissectors the user can define. A similar solution I have found here, but generally the lack of a this pointer makes things more complicated than necessary.

permanent link

answered 15 Dec '16, 10:43

Christian%20Eggers's gravatar image

Christian Eg...
6112
accept rate: 0%

  1. There are two solutions: 1) The preferred is to not write your dissector as a plugin but rather as a built-in dissector (in epan/dissectors/); in fact we generally don't accept new plugins. The main exception to this rule is for dissectors that end up requiring lots of files. 2) The second option (if there are good reasons for it to be a plugin) is to submit a patch that exports those APIs so they can be used by plugins.
  2. The old way (before any dissector calls had void *'s) was to use pinfo->private_data. That can be mildly tricky and painful (especially with regards to restoring the old private_data after an exception was thrown) but it can be made to work. You could probably do that if adding void *'s to the existing stuff isn't feasible.
(15 Dec '16, 11:29) JeffMorriss ♦
  1. This is clear for me. I would like to start with a plugin and submit a built-in later. For the built-in, I'll try to (de)register the new protocols immediately in the apply callback.

  2. How can I influence what Wireshark passes to my dissector function via the void* pointer when it is directly called by wireshark (e.g. as a result of dissector_table match)? I could only find many cases where NULL is passed. I would like to specify the value directly after registering the protocol / dissector.

(15 Dec '16, 14:06) Christian Eg...
  1. So you could always make the necessary APIs public/exported to plugins during development (and drop that part of the change when it's time to submit it).
  2. (I assume you mean the void * in the top-level dissector function.) You can't. That's why I was suggesting using pinfo->private_data instead. Or maybe I'm mis-understanding your question.
(16 Dec '16, 11:36) JeffMorriss ♦

I would suggest doing the general dissector first and submit that with sample traces. Them it would be easier to discuss the next step.

(16 Dec '16, 15:33) Anders ♦
Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×124
×16
×4
×2

question asked: 13 Dec '16, 22:25

question was seen: 2,179 times

last updated: 16 Dec '16, 15:33

p​o​w​e​r​e​d by O​S​Q​A