I would like rework the dissector for Google Protobuf found under the name "protobuf-wireshark-runtime-0.1.tar.gz" here: I cannot promise that I can finish the work for upstream, but I will try :-) Google Protobuf is not a protocol itself, it's more a generator for different protocols (message types in protobuf). Protobuf uses text files called protofiles which contain the definition for one or more message types. I think this may be comparable to Corba IDL or SOAP. Although encoded protobuf messages are somehow self contained (tags, length, wire type), they contain absolutely no information about the top level message type. So the message type must be known by wireshark (e.g. user defined mapping between TCP/UDP port numbers and the message type). As as first step I've defined preferences for the protobuf protocol so that the user can specify multiple (UAT) entries containing protofiles and the name of the messages types. Main question: How shall I implement the dissection of the different message types in wireshark? Alternative 1: Instantiate as much dissectors as there are different message types. Let wireshark dispatch the frame to the corresponding dissector (e.g. by TCP/UDP port number / decode as... dialog). Questions regarding alternative 1:
Alternative 2: Create only one dissector and differentiate between the different message type inside the dissector function. Questions regarding alternative 2:
asked 13 Dec '16, 22:25 Christian Eg... |
3 Answers:
Wireshark Generic Dissector (WSGD) uses a couple of text files to indicate what to dissect and how to format the dissection, a similar approach could be used for Google Protocol Buffers. answered 14 Dec '16, 05:38 grahamb ♦ Thank you for pointing to something which is similar to my purpose. It looks like WSGD registers exactly one protocol and one dissector per WSDG file. So lets think about alternative 3: Register one fixed protocol as a placeholder where the use can configure preferences for protobuf. I will check whether dynamic registration and deregistration of protocols is possible or the user has to restart wireshark after altering the list of protobuf message types. (14 Dec '16, 11:06) Christian Eg... |
Similar (I think) to Anders' suggestion, I think the right answer is probably close to Alternative 2 but slightly different (call it Alternative 3): one dissector per protocol where a protocol is effectively a collection of related messages. To answer your Alternative 2 questions: if your dissector is called based on a matching uint field (e.g., the TCP port number) then answered 14 Dec '16, 11:24 JeffMorriss ♦ Most protocols have either a fixed (iana assigned) port number or one particular TCP OR UDP port number can be configured in the protocols preferences. In the case of Google Protocol Buffers I don't like the scheme because ... ... protocol buffers are not a protocol itself, so they have no standard port numbers. ... protobuf messages may be sent via TCP, UDP or something completely different (e.g. HDLC or raw ethernet frames). ... I don't see a way to let the user configure this various possibilities via the protocols preferences page I would prefer when exclusively the "Decode As..." dialog can used to select protobuf as the dissector for data using a particular transport. Unfortunately it seems that I'll have to call dissector_add_for_decode_as() for each dissector table individually. In case my dissector is called this way, pinfo->match_uint may contain something like the port or protocol number, but this is not useful for me because if don't let the user configure the port number via the protocol preferences page, I also don't have any relation between pinfo->match_uint and the assigned protobuf message type (did anybody understand my problem?). Conclusion: I think that alternative 2 is not an ideal option. If multiple dissectors per protocol are not possible, I will have to register multiple protocols. (14 Dec '16, 12:16) Christian Eg... It's true that protocol buffers aren't a protocol themselves but generally people are implementing a protocol using protocol buffers (i.e., they make a protocol with N message types whose formatting happens to be the protocol buffers format). When they are implementing a protocol they often (in my experience) do things like choose a transport and a (probably not IANA registered) port. IOW I don't think just knowing the port number will tell you the message type. But it could tell you what group of message types the message will be from. If your dissector had a configuration file or UAT which allowed the user to specify the name of the protocol, the transport, the "port" (or equivalent), and the path to the proto file then you could establish a (dynamically configured) dissector for that protocol(**). If the port wasn't specified (or was 0) then the "protocol" would only be available via Decode-As. (**) Actually I think I'm starting to remember that the .proto file doesn't give you enough information. You need the "compiled" output to match the names to the numbers. Or something like that... (14 Dec '16, 12:39) JeffMorriss ♦ Perhaps I think too complicated, but how can I let the user choose a transport via UAT? Do I have to use UAT_VS_DEF() and code my own enum with transports which I think may be suitable for protobuf together with an additional uint field? Option 1: Offer a list of transports I personally use for protobuf --> Users who use another transport than me, will have to change the code. Option 2: Make protobuf available for ALL transports. No idea how to do this. In order to be available in "Decode As...", I would have to call dissector_add_for_decode_as() for ALL dissector tables. Not sure whether this makes sense. Both options depend on that my preferred/all dissector tables use an uint for matching the higher layer protocol. Decoding protobuf itself (by giving a textual proto file) is already solved in the original protobuf-runtime implementation. The protobuf definition is "compiled" on the fly inside wireshark and dissection is done by a protobuf feature called "reflection". (14 Dec '16, 13:28) Christian Eg... So as an example if you "told" the Tcp dissector to decode port x as protocol buffer and had a preference in the protocol buffer dissector to decode to port x as protocol type Y and loaded the protocol description for that port. That would work? In that case use a unit dissector table in the protocol buffer dissector per transport type and dynamically register a dissector with the Def there. (14 Dec '16, 14:32) Anders ♦ I'm not too familiar with UATs but I would think there would be a way to do it with an enum. Personally I don't think it's realistic to support ALL transports. Maybe there is a way to do it gracefully but I think trying to register for all dissector tables isn't very nice. You'll probably cover 99% of use cases by doing TCP and UDP and another 0.9% by adding SCTP. And anyway it's Open Source software: if someone actually wants to run protobufs over, say, Bluetooth, they can always add support when they find the need. And Wireshark doesn't get bloated with a bunch of stuff that nobody actually uses. (15 Dec '16, 05:59) JeffMorriss ♦ |
Thanks to all who gave information about this topic. I will try to summarize: 1. Similar workWireshark Generic Dissector does something similar. At startup it scans particular directories for configuration files and instantiates as much protocols+dissectors as there are configuration files. 2. Multiple dissectors per protocol?As nobody wrote something else, I assume that it is only possible to register one dissector per protocol. If I want to register more (e.g. dynamically) dissectors, I'll have to register them together with an individual protocol. 3. One or multiple dissectors at all?One protofile usually contains a "tree" of message type definitions. My intention for "alternative 1" was to have one dissector per "tree" of message types. So I think I agree to what JeffMorriss answered. It is most likely that I will register one "static" protocol+dissector for protobuf, which is able to perform "raw decoding" which means that no protofile is needed and only "raw" tag numbers + values can be shown in the proto tree. This "static" protocol will have a UAT in its preferences page, which allows the user to define further "dynamic" protocols (in contrast to let the user create configuration files as WSGD does). 4. Registration in dissector tables?Nobody wrote that it is usual/acceptable to register a dissector for all transport protocols. So the "normal" way is to add a enum preference with a selection of pre-defined transport names and a range preference for the (uint) port/protocol number. Remaining issues1. Dynamic deregistration is not possible.At least Because of this, I would also register "dynamic" protocol only on startup, so the user has to restart Wireshark after altering the list of dynamic protocols. 2. Missing this pointer in |
Hi, Disclaimer I don't know much about googles protocol buffers.
One way of looking at it is that Google protocol buffer provides you with the means of easily design a protocol which means that you actually have different protocols using the same frame work. E.G you need a dissector per protocol. It might be possible to use one dissector with configuration files or data structures for the protocol in question in which case there should be preference settings saying decode port xx as protocol yy or something along those lines or have a script creating a dissector from input files like asn1 based dissectors do.