This is a static archive of our old Q&A Site. Please post any new questions and answers at

Complex Lua TCP Reassembly Problem


Hello Wireshark Community!

I have been using lua to create various protocol dissectors for quite a while now and I am facing a problem I haven't been able to figure out. The way lua handles TCP reassembly is relatively simple however it seems like there is one shortcoming considering the following situation:

  • You are dissecting some streaming protocol FOO where a message unit is N byte long (N < 1500).
  • Your capture starts at the beginning of one message unit.
  • All the TCP packets you receive contains at least one cut off message (at the end of the packet)
  • And this keeps going on for as long as your capture is taken.
  • Let's say your whole capture contains 100 packets.
  • Let's say when you reassemble all those packets, you have 500 complete messages.

Now here is where it gets tricky.
CASE 1 If your last packet actually contains the bytes that complete the message cut off in the previous packet PLUS one complete message (and nothing else) then your Wireshark will display 99 TCP frames and 1 FOO frame. When looking at this FOO frame it will say something like "100 reassembled TCP segments.." with all the packet numbers composing this huge reassembled frame. You'll be able to look at your 500 decoded FOO messages and you'll be happy (so would I be).
CASE 2 If your last packet also contains the end of the previous message but also contains an incomplete message, then in Wireshark you will see 100 TCP frames marked as "TCP segment of a reassembled PDU" and basically even though your dissector has been able to decode all the messages (but the very last) in all those packets, no decoded FOO frame will be displayed. This is because the dissector "thinks" the dissection is not over because it is expecting the end of the last message in another packet (which is not available because the capture has ended.) This is a huge problem because it means that even though 499 FOO messages were successfully decoded, none of them will be displayed because the 500th one is incomplete!
So the question is how can you tell wireshark to just display what it has been able to dissect even if the last message was not complete? (Disabling dissector reassembly is not a solution because in this scenario you would only decode 1 message out of the hundreds actually available)

asked 10 Jul '15, 09:32

MarkoPaulo's gravatar image

accept rate: 0%

How are you performing the TCP reassembly in Lua, exactly? There are multiple ways to do it. Can you paste your script here, or something similar to your script?

(10 Jul '15, 09:50) Hadriel

Hey Hadriel, Thanks for your comment. Well I usually use 2 ways of reassembling my packets, but in this particular case I do something like this:

function isFooMessage(buf)    
//if header corresponds to FOO header then return (true, expected_length)    
//otherwise return false (heuristic dissection failed)

function dissectFOOMessage(buf, pinfo, rootTree)    
//doing some dissection    
//if successful return true otherwise return false (heuristic dissection)

function p_FOO.dissector(buf, pinfo, rootTree)
   local is_FOO_message, expected_length = isFooMessage(buf);

   if(~is_FOO_message) then
      //heuristic dissection failed
      return false;

   local offset = pkt.desegment_offset or 0
   if(expected_length > buf:len()) then
      pinfo.desegment_len = DESEGMENT_ONE_MORE_SEGMENT;
      pinfo.desegment_offset = offset
      return nil
   return dissectFOOMessage(buf, pinfo, rootTree);

//Makes the dissector heuristic : Dissects only if p_FOO.dissector
//does NOT return false
p_FOO:register_heuristic("tcp", function(...) p_FOO.dissector (...) end)

(10 Jul '15, 10:35) MarkoPaulo

So basically all I do here is check if the message looks like a FOO message, then if it does and it's larger than the packet, I tell wireshark that I'll need another packet to finish decoding. In this dissector I purposely don't handle the case where my capture starts in the middle of a message but it's not in the scope of the scenario I described in my question.

(10 Jul '15, 10:41) MarkoPaulo

That's not really enough detail to figure things out. :)

But anyway, doing a heuristic dissector with the end for TCP reassembly is quite tricky.

I'd suggest starting with it not being a heuristic dissector, but instead just add your p_FOO to the TCP port that your capture file has. Then you can work on the logic to make it work even when missing a last segment -which should be able to work, BTW.

What I don't see in the example script above is a while-loop, handling each of multiple potential messages inside a TCP segment.

For an example, see the 'fpm.lua' script and its associated capture file 'segmented_fpm.pcap' in the repository of Lua plugins.

(10 Jul '15, 12:54) Hadriel

Thanks for your reply Hadriel. This was just a sample of dummy code to keep things short. I also added the proto to the tcp table. I thought that all the heuristic registration did was to "cancel" the dissection if the dissector returns false, meaning that it turns out the packet was not of the expected format. I'm gonna try removing the heuristic part anyways and start from there.

I am not sure I understand what you mean with the "while loop thing". Are you talking about a while loop that would iterate through all the types of messages defined in the FOO protocol? If so then it would be in the function dissectFOOMessage(buf, pinfo, rootTree), which I did not develop to keep things short. I was putting what I thought was the right amount of code to show you how I handled the reassembly and nothing else.

I never found the page you linked in your comment, and I think I am going to find it very useful. I will look at your lua scripts there and see if I can get what I want to work.

I'll get back to you whether I am successful or not :). Thanks again!

(10 Jul '15, 13:11) MarkoPaulo

There can be multiple of your FOO messages inside a single TCP segment (inside a single Tvb buffer passed into the dissector function). So you need a while-loop to handle dissecting each of them. If the last one is only partial, then you should be setting the pinfo.desegment_offset to the beginning of that last partial one - but the full ones will have already been dissected, so even if your capture file doesn't have another TCP segment to be able to dissect that last partial one, you'll still have the complete messages dissected and in the tree, etc.

(10 Jul '15, 14:14) Hadriel
showing 5 of 6 show 1 more comments