Complex Lua TCP Reassembly Problem

How are you performing the TCP reassembly in Lua, exactly? There are multiple ways to do it. Can you paste your script here, or something similar to your script?

Hey Hadriel, Thanks for your comment. Well I usually use 2 ways of reassembling my packets, but in this particular case I do something like this:

function isFooMessage(buf)    
//if header corresponds to FOO header then return (true, expected_length)    
//otherwise return false (heuristic dissection failed)
end

function dissectFOOMessage(buf, pinfo, rootTree)    
//doing some dissection    
//if successful return true otherwise return false (heuristic dissection)
end

function p_FOO.dissector(buf, pinfo, rootTree)
   local is_FOO_message, expected_length = isFooMessage(buf);

   if(~is_FOO_message) then
      //heuristic dissection failed
      return false;
   end

   local offset = pkt.desegment_offset or 0
   if(expected_length > buf:len()) then
      pinfo.desegment_len = DESEGMENT_ONE_MORE_SEGMENT;
      pinfo.desegment_offset = offset
      return nil
   end
   return dissectFOOMessage(buf, pinfo, rootTree);
end

//Makes the dissector heuristic : Dissects only if p_FOO.dissector
//does NOT return false
p_FOO:register_heuristic("tcp", function(...) p_FOO.dissector (...) end)

So basically all I do here is check if the message looks like a FOO message, then if it does and it's larger than the packet, I tell wireshark that I'll need another packet to finish decoding. In this dissector I purposely don't handle the case where my capture starts in the middle of a message but it's not in the scope of the scenario I described in my question.

That's not really enough detail to figure things out. :)

But anyway, doing a heuristic dissector with the end for TCP reassembly is quite tricky.

I'd suggest starting with it not being a heuristic dissector, but instead just add your p_FOO to the TCP port that your capture file has. Then you can work on the logic to make it work even when missing a last segment -which should be able to work, BTW.

What I don't see in the example script above is a while-loop, handling each of multiple potential messages inside a TCP segment.

For an example, see the 'fpm.lua' script and its associated capture file 'segmented_fpm.pcap' in the repository of Lua plugins.

Thanks for your reply Hadriel. This was just a sample of dummy code to keep things short. I also added the proto to the tcp table. I thought that all the heuristic registration did was to "cancel" the dissection if the dissector returns false, meaning that it turns out the packet was not of the expected format. I'm gonna try removing the heuristic part anyways and start from there.

I am not sure I understand what you mean with the "while loop thing". Are you talking about a while loop that would iterate through all the types of messages defined in the FOO protocol? If so then it would be in the function dissectFOOMessage(buf, pinfo, rootTree), which I did not develop to keep things short. I was putting what I thought was the right amount of code to show you how I handled the reassembly and nothing else.

I never found the page you linked in your comment, and I think I am going to find it very useful. I will look at your lua scripts there and see if I can get what I want to work.

I'll get back to you whether I am successful or not :). Thanks again!