I have a packet capture, and we're trying to debug a problem with our application just terminating a session. While looking for our problem, we've noticed that the packet capture is out of order. Capture was taken via Cisco span-port, with a single network card. What we're seeing is the packets are out of order. Not via timestamps, or packet number, but just the packet conversation (SYN ->, ACK ->, <- SYN ACK) Specific example is at 15:58:00.62606500, frame #68417, time 880.210825000. (This is also an instance of when our App just terminated) I've used tracewrangler to try and correct the out of order packets. It reported timestamps out of order, so I let it fix them. This had no affect, since it only goes by the absolute time, and the absolute time appears correct in the conversation. I also used it to anonymize the packets. Thoughts on this? NIC card driver on the Capture PC? Mike asked 12 Sep '14, 09:11  Mpking edited 12 Sep '14, 09:22 |
One Answer:
I would suspect the SPAN port to be the origin of the out-of-order problem, because I have never seen a single NIC being responsible for swapping incoming frame order like that. SPAN ports are not really reliable when it comes to packet order and packet timing, and there are also cases where the SPAN session even had an impact on the "real" communication by delaying it slightly. Anyway. Let's assume the order was SYN -> SYN/ACK -> ACK -> RST. Basically the server accepts the TCP connection and tears it down right away. I have seen that kind of thing happen when a server application gets notified by the TCP stack that there is a new connection (right after the three way handshake is complete), and the application decides it doesn't like the client for whatever reason (too many connections, client IP blacklisted, etc.). When it shuts down the TCP socket the stack sends a RST to the client. So you might want to check if the application has any kind of mechanism to reject specific clients. answered 12 Sep '14, 09:27  Jasper ♦♦ |
I'm ok assuming the order is correct, and we've diagnosed what the issue was with the application.
We just have never seen the out of order get mangled like that, and were curious how it came about.
Thanks. Mind if I ask what the issue was? :-)
Not really sure myself. We develop the client application, another company develops the server, and they've been dropping the connection. They just say it's our fault. :-)
lol... yeah, right :)