This is actually more of a question of port-mirroring instead of Wireshark, but anyway, want to see if any one here has any thought on this. We have port-mirroring done in our client site, however all the traffic analyse is marked as decryption failed, finally we able to find one weird after doing a few tcpdump and look at each of them in Wireshark: Server Hello always comes before Client Hello. Seems like couldn't be the case that this is how the traffic really is in the production environment, otherwise all the banks wouldn't be able to operate anymore. It must be the port-mirror somehow went wrong, but I couldn't imagine what could be wrong with the port mirroring. Or let's said if there is nothing wrong with the port-mirroring(i.e what we see is what we get), then what's the possible explanation behind this? asked 17 Jul '17, 20:27 waikeatahlok edited 17 Jul '17, 23:02 Jaap ♦ |
Do you mirror the whole VLAN, or do you mirror on a single port looking towards the client or server and send each direction to a different capturing card? Some switches reportedly do not keep the order of frames while mirroring, and two network cards in a PC may have different processing delay.
I see significant out-of-order frames with Cisco switches on SPAN and always thought that it was a Cisco issue. One day while playing with offloading I discovered that disabling all of the offloading features of the host NIC and all of sudden the frames were suddenly in order... so maybe not a Cisco issue at all but my issue all along.
I doubt that the frames are really sent out of order so it is likely some capture anomaly as @sindy suggests.
As @Bob Jones has mentioned this, I've remembered recently seeing TCP packets out of order while tcpdumping directly on the receiving machine, i.e. even without any port mirroring. But it never came to my mind that this could be related to tcp offloading as only the order was wrong, no packet was missing, while I've always understood tcp offloading as direct access of the network card to memory without the CPU (and thus the network driver) ever seeing the data being transferred, so having no chance to capture them by tcpdump or Wireshark.
What Bob says makes me think that some NICs and their drivers which use TCP offloading may change their behaviour if set to promiscuous mode, in order to facilitate capturing, but it is just a blind shot. You can try to check whether disabling TCP offloading on the NICs you use for capturing changes the packet order in your case.
... and if so this would make a valuable addition to the Wiki regarding capture setup.