I'm seeing approx 60 different unknown ethertypes from a capture of about 5000 frames. We recently set our switch SPAN ports to trunks to capture dot1Q tags. Jumbo frames were also recently enabled. I'm not sure if these changes could have anything to do with it. Some frames do show vlan tags. The interfaces are error free. There is a SPAN aggregator device between the network switches and linux capture server. Any ideas why there would be so many of these? Thanks! Here's a list of them: 0xfe7f;0xfcfe;0xe741;0xe10e;0xe103;0xdee5;0xc4c2;0xc446;0xc0fe 0xbb2b;0xa477;0x9e11;0x980c;0x8eb6;0x8eb5;0x8ea8;0x85c2;0x8562 0x855f;0x81f9;0x8146;0x7ffe;0x7ffd;0x7d38;0x64c9;0x64a7;0x646a 0x6457;0x5f37;0x5ee9;0x5eb4;0x5d1b;0x5d03;0x5d02;0x5d00;0x5bbe 0x5baf;0x5ba7;0x5b97;0x5aa6;0x557f;0x557e;0x551b;0x50fe;0x504e 0x504a;0x4b01;0x4687;0x39ad;0x38ae;0x35a7;0x29ce;0x290b;0x2909 0x2125;0x0b92;0x0b47;0x0b21;0x0b1f asked 29 Nov '12, 19:50  ry6 |
One Answer:
As @Guy Harris already mentioned, there are allways 4 bytes in front of what looks like ethertype + IP header. As you mentioned it's 802.1q I thought it could be the VLAN tag and then I modified the (random) ethertype to 0x8100 (vlan tag). Now it looks reasonable, if those IP addresses are on your network. The whole ethernet frame (strange mac addresses, maybe wrong VLAN ID, etc.) seems to be kind of 'randomized'. http://cloudshark.org/captures/fee1e5f2926e So something is consistently modifying (randomizing) the ethernet frame, up to the 802.1q tag. As you mentioned a SPAN aggregator device, it could be that device. What do you see if you capture directly on a SPAN port? Regards answered 01 Dec '12, 01:54  Kurt Knochner ♦ edited 01 Dec '12, 02:08 I'm not able to capture at the SPAN port today. It looks like 50% or so are normal readable frames. It looks like the first 4 bytes of these frames are extraneous. I was able to get the true MACs from IPs in the capture Kurt provided. The actual frame starts at the 5th octet. I'm beginning to think its the 10G NIC card. A filter was used at the Gigamon SPAN aggregator to filter the true source MAC. tshark included frames matching the true MAC and frames that have 4 unknown bytes at the beginning. This filter was applied at a second Gigamon port outputting to a different NIC card and the frames looked normal there. I may try swapping ports at the Gigamon to see what happens. Kurt - how did you go about modifying the frames/ethertypes? (01 Dec '12, 14:19) ry6 1
Ah, so something just added four bytes at the beginning of each frame. Interesting. But, if those 4 bytes at the beginning are extraneous, then where is the 802.1q tag of your frames?
I used a HEX editor. BTW: I just saw, that you've tried several times to give me extra karma. That's only possible, if you have karma yourself. If you like my answer, you can click on the "thumbs up" icon. If you don't like it, click on the "thumbs down" icon (see FAQ). (01 Dec '12, 15:02) Kurt Knochner ♦ 1 The Gigamon devices might also be adding headers, especially if they're GigaSMART devices. The switch might be stripping VLAN tags. (01 Dec '12, 15:47) Guy Harris ♦♦ Port Labeling of the GigaSmart devices is a good find! However, they say, that Wireshark is able to decode their GigaSMART headers !?!
That's a message from 2010. (01 Dec '12, 16:53) Kurt Knochner ♦ If the GigaSMART devices, when they add their headers to the frame, change the Ethertype to 0x22E5 and put the real Ethertype into the Gigamon header, Wireshark will decode them. (We also decode trailers that they add to frames, but that doesn't appear to be this issue.) If, however, they (or the switch into which they're plugged) are just adding 4 extra bytes before the destination MAC address, we won't recognize that. I don't know whether they would do that, however. Seeing what happens when bypassing the Gigamon devices would help here. (01 Dec '12, 17:40) Guy Harris ♦♦ All but a couple SPAN links are tagged. I was thinking this may explain where the dot1Q tags went in the posted capture. However, I've noticed that most of the traffic is untagged (approx 800 of 10,000 frames) with tshark where I'd expect to see most tagged given the setup. A capture with a different NIC, at the same time and same sources, had mostly tagged frames (7,000 of 10,000). I suspect the problem is with the NIC card/driver not being able to handle the tags correctly? I don't recall seeing this problem before using tagged SPAN links. The Gigamon does have a GigaSMART card. It is only licensed for de-dup functionality, which afaik does not manipulate packets. Additionally, I mapped traffic directly from the Gigamon input to the output and bypassed the card completely. I do plan to capture directly off a SPAN once back in the office. (01 Dec '12, 20:32) ry6 Hmm. If the card on the Linux box doing the capturing is stripping out VLAN tags from the raw packet and supplying them out-of-band and the Linux networking stack is providing them to userland in packet metadata and libpcap is using the metadata and attempting to add the VLAN tags back to the raw packet data, there might be an error in the "attempting to add the VLAN tags back" stage. There was a bug that showed up when capturing in "cooked mode", but this isn't a "cooked mode" capture. What version of libpcap is the program doing the capturing using? (02 Dec '12, 15:11) Guy Harris ♦♦
what are the real MAC addresses? If there are always 4 bytes added in front of the frame, I cannot see meaningful MAC addresses if I ignore those four bytes. There is something that looks like a VRRP address (02:00:5e:*) but then there are prefixes, that don't match any vendor (d2:4b:81:*, 00:14:81:*). (02 Dec '12, 23:33) Kurt Knochner ♦ libpcap version is 1.0.0. The real MACs are in the original cap ignoring the first 4 octets. It appears the stated vendor OUIs are from the modified cap. (03 Dec '12, 19:22) ry6 showing 5 of 9 show 4 more comments |
Some questions about those frames
Kurt,
To answer your questions:
Thanks.
To be precise, there appear to be four octets after the source IP address and before the Ethernet type.
So you have the machine doing the traffic capture (the Linux capture server) plugged into a SPAN aggregator device (who makes it, and what's its product name/module number?) plugged into SPAN ports on some (Cisco?) switches?
And are there any normal Ethernet frames in the capture?
The IPv4 total length of frame 2 (94) is consistent with a 112-byte frame with an 18-byte (6 bytes of destination, 6 bytes of source, 4 bytes of ???, 2 bytes of type) header.
What happens if you plug the capture server directly into one of the SPAN ports? (I.e., does it appear that the switch, or the SPAN aggregator, is inserting those extra 4 bytes?)