I have an RSPAN setup on a network with limited resources on the switches. Monitoring both ports and vlans however I'm only seeing Layer 3 packet info and some Layer 2 Mac,vendor and NIC info. None of the data that has Modbus/TCP,DNP3,Ethernet/IP (not regular IP info) ...etc. Is this because I'm don't have any sensors on the actual access switch? asked 05 May '16, 10:54  msmorten showing 5 of 11 show 6 more comments |
This is a static archive of our old Q&A Site.
Please post any new questions and answers at ask.wireshark.org.
What does the network equipment supplier say? Does it have any forums?
The network equipment vendor is Cisco. SPAN'ing mostly 2950/2960 switches and the core is a 3850.. Based on this I should be able to get the PLC protocol packets.
The "access switch" to which no sensor is connected is Cisco?
The port/VLAN monitoring on a switch will only show you a copy of traffic which passes through that switch. The Modbus/TCP etc. packets can only start to flow after the ARP protocol has done its job, i.e. to inform the sender about the MAC address of the recipient. So if your controller is configured to query some sensors but these sensors aren't connected, it won't send the queries until it learns where it should send them.
Can you be more precise about the overall topology?
cisco core 3560 sw ------ Cisco 2960 Distribution stack ------ Cisco 2950 access switch ( this has the PLCs connected, Sieman and Rockwell).
My sensor is connected to the core and using RSPAN to monitor the ports on the access switch. After getting a longer pcap, i did notice ENIP packets coming from the monitoring on the access switch, but that is all. I expected to see Modbus/TCP and a number of other PLC protocols coming from the ports that I'm monitoring on the access switch, but there aren't any additional portocol. I thought this could be because I don't have my sensor directly connected to the access thus not able to see more detailed, Layer 2 protocols coming from my PLC(s).
Off topic: could you spend 5 seconds of your time to finally realize the difference between an Answer to the original Question and an answer to a previous comment, and stop posting the latter as the former?
Sorry, I've misunderstood the role of your "sensor"; I thought the PLC communicates with the sensor using the protocols you hunt for.
As "sensor" means the capturing machine, I'd ask you about your capture settings. While Wireshark captures whole packets by default, e.g. tcpdump doesn't - unless you specify that you want the packets complete (using
-s 0command line option, where 0 means "unlimited"), only first 60 bytes or so are captured, which is just enough to see the transport layers. You can set Wireshark to capture only the beginning of packets too, but it is not the default setting.If you apply a display filter
!(frame.cap_len == frame.len)on your existing capture, do any packets remain in the packet list? If yes, it means that this is the reason.I am using Wireshark to perform my captures, I am also using a proprietary program that gathers data from the RSPAN on the switch(s). Neither wireshark nor the software analyzing information directly from the RSPAN is picking up any other protocols than ENIP traffic in regards to the PLC(s) conversations. Where the software is looking for various PLC protocols. So this makes me think that the monitored ports at the access switch isn't sending the data back to the core where the sensor (data collecting PC) is setup. I am using RSPAN and a RSPAN vlan to get the mirrored traffic from the access switch to the core switch. I assume that the layer2 information is staying with the access switch and not getting to the sensor being sent over a RSPAN vlan.
I will give the filter a try. does it need to be applied in the desktop program or is this something I can run from the CLI using "tshark" or "tcpdump"?
The PLC doesn't communicate with the Sensor. the Sensor is collecting the mirrored traffic from the SPAN on the access switch. The sensor has discovered the PLC based on its conversation, but the PLC has no knowledge of the sensor. Sensor=Tap in this scenario.
You can run
tshark -r your_existing_capture -Y "!(frame.cap_len == frame.len)", if any packets get printed, the capture size issue exists.Otherwise:
I don't understand what you mean by "the layer2 information is staying with the access switch"? Does it mean that the part of each frame which contains the L2 information is missing, or that frames which only contain L2 information are not delivered to the sensor?
Normally, you configure the access switch to mirror the source (ports or VLAN) and send a copy of each frame to the RSPAN VLAN, and the monitoring port on the core switch forwards all these frames to the equipment connected to it. So if you can see some unicast frames in the capture, you should see all of them, otherwise it is an issue of the RSPAN implementation (or, less likely, configuration). Is the VLAN in the 2960 configured to RSPAN mode? If not, it may drop the frames, but again - if this should be the reason, I'd expect it to drop all unicast frames, not just some of them.
See requirements on the configuration at the intermediate switch (the 2960 in your case) here.
If by "L2 information" you mean the 802.1Q tags, then yes, using RSPAN causes the 802.1Q tags from the source VLAN to be replaced by those of the RSPAN VLAN and later stripped at the RSPAN output port, so your sensor gets the frames without tags. But source and destination MAC addresses must be there.
The PLC(s) aren't communicating with the sensor. The sensor is a PC I have setup, connected to the core switch. It runs linux and is basically running a proprietary software that analyzes the mirrored traffic. The sensor has discovered the PLC only thru the conversations happening on the mirrored ports or traffic captured on monitored vlan(s). The Sensor I have setup is actually looking for those additional PLC protocols, but if I use tshark to create a pcap of the RSPAN mirrored traffic. The only PLC related protocol is ENIP, where i was really hoping to see more like Modbus or DNP3. I was under the impression that PLC primarily used those protocols and not IP. So, I've come to the conclusion that this lower level data MODBUS, DNP3,...etc must stay on the access switch and isn't carried over the RSPAN VLAN that is analyzed at the core.
Yes, I've already understood the role of the sensor a couple of comments ago :-) What I was trying to say was that both tcpdump and tshark use the same libpcap (on linux) and as such do not themselves filter the received frames by protocols used (unless you ask them to do so, which you hopefully don't) while capturing them.
So if some of the expected frames are missing in the capture, they must
be missing at the source (i.e. the PLCs actually do not use those protocols),
be filtered out on their way from the RSPAN source switch (the "access" 2950) to the RSPAN destination (the "core" 3850)
be filtered out by the sensor's network card.
Some switches do have the possibility to filter frames by the "type" field of the ethernet header, but such filtering would normally occur regardless the monitoring, so if this would be the cause, the PLCs would not be able to communicate normally. Plus Modbus/TCP uses normal IP ethertype so it is not a likely reason.
To find out what actually happens:
can you temporarily move your sensor machine next to the "access" switch and check what you capture if you set up a plain SPAN on that switch, instead of RSPAN?
if the above gives the same results as you get now, can you use two NICs on the sensor PC (i.e. to add one if necessary), set up a software bridge between them, and "insert" the sensor between one of the PLCs and the access switch? This way, you'll be sure that the NIC gets the complete traffic of the PLC. If the NIC (or its driver) blocks some Ethernet types (modbus), the PLC would complain; if it doesn't, you should capture everything that the PLC really sends and receives.
Awesome I will give this a try.