We're trying to diagnose if we're being attacked, or what might be causing tons of NBSS continuation messages from our DNS servers to all our branches. We have 2 DNS servers, and various machines (always varies which do it at a given time) will get pounded by packets, and greatly slows down the network for that branch. Here is a capture from one of the end devices as it is happening: http://www.mtncom.net/mspv.pcapng asked 19 Jan '17, 07:41  CGRemakes edited 21 Jan '17, 08:15  packethunter |
One Answer:
Hi CGRemkaes The short trace file that you provided does not show obvious signs of an attack. Still, there are a few remarkable things. Dramatis Personae We have a few systems in 192.168.160.0/24, and at least one server at 192.168.100.0/24 and 192.168.110.0/24 each. The round trip time between the office is about 110 msec. TCP runs steady at a rate of approx. 2.8 MBit/sec. Problem 1: Retransmissions First of all, there are a few retransmissions. This would be normal, if the network link is really at 2.8 (or 3) MBit/sec. The retransmissions are not really slowing down the connection. Problem 2: DNS Server not responding The DNS requests are queries for Pointer Records. In packet 297 the client 192.168.160.27 is asking for the hostname of 192.168.160.23. This query goes to 192.168.110.254. We never see a response from the DNS server 192.168.110.254. This could be a result of the current capture setup, but it is more likely that the response never arrived. In packet 2147 we see the same query going to 192.168.100.253, again without answer. Note that we have lost 5 seconds between the two queries. Not sure, if a response came in. However, there is no third query. This might be, because there is no third DNS server configured, or because the query was answered. Obiously, at least one name server is not responding. The very minimum would be a return code like NXDOMAIN. Problem 3: Interfaces not properly registered in DNS Windows networks rely on a proper DNS configuration. The hostname should point to the proper IP address and the pointer record for that IP address should point to the hostname. The registration is happening either by the client or the DNS server. The exact duties are negotiated through DHCP, when an IP address is assigned to the client. As we are missing the pointer records I assume that something went wrong. Note: A DNS server integrated in a cable modem will do for home networks, but not for enterprise networks. Problem 4: Misconfigured NetBIOS Name Resolution In packet 5523 the client is trying to translate the hostname MSPV250 into an IP address using the "NetBIOS name service". This is a local broadcast message, which is answered by 192.168.160.250 and .27 Nowadays, Windows systems use DNS for name resolution. NBNS is available as a fallback. This client is using a local broadcast, where DNS would be preferable. The details of name resolution policies are a bit beyond the scope of an answer. Microsoft has a trove of information, like this article from the Cable Guy The NBNS request metioned in the previoud section triggers two identical responses. Each response lists 3 different IP addresses: 192.168.56.1, 192.168.160.27 and 192.168.160.250. Problem 5: Misconfigured IP helper In packet 5573 we see an ICMP port unreachable message. The message was triggered because the NBNS request in frame 5523 was forwarded 192.168.110.254. Remember that we saw this address as unresponsive DNS server from problem 2. The ICMP packet informs the client, that we have no UDP port 137 on this system. This allows the following educated guesses:
The small clue for your network: The ip helper configuration can be improved by excluding port 137 (and probably 138) from the list of forwarded ports. Problem 6: Disjunct networks Windows assumes, that all network interfaces of a computer are connected to the same network segment. For mail servers, for example, you might have one interface connected to the DMZ and the other interface connected to the servers network segment. These configurations require some fine tuning. I am not sure, if 192.168.56.1 is connected to the same backbone. If this is a VMware host, again, you want to fine tune MSPV250's name resolution. Problem 7: Lax security The client 192.168.160.23 is obtaining a Kerberos ticket. The list of supported encryption types includes DES with MD5. You might want to harden this. Problem 8: Strange application In packet 5812 the client tries to open the file with the name plv011. The file attribute mask in the create requests explicitly demands a file, NOT a directory. The file server signales the return code "File is a directory". Next we see a create request for a file with no name (Length 0 in the name field) Final remark The trace file holds a few more oddities, like SMB requets without response from .160.23 to 100.253. For now, I feel that you have quite some clean up to do. This is one of the situations where Wireshark reveals problems, that are not related to the network. Windows domains can be quite a complex beast. Please hire a consultant, if you feel unfamiliar with the details I have rarely seen such a large number of problems in such a small trace. This would make a great trace for a Wireshark workshop. Or for a job interview with candidates for that open position as network analysts. Thank you for sharing the trace file. I enjoyed the walk through. Good luck and good hunting. answered 20 Jan '17, 12:19 packethunter |
I just re-read my answer and hope, that you don't find the rather long list offending. A few of the issues are just briefly touched by giving the proper terms for your favorite search engine.
BTW, is 192.168.160.1 up and running? Because 192.168.160.27 keeps sending ARP requests for that IP-address. And then you might want to consider upgrading the software for your Dell N1548 switch.
No, your comments were spot on. We've hired an outside company to come in and help us clean some things up. Hopefully, that will help clear some underlying issues. Thanks for going above and beyond to dig into it.