Hi Friends, This is my first post and will try to make it as informative as I can. The issue is related to remote desktop connection on one of the windows 2008 R2 server. Some users are facing issue while connecting to remote desktop session of a server in VLAN A. For the same server remote desktop for other users are working fine. While troubleshooting Network team was able to see traffic flowing through the firewall and reaching to the server but never responded back. I did setup wireshark packet capture on the server side to analyse the traffic. I do see packets reaching to the server but are not being responded back. There is packet retransmission happening from client side due to no ACK. Can any one suggest what could be wrong here?
asked 17 Jun '15, 04:39  vjay |
One Answer:
I see three possible reasons:
Personally I believe it's option #2, although this is just a wild guess: I've seen that happen from time to time, because people give their servers several interfaces for "redundancy" reasons and then they simply assign an IP address from the same subnet (Windows does not prevent that)! This usually works on a local network without security devices (firewall, loadbalancers, etc.), but it can cause problems if those devices are in place. In your case, the firewall might well block the SYN-ACK, if it's coming from a different MAC address than the SYN was sent to (windows sends the SYN-ACK out the other interface). This depends on the firewall type and on the firewall configuration. Regards answered 17 Jun '15, 04:50  Kurt Knochner ♦ showing 5 of 27 show 22 more comments |
Thanks Kurt for your time. You are spot on. We do have two interfaces on that server over different VLAN but I couldn't capture syn/ack on other interface. I applied same filter as first interface which receives SYN packets. Is there anyway I can capture those SYN/ACK packets on second interface?
Set Wireshark to capture from all relevant interfaces. Simply check all the interfaces you wish to capture from in the Capture Options dialog.
to rule out problem #1 you should capture on a switch mirror/monitor/span port, which mirrors the traffic of both server interfaces.
Well, I am getting all the hand shakes and frames for other communication only for this particular communication I am seeing this behaviour. RDP connections are working for internal traffic but the issue is for external traffic where we are only getting SYN packets in capture. For internal traffic we are getting ACK packets from same interface. Do you think problem#1 and problem#3 can still be an issue?
O.K. can you please provide a capture file for the case where it does not work? I want to check if there is something wrong in the frame with the SYN flag.
Maybe. Can you asure that the frames in both cases (internal and 'external') are taking the same path/route?
Pardon me for ignorance, How can i upload a file here?
You can't. Please upload it to dropbox, google drive or cloudshark.org and then post the link here.
Thanks Kurt.
External Traffic : https://www.cloudshark.org/captures/7e47bcb89d32
Internal Traffic : https://www.cloudshark.org/captures/9d169624703d
The SYN frame looks good, but as you can see there is also no answer to the ping requests.
So, this leads me to further assumptions
Can you please post the output of the following command on the Windows server?
Pasting it in two parts.
Opps. Let me see any other way I can upload.
You have two gateways. One for 172.0.0.0/8 and one default route.
Now, what’s the IP address of the firewall and what’s the “other” gateway then?
The reply packets for the “external client” (10.6.4.125) should be coming in through gateway 172.16.87.5, as that’s the way back for your them (default GW). If that’s not the case, you have some form of asymmetric routing, which is a bad idea with statefull firewalls. But then you should at least be able to see the answer of your server on its other interface!
Do you mean to say the other interface 172.16.87.123 will send reply to 10.6.4.125? My filter was for both interfaces and via filtering 10.6.4.125 I could get only the response which I uploaded.
I can’t tell you, because I don’t know where the traffic came in. Can you please post the output of the following command on the server?
In network capture for both interfaces I only see traffic coming for 3389 from 10.6.4.125 IP but don’t see any traffic going out from 3389 port to that IP Address.
as I suspected, the traffic in your capture file was captured on the interface with IP address 172.16.84.121, (see Statistics -> Summary. You’ll find the ID of the interface there). Now, the default route will send out the answer through the other interface to gateway 172.16.87.5. This will result in a firewall seeing only parts of the traffic, or it will see the same session on different interfaces. In both cases a stateful firewall will deny to process the session.
Why you don’t see the SYN-ACK in Wireshark, I can’t tell you. Maybe there is something wrong with your capture setup. You should see the SYN-ACK and the ping response on the other interface (172.16.87.123).
There are two obvious solutions:
BTW: There is still a chance, that a filter on the Windows system blocks your frames, like the windows firewall or any other form of security software. Did you check that?
Yes, local firewall has been turned off and there is no antivirus software running on the server.
Unfortunately, Network Admin is stuck on the point that firewall is passing traffic but server is not responding, I will need some proof to get him make change on gateway or firewall. Is there a way I can capture response from other interface to 10.6.4.125? I do have whole capture without any filter available for both the interfaces but I don’t see any response for 10.6.4.125.
To make sure you are really capturing on both ports, please start Wireshark with the following command:
Then start the ping and connect to the RDP server. Wait 10-20 seconds. Then look at the data. You should see the ping response now and the SYN-ACK.
If you don’t see the answer packets, there is indeed a problem on your server and then you need to double check that nothing on the server blocks the frames (AV, IPS, Endpoint security, etc.).
Still getting same results. On Windows server also firewall has been turned off.
Don’t know what could be the issue.
I don’t know it either, but if you still don’t see the reply packets in the capture file (with the command I posted above), then I conclude that:
C:\Program Files\Wireshark>arp -a | find “172.16.87.5” 172.16.87.5 00-23-e9-6e-09-87 dynamic
How can I check ARP requests in capture file?
Use a display filter of “arp”.
You don’t have to look for ARP requests, as the server knows the MAC address of the gateway!
So, my conclusion is: something on the server blocks the incoming requests from 10.6.4.125. You say, that there is no security software enabled, so I can’t tell you what it might be. Please ask your local windows hero/guru to have a look at it.
Thanks Kurt for your extensive support. Really appreciated. I will check with Windows team.
You’re welcome!
Hint: If a supplied answer resolves your question can you please “accept” it by clicking the checkmark icon next to it. This highlights good answers for the benefit of subsequent users with the same or similar questions. For extra points you can up vote the answer (thumb up).
@Kurt Knochner, I have added persistent route for the destination and it is working fine now. I thought to put an update so it may help someone in future.