Here is a diagram of the network I am on.
The problem is with ANY secure or encrypted traffic from ANY computer on the 20.0 network going to the 2008 server on the 30.0 network. Why I say secure, is because I can transfer big files (100 – 300Mb files) to the 2008 server via windows copy or ftp while secure/encrypted connections break. (SSL, MAPI, RDP just to name a few)
Someone on MS forum (Link) with almost the same setup as mine was able to nail down the problem by changing his router Bandwidth Management setting from Priority to Rate Control. (explained below)
Making the suggested change in the routers didn’t fix my problem and I am wondering where I need to place wireshark computer to get a trace that would be helpful in troubleshooting this issue?
Thank you in advance
asked 30 Oct '10, 15:21
edited 30 Oct '10, 15:22
This sounds very much like a problem with fragmentation. Because traffic needs to be encapsulated in the VPN protocol (IPsec most probably), there will be an additional header to each packet. For packets that already are at maximum size, this would create packets that can't be transported over the network.
When nothing fancy is done by the VPN routers, the IP layer will take care of this by fragmenting the (too large) IP packets. But... if the DF (Don't Fragment) bit is set in the IP header of the packet, the router is not allowed to do that. The VPN router that wants to do fragmentation, but is not allowed to by the DF bit will send an "ICMP Fragmentation Needed, but DF bit set" message (ICMP type 3 code 4) back to the sender indicating this problem. If this is the case in your setup, you should be able to see those messages in Wireshark on the Win2008 server.
There are a few solutions for this situation:
As for the original question, I would place wireshark on the Win2008 server or in between the Win2008 server and RV042 and start looking for ICMP type 3 code 4 messages.
answered 31 Oct '10, 02:07
What do you mean "secure/encrypted connections" break?
First I'd take a trace at a client system over on the 20 network and look at the Expert Infos for errors and look at the TCP connection and communications sequence. I'd do this for both the "good" connection and the "broken" connection.
Next (or even simultaneously if possible) I'd get the traffic at the server to see what the traffic looks like at that point. You should be able to determine if the traffic along the path is having issues this way. You should also be able to tell if the problem occurs at the client or the server.
Make sure you set your time column to show large gaps in time (View > Time Display Format > Seconds Since Previous Displayed Packet. You might consider doing a few conversation filters to really focus in on one connection at a time. Sometimes I'll save the connections in separate trace files using File > Save As. Then open a separate instance of Wireshark and compare the two.
This should give you a good start - regardless of the cause of the problem, it's best to find out where it's occurring first and act accordingly.
answered 30 Oct '10, 18:45