This is our old Q&A Site. Please post any new questions and answers at ask.wireshark.org.

Hello,

I need to troubleshoot an issue which is occurring while the client is connected from home via VPN. While connected I can see two network cards. One is customer LAN connection let's call it LAN-2 and the other one seams to be the virtual NIC that got created at the time CISCO AnyConnect software was installed. Which interface should is listen on ?

Thank you !

asked 02 Aug '16, 00:59

adasko's gravatar image

adasko
86343842
accept rate: 0%

edited 02 Aug '16, 01:00

another update - need to confirm with you guys.

I've installed openSSL on the client to try to connect to the server on port 443:

Openssl.exe s_cleint –connect 172.16.5.245:443 –status –debug

I can see that the server is sending it's certificate but at the end in the command prompt I can see:

SSL3 alert read:warning:close notify SSL3 alert write:warning:close notify

I'm wondering if this is an error message? I'm asking because I was checking on my laptop and got the same "errors" but the client can download the policy from server.

I appreciate all feedback!

(11 Aug '16, 01:13) adasko
1

Adam, this is not the way to do it. Although for you it is a follow-up of the current question as you are troubleshooting the same thing, from the point of view this site, capturing at VPN interface and analyzing SSL/TLS negotiation are really different subjects. Please post it as a new question, possibly referring to the current one if you think such reference is useful.

(11 Aug '16, 01:19) sindy

  • if you want to analyse an issue with the VPN connection, you have to capture at the physical interface.

  • if you want to analyse an issue with the tunnelled data, you have to capture at the virtual interface.

Unfortunately, two points make your task really hard:

  • an issue with the tunnelled data may be a consequence of an issue with the VPN connection

  • not all frames on the virtual interface may be captured (my own experience with Cisco VPN client and both WinPcap and NPcap, just reconfirmed for the latter)

permanent link

answered 02 Aug '16, 01:22

sindy's gravatar image

sindy
6.0k4851
accept rate: 24%

Thank you @sindy.

It might be the case that not all frames are captured on the virtual interface. Let me describe the issue in grater detail, maybe you can point me in the right direction on how to troubleshoot further. We have a client software that needs to download a policy from server. It fails while connected via VPN. I've tried to perform a Wireshark capture but I cannot see any SYN packets coming from the client's IP address at all. I was capturing on the affected laptop but what's strange that I can see his IP as the destination and not as the source in the packets. Do you have any ideas?

Thank you Adam

(02 Aug '16, 02:23) adasko

Unfortunately, there is no way to affect how a proprietary code, like the one of the Cisco virtual interface, handles packet capturing. So if I've understood you right, you have already captured at server side and haven't seen there any incoming SYN packets from the client. As you cannot usefully capture at the virtual interface on the laptop, you have to use different approach to the analysis.

The first thing I would ask myself at your place would be "is the routing correct"? AFAIK the Cisco VPN supports two modes, one where the default route at the client PC is changed to the VPN tunnel, and another one where the default route remains as it was and only a list of IPs and subnets which should be routed to the tunnel is sent to the client.

So when the VPN connection is active, check your routing table and see whether the route to the server is the tunnel or to the LAN's router.

Or, as we are on a Wireshark site, capture at the client's LAN interface and start the application. If you can see SYN packets towards the server's IP address there, it means that they are misrouted.

If any of the two methods shows routing misconfiguration, fix it in client's settings at the VPN access gateway.

Another possibility could theoretically be that the client application somehow binds to the address of the LAN interface and uses that address as source for the packets it sends regardless the actual source interface, and the virtual interface filters out any packets bearing different source address than its own one.

(02 Aug '16, 02:52) sindy

@sindy I highly appreciate your feedback. Thank you. I was testing it with another VPN solution and it worked BUT the VPN solution i was testing with didn't install a virtual NIC unlike CISCO AnyConnect. Can it be somehow linked with what you wrote " Another possibility could theoretically be that the client application somehow binds to the address of the LAN interface and uses that address as source for the packets it sends regardless the actual source interface, and the virtual interface filters out any packets bearing different source address than its own one." ? If yes, how would I troubleshoot that ?

(04 Aug '16, 04:13) adasko

I wonder how a VPN solution could work without a virtual interface it would offer to the network stack. It is more likely that it creates one dynamically, but that doesn't help you solve your issue.

You haven't said how the routes looked like when the Cisco VPN connection was active; did you check that?

To confirm that the application chooses one IP address and uses it as source for all requests regardless from which interface they are actually sent, you would need to use another network card (an USB one should be easiest to deploy) and another router to which it would be connected; the "WAN" interface of the router would be connected to the same network like the other network card of the laptop.

                          internet
                             |
                [gateway router of subnet A]
                             |
Subnet A --------------------------------------------
             |                              |
             |                [gateway router of subnet B]
             |                              |
             |       Subnet B -----------------------------
             |                              |
      on-board NIC ... of the laptop ... USB NIC

Then you would set the default route of the PC first to the "LAN" IP addres of the gateway router of subnet A (or just check that it is set like that), start capturing at both network interfaces of the PC, and start the application. You should see the SYN packet on one of the interfaces. Next, you would change the default route of the PC to the "LAN" IP address of the gateway router of subnet B and do the same.

Now:

  • if in both cases the source IP of the SYN packet is the same (matching one of the two interfaces' IP addresses), the hypothesis is confirmed;

  • if in both cases the source IP of the SYN packet matches the IP of the interface from which it is physically sent, the hypothesis is denied.

(04 Aug '16, 04:52) sindy

@Sindy. I'm going to troubleshoot it further tomorrow but there's something i don't understand in relation to the routing hints. I mean, I can still ping / telnet to the server while connected via VPN. Wouldn't that mean that the routing is fine (I know that I didn't mention that ping and telnet work though). Or is your concern related to the issue that the application chooses one IP and uses it as source for all requests regardles from which interface they are actually sent ?

(04 Aug '16, 12:48) adasko

If telnet and ping work correctly, the routing must be fine. In that case, the theory that the application chooses one of local IPs as source for all its requests, regardless through which interface it actually sends them, is my only explanation of what happens. If some packets carrying payload would be lost, we could think of MTU mismatch, but a SYN packet is normally small enough to fit to any reasonable MTU.

(04 Aug '16, 13:02) sindy

And isn't there any other way to capture the packets like RawCap or something else ?

(04 Aug '16, 13:17) adasko

I was just checking right now, and you right. As soon I connected with the other VPN solution i can see now a new virtual NIC and i can see also the packets coming from source = ip addr assigned to that interface...

(04 Aug '16, 13:34) adasko

There sure is, yet I'm not sure whether these tools will make better friends with the virtual network adaptor from Cisco :-)

If there is normally no traffic through the VPN, you might also capture at the main interface and filter the IP of the VPN server. In that case, if there are several small packets after you launch the application and then silence again, it would indicate that the SYN is transmitted through the tunnel (and retransmitted several times before the TCP stack gives up).

(04 Aug '16, 13:36) sindy

My previous comment was related to the RawCap.

If with the other VPN the SYN packets are sent with that interface's IP rather than the primary one's one, my hypothesis about the application using the primary interface's IP to all packets is wrong. In that case, I'm out of ideas.

(04 Aug '16, 13:45) sindy

just to update, I was using ProcessMonitor and tcpview from sysinternals I was not able to see any TCP connection being initiated at all. I my lost :/

(05 Aug '16, 03:21) adasko
showing 5 of 11 show 6 more comments
Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×45

question asked: 02 Aug '16, 00:59

question was seen: 2,180 times

last updated: 11 Aug '16, 01:19

p​o​w​e​r​e​d by O​S​Q​A