This is a static archive of our old Q&A Site. Please post any new questions and answers at ask.wireshark.org.

SSL Communication fails with connection reset (RST,ACK)

0

alt textalt text I have this issue where when a connection is happening between a client and a server (both are hosted on Hyper V) server being windows server 2008 R2 and the client being Windows 8.1 R2 communication fails (both are in the same network). However when I try the same procedure with windows 8.1 physical machine (server still being 2008 R2 on hyper V), communication successfully happens.

I am not able to trouble shoot this issue at all, I have attached the screen shots of wireshark capture for both failed and successful communications.

kind of running in time crunch , any help much appreciated.

Thanks, GK

asked 08 May '14, 04:02

gk_vandamme's gravatar image

gk_vandamme
1●1●1●2
accept rate: 0%

edited 08 May '14, 04:04

2 Answers:

0

In the first session the client (192.168.0.6) closes the connection after having received the Server Hello, Certificate, Server Hello done from the server at 192.168.0.125. So it is probably the server's certificate that the client doesn't like.

In the second session the SSL handshake continues, so the client (192.168.0.141) is happy with the server's certificate.

So look at the clients certificate store and see if the certificate (chain) is trusted.

As Kurt mentioned the failing client is using TLS1.2 - The server responds with a TLS1.0 Server Hello in both cases.

Can you change it to use TLS1.0 also?

answered 08 May '14, 06:02

mrEEde's gravatar image

mrEEde
3.9k●15●22●70
accept rate: 20%

edited 14 May '14, 23:31

Checked that .. and the server certificate is added in all the trust stores of the client (Win 8.1) machine. still no luck.. :-(

(09 May '14, 03:38) gk_vandamme

You could export the certificate from the trace as .cer and doubleclick on it to have windows validate it ...

(09 May '14, 14:24) mrEEde2

0

and the server certificate is added in all the trust stores of the client (Win 8.1) machine

In that case the client does not accept the cert due to other reasons. Three possible reasons pop up in my mind.

  • Most likely and a problem that is often overseen: The date/time (month, year) on the client is outside of the valid time window of the server cert, and thus the client rejects it. Example: cert valid from 2012-2014. Date on client: 2011. Result: REJECT.
  • A mismatch of what is in the certs subject (common name) and what the client expected to get. Example: The client accessed 192.168.0.125 and got a cert with the cert subject: srv001.domain.local. Result: REJECT.
  • The server uses a TLS extension, that is unknown or unsupported by that specific client.

Regards
Kurt

answered 09 May '14, 12:05

Kurt%20Knochner's gravatar image

Kurt Knochner ♦
24.8k●10●39●237
accept rate: 15%

edited 09 May '14, 15:17

  1. I checked the subject of the certificate on the server , it is issued to xyz.abc.com and the client is accessing xyz.abc.com URL

  2. Date time of the certificate matched with that of the client (both of them are in the same time zone)

  3. I am attaching the actual wireshark captures of SSL hand shake on both successful and failing clients

  4. Please note that the Successful client is a WIN 8.1 physical machine connecting via a physical NIC where as the failing client is 8.1 hosted on hyper V connecting over a virtual switch.

would that make any difference ?

(14 May '14, 01:50) gk_vandamme

FAILED HANDSHAKE CAPTURE

http://www.filedropper.com/failedhandshakecapture

(14 May '14, 01:54) gk_vandamme

WORKING HANDSHAKE CAPTURE

http://www.filedropper.com/workinghandshakecapture

(14 May '14, 01:57) gk_vandamme

Two possible things:

  1. There is a CRL distribution point in the cert. Maybe the client does not like the ldap sytax or is unable to access the CRL.
  2. The difference between working and non working is: non working client suggests to use TLS 1.2, working client suggests to use SSL 2.0. Maybe the non working client does not like any of the TLS extensions sent by the server.

I can't see any clear sign for a problem in the capture file, so you'll probably have to enable some form of 'debugging/logging' on the client to figure out what's going wrong. As a first step, you could try to change the client config to use a different TLS version.

Regards
Kurt

(14 May '14, 15:10) Kurt Knochner ♦