This is a static archive of our old Q&A Site. Please post any new questions and answers at ask.wireshark.org.

Device sends FIN, ACK immediately after the SSL certificate is received

0

I am trying to establish an SSL Tunnel over TCP using a Lantronix Xport Pro network module.

The trouble is that certain websites are no allowing the connection for some reason. If i simply try to open a a secure session against, say Paypal or Google, it works fine and I can send data via a serial stream through the tunnel. (Note we do not have a web server).

The trouble comes when trying to connect to our servers which are based on the Microsoft Azure platform. We have a VIP address to use and it all looks fine and dandy, except i get the following sequence on wireshark:

---> Server Hello, Certificate, Server Hello Done

---> 42572→443 [FIN, ACK] Seq=65 Ack=716 Win=64240 Len=0

---> 443→42572 [RST, ACK] Seq=716 Ack=66 Win=0 Len=0

It looks like the server is sending our local module a certificate and it immediately closes the tunnel before it even attempts a key exchange. We then receive a reset.

I cant visually see anything wrong with the certificate that the server has sent, and we have disabled the certificate validation for now.

Anyone have any idea what could be happening here? I am not in my area of expertise and have run out of ideas.

asked 15 Apr '15, 11:01

jihenderson's gravatar image

jihenderson
6112
accept rate: 0%


One Answer:

1

Anyone have any idea what could be happening here?

There are "typical" reasons why a client would reject a server certificate

  1. The client is unable to verify the whole cert chain, meaning it does not know the root CA cert and/or any of the intermediate CA certs
  2. The client does not accept the cert, because the cert lifetime is invalid from the clients point of view. Possible reason: The date/time on the client is totally wrong (off by several months or years).
  3. The client does access the server via https://x.x.x.x (an IP address), but the common name of the cert contains a host name
  4. The client is unable to process the cert, because the public key is too large (4096 or even 8192 bit)
  5. The cert contains an X.509 extension that the client is unable to understand/process

and we have disabled the certificate validation for now.

If this means, that the connection works after you've disabled cert validation, then I would rule out 4. and 5. The other items are still good troublemaker candidates.

Without a capture file AND more detailed information about the client system (date/time, cert store, ssl stack, etc.) it's impossible to predict which of the items is causing the problems.

Regards
Kurt

answered 16 Apr '15, 12:54

Kurt%20Knochner's gravatar image

Kurt Knochner ♦
24.8k1039237
accept rate: 15%

edited 16 Apr '15, 12:55

Hi,

Thanks for the reply. Very useful.

Certainly on the current certificate (1) is true, although I imagine it might have ok with a previous one as we were trying different things. What happens if the chain is incomplete?

(2) I don’t think this is a problem because there are other websites which are connecting properly which are out of date by several years – unless both ends are out of date by the same amount?! This i will check.

(3) Not sure about this. We are activating a tunnel directly with an IP address. The system has not yet resolved the http(s) part so I'm not sure it is relevant.

However I have seen in a few places that the common name in the certificate should not be related to the domain in any way. Not really sure why this is as the explanations I have seen don’t seem to make a whole load of sense as it is quite well defined in the certificate structure and would imagine it would be hard to confuse.

(4) Don’t think this is it. Seems to be RSA2048 which is not too unusual and i have other websites that this seems to be ok with.

(5) Now this is a good point. Our module client is a bit (very) picky when we have to upload a cert onto the module and the only thing that seems to work is a .PEM format. It didn’t like any of the .cer, .crt, .der formats. Could the module not understand a cert if in the wrong format for the server end? Sounds stupid as it means that it simply won’t connect to many common https sites?

If disabling cert checking rules out 4&5, functionally that may be true, but I also think that it is not beyond belief that the module would at least try to read it, and if it cant…

(21 Apr '15, 01:04) jihenderson

Is it possible to attach a wireshark log on this forum?

(21 Apr '15, 05:47) jihenderson

Your "answer" has been converted to a comment as that's how this site works. Please read the FAQ for more information.

A capture can be shared from a publicly accessible spot, e.g. CloudShark, Google Drive, DropBox etc. and posting a link as a comment, or by editing your question.

(21 Apr '15, 06:09) grahamb ♦

(1) What happens if the chain is incomplete?

That depends on you SSL stack and it's configuration. I can't tell you, as I don't know the SSL stack and configuration of you Lantronix device!

(3) We are activating a tunnel directly with an IP address. The system has not yet resolved the http(s) part so I'm not sure it is relevant.

sounds relevant to me, because a web server cert usually does not have the IP address in it's common name. But again: If this is a problem or not depends solely on the SSL stack and it's configuration in the Lantronix device. Only the vendor will be able to help you with this.

(4) Don’t think this is it. Seems to be RSA2048 which is not too unusual and i have other websites that this seems to be ok with.

seems to be O.K. or are you sure it's O.K.??? ;-)

(5) Could the module not understand a cert if in the wrong format for the server end?

No, that's totally unrelated. After the cert is loaded into the server software, no matter in which format, it will send the same information, which is (basically) the signed public key with information to whom it belongs (common name) and how long it is valid. This information is called the cert ;-)

but I also think that it is not beyond belief that the module would at least try to read it, and if it cant…

I don't know. Only the vendor of that device will be able to answer that question.

(21 Apr '15, 14:19) Kurt Knochner ♦