This is our old Q&A Site. Please post any new questions and answers at ask.wireshark.org.

I am experiencing intermittent SSL decryption failures (10% success 90% failure) using Wireshark in accordance with the decryption protocol described here:

https://isc.sans.edu/forums/diary/Psst+Your+Browser+Knows+All+Your+Secrets/16415/

After very many attempts I have, I believe, narrowed the problem down to the following hypothesis: My browser (Firefox, Version 45.7.0) writes down several SSL keys in the SSLKey.log text file, many of which are unrelated to the TCP/TLSv1.2 stream of interest. It would appear that Wireshark is selecting the incorrect SSL key from the SSLkey.log file to decode the traffic of interest, and thus the decryption fails.

So, here are some intermediate questions to help me arrive at a final solution: (1) How does Wireshark select the SSl session key (from SSLkey.log) that it uses to decrypt a stream, which the user has selected for decryption ? (2) Is there a way to force Wireshark to cycle through and to use ALL the stored keys in SSLkey.log in succession, in order to decrypt the user selected stream ? (3) Is there a better method which will unfailing force Wireshark to correctly select the right SSL session key from the SSLkey.log file for its decryption attempt?

For analysing this problem, I am using Wireshark to decrypt the source of the video feed in the following site:

http://www.airclips.net/video/airbus-a400m-atlas-german-air-force-54-04-go-around-at-landsberg-airbase

Finally, In searching the Wireshark.org database for previously reported problems in this connection, the only entry which I found, is the following, as yet unanswered, question:

https://ask.wireshark.org/questions/46368/ssl-decryption-seems-intermittent

asked 22 Apr '17, 09:47

Utnapishtim's gravatar image

Utnapishtim
31336
accept rate: 100%


Your hypothesis

It would appear that Wireshark is selecting the incorrect SSL key from the SSLkey.log file to decode the traffic of interest, and thus the decryption fails.

is invalid. Wireshark selects the secret based on the Random field in the Client Hello message. This process is deterministic, the same secret is selected each time (but only if it is present).

More likely scenarios:

  • Firefox is too slow in writing the keylog file or Wireshark reads the packets before the key appears in the key log file. In this case, saving the packet capture and then reloading the file should work. Alternatively, you can try toggling a preference which hopefully results in a redissection (and thus re-reading the packet capture).
  • Out-of-order TCP segments are received and the decryption state is confused. (See https://bugs.wireshark.org/bugzilla/show_bug.cgi?id=9461). There is no solution for this at the moment other than using a tool to re-order TCP packets in a packet capture (I don't know any).

Given that you are analyzing a video stream, out-of-order TCP segments sounds more likely to me.

permanent link

answered 23 Apr '17, 05:35

Lekensteyn's gravatar image

Lekensteyn
2.2k3724
accept rate: 30%

Thank you for your helpful reply.

I tried Scenario 1 (saving the packet capture, then reloading it). It failed to correctly decrypt the video stream. This only leaves Scenario 2 and reinforces your comment "Given that you are analyzing a video stream, out-of-order TCP segments sounds more likely to me."

This result brings me back to sub-question (2) of my original question, namely: Is there a way to force Wireshark to cycle through and to use ALL the stored keys in SSLkey.log in succession, in order to decrypt the user selected stream ?

Doing this would unfailingly solve the decryption problem under both of your scenarios. If this exhaustive search is not possible in the 'standard' Wireshark, would it be difficult to tweak the source code to accomplish it?

(23 Apr '17, 12:34) Utnapishtim

TLS does not work like that, you cannot just "decrypt" each packet with the same key. At the beginning of a handshake, a key is agreed on, but decryption of further records depends on earlier ones. If the order is wrong, then the correct decryption keys for the following records cannot be derived. Brute-forcing through an exhaustive search would now work.

What would work is fixing up the TCP dissector to put the TCP segments in the correct order, but this is still an open issue (see the bug I linked in the post).

(24 Apr '17, 13:45) Lekensteyn

Once again, thank you for your helpful reply. Please excuse the delay in my response. I am unfamiliar with SSL/TLS and had to spend time reading up on them.

Also, kindly bear with me but I am having difficulty following your statement that: " the correct decryption keys for the following records cannot be derived"

To speedily allow you to point out the error(s) in my understanding, I have enumerated below, the events which I believe are occurring during the TLS session key generation process. This understanding is based on my reading of references [1, 2, 3]. I have not been able to determine exactly what my browser is recording in the SSLkey.log text file (step 7 below). The best information which I have managed to obtain is given in reference [4]. Accordingly, there are two possibilities (a) only the session keys themselves are recorded (b) the session keys as well as the intermediate values ie client's random value, server's random value, client's premaster secret and server's premaster secret, are being recorded. Here are the steps per my understanding:

1)  My Firefox Browser (the client), sends a "Client hello" message to the server hosting the website containing video (the URL of which I wish to know), along with the client's random value and supported cipher suites.
2)  The server responds by sending a "Server hello" message to the client, along with the server's random value.
3)  The server sends its certificate to the client for authentication and may request a certificate from the client. The server sends the "Server hello done" message.
4)  If the server has requested a certificate from the client, the client sends it.
5)  The client creates a random Pre-Master Secret and encrypts it with the public key from the server's certificate, sending the encrypted Pre-Master Secret to the server.
6)  The server receives the Pre-Master Secret. The server and client each generate the Master Secret and session keys based on the Pre-Master Secret.
7)  The client records the new session keys, one for hashing and a second for encrypting messages and perhaps intermediate values, in the SSLkey.log text file.
8)  The client sends "Change cipher spec" notification to server to indicate that the client will start using the new session keys for hashing and encrypting messages. Client also sends "Client finished" message.
9)  Server receives "Change cipher spec" and switches its record layer security state to symmetric encryption using the session keys. Server sends "Server finished" message to the client.
10) Client and server can now exchange application data over the secured channel they have established. All messages sent from client to server and from server to client are encrypted using session key.
11) It is possible that the website is 'nested' ie transfers the client to one or more subsequent websites which all require steps 1-10 to be completed.

Now, these SSLkey.log entries are being recorded by the Browser, which always correctly decrypts the stream of interest (video feed in my case), ie its unaffected by the incorrect byte order issue which is confusing Wireshark. Hence these entries are entirely correct. Also, regardless of the two possibilities which I mentioned for the contents of SSLkey.log, one of the entries must be the correct session key for decrypting the video stream. Moreover, per the TLS protocol described in steps 1 to 11, the session key for decryption remains unchanged throughout the decryption process ie shuffling the videofeed byte order would simply mess up the video but would not affect the decryption session key.

If the aforementioned is true, then using each of the entries of the SSLkey.log file in succession (exhaustive search), will unfailingly result in successfully decrypting the stream of interest, regardless of shuffles in byte order.

References [1] https://msdn.microsoft.com/en-us/library/windows/desktop/aa380513(v=vs.85).aspx [2] https://uk.sans.org/reading-room/whitepapers/protocols/ssl-tls-beginners-guide-1029 [3] https://tlswg.github.io/tls13-spec/ [4] https://enterprise.cloudshark.org/blog/2015-05-14-all-about-ssl-key-logging/

(25 Apr '17, 16:51) Utnapishtim
Now, these SSLkey.log entries are being recorded by the Browser, which always correctly decrypts the stream of interest (video feed in my case), ie its unaffected by the incorrect byte order issue which is confusing Wireshark.

The browser won't see an out of order TCP packet, the host OS will present the TCP stream in order to the application.

(26 Apr '17, 02:41) grahamb ♦
1

There are some inaccuracies in your assumptions (e.g. the Client Hello does not store the full URL for example, only the hostname), but let's disregard them and focus on the key material. Your step 7 is about the derivation of MAC (integrity), key/IV (symmetric encryption) as can be found in RFC 5246 - TLS 1.2, Section 6.3. Key Calculation. This key is derived from the Master Secret (which is derived from the pre-master secret) combined with the Client and Server Random values.

The key used in step 10 (for Application Data) is based on the key/IV as can be seen in the subsections after Section 6.2.3. Record Payload Protection. The integrity key depends on a sequence number. Out-of-order packets will result in the wrong integrity check and as a result decryption will fail.

The SSL keylog file stores a mapping from the Client Random value to the Master Secret. Thus it should become clear now that enumerating all entries in the SSL keylog file will not help since the Client Random value is unique and we can figure out whether the key exists or not.

Why does it work for the web browser and not Wireshark? The web browser relies on the operating system to buffer TCP segments and re-order to ensure a stream of data that fixes out-of-order packets. Wireshark (at least for SSL/TLS) currently does not do this re-ordering yet (see the linked bug) and will thus fail to provide the required ordering that is necessary for TLS decryption to function.

(26 Apr '17, 02:54) Lekensteyn
Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×319
×165
×7

question asked: 22 Apr '17, 09:47

question was seen: 1,270 times

last updated: 26 Apr '17, 02:54

p​o​w​e​r​e​d by O​S​Q​A