Intermittent SSL Decryption

Thank you for your helpful reply.

I tried Scenario 1 (saving the packet capture, then reloading it). It failed to correctly decrypt the video stream. This only leaves Scenario 2 and reinforces your comment "Given that you are analyzing a video stream, out-of-order TCP segments sounds more likely to me."

This result brings me back to sub-question (2) of my original question, namely: Is there a way to force Wireshark to cycle through and to use ALL the stored keys in SSLkey.log in succession, in order to decrypt the user selected stream ?

Doing this would unfailingly solve the decryption problem under both of your scenarios. If this exhaustive search is not possible in the 'standard' Wireshark, would it be difficult to tweak the source code to accomplish it?

TLS does not work like that, you cannot just "decrypt" each packet with the same key. At the beginning of a handshake, a key is agreed on, but decryption of further records depends on earlier ones. If the order is wrong, then the correct decryption keys for the following records cannot be derived. Brute-forcing through an exhaustive search would now work.

What would work is fixing up the TCP dissector to put the TCP segments in the correct order, but this is still an open issue (see the bug I linked in the post).

Once again, thank you for your helpful reply. Please excuse the delay in my response. I am unfamiliar with SSL/TLS and had to spend time reading up on them.

Also, kindly bear with me but I am having difficulty following your statement that: " the correct decryption keys for the following records cannot be derived"

To speedily allow you to point out the error(s) in my understanding, I have enumerated below, the events which I believe are occurring during the TLS session key generation process. This understanding is based on my reading of references [1, 2, 3]. I have not been able to determine exactly what my browser is recording in the SSLkey.log text file (step 7 below). The best information which I have managed to obtain is given in reference [4]. Accordingly, there are two possibilities (a) only the session keys themselves are recorded (b) the session keys as well as the intermediate values ie client's random value, server's random value, client's premaster secret and server's premaster secret, are being recorded. Here are the steps per my understanding:

1)  My Firefox Browser (the client), sends a "Client hello" message to the server hosting the website containing video (the URL of which I wish to know), along with the client's random value and supported cipher suites.
2)  The server responds by sending a "Server hello" message to the client, along with the server's random value.
3)  The server sends its certificate to the client for authentication and may request a certificate from the client. The server sends the "Server hello done" message.
4)  If the server has requested a certificate from the client, the client sends it.
5)  The client creates a random Pre-Master Secret and encrypts it with the public key from the server's certificate, sending the encrypted Pre-Master Secret to the server.
6)  The server receives the Pre-Master Secret. The server and client each generate the Master Secret and session keys based on the Pre-Master Secret.
7)  The client records the new session keys, one for hashing and a second for encrypting messages and perhaps intermediate values, in the SSLkey.log text file.
8)  The client sends "Change cipher spec" notification to server to indicate that the client will start using the new session keys for hashing and encrypting messages. Client also sends "Client finished" message.
9)  Server receives "Change cipher spec" and switches its record layer security state to symmetric encryption using the session keys. Server sends "Server finished" message to the client.
10) Client and server can now exchange application data over the secured channel they have established. All messages sent from client to server and from server to client are encrypted using session key.
11) It is possible that the website is 'nested' ie transfers the client to one or more subsequent websites which all require steps 1-10 to be completed.

Now, these SSLkey.log entries are being recorded by the Browser, which always correctly decrypts the stream of interest (video feed in my case), ie its unaffected by the incorrect byte order issue which is confusing Wireshark. Hence these entries are entirely correct. Also, regardless of the two possibilities which I mentioned for the contents of SSLkey.log, one of the entries must be the correct session key for decrypting the video stream. Moreover, per the TLS protocol described in steps 1 to 11, the session key for decryption remains unchanged throughout the decryption process ie shuffling the videofeed byte order would simply mess up the video but would not affect the decryption session key.

If the aforementioned is true, then using each of the entries of the SSLkey.log file in succession (exhaustive search), will unfailingly result in successfully decrypting the stream of interest, regardless of shuffles in byte order.

References [1] https://msdn.microsoft.com/en-us/library/windows/desktop/aa380513(v=vs.85).aspx [2] https://uk.sans.org/reading-room/whitepapers/protocols/ssl-tls-beginners-guide-1029 [3] https://tlswg.github.io/tls13-spec/ [4] https://enterprise.cloudshark.org/blog/2015-05-14-all-about-ssl-key-logging/

Now, these SSLkey.log entries are being recorded by the Browser, which always correctly decrypts the stream of interest (video feed in my case), ie its unaffected by the incorrect byte order issue which is confusing Wireshark.

The browser won't see an out of order TCP packet, the host OS will present the TCP stream in order to the application.

There are some inaccuracies in your assumptions (e.g. the Client Hello does not store the full URL for example, only the hostname), but let's disregard them and focus on the key material. Your step 7 is about the derivation of MAC (integrity), key/IV (symmetric encryption) as can be found in RFC 5246 - TLS 1.2, Section 6.3. Key Calculation. This key is derived from the Master Secret (which is derived from the pre-master secret) combined with the Client and Server Random values.

The key used in step 10 (for Application Data) is based on the key/IV as can be seen in the subsections after Section 6.2.3. Record Payload Protection. The integrity key depends on a sequence number. Out-of-order packets will result in the wrong integrity check and as a result decryption will fail.

The SSL keylog file stores a mapping from the Client Random value to the Master Secret. Thus it should become clear now that enumerating all entries in the SSL keylog file will not help since the Client Random value is unique and we can figure out whether the key exists or not.

Why does it work for the web browser and not Wireshark? The web browser relies on the operating system to buffer TCP segments and re-order to ensure a stream of data that fixes out-of-order packets. Wireshark (at least for SSL/TLS) currently does not do this re-ordering yet (see the linked bug) and will thus fail to provide the required ordering that is necessary for TLS decryption to function.