unable to decrypt HTTPS traffic with Wireshark

Question

Hello,

I have tried the latest developers edition (2.0.0rc1), 1.99 and the latest stable, 1.12.8 and I am unable to decrypt traffic from one particular certificate.

I am able to decrypt traffic from another website with another key so I believe my wireshark settings are set to a working state.

The certificate was created in IIS and exported to a PFX file. I have extracted the key with openssl and removed the password.

I created a test site in a new windows server install and bound the certificate. The site is a basic IIS under construction page.

My debug shows these messages (edited down):

ssl_init private key file C:\temp\key.key successfully loaded.
ssl_find_private_key server 10.1.1.1:443
ssl_find_private_key: testing 1 keys
dissect_ssl enter frame #1349 (first time)
packet_from_server: is from server - TRUE
  conversation = 000000000464CAF8, ssl_session = 0000000007998780
  record: offset = 0, reported_length_remaining = 3778
dissect_ssl3_record found version 0x0301(TLS 1.0) -> state 0x11
dissect_ssl3_record: content_type 22 Handshake
decrypt_ssl3_record: app_data len 3773, ssl state 0x11
packet_from_server: is from server - TRUE
decrypt_ssl3_record: using server decoder
decrypt_ssl3_record: no decoder available
dissect_ssl3_handshake iteration 1 type 2 offset 5 length 81 bytes, remaining 3778 
dissect_ssl3_hnd_hello_common found SERVER RANDOM -> state 0x13
dissect_ssl3_hnd_srv_hello found CIPHER 0x002F -> state 0x17
dissect_ssl3_hnd_srv_hello trying to generate keys
ssl_generate_keyring_material not enough data to generate key (0x17 required 0x37 or 0x57)
dissect_ssl3_hnd_srv_hello can't generate keyring material
ssl_decrypt_pre_master_secret wrong pre_master_secret length (256, expected 48)
ssl_generate_pre_master_secret: can't decrypt pre master secret
trying to use SSL keylog in 
failed to open SSL keylog
dissect_ssl3_handshake can't generate pre master secret
  record: offset = 267, reported_length_remaining = 59
dissect_ssl3_record: content_type 20 Change Cipher Spec
dissect_ssl3_change_cipher_spec
packet_from_server: is from server - FALSE
ssl_change_cipher CLIENT
  record: offset = 273, reported_length_remaining = 53
dissect_ssl3_record: content_type 22 Handshake
decrypt_ssl3_record: app_data len 48, ssl state 0x17
packet_from_server: is from server - FALSE
decrypt_ssl3_record: using client decoder
decrypt_ssl3_record: no decoder available

Answer 1

0

ssl_generate_keyring_material not enough data to generate key (0x17 required 0x37 or 0x57)

That's a possible sign, that you have the wrong private key for the certificate.

Please check the Modulus of the private key and the public key (cert). See my answer to a similar question:

https://ask.wireshark.org/questions/46788/cant-decode-ssl-session-even-though-the-cipher-is-not-diffie-hellman

BTW: What's the Wireshark version that created the ssl debug file? 2.0.0rc should have some code to match the private/public key based on the Modulus. See the comment of @Lekensteyn in the following question.

https://ask.wireshark.org/questions/46834/ssl-failure-to-decrypt-pre-secret

Regards
Kurt

answered 26 Oct '15, 14:49

Kurt Knochner ♦
24.8k●10●39●237
accept rate: 15%

edited 26 Oct '15, 15:06

Thanks so much for responding Kurt

Regarding the modulus, I ran the following commands as per another post I saw before yours (https://ask.wireshark.org/questions/22813/not-able-to-decrypt-ssl-data-with-private-keys):

openssl x509 -noout -modulus -inform DER -in certfile.cert

openssl rsa -noout -modulus -in privatekey.key

The two outputs were identical which tell me I’m using the right private key.

I’ve gone between the latest stable release and the latest dev release so I can’t remember which release I was on when I posted the debug but I’ll re-post as I’ve re-installed 2.0.0rc1. The capture was done with 1.12.8 but the debug logs is from opening the file with 2.0.0rc1.

Went back to Edit this so its readable..I’m not used to the formatting in this forum..

Wireshark SSL debug log
ssl_association_remove removing TCP 443 - http handle 0000000004BF2BC0
KeyID[20]:
| f9 bd d3 76 37 69 bd 3d e6 db f1 90 cf 11 c7 da |…v7i.=……..|
| b5 9d 13 90                                     |….            |
ssl_init private key file C:\temp\keynopwd.key successfully loaded.
ssl_init port '443' filename 'C:\temp\keynopwd.key' password(only for p12 file) ''
association_add TCP port 443 protocol http handle 0000000004BF2BC0
KeyID[20]:
| f9 bd d3 76 37 69 bd 3d e6 db f1 90 cf 11 c7 da |…v7i.=……..|
| b5 9d 13 90                                     |….            |
ssl_init private key file C:\temp\keynopwd.key successfully loaded.
ssl_init port '443' filename 'C:\temp\keynopwd.key' password(only for p12 file) ''
association_add TCP port 443 protocol http handle 0000000004BF2BC0
dissect_ssl enter frame #824 (first time)
association_find: TCP port 49309 found 0000000000000000
packet_from_server: is from server - FALSE
conversation = 0000000008BBD6B0, ssl_session = 0000000008BC0E10
record: offset = 0, reported_length_remaining = 182
dissect_ssl3_record: content_type 22 Handshake
Calculating hash with offset 5 177
decrypt_ssl3_record: app_data len 177, ssl state 0x00
association_find: TCP port 49309 found 0000000000000000
packet_from_server: is from server - FALSE
decrypt_ssl3_record: using client decoder
decrypt_ssl3_record: no decoder available
dissect_ssl3_handshake iteration 1 type 1 offset 5 length 173 bytes, remaining 182
ssl_dissect_hnd_hello_common found CLIENT RANDOM -> state 0x01
dissect_ssl enter frame #825 (first time)
packet_from_server: is from server - TRUE
conversation = 0000000008BBD6B0, ssl_session = 0000000008BC0E10
record: offset = 0, reported_length_remaining = 1460
need_desegmentation: offset = 0, reported_length_remaining = 1460
dissect_ssl enter frame #829 (first time)
packet_from_server: is from server - TRUE
conversation = 0000000008BBD6B0, ssl_session = 0000000008BC0E10
record: offset = 0, reported_length_remaining = 4109
dissect_ssl3_record found version 0x0301(TLS 1.0) -> state 0x11
dissect_ssl3_record: content_type 22 Handshake
Calculating hash with offset 5 4104
decrypt_ssl3_record: app_data len 4104, ssl state 0x11
packet_from_server: is from server - TRUE
decrypt_ssl3_record: using server decoder
decrypt_ssl3_record: no decoder available
dissect_ssl3_handshake iteration 1 type 2 offset 5 length 81 bytes, remaining 4109
ssl_dissect_hnd_hello_common found SERVER RANDOM -> state 0x13
ssl_dissect_hnd_srv_hello found CIPHER 0xC014 -> state 0x17
dissect_ssl3_handshake iteration 0 type 11 offset 90 length 2275 bytes, remaining 4109
lookup(KeyID)[20]:
| f9 bd d3 76 37 69 bd 3d e6 db f1 90 cf 11 c7 da |…v7i.=……..|
| b5 9d 13 90                                     |….            |
ssl_find_private_key_by_pubkey: lookup result: 0000000005BCFAC0
dissect_ssl3_handshake iteration 0 type 22 offset 2369 length 1401 bytes, remaining 4109
dissect_ssl3_handshake iteration 0 type 12 offset 3774 length 327 bytes, remaining 4109
dissect_ssl3_handshake iteration 0 type 14 offset 4105 length 0 bytes, remaining 4109
dissect_ssl enter frame #831 (first time)
packet_from_server: is from server - FALSE
conversation = 0000000008BBD6B0, ssl_session = 0000000008BC0E10
record: offset = 0, reported_length_remaining = 134
dissect_ssl3_record: content_type 22 Handshake
Calculating hash with offset 5 70
decrypt_ssl3_record: app_data len 70, ssl state 0x17
packet_from_server: is from server - FALSE
decrypt_ssl3_record: using client decoder
decrypt_ssl3_record: no decoder available
dissect_ssl3_handshake iteration 1 type 16 offset 5 length 66 bytes, remaining 75
ssl_load_keyfile dtls/ssl.keylog_file is not configured!
ssl_generate_pre_master_secret: found SSL_HND_CLIENT_KEY_EXCHG, state 17
ssl_restore_master_key can't find pre-master secret by Unencrypted pre-master secret
ssl_decrypt_pre_master_secret key exchange 24 different from KEX_RSA (30)
ssl_generate_pre_master_secret: can't decrypt pre-master secret
ssl_restore_master_key can't find pre-master secret by Encrypted pre-master secret
dissect_ssl3_handshake can't generate pre master secret
record: offset = 75, reported_length_remaining = 59
dissect_ssl3_record: content_type 20 Change Cipher Spec
dissect_ssl3_change_cipher_spec
ssl_load_keyfile dtls/ssl.keylog_file is not configured!
ssl_finalize_decryption state = 0x17
ssl_restore_master_key can't find master secret by Session ID
ssl_restore_master_key can't restore master secret using an empty Session Ticket
ssl_restore_master_key can't find master secret by Client Random
Cannot find master secret
packet_from_server: is from server - FALSE
ssl_change_cipher CLIENT
record: offset = 81, reported_length_remaining = 53
dissect_ssl3_record: content_type 22 Handshake
Calculating hash with offset 86 48
decrypt_ssl3_record: app_data len 48, ssl state 0x17
packet_from_server: is from server - FALSE
decrypt_ssl3_record: using client decoder
decrypt_ssl3_record: no decoder available
dissect_ssl3_handshake iteration 1 type 147 offset 86 length 1415968 bytes, remaining 134
dissect_ssl enter frame #832 (first time)
packet_from_server: is from server - TRUE
conversation = 0000000008BBD6B0, ssl_session = 0000000008BC0E10
record: offset = 0, reported_length_remaining = 59
dissect_ssl3_record: content_type 20 Change Cipher Spec
dissect_ssl3_change_cipher_spec
ssl_load_keyfile dtls/ssl.keylog_file is not configured!
ssl_finalize_decryption state = 0x17
ssl_restore_master_key can't find master secret by Session ID
ssl_restore_master_key can't restore master secret using an empty Session Ticket
ssl_restore_master_key can't find master secret by Client Random
Cannot find master secret
packet_from_server: is from server - TRUE
ssl_change_cipher SERVER
record: offset = 6, reported_length_remaining = 53
dissect_ssl3_record: content_type 22 Handshake
Calculating hash with offset 11 48
decrypt_ssl3_record: app_data len 48, ssl state 0x17
packet_from_server: is from server - TRUE
decrypt_ssl3_record: using server decoder
decrypt_ssl3_record: no decoder available
dissect_ssl3_handshake iteration 1 type 52 offset 11 length 16100397 bytes, remaining 59
dissect_ssl enter frame #848 (first time)
packet_from_server: is from server - FALSE
conversation = 0000000008BBD6B0, ssl_session = 0000000008BC0E10
record: offset = 0, reported_length_remaining = 410
dissect_ssl3_record: content_type 23 Application Data
decrypt_ssl3_record: app_data len 32, ssl state 0x17
packet_from_server: is from server - FALSE
decrypt_ssl3_record: using client decoder
decrypt_ssl3_record: no decoder available
association_find: TCP port 49309 found 0000000000000000
association_find: TCP port 443 found 0000000005D8BA10
record: offset = 37, reported_length_remaining = 373
dissect_ssl3_record: content_type 23 Application Data
decrypt_ssl3_record: app_data len 368, ssl state 0x17
packet_from_server: is from server - FALSE
decrypt_ssl3_record: using client decoder
decrypt_ssl3_record: no decoder available
dissect_ssl enter frame #855 (first time)
packet_from_server: is from server - TRUE
conversation = 0000000008BBD6B0, ssl_session = 0000000008BC0E10
record: offset = 0, reported_length_remaining = 245
dissect_ssl3_record: content_type 23 Application Data
decrypt_ssl3_record: app_data len 240, ssl state 0x17
packet_from_server: is from server - TRUE
decrypt_ssl3_record: using server decoder
decrypt_ssl3_record: no decoder available
dissect_ssl enter frame #824 (already visited)
packet_from_server: is from server - FALSE
conversation = 0000000008BBD6B0, ssl_session = 0000000000000000
record: offset = 0, reported_length_remaining = 182
dissect_ssl3_record: content_type 22 Handshake
dissect_ssl3_handshake iteration 1 type 1 offset 5 length 173 bytes, remaining 182
dissect_ssl enter frame #829 (already visited)
packet_from_server: is from server - TRUE
conversation = 0000000008BBD6B0, ssl_session = 0000000000000000
record: offset = 0, reported_length_remaining = 4109
dissect_ssl3_record: content_type 22 Handshake
dissect_ssl3_handshake iteration 1 type 2 offset 5 length 81 bytes, remaining 4109
dissect_ssl3_handshake iteration 0 type 11 offset 90 length 2275 bytes, remaining 4109
dissect_ssl3_handshake iteration 0 type 22 offset 2369 length 1401 bytes, remaining 4109
dissect_ssl3_handshake iteration 0 type 12 offset 3774 length 327 bytes, remaining 4109
dissect_ssl3_handshake iteration 0 type 14 offset 4105 length 0 bytes, remaining 4109
dissect_ssl enter frame #831 (already visited)
packet_from_server: is from server - FALSE
conversation = 0000000008BBD6B0, ssl_session = 0000000000000000
record: offset = 0, reported_length_remaining = 134
dissect_ssl3_record: content_type 22 Handshake
dissect_ssl3_handshake iteration 1 type 16 offset 5 length 66 bytes, remaining 75
record: offset = 75, reported_length_remaining = 59
dissect_ssl3_record: content_type 20 Change Cipher Spec
dissect_ssl3_change_cipher_spec
record: offset = 81, reported_length_remaining = 53
dissect_ssl3_record: content_type 22 Handshake
dissect_ssl3_handshake iteration 1 type 147 offset 86 length 1415968 bytes, remaining 134
dissect_ssl enter frame #832 (already visited)
packet_from_server: is from server - TRUE
conversation = 0000000008BBD6B0, ssl_session = 0000000000000000
record: offset = 0, reported_length_remaining = 59
dissect_ssl3_record: content_type 20 Change Cipher Spec
dissect_ssl3_change_cipher_spec
record: offset = 6, reported_length_remaining = 53
dissect_ssl3_record: content_type 22 Handshake
dissect_ssl3_handshake iteration 1 type 52 offset 11 length 16100397 bytes, remaining 59
dissect_ssl enter frame #848 (already visited)
packet_from_server: is from server - FALSE
conversation = 0000000008BBD6B0, ssl_session = 0000000000000000
record: offset = 0, reported_length_remaining = 410
dissect_ssl3_record: content_type 23 Application Data
record: offset = 37, reported_length_remaining = 373
dissect_ssl3_record: content_type 23 Application Data
dissect_ssl enter frame #855 (already visited)
packet_from_server: is from server - TRUE
conversation = 0000000008BBD6B0, ssl_session = 0000000000000000
record: offset = 0, reported_length_remaining = 245
dissect_ssl3_record: content_type 23 Application Data

(28 Oct ‘15, 07:13) Sandyw

1

ssl_dissect_hnd_srv_hello found CIPHER 0xC014 -> state 0x17

I was looking for that line! 0xC014 is a cipher with Diffie Hellman key exchange (TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA) and that means you cannot decrpyt it by using the RSA key of the server. That’s why DH is being used.

So, unless you can convince the browser (or server) to export the session keys, there is no way to decrypt this session.

See here:

https://jimshaver.net/2015/02/11/decrypting-tls-browser-traffic-with-wireshark-the-easy-way/

https://www.google.com/?q=site%3Aask.wireshark.org+SSLKEYLOGFILE

and my answer to a similar question:

https://ask.wireshark.org/questions/29936/decrypting-ssl-traffic-in-wireshark-processed-by-sslsniff

Regards
Kurt

(28 Oct '15, 08:57) Kurt Knochner ♦

Answer 2

Ok I feel rather stupid about that..I thought I fixed that yesterday

I originally posted that it didn't work but it just did! Doesn't work with 1.12.5 but it just worked with the latest dev release.

The strange part is that I was testing a similar but different certificate, bought from the same place but just a few months prior, also SHA256 and I was able to decrypt with 1.12.5 of Wireshark. Same web server, same client.

Thank you so much!