This is a static archive of our old Q&A Site. Please post any new questions and answers at ask.wireshark.org.

Problem with SSL, maybe IIS7?

1

Ok, so I have tried and tried and I cannot get the SSL connection to be decrypted.

I export the key using the certificate manager on my Windows 2008 R2 server. I tell it to export the private key, do no use strong encryption, and then I give it a password. I export that out, then do openssl with:

openssl pkcs12 -nodes -nocerts -in "mycer.pfx" -out "mycer.pem"

I then put that on the server and put the following in the wireshark SSL protocols

10.100.101.35,443,http,C:\mycer.pem

and load it up and it doesn't work. Here's the data from the debug:

ssl_init keys string:
10.100.101.35,443,http,C:\mycer.pem;
ssl_init found host entry 10.100.101.35,443,http,C:\mycer.pem
ssl_init addr '10.100.101.35' port '443' filename 'C:\mycer.pem' password(only for p12 file) '(null)'
Private key imported: KeyID b5:8a:14:21:92:bd:79:c8:06:fd:57:c4:56:f4:0b:34:...
ssl_init private key file C:\mycer.pem successfully loaded
association_add TCP port 443 protocol http handle 000000000677A670
ssl_init found host entry 
ssl_init entry malformed can't find port in ''

and I don't know if that's an error or not. If I try and do the follow SSL it just shows a blank screen. Anybody have any ideas?

============= UPDATE =============

Ok, I had a semicolon on the end (I had removed it before posting here, thinking it wasn't relevant). Anyways, I removed that and now I just get this in the log

ssl_init keys string:
10.100.101.35,443,http,C:\mycer.pem
ssl_init found host entry 10.100.101.35,443,http,C:\mycer.pem
ssl_init addr '10.100.101.35' port '443' filename 'C:\mycer.pem' password(only for p12 file) '(null)'
Private key imported: KeyID b4:8a:14:21:95:bd:79:c8:06:fd:54:c4:56:f3:0b:34:...
ssl_init private key file C:\mycer.pem successfully loaded
association_add TCP port 443 protocol http handle 00000000066DDF40

dissect_ssl enter frame #4 (first time) ssl_session_init: initializing ptr 0000000009051C88 size 672 conversation = 0000000009051948, ssl_session = 0000000009051C88 record: offset = 0, reported_length_remaining = 90 dissect_ssl3_record: content_type 22 decrypt_ssl3_record: app_data len 85, ssl state 0x00 association_find: TCP port 1431 found 0000000000000000 packet_from_server: is from server - FALSE decrypt_ssl3_record: using client decoder decrypt_ssl3_record: no decoder available dissect_ssl3_handshake iteration 1 type 1 offset 5 length 81 bytes, remaining 90 packet_from_server: is from server - FALSE ssl_find_private_key server 10.100.101.35:443 dissect_ssl3_hnd_hello_common found CLIENT RANDOM -> state 0x01

dissect_ssl enter frame #5 (first time) conversation = 0000000009051948, ssl_session = 0000000009051C88 record: offset = 0, reported_length_remaining = 71 dissect_ssl3_record found version 0x0300 -> state 0x11 dissect_ssl3_record: content_type 20 dissect_ssl3_change_cipher_spec packet_from_server: is from server - FALSE ssl_change_cipher CLIENT record: offset = 6, reported_length_remaining = 65 dissect_ssl3_record: content_type 22 decrypt_ssl3_record: app_data len 60, ssl state 0x11 packet_from_server: is from server - FALSE decrypt_ssl3_record: using client decoder decrypt_ssl3_record: no decoder available dissect_ssl3_handshake iteration 1 type 60 offset 11 length 14578034 bytes, remaining 71

dissect_ssl enter frame #6 (first time) conversation = 0000000009051948, ssl_session = 0000000009051C88 record: offset = 0, reported_length_remaining = 608 dissect_ssl3_record: content_type 23 decrypt_ssl3_record: app_data len 603, ssl state 0x11 packet_from_server: is from server - FALSE decrypt_ssl3_record: using client decoder decrypt_ssl3_record: no decoder available association_find: TCP port 1431 found 0000000000000000 association_find: TCP port 443 found 0000000007455440

dissect_ssl enter frame #4 (already visited) conversation = 0000000009051948, ssl_session = 0000000000000000 record: offset = 0, reported_length_remaining = 90 dissect_ssl3_record: content_type 22 dissect_ssl3_handshake iteration 1 type 1 offset 5 length 81 bytes, remaining 90

dissect_ssl enter frame #5 (already visited) conversation = 0000000009051948, ssl_session = 0000000000000000 record: offset = 0, reported_length_remaining = 71 dissect_ssl3_record: content_type 20 dissect_ssl3_change_cipher_spec record: offset = 6, reported_length_remaining = 65 dissect_ssl3_record: content_type 22 dissect_ssl3_handshake iteration 1 type 60 offset 11 length 14578034 bytes, remaining 71

dissect_ssl enter frame #6 (already visited) conversation = 0000000009051948, ssl_session = 0000000000000000 record: offset = 0, reported_length_remaining = 608 dissect_ssl3_record: content_type 23 association_find: TCP port 1431 found 0000000000000000 association_find: TCP port 443 found 0000000007455440

dissect_ssl enter frame #8 (first time) conversation = 0000000009051948, ssl_session = 0000000009051C88 record: offset = 0, reported_length_remaining = 555 dissect_ssl3_record: content_type 23 decrypt_ssl3_record: app_data len 550, ssl state 0x11 packet_from_server: is from server - FALSE decrypt_ssl3_record: using client decoder decrypt_ssl3_record: no decoder available association_find: TCP port 1431 found 0000000000000000 association_find: TCP port 443 found 0000000007455440

dissect_ssl enter frame #8 (already visited) conversation = 0000000009051948, ssl_session = 0000000000000000 record: offset = 0, reported_length_remaining = 555 dissect_ssl3_record: content_type 23 association_find: TCP port 1431 found 0000000000000000 association_find: TCP port 443 found 0000000007455440

as far as the cipher, I don’t see any DH. The “Client Hello” Info message says “TLSV1” next to it. Even if I force it to use SSLv3 though it still does not decrypt properly.

And no, I never see any Info message that says “Certificate” specifically. All I see if this:

1   2010-10-01 09:16:55.089510  firewal_0e:55:3a    Broadcast   ARP Who has 10.100.101.35?  Tell 10.100.101.1
2   2010-10-01 09:16:55.128255  xxx.xxx.xxx.xxx 10.100.101.35   TCP rgtp > https [SYN] Seq=0 Win=8192 Len=0 MSS=1460 WS=2 SACK_PERM=1
3   2010-10-01 09:16:55.229766  xxx.xxx.xxx.xxx 10.100.101.35   TCP rgtp > https [ACK] Seq=1 Ack=1 Win=262140 Len=0
4   2010-10-01 09:16:55.229807  xxx.xxx.xxx.xxx 10.100.101.35   SSLv3   Client Hello
5   2010-10-01 09:16:55.331682  xxx.xxx.xxx.xxx 10.100.101.35   SSLv3   Change Cipher Spec, Encrypted Handshake Message
6   2010-10-01 09:16:55.337238  xxx.xxx.xxx.xxx 10.100.101.35   SSLv3   Application Data
7   2010-10-01 09:16:55.441479  xxx.xxx.xxx.xxx 10.100.101.35   TCP rgtp > https [ACK] Seq=770 Ack=1628 Win=262140 Len=0
8   2010-10-01 09:16:55.457850  xxx.xxx.xxx.xxx 10.100.101.35   SSLv3   Application Data
9   2010-10-01 09:16:55.568862  xxx.xxx.xxx.xxx 10.100.101.35   TCP rgtp > https [ACK] Seq=1325 Ack=4548 Win=262140 Len=0
10  2010-10-01 09:16:55.670162  xxx.xxx.xxx.xxx 10.100.101.35   TCP rgtp > https [ACK] Seq=1325 Ack=7468 Win=262140 Len=0
11  2010-10-01 09:16:55.670353  xxx.xxx.xxx.xxx 10.100.101.35   TCP rgtp > https [ACK] Seq=1325 Ack=10388 Win=262140 Len=0
12  2010-10-01 09:16:55.670382  xxx.xxx.xxx.xxx 10.100.101.35   TCP rgtp > https [ACK] Seq=1325 Ack=11982 Win=262140 Len=0
13  2010-10-01 09:18:00.641878  xxx.xxx.xxx.xxx 10.100.101.35   TCP rgtp > https [RST, ACK] Seq=1325 Ack=11982 Win=0 Len=0

Note: I changed the source IP to xxx.xxx.xxx.xxx just for security reasons, it is shown as a real registered IP on my screen.

Hopefully all that data helps to figure this out. Let me know if I need to provide anything else.

Thanks for the help!!

asked 30 Sep ‘10, 13:51

Buck's gravatar image

Buck
21115
accept rate: 0%

edited 01 Oct ‘10, 06:22


2 Answers:

1

Looks like you have an additional ";" at the end of this key definition that is generating the last two lines of the ssl-debug that you posted. This should however not influence the decryption of your traffic.

Three main sources for not being able to decrypt are:

  • The key can't be loaded (which is not true in your case), due to the wrong file format
  • The capture file does not contain the full ssl handshake (does your trace show the "Certificate" handshake message from the server?)
  • The chosen cipher is a DH cipher (you can verify the chosen cipher in the ServerHello message, does it contain DH in the name? if it does, try to force your testclient to not use DH ciphers)

Response to your update:

In your trace, only the traffic from the client to the server is shown. Wireshark also needs the traffic from the server to the client to do the decryption.

answered 30 Sep '10, 14:03

SYN-bit's gravatar image

SYN-bit ♦♦
17.1k957245
accept rate: 20%

edited 01 Oct '10, 06:54

The problem is I have a lot of different web sites on the server, so I'm trying to filter out the other website data. What would be the best filter? I have tried

(ip.src_host == xxx.xxx.xxx.xxx && ip.dst_host == 10.100.101.35) || (ip.dst_host == xxx.xxx.xxx.xxx && ip.src_host == 10.100.101.35)

and

host 10.100.101.35 or host xxx.xxx.xxx.xxx as the capture filter. What should I be using to correctly filter out the data?

(01 Oct '10, 07:06) Buck

Looks like you are using the right filters (well, it's ip.src and ip.dst, but I'm sure you used those).

Where did you make the tracefile, it could be that traffic to and from the server are taking other paths. In that case you need to either take captures at two points or find a place where both flows travel the same path.

Or it could be that one side of the connection is vlan tagged while the other is not. In that case, could you try the following capture filter?

host 10.100.101.35 or host xxx.xxx.xxx.xxx or (vlan and (host 10.100.101.35 or host xxx.xxx.xxx.xxx))

(01 Oct '10, 07:14) SYN-bit ♦♦

Well, I don't think the filters are causing it, cause I removed them all and just ran the trace real quick and I still could not decipher the SSL.

I do have two network cards in my server though, one for incoming traffic (no gateway specified), and one for outgoing traffic (gateway specified). I guess that means that's where the problem is occurring? If that's the case though, how do I capture from both interfaces at the same time?

(01 Oct '10, 07:20) Buck

If I switch it to the other interface, I seem to get both sides talking, although every request from my server to the destination is a black line and it says the header checksum is incorrect. I still can't decrypt the traffic, but I suppose that's because some packets must still be on the other interface? I do see the "Server Hello, Certificate, Server Hello Done" now

(01 Oct '10, 07:28) Buck

When Wireshark detects bad checksums, it will not perform decryption, as the data is not trustworthy. However, if you have bad checksums on all outgoing packets, that is most likely the result of TCP checksum offloading. In that case disabling checksum checking at the TCP layer will help.

You can disable TCP checksum checking in the TCP protocol preferences.

(01 Oct '10, 07:39) SYN-bit ♦♦

Ok, I didn't have it on the TCP, but I disabled it on the IP protocol and that seems to have removed the black lines, still can't decipher the SSL though.

(01 Oct '10, 07:48) Buck

Sounds like it might be checksum offloading at the IP layer. You might want to disable Checksum Checking in the IP protocol preferences too, although I'm not sure whether that will actually make a difference on whether Wireshark will decrypt or not.

Maybe it's easier to capture on a spanport that mirrors both interfaces?

(01 Oct '10, 07:51) SYN-bit ♦♦

Maybe it's still the cipher being a DH cipher. Can you look at the details of the "ServerHello" message and tell me what cipher has been chosen (BTW the cipher is not the SSL/TLS version that you mentioned earlier).

(01 Oct '10, 07:55) SYN-bit ♦♦

I see Cipher Suite: TLS_RSA_WITH_AES_128_CBC_SHA (0x002f)

(01 Oct '10, 07:57) Buck

Wireshark should be able to decrypt that, so it's something else. I think it will be very hard to help you any further without being able to look at the tracefiles themselves.

My bet would be that the setup with two interfaces is interfering, so if you are able to capture on a spanport that mirrors both interfaces, that would be my best bet.

(01 Oct '10, 08:01) SYN-bit ♦♦

This server runs on a VMWare ESXi box as well, do you think that would cause any sort of problem?

(01 Oct '10, 08:16) Buck

ok, think I got this sorted out. It was really a combination of the problem above combined with an error with the client who was connecting (which was what I was double checking in the first place). Now I just have a different question, but I'll make a new thread for that. Thanks for all your help SYNbit - I'm able to successfully decrypt SSL connections from different clients now :)

(01 Oct '10, 13:29) Buck

Glad to be of help!

Just out of curiosity, what was the combination of problems?

(01 Oct '10, 14:01) SYN-bit ♦♦
showing 5 of 13 show 8 more comments

0

There's an empty entry trailing the first one. That's what this says:

ssl_init found host entry

That's what the parser barfs on (it shouldn't). Clean up the preference entry (no trailing spaces, etc) and try again.

answered 30 Sep '10, 14:08

Jaap's gravatar image

Jaap ♦
11.7k16101
accept rate: 14%