SeaCat and OpenSSL Heartbleed Bug

Today, on 7th April 2014, a serious vulnerability in the popular OpenSSL has been discovered and published. This bug is especially nasty since it can disclosure important secret information in an undetectable way.

Important update (10th April 2014): Original content of this blog entry stated that one of our SeaCat server detected Heartbleed bug attack prior its actual disclosure. EFF correctly pointed out that there are other tools, that can produce the same pattern in the SeaCat server log (see http://blog.erratasec.com/2014/04/no-we-werent-scanning-for-hearbleed.html#.U0Xq4OaSz-l ). I don't have any hard data evidence to support or reject this statement. Since there is a risk that our finding is false positive, I have modified this entry to neutral tone, removing any conclusions. There are real honeypots in the Internet that should provide final evidence when Heartbleed has been broadly exploited for a first time.

Please also see another update attached at the end of this post. There is a lot of attention around this entry.

SeaCat server is using OpenSSL and therefore it is exposed to this vulnerability. However thanks to very picky SSL handshake implementation it seems that SeaCat server rejects to respond to TLS heartbeat request till authorisation is completed making it somehow resistant to this threat.

Every fresh distribution of the SeaCat server is using fixed version of OpenSSL, mitigating this issue completely.

SeaCat client for iOS and Java/Android is not affected since it is using other SSL implementation than OpenSSL (platform specific one) and it is running in a client mode.

Test of SeaCat server that uses compromised OpenSSL version

We have tested this vulnerability using simple detection tool hb-test.py.

This is an output:

$ python hb-test.py 192.241.86.55 -p 443
Connecting...
Sending Client Hello...
Waiting for Server Hello...
 ... received message: type = 22, ver = 0301, length = 58
 ... received message: type = 22, ver = 0301, length = 2914
 ... received message: type = 22, ver = 0301, length = 14

Unexpected EOF receiving record header - server closed connection
Server closed connection without sending Server Hello.

SeaCat server refuses to communicate and closes the connection after a while. Moreover, there is a warning in the log file (see bellow). No internal information has been exposed since no heartbeat packet has been returned back.

You should upgrade your OpenSSL library anyway!

Log entries

Since SeaCat server is creating log entry for such an attack attempt, it is actually representing a kind of honeypot. The same entry is also created for any other non-standard SSL handshake that is not necessarily an attack (e.g. mass scan tool).

Here are log entries observed by two of our SeaCat servers. These entries started on 23rd March and culminated on 7th April 2014 (the day OpenSSL vulnerability has been announced publicly). IP addresses of these servers are quite private (only DNS entries exist) and service is running on port TCP/443 (standard HTTPS), so these attempts have to be result of broad automated scans.

Server 1

2014:03:23 07:24:48     691  WARN: Unexpected SSL error during handshake (5): error:00000005:lib(0):func(0):DH lib
2014:03:24 11:42:42     691  WARN: Unexpected SSL error during handshake (1): error:00000001:lib(0):func(0):reason(1)
2014:03:24 12:43:26     691  WARN: Unexpected SSL error during handshake (5): error:00000005:lib(0):func(0):DH lib
2014:03:25 03:23:03     691  WARN: Unexpected SSL error during handshake (5): error:00000005:lib(0):func(0):DH lib
2014:03:28 09:22:30     691  WARN: Unexpected SSL error during handshake (5): error:00000005:lib(0):func(0):DH lib
2014:03:31 02:18:55     691  WARN: Unexpected SSL error during handshake (1): error:00000001:lib(0):func(0):reason(1)
2014:03:31 08:06:13     682  WARN: Unexpected SSL error during handshake (5): error:00000005:lib(0):func(0):DH lib
2014:03:31 21:19:07     691  WARN: Unexpected SSL error during handshake (1): error:00000001:lib(0):func(0):reason(1)
2014:04:04 20:46:17     687  WARN: Unexpected SSL error during handshake (1): error:00000001:lib(0):func(0):reason(1)
2014:04:07 08:20:36     691  WARN: Unexpected SSL error during handshake (5): error:00000005:lib(0):func(0):DH lib
2014:04:07 11:25:46     691  WARN: Unexpected SSL error during handshake (5): error:00000005:lib(0):func(0):DH lib
2014:04:07 11:25:47     691  WARN: Unexpected SSL error during handshake (1): error:00000001:lib(0):func(0):reason(1)
2014:04:07 11:25:49     682  WARN: Unexpected SSL error during handshake (1): error:00000001:lib(0):func(0):reason(1)
2014:04:07 11:25:53     691  WARN: Unexpected SSL error during handshake (1): error:00000001:lib(0):func(0):reason(1)
2014:04:07 11:25:55     691  WARN: Unexpected SSL error during handshake (5): error:00000005:lib(0):func(0):DH lib
2014:04:07 11:25:58     687  WARN: Unexpected SSL error during handshake (5): error:00000005:lib(0):func(0):DH lib
2014:04:08 13:20:36     691  WARN: Unexpected SSL error during handshake (5): error:00000005:lib(0):func(0):DH lib
2014:04:08 21:25:54     687  WARN: Unexpected SSL error during handshake (5): error:00000005:lib(0):func(0):DH lib
2014:04:08 21:27:11     691  WARN: Unexpected SSL error during handshake (5): error:00000005:lib(0):func(0):DH lib

Server 2

This server has luckily debug level logging enabled.

2014:03:24 09:41:52     690 DEBUG: New client connection accepted from 54.186.135.68 49769
2014:03:24 09:42:12     690  WARN: Unexpected SSL error during handshake (5): error:00000005:lib(0):func(0):DH lib

2014:03:24 10:12:42     696 DEBUG: New client connection accepted from 220.181.159.76 48950
2014:03:24 10:12:42     696  WARN: Unexpected SSL error during handshake (1): error:00000001:lib(0):func(0):reason(1)

2014:03:25 06:36:56     699 DEBUG: New client connection accepted from 54.193.75.203 36633
2014:03:25 06:37:17     699  WARN: Unexpected SSL error during handshake (5): error:00000005:lib(0):func(0):DH lib

2014:03:27 08:33:30     696 DEBUG: New client connection accepted from 5.61.43.116 43177
2014:03:27 08:34:13     696  WARN: Unexpected SSL error during handshake (5): error:00000005:lib(0):func(0):DH lib

2014:03:31 05:43:09     690 DEBUG: New client connection accepted from 173.45.70.84 44362
2014:03:31 05:43:09     690  WARN: Unexpected SSL error during handshake (1): error:00000001:lib(0):func(0):reason(1)

2014:03:31 10:01:10     699 DEBUG: New client connection accepted from 54.72.158.29 44537
2014:03:31 10:01:30     699  WARN: Unexpected SSL error during handshake (5): error:00000005:lib(0):func(0):DH lib

2014:03:31 21:19:07     685 DEBUG: New client connection accepted from 113.64.221.42 1919
2014:03:31 21:19:07     685  WARN: Unexpected SSL error during handshake (1): error:00000001:lib(0):func(0):reason(1)

2014:04:04 20:45:20     685 DEBUG: New client connection accepted from 83.222.250.151 58947
2014:04:04 20:45:20     685  WARN: Unexpected SSL error during handshake (1): error:00000001:lib(0):func(0):reason(1)

2014:04:06 04:19:09     699 DEBUG: New client connection accepted from 62.210.125.141 33877
2014:04:06 04:19:09     699  WARN: Unexpected SSL error during handshake (5): error:00000005:lib(0):func(0):DH lib

2014:04:07 09:32:16     696 DEBUG: New client connection accepted from 54.197.30.237 60999
2014:04:07 09:32:36     696  WARN: Unexpected SSL error during handshake (5): error:00000005:lib(0):func(0):DH lib

2014:04:07 11:32:32     696 DEBUG: New client connection accepted from 80.82.70.118 55409
2014:04:07 11:32:32     696  WARN: Unexpected SSL error during handshake (5): error:00000005:lib(0):func(0):DH lib

2014:04:07 11:32:34     696 DEBUG: New client connection accepted from 80.82.70.118 60641
2014:04:07 11:32:34     696  WARN: Unexpected SSL error during handshake (1): error:00000001:lib(0):func(0):reason(1)

2014:04:07 11:32:42     690 DEBUG: New client connection accepted from 80.82.70.106 49246
2014:04:07 11:32:42     690  WARN: Unexpected SSL error during handshake (1): error:00000001:lib(0):func(0):reason(1)

2014:04:07 11:32:42     685 DEBUG: New client connection accepted from 80.82.70.106 49322
2014:04:07 11:32:44     685  WARN: Unexpected SSL error during handshake (5): error:00000005:lib(0):func(0):DH lib

2014:04:07 11:32:45     690 DEBUG: New client connection accepted from 80.82.70.106 50842
2014:04:07 11:32:47     690  WARN: Unexpected SSL error during handshake (5): error:00000005:lib(0):func(0):DH lib

2014:04:08 13:19:11     699 DEBUG: New client connection accepted from 209.126.230.70 59259
2014:04:08 13:19:23     699  WARN: Unexpected SSL error during handshake (5): error:00000005:lib(0):func(0):DH lib

2014:04:08 18:23:44     690 DEBUG: New client connection accepted from 183.60.244.46 43715
2014:04:08 18:23:49     690  WARN: Unexpected SSL error during handshake (5): error:00000005:lib(0):func(0):DH lib

Update (10th April 2014)

There is quite a large number of people reading this post and asking similar questions.
Here is a small FAQ:

What are these servers?

These servers are test/demo instances of our product. It is one virtual server with multiple public IPs that are exposed to the Internet (it is a part of a test).
It is a long-running stability test of a new SeaCat product: people from our Czech branch have dedicated mobile application on their phones that communicates with these servers via the Internet.
Time to time we also observe random scans (since it is running on TCP/443 it is not surprising).
This mobile application is not published to any store and it is quite limited to specific group of people. Current configuration has been enabled during Jan 2014 and last significant change has been done there in Feb 2014.
Since that time it is running quite smoothly. I actually noticed these log entries even before 'Heartbleed bug announcement' and they made it even to our development backlog but not much priority has been assigned to them since server is stable.

Can you describe architecture of the server software?

It is built using libev, this means that it is asynchronous in terms of IO.
This implicates that OpenSSL is not reading data directly from a network socket (and not writing data to socket) but there is a layer between actual socket and OpenSSL.
Also OpenSSL handshake is coordinated by our code. This is not anyhow extraordinary design - a lot of a little bit advanced servers are designed this way.
However, this gives you a possibility to 1) log activities that deviates from 'normal' connection 2) insert additional checks into standard OpenSSL handshake.
Since our product consists also from client implementation, we narrowed down a way how SSL establishes secure session, exclude techniques that we don't need or want to support.
I guess this is way it actually complains about heartbleed attack attempts - it is not following expected path that our clients do. This also applies for any other non-standard SSL handshake, I'm not able retrospectively see a differene.

What does it means?

Can I test that personally?

Yes, you can actually test that on your own: SeaCat is available as a trial for Mac, you can virtually recreate the whole story on your own.