Analyzing JDBC connection issues
Some customers do have connection issues between their SAP NetWeaver Java application server and SQL Server. Especially when they are trying to update or upgrade their SAP systems because some issues only occur if you use a more recent JVM patch level.
The SUM logs contain exceptions like
The driver could not establish a secure connection to SQL Server by using Secure Sockets Layer (SSL) encryption. Error: SQL Server returned an incomplete response. The connection has been closed.
or
The driver could not establish a secure connection to SQL Server by using Secure Sockets Layer (SSL) encryption. Error: SQL Server did not return a response. The connection has been closed.
The reason for these issues are mostly related to SSL. We already discussed one issue that can occur when you try to connect to SQL Server using a Microsoft JDBC 1.2 driver in an earlier blog and SQL Server uses a certificate that is stored in the Windows certificate store and which is larger than 4KB.
In this blog post, we want to analyze connection issues that are caused by a misconfiguration of SCHANNEL. The configuration of SCHANNEL is described in Microsoft KB 245030.
Test setup
To be able to trace the communication between the JDBC driver and SQL Server, we execute the Java process on a different host.
The first thing we need to know is which SSL version the client uses and which Ciphers it offers the server. If we trace the SSL handshake, we can see that the client offers the following Ciphers (see MSDN: Cipher Suites in Schannel):
| Name | Encryption SCHANNEL\Ciphers\ | Hash SCHANNEL\Hashes\ | Exchange SCHANNEL\ KeyExchangeAlgorithms\ | Protocol |
| TLS_RSA_WITH_RC4_128_MD5 | RC4 128/128 | MD5 | PKCS | TLS 1.2 TLS 1.1 TLS 1.0 SSL 3.0 |
| SSL_CK_RC4_128_WITH_MD5 | RC4 128/128 | MD5 | PKCS | SSL 2.0 |
| TLS_RSA_WITH_RC4_128_SHA | RC4 128/128 | SHA | PKCS | TLS 1.2 TLS 1.1 TLS 1.0 SSL 3.0 |
| TLS_RSA_WITH_AES_128_CBC_SHA | AES 128/128 | SHA | PKCS | TLS 1.2 TLS 1.1 TLS 1.0 |
| TLS_DHE_RSA_WITH_AES_128_CBC_SHA | ||||
| TLS_DHE_DSS_WITH_AES_128_CBC_SHA | AES 128/128 | SHA | Diffie-Hellman | TLS 1.2 TLS 1.1 TLS 1.0 |
| TLS_RSA_WITH_3DES_EDE_CBC_SHA | Triple DES 168/168 | SHA | PKCS | TLS 1.2 TLS 1.1 TLS 1.0 |
| SSL_CK_DES_192_EDE3_CBC_WITH_MD5 | SSL 2.0 | |||
| TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA | TLS | |||
| TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA | Triple DES 168/168 | SHA | Diffie-Hellman | TLS 1.2 TLS 1.1 TLS 1.0 SSL 3.0 |
| TLS_RSA_WITH_DES_CBC_SHA | ||||
| SSL_CK_DES_64_CBC_WITH_MD5 | ||||
| TLS_DHE_RSA_WITH_DES_CBC_SHA | ||||
| TLS_DHE_DSS_WITH_DES_CBC_SHA | ||||
| TLS_RSA_EXPORT_WITH_RC4_40_MD5 | ||||
| SSL_CK_RC4_128_EXPORT40_WITH_MD5 | ||||
| TLS_RSA_EXPORT_WITH_DES40_CBC_SHA | ||||
| TLS_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA | ||||
| TLS_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA |
The server then chooses which Cipher it want to use. From the trace we can see that the SQL Server chooses TLS_RSA_WITH_AES_128_CBC_SHA. The server can only choose this cipher, if the following SCHANNEL configuration in the registry is configured like (or the keys do not exist):
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Ciphers\AES 128/128]
"enabled"=dword:0xffffffff
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Hashes\SHA]
"enabled"=dword:0xffffffff
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\KeyExchangeAlgorithms\PKCS]
"enabled"=dword:0xffffffff
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.0\Server]
"enabled"=dword:0xffffffff
If you change the configuration for SCHANNEL, each key may disable or enable multiple ciphers – see https://support.microsoft.com/kb/245030/en-us
If we disable TLS by setting
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.0\Server]
"enabled"=dword:0x00000000
all TLS related ciphers cannot be used by SQL Server and it will choose a different cipher. In our case this would be SSL_RSA_WITH_RC4_128_SHA but since this cipher is not supported by the client, the JDBC connection fails with
The driver could not establish a secure connection to SQL Server by using Secure Sockets Layer (SSL) encryption. Error: SQL Server returned an incomplete response. The connection has been closed.
Let’s disable something different. If we enable TLS again by deleting the TLS 1.0 key and disabling the RSA key exchange by adding the following registry key:
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\KeyExchangeAlgorithms\PKCS]
"enabled"=dword:0x00000000
we get the following error:
The driver could not establish a secure connection to SQL Server by using Secure Sockets Layer (SSL) encryption. Error: SQL Server did not return a response. The connection has been closed.
Both errors create an error in the Windows Server system event log.
Conclusion
If you encounter connection issues from your Java application and SQL Server that are related to SSL, first check the Windows Server system event log if it contains SCHANNEL errors or warnings. If you see such entries you can test if SCHANNEL is misconfigured by first exporting the current registry key and then delete the complete
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL]
key. Restart the computer and check if the JDBC connection works. If the connection works, ask your IT department, if they set these keys via group policy or if they use tools that configure SCHANNEL.