HTTPS Communication between an Android App and Tomcat7 using Self-Signed Certificates

Technology Overview

HTTPS and TLS (SSL)

HTTPS (HTTP Secure) uses TLS (previously SSL) to provide authenticated and encrypted communication between a client and a server. Typically the authenticity of the server is verified by the client but occasionally you may also want the server to verify the authenticity of the client. This post describes the former and can be extended to provide the latter.

TLS typically uses digital certificates based on the X.509 standard to verify the authenticity of one or both communicating entities (server and/or client). A digital certificate typically contains:

Owner’s public key
Owner’s name
Expiration date of the public key
Name of the Certificate Authority (CA) that issued the Digital Certificate
Serial number of the Digital Certificate
Digital signature of the issuer

TLS is a hybrid protocol that uses

Assymetric cryptography based on the above certificates to verify the authenticity of a communicating entity
Symmetric cryptography (shared keys) to ensure secrecy of the communicated messages via shared key encryption

Ensuring Authenticity

In the public key infrastructure (PKI), authenticity verification is implemented in the following way:

The Sender creates the message to send
The Sender computes a hashed message digest of the message using a hashing algorithm, e.g., SHA-2
- Think of this as a summary of the message that will change if the message changes
The Sender encrypts the hashed message digest using its private key
- This is the digital signature of the message
The Sender sends the message and the digital signature to the Receiver
The Receiver creates a hashed message digest of the message using the same algorithm as the Sender
The Receiver decrypts the digital signature using the public key of the Sender
- This (and PKI) works because
  - Only the Sender’s public key can correctly decrypt data encrypted with the Sender’s private key, and because
  - It is impossible (computationally infeasible) to reverse engineer (create) the private key from the public key
- We typically use keys that are 2048 bits long or longer
If the decrypted digital signature (from step 6) matches the hashed message digest (from step 5), then
- The message has not been changed in transit – since the two independently created hashed message digests match
- The digital signature was encrypted using the Sender’s private key and hence the Sender is authenticated as the source of the message

Certificate Chains, Root Certificates, and Self-Signed Certificates

There are two types of Certificate Authorities – Intermediate CAs and Root CAs.

Root CAs form what are called, trust anchors, as they are independently trusted with no one higher to verify their authenticity. Organizations like Symantec and Network Solutions fall into this category. Certificates issued by Root CAs are called root certificates.

Intermediate CAs are CAs whose authenticity is verified by another Intermediate CA or a Root CA.

A Certificate Chain is a chain of trust where the Root CA issues a certificate to an Intermediate CA who issues a certificate to another Intermediate CA and so on until finally the last Intermediate CA issues a certificate to the ultimate client or server who has purchased a certificate verifying their own authenticity.

A verifier (lets say the Client or the Receiver) has to verify this entire chain of trust before it can accept the authenticity of the Server (or Sender).

The verifier (the Client or Receiver – typically your home desktop / laptop computer) has a trusted store of certificates. This trusted store is typically pre-loaded by your Operating System provider (Windows, Ubuntu, Apple, etc.) with root certificates from Root CAs and maybe some certificates from Intermediate CAs.

Root certificates are either unsigned or self-signed by the Root CA. Their authenticity is established by other means. They typically come pre-loaded on computers as the Operating System manufacturer gets them directly from the Root CAs.

Certificates issued by a Root CA or an Intermediate CA are digitally signed by the issuing CA.

Self-signed certificates are certificates which are signed by the issuing authority. This is self-certification. If you are not a certified CA, then you can create your own self-signed certificates. These are typically used for testing or when you control both the client and the server but need to provide for authenticated and encrypted communication over an open network.

Verifying a Digital Certificate

As a business, digital certificate verification works because

The issuing authority independently verifies the authenticity of the entity to which it issues the certificate
- The issuing authority’s agents may verify the physical address, phone number, organization name, etc. of the entity
The public at large trusts the issuing authority, e.g., Symantec and Network Solutions

Digitally, verification works as per the previous section with a few additions

The issuing authority’s public key is known to the verifier
- This is usually because the certificate establishing the identity of the issuing authority is present in the verifier’s trust store and the verifier can obtain the issuing authority’s public key from it
The verifier decrypts the digital signature on the certificate using the issuer’s public key
- This decrypted digital signature is a hashed message digest of the data contained in the certificate
The verifier computes a hashed message digest of the data contained in the certificate
If the hashed message digests from steps 2 and 3 are the same, then the certificate is verified
Additionally the verifier will check if the certificate’s validity date has been passed, whether the server URL from which it received the certificate matches the server URL in the certificate, whether the certificate has been revoked, etc.

Verifying a Digital Certificate Chain

Verification of a digital certificate chain works as above except that typically the server provides the complete chain of certificates but does not include the root certificate.

The client verifies the digital signature of a certificate using the public key from the previous certificate in the chain. The previous certificate in the chain belong’s to the issuing authority of the certificate, and so on, up the chain.

Verification stops when the verifoer finds one of the certificates in the chain or the root certificate in its trusted store. If a matching certificate is not found in the trusted store, then verification fails, and communication with the server is aborted.

Encryption in TLS (SSL)

Once the client verifies the authenticity of the server, it selects a shared key and encrypts it with the public key of the server (from the server’s certificate). It sends encrypted shared key to the server. The shared key can be decrypted only by using the private key of the server (asymmetric encryption), hence it is protected from eavesdropping. The server saves this shared key for the duration of the session. All subsequent communications in the session are encrypted and decrypted with this shared key (symmetric encryption).

Making it Work

In this post, we will store your server’s self-signed certificate in the trusted store of the Android App. Thus, when the Android App receives your server’s certificate, as part of the HTTPS handshake, it will find the certificate in it’s own trusted store and thus accept it.

We used the following tools and SDKs while developing and testing the setup:

All development was done on a Mac OS X version 10.9.2
Eclipse Kepler Service Release 2 was the IDE
Android 4.4.2 API Level 19 SDK
Open SSL version 1.0.1g 7
keytool from Java 1.7.0_55
Bouncy Castle JAR – bcprov-jdk15on-146.jar

Create the Server’s Private Key in PEM format

$ openssl genrsa -des3 -out web_server_private_key.pem 2048

Create the Server’s Self-Signed Digital Certificate in PEM format

$ openssl req -new -x509 -key web_server_private_key.pem -out web_server_ssl_certificate.pem -days 365

Create the Server’s Key Store containing its Self-Signed Digital Certificate

Java’s keytool does not allow importing an existing private key into a key store.

Hence, we first use openssl to import the private key into a PKCS12 format key store.

$ openssl pkcs12 –export –inkey web_server_private_key.pem –in web_server_ssl_certificate.pem –out web_server_key_store.p12

We then convert the PKCS12 format key store into JKS format that can be used by Tomcat7.

$ keytool –importkeystore –srckeystore web_server_key_store.p12 –srcstoretype pkcs12 –destkeystore web_server_key_store.jks –deststoretype jks

Create the Client’s (Android App’s) Trust Store containing the Server’s Self-Signed Digital Certificate

Android supports the BKS format for Trust Stores via the Bouncy Castle library. You can find the JAR here:

http://www.bouncycastle.org/download/bcprov-jdk15on-146.jar

$ keytool -import -v -trustcacerts -alias ‘Web Server Key’ -file web_server_ssl_certificate.pem -keystore apptruststore.bks -storetype BKS -providerclass org.bouncycastle.jce.provider.BouncyCastleProvider -providerpath bcprov-jdk15on-146.jar -storepass S3cr3t

Configure Tomcat7 for HTTPS with the Server’s Key Store

Copy server keystore web_server_key_store.jks to the web server

Place the file in /etc/tomcat7 with the other tomcat7 configuration files

2. Edit /etc/tomcat7/server.xml and add the HTTPS connector to it

<Connector port="443" protocol="HTTP/1.1" SSLEnabled="true"
               maxThreads="150" scheme="https" secure="true"
               clientAuth="false" sslProtocol="TLS" 
               keystoreFile="/etc/tomcat7/web_server_key_store.jks"
               keystorePass="S3cr3t"/>

3. Comment out the plain HTTP connectors, e.g.,

<!--
    <Connector port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               URIEncoding="UTF-8"
               redirectPort="8443" />
    -->

4. Restart tomcat7

$ sudo service tomcat7 restart

Use the Client Trust Store in the Android App

Place the client trust store apptruststore.bks in the assets folder of your Android App
When you want to open a connection to your server, you would

Load the Trust Store into memory

private KeyStore loadTrustStore() throws KeyStoreException, IOException, NoSuchAlgorithmException, CertificateException {

        String TRUST_STORE_FORMAT = "BKS";
        String TRUST_STORE_FILENAME = "apptruststore.bks";
        String TRUST_STORE_PASSWORD = "S3cr3t";
        KeyStore trustStore = null;

        // Create a new trustStore object
        try {
            trustStore = KeyStore.getInstance(TRUST_STORE_FORMAT);
        } catch (KeyStoreException e) {
            throw e;
        }

        // Open the trustStore file for reading
        AssetManager assetManager = this.getContext().getAssets();

        InputStream inputStream = null;
        try {
            inputStream = assetManager.open(TRUST_STORE_FILENAME,
                    AssetManager.ACCESS_UNKNOWN);
        } catch (IOException e) {
            throw e;
        }

        // Load the trustStore
        try {
            trustStore.load(inputStream,
                    TRUST_STORE_PASSWORD.toCharArray());
        } catch (NoSuchAlgorithmException e) {
            throw e;
        } catch (CertificateException e) {
            throw e;
        }

        inputStream.close();

        return trustStore;
    }

Create a TrustManagerFactory based on the Trust Store

private TrustManagerFactory initializeTrustManagerFactory(KeyStore trustStore) throws NoSuchAlgorithmException, KeyStoreException {

        TrustManagerFactory trustManagerFactory = TrustManagerFactory
                .getInstance(TrustManagerFactory.getDefaultAlgorithm());

        trustManagerFactory.init(trustStore);

        return trustManagerFactory;
    }

Create an SSLContext based on the TrustManagerFactory

private SSLContext initializeSSLContext(TrustManagerFactory trustManagerFactory) throws NoSuchAlgorithmException, KeyManagementException {

        String CRYPTO_PROTOCOL = "TLS";

        SSLContext sslContext = SSLContext.getInstance(CRYPTO_PROTOCOL);

        sslContext.init(null, trustManagerFactory.getTrustManagers(),
                null);

        return sslContext;
    }

Create and use the HttpsUrlConnection based on the SSLContext

public JSONObject postToAppWebServer(Context context, String url,
            String urlEncodedKeyValuePairsToPost, SSLContext sslContext) throws IOException,
            NonHttp200Exception, JSONException, DisconnectedNetworkException {

        String HTTP_METHOD_POST = "POST";
        String HTTP_REQUEST_PROPERTY_CONTENT_TYPE = "Context-Type";
        String HTTP_REQUEST_PROPERTY_CONTENT_TYPE_VALUE_URL_ENCODED = "application/x-www-form-urlencoded";

        JSONObject serverResponse = null;
        HttpsURLConnection connection = null;
        String responseString = null;

        try {

            URL url = new URL(url);

            connection = (HttpsURLConnection) url.openConnection();

            connection.setSSLSocketFactory(sslContext.getSocketFactory());

            connection.setRequestMethod(HTTP_METHOD_POST);
            connection
                    .setRequestProperty(
                            HTTP_REQUEST_PROPERTY_CONTENT_TYPE,
                            HTTP_REQUEST_PROPERTY_CONTENT_TYPE_VALUE_URL_ENCODED);

            connection.setDoOutput(true);
            connection.setUseCaches(false);
            connection.setAllowUserInteraction(false);

            DataOutputStream requestData = new DataOutputStream(
                    connection.getOutputStream());
            requestData.writeBytes(urlEncodedKeyValuePairsToPost);
            requestData.flush();
            requestData.close();

            int responseCode = connection.getResponseCode();

            if (responseCode == NetConstants.HTTP_RESPONSE_CODE_SUCCESS) {
                responseString = StringUtilities
                        .getStringFromInputStream(connection
                                .getInputStream());

                serverResponse = new JSONObject(responseString);
            } else {
                String errorString = StringUtilities
                        .getStringFromInputStream(connection
                                .getErrorStream());

                throw new NonHttp200Exception(errorString,
                        connection.getResponseCode(),
                        connection.getResponseMessage(), errorString);
            }
        } catch (IOException e) {
            throw e;
        } catch (JSONException e) {
            throw e;
        } finally {
            if (connection != null) {
                connection.disconnect();
            }
        }

        return serverResponse;
    }

Thats all! Under the covers Java / Android will handle the HTTPS handshaking and establish the connection.

References

Thanks to the following web postings for the valuable information, procedures, and insights