My website can still be found at, but I have not been posting on this blog as I've been primarily focused on - please head over there and take a look!

Monday 10 October 2016

Without or without certificates - an idiot's guide to end-to-end web encryption

There are many cases where HTTPS / SSL isn't an option, and usually when that happens everything, including sensitive data, is sent plain text. Additionally, most websites still use passwords instead of external authentication and password policies and storage that were outdated years ago.

The risk is entirely unnecessary.

Implementing the most crucial elements of HTTPS with a combination of client and server security is non-trivial but inexpensive, and can protect your users and your organization from most types of attacks. The following protocol is based on existing protocols and algorithms - the first rule of crypto is "don't roll your own" - and as I am a crypto amateur I believe I can explain the steps in terms simple enough for most developers.

I have illustrated the implementation of this protocol in pure Javascript and Node.js here.

1. Session initiation

The only plaintext transmission is the initial transfer of the username / email address.

As a session initiation request, the client transmits the username / email address to the server, which is used to create a server-side session object identified by a token (preferably a GUID).

The server looks up the user's email address and any other authentication factor that can receive messages (ie other email addresses, mobile devices, etc) and sends them a verification code which will be used as a shared secret when authenticating. To this end the verification code must be stored in the session object.

This verification code must never be returned to the client directly.

The server responds to the session initiation request with the session identification token.

2. Session authentication

The first step to properly securing the communication is exchanging RSA public keys which will be used to protect the AES passphrases. Because the public keys must be exchanged before they can be used, the verification code sent to the user in the previous step will be the first AES passphrase.

AES 256 requires a passphrase 32 bytes (256 bits) long. We convert the verification code to 32 bytes by hashing it with the MD5 algorithm.

From this point on, every transmission will be encrypted using AES (specifically AES-256 in CTR mode). Every transmission must include the session token and an encrypted payload object. The payload object must include a timestamp, to ensure that the transmission was received in an acceptable amount of time, and a nonce, which is a randomly generated string used to prevent replay attacks.
The client generates an RSA private / public key pair and adds it to the payload. For browser calls, store the client keys in an HTML Local Storage Object.

At this point it is possible to include the user's password in the payload if one is required for authentication, although it is advisable to rely on other forms of authentication instead such as external applications (Duo, Authy), OAuth providers (Google, Facebook) or whatever else is available and practical with regards the user experience.

The payload is encrypted using the hashed verification code and is sent to the server along with the session token.

The server decrypts the payload using the hashed verification code for the session and validates the payload. Validating the payload includes testing that the timestamp isn't too old and the nonce against a list of nonces that the session has accepted.

If  validation is successful the session stores the client's public key and generates its own RSA key pair which will be used until the session expires.

The server generates a new AES passphrase and uses it to encrypt the session's public key. This will be returned to the client along with the AES passphrase encrypted using the client's public key. At this point only the client will be able to decrypt the AES passphrase protecting the session's public key.

3. Closure

The client decrypts the AES passphrase and stores the session's public key. From this point on, every transmission to the server will include the following:

a) the session token

b) a newly generated AES passphrase (or key) encrypted using the session's public key

c) the payload, including timestamp and nonce, encrypted using the generated AES passphrase

And from this point on the response format will be as follows:

a) a newly generated AES passphrase (or key) encrypted using the client's public key

b) the payload encrypted using the generated AES passphrase

Even when HTTPS is available, with the current computing power available the performance cost of this form of encryption is quite small and can add an extra layer of protection.

I have illustrated the implementation of this protocol in pure Javascript and Node.js here.

If you have any questions, suggestions or criticism, please drop me a line in the comments!