Web applications with TypeScript and Sequelize

I've been using TypeScript professionally for a short while now and enjoy working with it. I think it really smooths out some of the rough parts of developing server software in JavaScript. At Colibri, I've been working on a NodeJS project that is starting to move to TypeScript with some success. One part of that project, the storage subsystem, hasn't been ported yet and I've done some research to best understand how to tackle it.

With that, I created a small proof of concept application that demonstrates how Express web applications that use Sequelize (ORM) on top of Postgres can be written in TypeScript.

https://github.com/ngerakines/express-typescript-sequelize

I think there is a time and a place for tools like gulp and grunt, but most of the time they aren't necessary. For this project, I'm using the scripts block to create a chain of actions. The lint target uses the tslint tool to verify that standards are enforced. That target is dependency of the build target the runs the tsc command that compiles the TypeScript code into JavaScript. The build target is a dependency of the test target that executes the unit tests through mocha and istanbul. After tests are run, that same tool verifies that coverage requirements are met.

The end result is that I use two commands to build and run this application:

$ npm run build && npm start

This application has a handful of dependencies but the main ones are express, sequelize, and TypeScript. I've also gotten pretty used to including bluebird, moment, lodash, and node-uuid by default in everything that do. For template rendering, I'm using dustjs, which I had never used before and find appealing.

The application code is split into three areas:

  • src/index.ts is where the express application is constructed and configured.
  • src/routes.ts contains the definition of the ApplicationController that does all of the request handling work.
  • src/storage.ts contains the storage manager, including the sequelize implementation.

The storage manager is a fairly simple interface that is used to define and interact with the Account and Address objects. For this small example application, I just have the two. An account is a registered site user (register, login, logout, etc) and they have one or more addresses as managed on the settings page. The relationship between accounts and addresses is one to many.

When using the Sequelize library in a TypeScript application, you need to know how the object interfaces are defined and relate to each other. The important thing to know is that for each application object (account, address, etc) there are 3 interfaces that need to be defined: An attribute definition interface, an instance interface, and a model interface.

export interface AccountAttribute {
    id?:string;
    name?:string;
    email?:string;
    password?:string;
}

export interface AccountInstance extends Sequelize.Instance<AccountAttribute>, AccountAttribute {
}

export interface AccountModel extends Sequelize.Model<AccountInstance, AccountAttribute> { }

In the above code block, the AccountAttribute is defined. That object has 4 managed fields including the id (a generated uuid), name, email, and password. That interface is then referenced by the AccountInstance and AccountModel interfaces. The instance interface is used to describe what an instantiated instance of an account object looks like and how it behaves. It includes all of the attributes of the attribute interface but also some sequelize specific methods like updateAttributes and save. The model interface is used to describe how, through sequelize, instances of objects are managed. The model interface includes methods like find and create.

When the sequelize implementation of the storage manager is created, within the constructor the define method is called which binds the schema definition and model interface to an instance of the model and it can be used.

this.Account = this.sequelize.define<AccountInstance, AccountAttribute>("Account", {
        "id": {
            "type": Sequelize.UUID,
            "allowNull": false,
            "primaryKey": true
        },
        "name": {
            "type": Sequelize.STRING(128),
            "allowNull": false
        },
        "email": {
            "type": Sequelize.STRING(128),
            "allowNull": false,
            "unique": true,
            "validate": {
                "isEmail": true
            }
        },
        "password": {
            "type": Sequelize.STRING(128),
            "allowNull": false
        }
    },
    {
        "tableName": "accounts",
        "timestamps": true,
        "createdAt": "created_at",
        "updatedAt": "updated_at",
    });

In the above code block, the private Account variable is set and the schema defined. In the third param I'm instructing sequelize to manage the created at and updated fields, but I'm specifying what the column names should be. Later in that same storage manager implementation, I reference those model instances like this:

register(name:string, email:string, rawPassword:string):Promise<any> {
    return this.sequelize.transaction((transaction:Sequelize.Transaction) => {
        let accountId = uuid.v4();
        return this.hashPassword(rawPassword)
            .then((password) => {
                return this.Account
                    .create({
                        id: accountId,
                        name: name,
                        email: email,
                        password: password
                    }, {transaction: transaction})
            });
    });
}

The rest of the application is pretty standard. The storage manager is created early and used in both creating the express application as well as the application handler. Open an issue on GitHub or message me on twitter @ngerakines if you've got any questions or comments.

Warrant Canaries

Wikipedia defines a warrant canary as:

... a method by which a communications service provider informs its users that the provider has not been served with a secret United States government subpoena.

Practically, this ends up being a file or web location that states something to the effect of, "As of date, we have not received a subpoena." The notice usually includes a disclosure stating that no warrants have been served to the entity or its employees and no searches or seizures have been performed on the entity or the employees of the entity's assets. It will also include a date as to when the notice was updated and may also include links to external websites with time-relevant information such as news articles, major headlines, tweets, etc.

The most important part of the warrant canary is the signature and signed content. All of the above information is cryptographically signed, and the public key made available to verify the signature. The act of signing the notice increases the difficulty in forging a warrant canary.

There are many cases where warrant canaries exist and are used by commercial and non-commercial entities. One of the oldest and well-known instances is the rsync.net warrant canary. Other examples include:

There is, however, speculation that warrant canaries have questionable legal ground or could be used as an effective way to indirectly communicate said legal action by a government agency or court. At this time, there have been no cases where warrant canaries have been upheld. For more information, see the EFF Warrant Canary FAQ.

Creating A Warrant Canary

Creating a warrant canary is a fairly simple process. It requires just a small amount of time to become familiar with tools like GPG. After creating a warrant canary notice, it can be published by anyone with access to your website.

Before You Begin

To begin, you will need to install GPG and create a signing key. Create the signing key by following the official GPG Getting Started guide. A key is only created once and will be used to update your canary in the future; it is crucial that the same key be used for subsequent canary updates.

Creating The Notice

As with previous examples, the notice should contain the disclosures that are most relevant to your needs as well as information and data that can sufficiently be determined as both accurate and time relevant. This often includes the current date, sports scores, weather information, etc. For example:

It is Friday, December 26th, 2014 at 4:50 pm EST.

To this date no warrants, searches or seizures of any kind have ever been performed on my assets or any assets belonging to members of my household.

Headlines from http://www.npr.org/sections/news/archive?date=12-31-2014
Body Of Catholic Priest Found In Southern Mexico
Businesses Buzz With Anticipation In Wake Of U.S.-Cuba Thaw
Military Policy Impedes Research On Traumatic Brain Injuries
In The Nation's Capital, A Signature Soup Stays On The Menu
Already Bleak Conditions Under ISIS Deteriorating Rapidly

Week 16 NFL Scores
Giants 37 Rams 27
Cols 7 Cowboys 42
Bills 24 Raiders 26
Seahawks 35 Cardinals 6

You can verify this document using the public key 953023D848C35059A2E2488833D43D854F96B2E4.

With your notice saved as warrant_canary.txt, sign it with your GPG key.

$ gpg --clearsign warrant_canary.txt

Running this command will create a file named warrant_canary.txt.asc.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

It is Friday, December 26th, 2014 at 4:50 pm EST.

To this date no warrants, searches or seizures of any kind have ever been performed on my assets or any assets belonging to members of my household.

Headlines from http://www.npr.org/sections/news/archive?date=12-31-2014
Body Of Catholic Priest Found In Southern Mexico
Businesses Buzz With Anticipation In Wake Of U.S.-Cuba Thaw
Military Policy Impedes Research On Traumatic Brain Injuries
In The Nation's Capital, A Signature Soup Stays On The Menu
Already Bleak Conditions Under ISIS Deteriorating Rapidly

Week 16 NFL Scores
Giants 37 Rams 27
Cols 7 Cowboys 42
Bills 24 Raiders 26
Seahawks 35 Cardinals 6

You can verify this document using the public key 953023D848C35059A2E2488833D43D854F96B2E4.
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - https://gpgtools.org

iQIcBAEBCAAGBQJUndq1AAoJEDPUPYVPlrLk2fYP/jGFb1vxR2sXEu5DzHJU9urd
Q8ia1srhm4UogchTuN6nGv39zlBgpT1H75xwLYYSyiEbjpV7CYPqwYOgZvv8xF5D
hMRGoHu2WE7RCllQr49cKyzro0m9TEWHUt8HLxlaV/Go58Q2i3TbiKo5z0QdlB7B
XXyQSA5ZDFSKqrdMl6oVqHI1dJhM3TRGpxmkrF/mD7RRpdqw0yJKMefqxGRFLavI
Vg8su3XlYgl6xmlL+BAcd0Pc0SiSCH/IIiLbpBrNaWeOFeEnaAbeC4apYn45np5G
jXPQ7+xdfcxmyt+VUSJ9aSw6WxHSYYBR2YhOvnunssCI6dev06Ot3p5+zOkgsFZt
2rqvNFKjp92J/vB8cKCoFi8UwizftcyvrwZHHtzFcLPEg4mhqWQp4DE3ToMOp37o
wieVqWbYhqRDMlFgQGr9Zdx0xPipnz5JwcSeaJuUZTOYUbN2L4w5s25yvCtuyT4p
yac0D+mxoFhG96UuSXsQjtwbiot7Kddt0TeaXzfbR7nk7n9Cv5thEEQlgtoV4Htv
f8jXua2/L3+Cl8j+WM+C9S5lXXR3t3RGy555lYcssDXAAcWsSY4UJasHaVU0vRTu
CqDPfOJmCnqI9Pv7tlP4iBWMkkAVV9ToqyRoM4fIQ41jTDn+ncc52du4M1+LZNJq
2tQPWQHVW8/oQtwo2W7W
=HQN5
-----END PGP SIGNATURE-----

That file is your current warrant canary and should be made available as you see fit. The most common url used to present your canary is "/canary". In this case, the canary is available at http://ngerakines.me/canary.

Next Steps

With your canary online and available, you'll need to be sure that the signing key used to sign the notice is also available. Please refer to the Exchanging Keys and Distributing Keys documentation to export your key to share with others and make available through a GPG key server.

It may also be in your interest to have third parties verify your key and identity. This allows other key owners to demonstrate trust. More information can be found on the GPG documentation: Validating other keys on your public keyring.

Securing Your Canary

When a canary is not updated or is removed, it means that several things may have happened.

The first is simply human error. For safety and security purposes, the act of signing a warrant canary is a manual process. That means that a human has to be at a computer and run the commands to create and sign the warrant canary notice. There are plenty of reasons from sickness to changing companies and even simply forgetfulness that could be the reason why a canary is not updated.

It could also mean that the the entity no longer wants to include the notice. A change in management or ownership could result in the canary is neglected or removed.

Lastly, it could mean that harm, detention or a lack of control is preventing the canary from being updated.

A watchdog or Dead man's switch can be used mitigate damage or loss of data or reputation.

Dead Man Switch: Revocation

Using a revocation certificate, a trusted third party can publicly revoke the key used to cryptographically sign canaries.

A core component of the warrant canary is the signature. The signature is used to determine that the contents of the canary have not been tampered with and provide a way to identify the owner of the signing key through the web of trust. When the GPG key used to sign the canary is created, a revocation certificate should be created along with it.

If you forget your passphrase or if your private key is compromised or lost, this revocation certificate may be published to notify others that the public key should no longer be used. -- The GNU Privacy Handbook

A revocation certificate can be securely given to a trusted third party responsible for publishing the revocation certificate under certain conditions.

Conditions could range from:

  • The warrant canary not being updated after a certain period of time.
  • Unusual behavior or contact with the company.
  • A cue or hint that it should be done so through information contained in a dead drop or press release.

Multiple Signers

As a way to reduce the risk of human error from raising false concern, multiple signers can sign a canary or multiple canaries can be published used. The most common way to do this would be to have two or more members of an organization create signatures of the canary and append it to the notice.

This can be done by creating one or more detached signatures along with the canary.

$ gpg --output canary.sig1 --detach-sig canary

When the above command is run, a file named canary.sig1 is created that contains a signature of the canary file. You can publish these additional signatures along-side the canary or append them to the bottom of the canary file.

IOT and Home automation, 10 months later

In December of 2013 I was given a SmartThings kit and that kicked off a home automation project. I didn't go all-out and try to automate all the things, but instead tackled a single area, specifically my home office. Nearly 10 months later, I hardly think about it but use it every single day.

  • If the office lights are on and I leave the house, the lights are automatically turned off.
  • When I enter the house from being away, turn on the lights in the office.
  • When I'm at home, but there isn't any movement in my office after 15 minutes, turn the lights off automatically.
  • When I'm home, there is movement in my office and the lights are off then turn the lights on.

This started with some z-wave light switches and the SmartThings kit. I use a motion/presence detector (battery powered) that detects movement in the office. The z-wave light switches are used to software control the lights and then my phone is connected to the SmartThings system to determine when I'm nearby.

That isn't the only system in place, but it is the one that I use the most. Additionally, I've got motion sensors in other parts of the house, including on the garage door, and presense fobs in the cars. I also have a nest installed as well as door locks that support z-wave.

So what is next? I'd like to get the rest of the light switches in the house replaced with z-wave switches and find a way to automate the garage door. Having an "away mode" able to turn off all of the lights, lock the doors and set the nest tempature would be nice too. Maybe someday.

Parts:

Clean Build Versions With DocOpt

With some recent Go projects, I've been using docopt.go for command line argument parsing. It greatly reduces the complexity of dealing with arguments and options. On it's own, arguments can be processed without much work:

package main

import (
    "github.com/docopt/docopt.go"
)

var (
    githash string = ""
)

func main() {
    usage := `Awesome

Usage: awesome [--help --version --config=<file>]
       awesome daemon [--help --version --config <file>]
       awesome thing [--verbose... ] <with> <more>...

Options:
  --help     Show this screen.
  --version  Show version.
  --verbose  Verbose
`

    arguments, _ := docopt.Parse(usage, nil, true, version(), false)
    // ... do something with arguments
}

func version() string {
    previewVersion := "1.0.0"
    if len(githash) > 0 {
        return previewVersion + "+" + githash
    }
    return previewVersion
}

What I've also been doing is using a small Makefile to add extra information to the version:

all:
    go build -ldflags "-X main.githash `git rev-parse --short HEAD`"

If the go build command is used, then the version given to docopt is just 1.0.0, but if the main.githash is set as it is in the Makefile, then the version ends up being something like "1.0.0+b74276b".

$ ./awesome --version
1.0.0+b74276b

You can take it one step further and have the version set as a var that can easily be updated with sed or awk. An example would look something like:

package main

var (
    AWESOME_VERSION = "1.0.0"
)

Using awk to update the version would look like this:

$ awk '/AWESOME_VERSION/ { sub($3, "\"2.0.0\""); print; next}1' version.go
package main

var (
    AWESOME_VERSION = "2.0.0"
)

Secure Email On OSX

There are a few really good articles out there on how to send and receive secure email using Thunderbird and GPG. This small guide will show you how you can use Mail app along with GPG Tools for the same result.

Install GPG Tools

First, head over to https://gpgtools.org/ and download the latest release. At the time of this article, the latest stable release is 2.1.

To start off on the right foot, before you install it, open up terminal and verify that you've downloaded a package that matches the following signature: ac7a636bfee1027d8f43a12a82eea54e7566dcb8. This can be accomplished with the following commands:

$ cd ~/Downloads
$ shasum GPG\ Suite\ -\ 2013.10.22.dmg
ac7a636bfee1027d8f43a12a82eea54e7566dcb8  GPG Suite - 2013.10.22.dmg

Once you can verify that the dmg file that you've downloaded hasn't been tampered with during transfer, go ahead and open it and run through the install process. This will install the base GPG tools, a graphical key manager and a plugin for Mail.

Create A Key

Next you'll want to create a key. Open the newly installed "GPG Keychain Access" application and click the "New" button to create a key. You'll be prompted for your full name and email address, which you should fill in. Be sure to also check the box to have the public key uploaded once generated. Having accurate information is vital and if this is your first time going through this process. I strongly recommend setting the comment to your website or twitter under the advanced options.

Next, you'll be prompted to set a password for your key. Choose a strong password. Depending on your system, it may take a few moments for the key to be generated after your password is accepted. Don't be alarmed.

Configure Mail

There shouldn't be anything extra needed to send and receive encrypted and or signed email through the Mail app now. In the Mail app preferences is a "GPGMail" section that should indicate that GPGMail is ready for use. It is set to encrypt/sign drafts and sign all new messages by default.

Test Sending Signed Mail

From Mail, create a new message to send to a loved one, friend, coworker or the like. Once you fill in the To, Subject, and Body, ensure that the message is signed by clicking the checkmark box button within the new mail window. If you have Mail configured to sign by default, you may be prompted within a few seconds to give the password for the key.

It is important to note that you can sign outbound email to anyone, but you can only encrypt email messages to people who have given you their public key. This is where the GPG Keychain Access app comes into play.

With the GPG Keychain Access app you can also import key files given to you and search for keys for people you may know. If someone sends you their public key you can use the "import" feature to load the key into your keyring. Alternatively, if you know the email address or name, you can attempt to search for keys associated with them on public key servers.

When composing emails to addresses that have public keys associated with them, you'll have the option of encrypting the email messages being sent. If you don't have any other public keys in your key ring, you can test this by sending an encrypted email to yourself.

Tips

Guard your private key. It is critical that you ensure your private key is safe and secure. For everyday use, keeping it on a personal, non-public computer is probably enough. If you feel that a computer that has your private key on it has been compromised, infected by a virus or malware, etc then you revoke the key and create a new one.

Find a thumbdrive that you don't use and back up your keyring to it. This should also include a revocation certificate. A revocation certificate will allow you to revoke the key if the key is lost or compromised.

When backing up your private key, consider using symmetric encryption using a password to encrypt the backup file. This can be done with GPG using the following command:

$ cd /Volumes/thumbdrive
$ gpg --output backup.zip.gpg --symmetric backup.zip

When you need to decyrpt your backup, you can use the following command:

$ gpg --output backup.zip -d backup.zip.gpg

When publishing your key on your blog or website, you can export a plain text version of your key that can be read as text and imported easily using the following command:

gpg --armor --export person@wherever.place

The output of that command can be placed inside of a pre block as-is. It is the most direct way to share your key wih someone viewing your blog or website. An alternative would be to create a small signature block telling people how to find your key.

$ gpg --fingerprint nick@gerakines.net
pub   4096R/4F96B2E4 2013-06-15
      Key fingerprint = 9530 23D8 48C3 5059 A2E2  4888 33D4 3D85 4F96 B2E4
...
$ gpg --clearsign

You need a passphrase to unlock the secret key for
user: "Nick Gerakines (http://ngerakines.me/) <nick@gerakines.net>"
4096-bit RSA key, ID 4F96B2E4, created 2013-06-15

9530 23D8 48C3 5059 A2E2  4888 33D4 3D85 4F96 B2E4
nick@gerakines.net

^D
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

9530 23D8 48C3 5059 A2E2  4888 33D4 3D85 4F96 B2E4
nick@gerakines.net

-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.22 (Darwin)
Comment: GPGTools - http://gpgtools.org

iQIcBAEBCAAGBQJTi8pOAAoJEDPUPYVPlrLkwZkP/3PxOQfNAlF0W5JVImPltVMr
9rqNK/9T07cU8qCugECX0U+CPsz5+fY9t6KuPb9XQv1SZT/s0Cdu0NoV83/zyTJe
VmCpnDwDYa1k8PsfiYHziM/BQ4N8HFlc/rNwsyfS+v9o2Pa2nEJA6OmU+jsVg25A
vyGfgH6fK/QeWRIlFIMfuh5b0+XSOA0E6/xTHFSNHdn3oYA4xjNsE6AajHekcYAS
l99uZZhqu+bnKLaCpxLHjZbTcjGuZcacIyTXNh20VcHtgZS0VvUWKyRvJ9PPZcwJ
oidbGTQkx5GJJJrXREoncHsh5uVt0SUJk/Cb2B43sICzTD1+5tENpK6kUnxlo2bi
O0rzEFSZRVme3GiDTZc5pV7DoWUS28EiJl6LLc7hU7d8lwsme69/3tV85mEdyDzJ
4OnFDQ39qIHfHhnswyumTAYnI/31GWrWfCl/UL3MOd4HKQhxsuQWi/zOWVAlvHJN
/lwIh3yiH5PGJsOUKs04XoOgNaZLC2A2vq9FUng+hi7WfGBzYkPc/RLgNxI9cU9H
dADC+Np4DRQ71YMSX9oYpUpybq6IdA68rrWbdjfDMc+ZQBDZz83zk7xRMLfws1ut
u2n6uzAVvYe/FjGjBaNXJ++yE8oIC38RDBG14nJDBK+cdqZpBP0Lxd+nGRB6VxcX
ZBOr7eKH1bpVjSbuOX1S
=OhTU
-----END PGP SIGNATURE-----