Greg's Blog

helping me remember what I figure out

New Year’s Resolution

| Comments

First post of 2015… Yay. It’s that time of the year for reflection and fresh starts. I guess most folks would have done that last month. I have been feeling quite sheepish about not having made any resolutions for the new year. However after having listened to a few Podcasts and read bunch of posts about what folks have done in 2014 on the re-commenced daily commute I feel even more sheepish… So I have pondered what I would like to get out of 2015 and do/achieve.

For starters I would like to be a tad more analytical/critical. I have realised that I just really consume content, without reflecting on it too deeply or applying it in earnest to see if it works for me. Kind of like consuming an entire box set of some TV series in one or a couple of sittings, to the point where all episodes just blend into one. To help me with that I want to put up least one blog post a week, and yes I realise it’s the second week of January and I am already behind.. The idea of course is to write more and get better at it, but I don’t just want to say share something I learned, but also demonstrate why it’s useful. Or should I have read something that provoked some thoughts share and discuss these. Well that’s the intention anyway…

I actually really, really want to make progress on some of my side projects, that have suffered from fits and starts over the years, right at the top of that list is making good progress on my book. Short aside for those that are reading this and following it’s progress, I have started work on the model and testing it.

I have also been tinkering with a few apps/games over the years. For some reason a few months back, I had been reminiscing about some old computer games from the Spectrum days that I used to play as a kid. One that sticks out was Football Manager. When you play it now, well let’s just say… Nostalgia… Regardless I thought it would be fun to try and build a clone, as I see the potential for some interesting challenges and applications of tools/technologies and services. I believe my friend Jack termed it as over engineered when I told him about all the things I wanted to try out as part of it.

The site itself also needs some love, it’s been two years, so it is time for a little refresh.

Let’s see how this all pans out.

Change of Tack

| Comments

Another one of those quick posts on state of play about the book project. First off while updates have been a little slow of late (summer, holidays, work, etc…) I have been busy-ish planning the next chapters of the book and hope to push some of these out by the end of the month.

I am also please to reveal that over at GitBook, some 130 people have been viewing the book, which is just awesome and roughly 129 more people than I had hoped for! I also see that one person would be willing to buy the book over at LeanPub.

One thing though is that I have had 0 feedback on the book and it’s content. I have pondered this for some time now and I have decided to change the book from free to paid. The reasoning being that paying customers might speak up some more about any issues or better yet things that they like! So starting today I am changing the book to paid on GitBook, starting at $5.00 for the first section. If you purchase it now you will of course get the updates/fixes and subsequent chapters as they are written. I have also published a copy over at LeanPub. You can still get a free version of the book over at my gihub repository, reading it that way won’t be nearly as enjoyable as using Gitbook’s reader or the many eBook options you can get with GitBook or LeanPub. Of course there always the blog posts of the chapters, however I do ask suggest that if you like the book and it’s content, that maybe buying it is not such a bad idea after all :)

As always please let me know your thoughts and feedback.

Development Guided by Tests

| Comments

Time for a sneak peak of some work in progress. Here we cover setting up Karma to run our unit tests, As always you can read the chapter here , as well as the full book.

Here’s also a list of chapters you can find as blog posts on the site:


Up until now we have been very much focused on setting up our build pipeline and writing a high level feature tests. And while I promised that it was time to write some code, we do have do a few more setup steps before we can get stuck in. To get confidence in our code we will be writing JavaScript modules using tests and we want those tests to run all the time (i.e. with each save). To that end we need to set up some more tasks to run those tests for us and add them to our our deployment process. Furthermore we want these test to run during our build process.

Setting up our unit test runner using karma

I have chosen Karma as our Unit test runner, if you are new to Karma I suggest you take a peak at some of the videos on the site. It comes with a variety of plugins and supports basically all of the popular unit test frameworks. As our testing framework we will use Jasmine.

Before going to far, let’s quickly create a few folders in the root of our project. src/js is where we will store all of our JavaScript source code, later on we will create a task to concatenate/minify and move it to our app folder:

-> tests
    -> unit
-> src
    ->js

TODO: this for now but really I want to do commonJS

As with all tasks, let’s create a new branch:

> git checkout -b test-runner

And then let’s install the package and add it to our package.json file:

> npm install karma --save-dev

Ok time to create our Karma configuration file, typically you would type in the root of your project:

> karma init karma.conf.js

This would guide you through the process of setting up your test runner, here’s how I answered the setup questions:

Which testing framework do you want to use ?
Press tab to list possible options. Enter to move to the next question.
> jasmine

Do you want to use Require.js ?
This will add Require.js plugin.
Press tab to list possible options. Enter to move to the next question.
> no

Do you want to capture any browsers automatically ?
Press tab to list possible options. Enter empty string to move to the next question.
> PhantomJS
>

What is the location of your source and test files ?
You can use glob patterns, eg. "js/*.js" or "test/**/*Spec.js".
Enter empty string to move to the next question.
> src/js/**/*.js
WARN [init]: There is no file matching this pattern.

> tests/unit/**/*.js
WARN [init]: There is no file matching this pattern.

>

Should any of the files included by the previous patterns be excluded ?
You can use glob patterns, eg. "**/*.swp".
Enter empty string to move to the next question.
>

Do you want Karma to watch all the files and run the tests on change ?
Press tab to list possible options.
> no

Config file generated at "/Users/writer/Projects/github/weatherly/karma.conf.js".

And here’s the corresponding configuration that was generated:

// Karma configuration
// Generated on Sun Jul 20 2014 16:18:54 GMT+0100 (BST)

module.exports = function (config) {
    config.set({

        // base path that will be used to resolve all patterns (eg. files, exclude)
        basePath: '',


        // frameworks to use
        // available frameworks: https://npmjs.org/browse/keyword/karma-adapter
        frameworks: ['jasmine'],


        // list of files / patterns to load in the browser
        files: [
            'src/js/**/*.js',
            'tests/unit/**/*.js'
        ],


        // list of files to exclude
        exclude: [
        ],


        // preprocess matching files before serving them to the browser
        // available preprocessors: https://npmjs.org/browse/keyword/karma-preprocessor
        preprocessors: {
        },


        // test results reporter to use
        // possible values: 'dots', 'progress'
        // available reporters: https://npmjs.org/browse/keyword/karma-reporter
        reporters: ['progress'],


        // web server port
        port: 9876,


        // enable / disable colors in the output (reporters and logs)
        colors: true,


        // level of logging
        // possible values: config.LOG_DISABLE || config.LOG_ERROR || config.LOG_WARN || config.LOG_INFO || config.LOG_DEBUG
        logLevel: config.LOG_INFO,


        // enable / disable watching file and executing tests whenever any file changes
        autoWatch: true,


        // start these browsers
        // available browser launchers: https://npmjs.org/browse/keyword/karma-launcher
        browsers: ['PhantomJS'],


        // Continuous Integration mode
        // if true, Karma captures browsers, runs the tests and exits
        singleRun: true
    });
};

Let’s take it for a spin:

> karma start
> INFO [karma]: Karma v0.12.17 server started at http://localhost:9876/
> INFO [launcher]: Starting browser PhantomJS
> WARN [watcher]: Pattern "/Users/writer/Projects/github/weatherly/tests/unit/**/*.js" does not match any file.
> INFO [PhantomJS 1.9.7 (Mac OS X)]: Connected on socket iqriF61DkEH0qp-sXlwR with id 10962078
> PhantomJS 1.9.7 (Mac OS X): Executed 0 of 0 ERROR (0.003 secs / 0 secs)

So we got an error, but that is because we have no tests. Let’s wrap this into a grunt task:

> npm install grunt-karma --save-dev

And update our Gruntfile to load the task:

module.exports = function (grunt) {
    grunt.initConfig({
        express: {
            test: {
                options: {
                    script: './server.js'
                }
            }
        },
        cucumberjs: {
            src: 'tests/e2e/features/',
            options: {
                steps: 'tests/e2e/steps/'
            }
        },
        karma: {
            unit: {
                configFile: 'karma.conf.js'
            }
        }
    });

    grunt.loadNpmTasks('grunt-express-server');
    grunt.loadNpmTasks('grunt-selenium-webdriver');
    grunt.loadNpmTasks('grunt-cucumber');
    grunt.loadNpmTasks('grunt-karma');

    grunt.registerTask('e2e', [
        'selenium_start',
        'express:test',
        'cucumberjs',
        'selenium_stop',
        'express:test:stop'
    ]);
};

Let’s try this out our new grunt task:

> grunt karma:unit

> Running "karma:unit" (karma) task
> INFO [karma]: Karma v0.12.17 server started at http://localhost:9876/
> INFO [launcher]: Starting browser PhantomJS
> WARN [watcher]: Pattern "/Users/gregstewart/Projects/github/weatherly/src/js/**/*.js" does not match any file.
> WARN [watcher]: Pattern "/Users/gregstewart/Projects/github/weatherly/tests/unit/**/*.js" does not match any file.
> INFO [PhantomJS 1.9.7 (Mac OS X)]: Connected on socket QO4qLCSO-4DZVO7eaRky with id 9893379
> PhantomJS 1.9.7 (Mac OS X): Executed 0 of 0 ERROR (0.003 secs / 0 secs)
> Warning: Task "karma:unit" failed. Use --force to continue.

> Aborted due to warnings.

Similar output, with the difference that our process terminated this time because of the warnings about no files macthing our pattern. We’ll fix this issue by writing our very first unit test!

Writing and running our first unit test

In the previous chapter we created a source folder and added a sample module, to confirm our build process for our JavaScript assets worked. Let’s go ahead and create one test file, as well as some of the folder structure for our project:

> mkdir tests/unit/
> mkdir tests/unit/model/
> touch tests/unit/model/TodaysWeather-spec.js

What we want to do know is validate our Karma configuration before we starting our real tests, so let’s add a sample test to our TodaysWeather-spec.js:

describe('Today \'s weather', function () {
    it('should return 2', function () {
        expect(1+1).toBe(2);
    });
});

We could try and run our Karma task again, but this would only result in an error, because we are using the CommonJS module approach and we would see an error stating that module is not defined, because our module under tests uses:

module.exports = TodaysWeather;

In order to fix this we need run our browserify task before our karma task, so let’s register a new task unit in our grunt file to handle this:

module.exports = function (grunt) {
    grunt.initConfig({
        express: {
            test: {
                options: {
                    script: './server.js'
                }
            }
        },
        cucumberjs: {
            src: 'tests/e2e/features/',
            options: {
                steps: 'tests/e2e/steps/'
            }
        },
        less: {
            production: {
                options: {
                    paths: ['app/css/'],
                    cleancss: true
                },
                files: {
                    'app/css/main.css': 'src/less/main.less'
                }
            }
        },
        copy: {
            fonts: {
                expand: true,
                src: ['bower_components/bootstrap/fonts/*'],
                dest: 'app/fonts/',
                filter: 'isFile',
                flatten: true
            }
        },
        bower: {
            install: {
                options: {
                    cleanTargetDir:false,
                    targetDir: './bower_components'
                }
            }
        },
        browserify: {
            dist: {
                files: {
                    'app/js/main.min.js': ['src/js/**/*.js']
                }
            },
            options: {
                transform: ['uglifyify']
            }
        },
        karma: {
            unit: {
                configFile: 'karma.conf.js'
            }
        }
    });

    grunt.loadNpmTasks('grunt-express-server');
    grunt.loadNpmTasks('grunt-selenium-webdriver');
    grunt.loadNpmTasks('grunt-cucumber');
    grunt.loadNpmTasks('grunt-contrib-less');
    grunt.loadNpmTasks('grunt-contrib-copy');
    grunt.loadNpmTasks('grunt-browserify');
    grunt.loadNpmTasks('grunt-bower-task');
    grunt.loadNpmTasks('grunt-karma');

    grunt.registerTask('generate', ['less:production', 'copy:fonts', 'browserify']);
    grunt.registerTask('build', ['bower:install', 'generate']);
    grunt.registerTask('unit', ['browserify', 'karma:unit']);

    grunt.registerTask('e2e', [
        'selenium_start',
        'express:test',
        'cucumberjs',
        'selenium_stop',
        'express:test:stop'
    ]);

    grunt.registerTask('heroku:production', 'build');
};

And modify our karma.conf.js to point to the built version of our JavaScript code by updating the files block to point to app/js/**/*.js instead of src/js/**/*.js.

// Karma configuration
// Generated on Sun Jul 20 2014 16:18:54 GMT+0100 (BST)

module.exports = function (config) {
    config.set({

        // base path that will be used to resolve all patterns (eg. files, exclude)
        basePath: '',


        // frameworks to use
        // available frameworks: https://npmjs.org/browse/keyword/karma-adapter
        frameworks: ['jasmine'],


        // list of files / patterns to load in the browser
        files: [
            'app/js/**/*.js',
            'tests/unit/**/*.js'
        ],


        // list of files to exclude
        exclude: [
        ],


        // preprocess matching files before serving them to the browser
        // available preprocessors: https://npmjs.org/browse/keyword/karma-preprocessor
        preprocessors: {
        },


        // test results reporter to use
        // possible values: 'dots', 'progress'
        // available reporters: https://npmjs.org/browse/keyword/karma-reporter
        reporters: ['progress'],


        // web server port
        port: 9876,


        // enable / disable colors in the output (reporters and logs)
        colors: true,


        // level of logging
        // possible values: config.LOG_DISABLE || config.LOG_ERROR || config.LOG_WARN || config.LOG_INFO || config.LOG_DEBUG
        logLevel: config.LOG_INFO,


        // enable / disable watching file and executing tests whenever any file changes
        autoWatch: true,


        // start these browsers
        // available browser launchers: https://npmjs.org/browse/keyword/karma-launcher
        browsers: ['PhantomJS'],


        // Continuous Integration mode
        // if true, Karma captures browsers, runs the tests and exits
        singleRun: true
    });
};

Now let’s our setup:

> grunt unit
> Running "browserify:dist" (browserify) task

> Running "karma:unit" (karma) task
> INFO [karma]: Karma v0.12.17 server started at http://localhost:9876/
> INFO [launcher]: Starting browser PhantomJS
> INFO [PhantomJS 1.9.7 (Mac OS X)]: Connected on socket 8eDrHt--bJFzVxSN36mN with id 14048651
> PhantomJS 1.9.7 (Mac OS X): Executed 1 of 1 SUCCESS (0.002 secs / 0.002 secs)

> Done, without errors.

Perfect!

Running our tests as part of the build

Now that we have our test runner set up’ let’s add it to our build process. This is going to require us to register a new task as we will need to do a few things:

  • build our assets
  • run our unit tests
  • run our end to end tests

Let’s go ahead and create a task called test in our Gruntfile and configure it to execute these tasks:

module.exports = function (grunt) {
    grunt.initConfig({
        express: {
            test: {
                options: {
                    script: './server.js'
                }
            }
        },
        cucumberjs: {
            src: 'tests/e2e/features/',
            options: {
                steps: 'tests/e2e/steps/'
            }
        },
        less: {
            production: {
                options: {
                    paths: ['app/css/'],
                    cleancss: true
                },
                files: {
                    'app/css/main.css': 'src/less/main.less'
                }
            }
        },
        copy: {
            fonts: {
                expand: true,
                src: ['bower_components/bootstrap/fonts/*'],
                dest: 'app/fonts/',
                filter: 'isFile',
                flatten: true
            }
        },
        bower: {
            install: {
                options: {
                    cleanTargetDir:false,
                    targetDir: './bower_components'
                }
            }
        },
        browserify: {
            dist: {
                files: {
                    'app/js/main.min.js': ['src/js/**/*.js']
                }
            },
            options: {
                transform: ['uglifyify']
            }
        },
        karma: {
            unit: {
                configFile: 'karma.conf.js'
            }
        }
    });

    grunt.loadNpmTasks('grunt-express-server');
    grunt.loadNpmTasks('grunt-selenium-webdriver');
    grunt.loadNpmTasks('grunt-cucumber');
    grunt.loadNpmTasks('grunt-contrib-less');
    grunt.loadNpmTasks('grunt-contrib-copy');
    grunt.loadNpmTasks('grunt-browserify');
    grunt.loadNpmTasks('grunt-bower-task');
    grunt.loadNpmTasks('grunt-karma');

    grunt.registerTask('generate', ['less:production', 'copy:fonts', 'browserify']);
    grunt.registerTask('build', ['bower:install', 'generate']);
    grunt.registerTask('unit', ['browserify', 'karma:unit']);

    grunt.registerTask('e2e', [
        'selenium_start',
        'express:test',
        'cucumberjs',
        'selenium_stop',
        'express:test:stop'
    ]);

    grunt.registerTask('test', ['build', 'karma:unit', 'e2e']);

    grunt.registerTask('heroku:production', 'build');
};

And let’s make sure everything runs as intended:

> grunt test
> Running "less:production" (less) task
> File app/css/main.css created: 131.45 kB → 108.43 kB

> Running "copy:fonts" (copy) task
> Copied 4 files

> Running "browserify:dist" (browserify) task

> Running "karma:unit" (karma) task
> INFO [karma]: Karma v0.12.17 server started at http://localhost:9876/
> INFO [launcher]: Starting browser PhantomJS
> INFO [PhantomJS 1.9.7 (Mac OS X)]: Connected on socket 1kikUD-UC4_Gd6Qh9T49 with id 53180162
> PhantomJS 1.9.7 (Mac OS X): Executed 1 of 1 SUCCESS (0.002 secs / 0.002 secs)

> Running "selenium_start" task
> seleniumrc webdriver ready on 127.0.0.1:4444

> Running "express:test" (express) task
> Starting background Express server
> Listening on port 3000

> Running "cucumberjs:src" (cucumberjs) task
> ...

> 1 scenario (1 passed)
> 3 steps (3 passed)

> Running "selenium_stop" task

> Running "express:test:stop" (express) task
> Stopping Express server

> Done, without errors.

If you recall we configured our build to execute grunt e2e, we need to update this now to execute grunt test. Log into to your Codeship dashboard and edit the test configuration:

Codeship dashboard with updating test configuration

Ready to give this is a spin?

> git status
> git add .
> git commit -m "Karma test configuration added and new build test task created"
> git checkout master
> git merge test-runner
> git push

If we keep an eye on our dashboard we should see a build kicked-off and test task being executed:

Codeship dashboard with updating test configuration


You can read the full book over at GitBook or LeanPub. Updated content for this chapter can be found on GitHub.

Continuous Delivery

| Comments

This is the chpater on setting our continuous delivery pipeline, as always you can read the chapter of the book here, as well as the full book.

Here’s also a list of chapters you can find as blog posts on the site:


In the previous part we wrote our first functional test (or feature test or end 2 end test) and automated the running using a set of Grunt tasks. Now we will put these tasks to good use and have our Continuous Integration server run the test with each commit to our remote repository. There are two parts two Continuous Delivery: Continuous Integration and Continuous Deployment. These two best practices were best defined in the blog post over at Treehouse, do read the article, but here’s the tl;rd:

Continuous Integration is the practice of testing each change done to your codebase automatically and as early as possible. But this paves the way for the more important process: Continuous Deployment.

Continuous Deployment follows your tests to push your changes to either a staging or production system. This makes sure a version of your code is always accessible.

In this ection we’ll focus on Continuous Integration. As always before starting we’ll create a dedicated branch for our work:

git checkout -b ci

Setting up our Continuous Integration environment using Codeship

In the what you will need section I suggested signing up for a few services, if you haven’t by now created an account with either Github and Codeship now is the time! Also if you haven’t already now is the time to connect your Githuib account with Codeship. You can do this by looking under your account settings for connected services:

Link your Github account to Codeship

To get started we need to create a new project:

Create a new project

This starts starts a three step process:

  1. Connect to your source code provider
  2. Choose your repository
  3. Setup test commands

The first step is easy, choose the Github option, then for step two choose the weatherly repository from the list.

If you hadn’t already signed up for Github and hadn’t pushed your changes to it, then the repository won’t be showing up in the list. Link your local repository and push all changes up before continuing.

Not it’s time to set up the third step, set up out test commands. From the drop down labelled Select your technology to prepopulate basic commands choose node.js.

Next we need to tackle the section: Modify your Setup Commands. The instructions tell us that it can use the Node.js version specified in our package.json file, given that we have not added this information previously let’s go ahead and do that now. If you are unsure of the version of Node.js simply type:

node --version

In my case the output was 0.10.28, below is my package.json file, look for the block labelled with engines:

{
    "name": "weatherly",
    "version": "0.0.0",
    "description": "Building a web app guided by tests",
    "main": "index.js",
    "engines" : {
        "node" : "~0.10.28"
    },
    "scripts": {
        "test": "grunt test"
    },
    "repository": {
        "type": "git",
        "url": "https://github.com/gregstewart/weatherly.git"
    },
    "author": "Greg Stewart",
    "license": "MIT",
    "bugs": {
        "url": "https://github.com/gregstewart/weatherly/issues"
    },
    "homepage": "https://github.com/gregstewart/weatherly",
    "dependencies": {
        "express": "^4.4.5"
    },
    "devDependencies": {
        "chai": "^1.9.1",
        "cucumber": "^0.4.0",
        "grunt": "^0.4.5",
        "grunt-cucumber": "^0.2.3",
        "grunt-express-server": "^0.4.17",
        "grunt-selenium-webdriver": "^0.2.420",
        "webdriverjs": "^1.7.1"
    }
}

With that added we can edit the set up commands to look as follows:

npm install
npm install grunt-cli

Now let’s edit the Modify your Test Commands section. In the previous chapter we created a set of tasks to run our tests and wrapped them in a grunt command grunt e2e. Let’s add this command to our configuration:

grunt e2e

That’s hit the big save button. Right now we are ready to push some changes to our repository. Luckily we have a configuration change ready to push!

git add package.json
git commit -m "Added node version to the configuration for CI"
git checkout master
git merge ci
git push

And with that go over to your codeship dashboard and if it all went well, then you should see something like this:

First CI run!

You have to admit that setting this up was a breeze. Now we are ready to configure our Continous Deployment to Heroku.

Setting up Continous Deployment to Heroku

Before we configure our CI server to to deploy our code to Heroku on a successful build, we’ll need to create a new app through our Heroku dashboard:

Heroku dashboard

And click on the Create a new app link and complete the dialogue box.

Creating a new app

The name weatherly was already taken so I left it blank to get one assigned, if you do this as well, just be sure to make a note of it as we’ll need it shortly. I choose Europe, well because I live in Europe, so feel free to choose what ever region makes sense to you.

Confirmation screen

Armed with this information let’s head back to our project on Codeship and let’s configure our deployment. From the project settings choose the Deployment tab and from the targets select Heroku. You will need your Heroku app name (see above) and your Heroku api key which you can find under your account settings under the Heroku dashboard:

Codeship settings for heroku deployment

We will be deploying from our master branch. Once you are happy with your settings click on the little green tick icon to save the information. Time to test our set up! We just need to make one little change to our app configuration which is handy because that will allow us to commit and a change and verify the whole process from start to finish. In the previous section we have configured our web server to listen on port 3000, well Heroku assigns a part dynamically, so we to account for that by editing our server.js file by adding process.env.PORT to our listen function:

var express = require('express');
var app = express();

app.use(express.static(__dirname + '/app'));

var server = app.listen(process.env.PORT || 3000, function() {
    console.log('Listening on port %d', server.address().port);
});

Now let’s commit the change:

git add server.js
git commit -m "Server configured to handle dynamic port allocation"
git push

If we check our build dashboard we should see a succesful build and deployment to our Heroku instance:

Successful build and deployment

The build process checks that we get a 200 response back and marks the build as successful, so let’s open up our browser to see the results of our work:

Weatherly running on Heroku

And there you are your Continuous Delivery pipeline has been created and in less than a minute we go from commit to production!

Recap

In this last section we:

  • configued our ci envinronment
  • it runs our feature test
  • created a Heroku app
  • configured our CI environment to deploy to that instance
  • modified our web server to handle dynamic port allocation

And now it’s time to write some code!


You can read the full book over at GitBook or LeanPub. Updated content for this chapter can be found on GitHub.

Writing Our First Functional Test

| Comments

Time for the chapter on writing our first functional test of the book, as always you can read the chapter of the book here, as well as the full book.

Here’s also a list of chapters you can find as blog posts on the site:


_TODO: What is a functional test?

I order to write our first functional test we needed a test page, which we built in the previous section. Now let’s set up a simple Node.js webserver. Throughout this section and in future you may find me refering to these kind of tests as Functional Tests, End-to-end, Feature tests or indeed User Journey tests. Confusing I know, I will try my best to keep it to feature and end-to-end tests, but be prepared when discussing this topic to come across the same variety of breadth of names for Functional tests.

Web server: Express

A good practice to follow while working with Git is to create a branch for each feature that you are working on, so let’s go ahead and create a new branch for this item of work.

git checkout -b web-server

Make sure you are in the root of our project and not in the app/ folder

Since we’ll be using Node.js we can use NPM to manage the dependencies. These dependencies are stored in a folder called node_modules. Since we don’t want to check any node modules/packages into our repository we added that folder to our .gitignore file in when we set up the project. If we don’t add those packages to our repository you may be wondering how our CI and Heroku instance will now how to run the app. To that end we’ll use a handy file called package.json. When we run NPM we can not only install dependencies, we can also add them to our package.json file and our target envinronments can read this file and install these packages for us.

Typing npm init give us a way to create our package.json, here’s what I answered when prompted:

This utility will walk you through creating a package.json file. It only covers the most common items, and tries to guess sane defaults.

See npm help json for definitive documentation on these fields and exactly what they do.

Use npm install <pkg> --save afterwards to install a package and save it as a dependency in the package.json file.

Press ^C at any time to quit.
name: (weatherly)
version: (0.0.0)
description: Building a web app guided by tests
entry point: (index.js)
test command: grunt test
git repository: (https://github.com/gregstewart/weatherly.git)
keywords:
author: Greg Stewart
license: (ISC) MIT
About to write to /Users/gregstewart/Projects/github/weatherly/package.json:

{
    "name": "weatherly",
    "version": "0.0.0",
    "description": "Building a web app guided by tests",
    "main": "index.js",
    "scripts": {
        "test": "grunt test"
    },
    "repository": {
        "type": "git",
        "url": "https://github.com/gregstewart/weatherly.git"
    },
    "author": "Greg Stewart",
    "license": "MIT",
    "bugs": {
        "url": "https://github.com/gregstewart/weatherly/issues"
    },
    "homepage": "https://github.com/gregstewart/weatherly"
}

Is this ok? (yes) yes

As you can see it autocompleted a bunch of information for you, such as the project name, version number and Git details. Let’s add that file to our repo before going any further:

git add package.json
git commit -m "Created package.json file"

Now let’s go ahead and install a web server module. We’ll just use express.

npm install express --save

By specifying --save the dependecy was added to our package.json file, if you open it up you should see the following toward the end of the file:

"dependencies": {
        "express": "^4.4.5"
}

Next create a new file called server.js in the root of our project and add the following content:

var express = require('express');
var app = express();

app.use(express.static(__dirname + '/app'));

var server = app.listen(3000, function() {
  console.log('Listening on port %d', server.address().port);
});

And to start our server type:

npm start

If you now open your browser and hit http://localhost:3000 you should once again see:

Rendered HTML hosted by our Connect server

The process that runs our server is not daemonised and will continue to run until we close the console or type ^C. Go ahead and kill the server and add those changes to our repository, merge these changes back into master and finally push to origin:

git add server.js
git add package.json
git commit -m "Installed Connect and created a very basic web server for our app"
git checkout master
git merge web-server
git push

Cucumber, WebDriver and Selenium

For our functional tests I have chosen Cucumber.js and WebDriver.js with Selenium. I chose this combination because I believe this will give you greater felxibility in the long wrong, especially if you plan on using different languagesin your toolchain. You can find Ruby, Java and .Net versions of Cucumber, WebDriver and Selenium.

Once again we’ll create a dedicated branch for this work:

git checkout -b functional-test

Selenium

Selenium uses Java, so you will need to make sure you have it installed.

We could install the binaries manually, but since I plan using Grunt to automate tasks around starting and stopping the server, we might as well use [grunt-selenium-webdriver] (https://www.npmjs.org/package/grunt-selenium-webdriver) module as this includes everything that we need, including the jar file for the Selenium Server.

npm install grunt-selenium-webdriver --save-dev

We use the --save-dev flag to indicate that we want to add this dependency to our package.json file, however only for development purposes. With that done let’s create a Grunt task to start the server (you can find more information on Grunt and tasks over at the official Grunt.js website). The first thing we’ll need is a Gruntfile.js, so add one to the root of your project and edit it to contain the following:

module.exports = function(grunt) {
    grunt.initConfig({
    });

    grunt.loadNpmTasks('grunt-selenium-webdriver');

    grunt.registerTask('e2e', [
        'selenium_start',
        'selenium_stop'
    ]);
};

Save the changes and at the command line type: grunt e2e and you should see something like this:

Running "selenium_start" task
seleniumrc webdriver ready on 127.0.0.1:4444

Running "selenium_stop" task

Done, without errors.

This told grunt to execite a task called e2e and confirms that the selenium server started properly at the following address 127.0.0.1:4444 and then was shutdown again (apparently it is not necessary to shutdown the server with a stop task).

Using Grunt to start and stop the server

Let’s also add a step to stop and start our web server when we are running our frunctional tests. To that end we’ll install another grunt module:

npm install grunt-express-server --save-dev

And we’ll edit our Grunt file so that it looks for our server.js and we can control the starting and stopping of our server:

module.exports = function(grunt) {
    grunt.initConfig({
        express: {
            test: {
                options: {
                    script: './server.js'
                }
            }
        }
    });

    grunt.loadNpmTasks('grunt-express-server');
    grunt.loadNpmTasks('grunt-selenium-webdriver');

    grunt.registerTask('e2e', [
        'selenium_start',
        'express:test',
        'selenium_stop',
        'express:test:stop'
    ]);
};

If you now run grunt e2e, you should see the following output:

Running "selenium_start" task
seleniumrc webdriver ready on 127.0.0.1:4444

Running "express:test" (express) task
Starting background Express server
Listening on port 3000

Running "selenium_stop" task

Running "express:test:stop" (express) task

WebDriver

The next thing we need to do is install WebDriver.js and we are then nearly ready to write our first feature test:

npm install webdriverjs --save-dev

Cucumber

The final piece of the puzzle is Cucumber.js:

npm install cucumber --save-dev

Our first test

Features are written using the Gherkin syntax, and this is what our first feature looks like:

Feature: Using our awesome weather app
    As a user of weatherly
    I should be able to see the weather information for my location

    Scenario: Viewing the homepage
        Given I am on the home page
        When I view the main content area
        Then I should see the temperature for my location

I like to store these and the associated code in a e2e directory under a parent tests folder. So go ahead and create that folder structure under the root of our project. Then create a features folder and save the above feature contents to a file called using-weatherly.feature.

If we were to run our cucumber tests now using cucumber.js tests/e2e/features/using-weatherly.feature we would see the following output:

UUU

1 scenario (1 undefined)
3 steps (3 undefined)

You can implement step definitions for undefined steps with these snippets:

this.Given(/^I am on the home page$/, function (callback) {
    // express the regexp above with the code you wish you had
    callback.pending();
});

this.When(/^I view the main content area$/, function (callback) {
    // express the regexp above with the code you wish you had
    callback.pending();
});

this.Then(/^I should see the temperature for my location$/, function (callback) {
    // express the regexp above with the code you wish you had
    callback.pending();
});

This is extremely useful output. While it’s clear that the code to execute the steps in the feature are undefined, the output actually gives snippets to create our step definitions. So let’s go ahead and create our step definition. Inside of our functional test folder, create a steps folder and add a file called using-weather-steps.js with the following content:

var UsingWeatherlyStepDefinitions = function () {

    this.Given(/^I am on the home page$/, function (callback) {
        // express the regexp above with the code you wish you had
        callback.pending();
    });

    this.When(/^I view the main content area$/, function (callback) {
        // express the regexp above with the code you wish you had
        callback.pending();
    });

    this.Then(/^I should see the temperature for my location$/, function (callback) {
        // express the regexp above with the code you wish you had
        callback.pending();
    });
};

module.exports = UsingWeatherlyStepDefinitions;

Let’s try and execute our feature test again with cucumber.js tests/e2e/features/using-weatherly.feature --require tests/e2e/steps/using-weatherly-step-definitions.js and now we should see:

P--

1 scenario (1 pending)
3 steps (1 pending, 2 skipped)

Time to flesh out the steps to do some work and check for elements on the page while the tests are running. We’ll make use of Chai.js as our assertion library, so let’s go ahead and install this module:

npm install chai --save-dev

The first bit of code we’ll add to our tests is a World object, which willl initialise our browser (read WebDriver) and add a few helper methods (visit and hasText). As our browser we are using phantomjs, but if you would like to see the test running simply replace browserName: 'phantomjs' with say browserName: 'firefox'.

Note that other browsers such as Chrome and IE require special drivers which you can download from the Selenium website

Here’s our world object (world.js), which we save into a folder called support under tests/e2e:

var webdriverjs = require('webdriverjs');
var expect = require('chai').expect;
var assert = require('chai').assert;

var client = webdriverjs.remote({ desiredCapabilities: {browserName: 'phantomjs'}, logLevel: 'silent' });

client.addCommand('hasText', function (selector, text, callback) {
    this.getText(selector, function (error, result) {
        expect(result).to.have.string(text);
        callback();
    });
});

client.init();


var World = function World(callback) {
    this.browser = client;

    this.visit = function(url, callback) {
        this.browser.url(url, callback);
    };

    callback(); // tell Cucumber we're finished and to use 'this' as the world instance
};

exports.World = World;

Please note that if when running the tests you come across the message shown below (logging in verbose mode here), this simply (indeed simply…) means that webdriverjs cannot find phatomjs in your PATH.

====================================================================================
Selenium 2.0/webdriver protocol bindings implementation with helper commands in nodejs.
For a complete list of commands, visit http://webdriver.io/docs.html.

====================================================================================

[09:51:45]:  ERROR      Couldn't get a session ID - undefined
Fatal error: [init()] <=
An unknown server-side error occurred while processing the command.

Now let’s re-visit our using-weatherly-step-definitions.js and replace the contents with the following code:

var UsingWeatherlyStepDefinitions = function () {
    this.World = require("../support/world.js").World;

    this.Given(/^I am on the home page$/, function (callback) {
        this.visit('http://localhost:3000/', callback);
    });

    this.When(/^I view the main content area$/, function (callback) {
        this.browser.hasText('.jumbotron h1', 'London Right Now', callback);
    });

    this.Then(/^I should see the temperature for my location$/, function (callback) {
        this.browser.hasText('p.temperature', '14 degrees', callback);
    });
};

module.exports = UsingWeatherlyStepDefinitions;

The first step opens the site, and then we assert that the header element displays London Right Now and that the element with our temperature shows 14 degrees

If we were to once again try and execute our feature test, we would get an error telling us that it can’t connect to the selenium server. So let’s wrap all of this into our e2e grunt task. Let’s start by adding another module to our setup:

npm install grunt-cucumber --save-dev

And let’s edit our Gruntfile.js to look like this now:

module.exports = function(grunt) {
    grunt.initConfig({
        express: {
            test: {
                options: {
                    script: './server.js'
                }
            }
        },
        cucumberjs: {
            src: 'tests/e2e/features/',
            options: {
                steps: 'tests/e2e/steps/'
            }
        }
    });

    grunt.loadNpmTasks('grunt-express-server');
    grunt.loadNpmTasks('grunt-selenium-webdriver');
    grunt.loadNpmTasks('grunt-cucumber');

    grunt.registerTask('e2e', [
        'selenium_start',
        'express:test',
        'cucumberjs',
        'selenium_stop',
        'express:test:stop'
    ]);
};

Now type grunt e2e and you should see the following output:

Running "selenium_start" task
seleniumrc webdriver ready on 127.0.0.1:4444

Running "express:test" (express) task
Starting background Express server
Listening on port 3000

Running "cucumberjs:src" (cucumberjs) task
...

1 scenario (1 passed)
3 steps (3 passed)

Running "selenium_stop" task

Running "express:test:stop" (express) task
Stopping Express server

Done, without errors.

With that done we can commit our changes to our repository:

git add .
git commit -m "Scenario: Viewing the homepage, created and implemented"
git checkout master
git merge functional-test
git push

Recap

To sum things up in this section we created a set of grunt tasks that:

  • start our selenium server
  • start our express server that hosts our page
  • execute the features and steps we defined with cucumberjs
  • output the result to the console
  • closes down the services after finishing the tests

We also wrote a feature test that:

  • open a browser
  • check the contents for a header
  • check for an element that holds the current temperature

You can read the full book over at GitBook or LeanPub. Updated content for this chapter can be found on GitHub.

Getting Started

| Comments

Here’s the getting started chapter of the book, you can read the chapter of the book here, as well as the full book.

Here’s also a list of chapters you can find as blog posts on the site:


The first thing I like to do with any project is to get our build pipeline set up and start deploying code to our ‘production’ environment. To that end we need to look at building the simplest thing possible to validate our testing infrastructure works, our CI envinronment can pick up changes on commit and after a succesful build deploy the changes.

I will assume you have signed up and installed all of the software outlined in the What you will need section.

Setting up our project

Open up a terminal window or code to the command line and navigate to the location you want to store your project files. Once there let’s create a project folder:

mkdir weatherly && cd weatherly

And initilise our github repository:

git init
Initialized empty Git repository in /Users/gregstewart/Projects/github/weatherly/.git/

Before we go any further let’s create a .gitignore file in the root of our project and add the following to it:

node_modules
.idea

Let’s commit this change quickly:

git add .gitignore
git commit -m "Adding folders to the ignore list"

From a folder perspective I like to create a distribution folder and an app folder to hold the source, so let’s go ahead and add these folders as well.

mkdir app
mkdir dist

We’ll start by using bower to grab some of our front end dependencies. Let’s start by creating a bower.json file by typing: bower init, fill in the details as you see fit, but here’s what I selected:

{
    name: 'weatherly',
    version: '0.0.0',
    authors: [
        'Greg Stewart <gregs@tcias.co.uk>'
    ],
    description: 'Building a web app guided by tests',
    moduleType: [
        'amd'
    ],
    license: 'MIT',
    homepage: 'http://www.tcias.co.uk/',
    private: true,
    ignore: [
        '**/.*',
        'node_modules',
        'bower_components',
        'test',
        'tests'
    ]
}

Everybody likes a bit of Bootstrap so let’s start with that package :

bower install bootstrap --save

The --save flag at the end of the command means that the dependecy will be added to our bower.jon file. We are doing this because we do not want to check in any external dependencies into our repository. Instead at build/CI time we’ll restore these using bower.

So let’s edit our .gitignore file to make sure we don’t accidentally commit these files:

node_modules
.idea
bower_components

And let’s add this change to our repo:

git add .gitignore
git commit -m "Adding bower_components to the ignore list"

To round things off let’s install HTML5 boilerplate

bower install html5-boilerplate

You may have noticed that I decided not to add this package to our bower.json file, simply because we’ll copy the files we need into our app folder:

mv bower_components/html5-boilerplate/css app/
mv bower_components/html5-boilerplate/img app/
mv bower_components/html5-boilerplate/*.html app/
mv bower_components/html5-boilerplate/*.png app/
mv bower_components/html5-boilerplate/*.xml app/
mv bower_components/html5-boilerplate/*.ico app/
mv bower_components/html5-boilerplate/*.txt app/

In your favourite editor open up the app/index.html file and add the following:

<!DOCTYPE html>
<!--[if lt IE 7]>      <html class="no-js lt-ie9 lt-ie8 lt-ie7"> <![endif]-->
<!--[if IE 7]>         <html class="no-js lt-ie9 lt-ie8"> <![endif]-->
<!--[if IE 8]>         <html class="no-js lt-ie9"> <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js"> <!--<![endif]-->
    <head>
        <meta charset="utf-8">
        <meta http-equiv="X-UA-Compatible" content="IE=edge">
        <title>Weatherly - Forecast for London</title>
        <meta name="description" content="">
        <meta name="viewport" content="width=device-width, initial-scale=1">

        <!-- Place favicon.ico and apple-touch-icon.png in the root directory -->

        <link rel="stylesheet" href="css/main.css">
    </head>
    <body>
        <!--[if lt IE 7]>
            <p class="browsehappy">You are using an <strong>outdated</strong> browser. Please <a href="http://browsehappy.com/">upgrade your browser</a> to improve your experience.</p>
        <![endif]-->

        <!-- Add your site or application content here -->
        <div class="container">
            <div class="header">
                <ul class="nav nav-pills pull-right">
                    <li class="active"><a href="#">Home</a></li>
                    <li><a href="#">About</a></li>
                    <li><a href="#">Contact</a></li>
                </ul>
                <h3 class="text-muted">test</h3>
            </div>

            <div class="jumbotron">
                <h1>London Right Now</h1>
                <p class="temperature">14 degrees</p>
                <p>Mostly cloudy - feels like 14 degrees</p>
            </div>

            <div class="row marketing">
                <div class="col-lg-6">
                    <h4>NEXT HOUR</h4>
                    <p>Mostly cloudy for the hour.</p>

                    <h4>NEXT 24 HOURS</h4>
                    <p>Mostly cloudy until tomorrow afternoon.</p>
                </div>
            </div>

            <div class="footer">
                <p><span class="glyphicon glyphicon-heart"></span> from Weatherly</p>
            </div>

        </div>
        <p></p>

        <script src="//ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
        <script>window.jQuery || document.write('<script src="js/vendor/jquery-1.10.2.min.js"><\/script>')</script>

        <!-- Google Analytics: change UA-XXXXX-X to be your site's ID. -->
        <script>
            (function(b,o,i,l,e,r){b.GoogleAnalyticsObject=l;b[l]||(b[l]=
                 function(){(b[l].q=b[l].q||[]).push(arguments)});b[l].l=+new Date;
                e=o.createElement(i);r=o.getElementsByTagName(i)[0];
                e.src='//www.google-analytics.com/analytics.js';
                r.parentNode.insertBefore(e,r)}(window,document,'script','ga'));
            ga('create','UA-XXXXX-X');ga('send','pageview');
        </script>
    </body>
</html>

If you open up the file in your browser you should see something like this hopefully:

Rendered HTML

Not exactly something to write home about, but it’s enough for us to get started setting up our little server, writing a functional test and deploying something to our Heroku instance.

We’ll make this a lot prettier later on in the book when we deal with setting up Grunt to build our JavaScript and CSS assets.

The last thing we’ll do is commit all of our changes to our local repos:

git add .
git commit -m "Added Bootstrap/Modernizr to bower.json, moved the skeleton of the HTML5 boilerplate to the app folder and created a base index page for our weather forecat app."

At this stage it’s a good idea to also push the changes to our remote repository. If you have followed the What you will need section, you will hopefully have created a Github account. If not go ahead and to that now. Then create a repository called weatherly, here’s what I entered:

Creating your weatherly repository

To push our changes to the remote repository, you will need to tell your local repo where it is:

git remote add origin https://github.com/<account_name>/weatherly.git

Now you can push your changes:

git push -u origin master

Recap

Before we move on let’s just quickly recap what we have done so far:

  • created our app folder structure
  • initialised our git repo
  • created a Git ignore file
  • used bower to manage some of our front end dependencies:
    • Bootstrap
    • Modernizr
    • HTML5 boilerplate
  • created a very basic index.html page
  • pushed all of the changes to our remote git repository

You can read the full book over at GitBook or LeanPub. Updated content for this chapter can be found on GitHub.

Book Introduction

| Comments

Just a quick post to share the initial draft to the introductory chapter to my book, which covers the pre-requisites, set up steps and accounts yo will need to sign up for in order to follow along.

By the way you can read the chapter of the book here (well work in progress), as well as the full book.

Here’s also a list of chapters you can find as blog posts on the site:


Introduction

The idea for this book started with a series of blog posts I wrote some time ago exploring building a Backbone.js app using tests. I initially wanted to just consolidate these blog posts into a book, however once I started jotting down what I wanted to put into this series, I decided there was more to write about.

In the Front end and JavaScript world we have come a long way since the heady days of table based layouts sprinkled with Macromedia roll over scripts to make web pages interactive. The lines between back end and front end application code has almost blurred completely, you could argue it no longer exists when you consider Isomorpic apps. Our tooling has changed substantially as well: at our disposal we have package managers (for both back and front end), build tools, unit test frameworks and deployment tools.

What we will be building

Over the coming pages and posts we will explore how to build a modern JavaScript web app guided by tests, using a toolset that will allow you to deploy with each commit to the cloud. We’ll be building a small JavaScript weather app using forecast.io’s API. The app itself will use the browser’s Geolocation API to figure out where your user is and retrieve a weekly weather forecast for that location.

How we will build it

Plain and simple, we’ll build this app guided by tests and using a continuous delivery model, by having it deployed using Codeship’s CD service to a Heroku instance.

The focus of this book is really about setting you on the right path to delivering quality software reliably and continuously. By the end you should have the confidence to push every commit to ‘production’.

We will be making extensive use of the following tools: * Node.js * Grunt * Karma * Jasmine * Cucumber * Selenium

What you will need

There are few pre-requisits you will need to get this app built. You will need to sign up for some services, grab a code/etxt editor, set up a version control system and finally get your Node.js environment configured.

Services you wil need to sign up for

As I mentioned for our weather forecast API, we’ll be using forecast.io, so you might want to go and sign up for a developer account as you will need a key to access the API.

You should also sign up for a Github or Bitbucket account if you don’t already have one, we’ll need this for our CI service.

So that we can reliably deploy our app, we’ll make use of Codeship’s hosted Continuous Integration service. Sign up for the [free service] (https://www.codeship.io/pricing) to get started.

To host our app we’ll make use of Heroku cloud computing service. They also offer a free service to help you get started.

That should cover the things you need to sign up for.

Code editor

You will need a decent IDE (I recommend WebStorm) or Text Editor (Sublime is very popular with many of my co-workers)

Version control: Git

I recommend using Version Control for every project, regardless of size or complexity. If don’t already have Git installed you should do so. There are many ways to install the necessary binaries and the Git website has all the necessary links. If you are on a Mac though, then I would recommend using Homebrew to install the binaries.

If you are new to Git then I recommend taking the Git Immersion guided tour.

Node.js and NPM

We’ll be making extensive use of JavaScript throughout the book, so you will need to install Node.js and NPM. Once again if you are on a Mac though, then I would recommend using Homebrew to install the binaries.

NPM will allow us to resolve all of the dependencies we need in order to achieve our goal of building and delivering a web app guided by tests.

Bower

Bower is handy tool to manage your front end library dependencies, so I recommed installing it as well.

Grunt

Grunt will be our build and automation tool. If you haven’t used Grunt before, be sure to check out the Getting Started guide, as it explains how to create a Gruntfile as well as install and use Grunt plugins.

With all that installed and configured, it’s time to get started!


You can read the full book over at GitBook or LeanPub. Updated content for this chapter can be found on GitHub.

Building Your Assets Using Grunt

| Comments

So last time I posted I gave a quick update on the book. This time is no different, I just wanted to share that I finished drafting the chapter on building less and JavaScript assets as part of the deployment process. What follows is the chapter, hope you find it useful.

By the way you can read the chapter here.

Here’s also a list of chapters you can find as blog posts on the site:


Building our assets

Now that we have a pipeline up and running it’s time to turn our attention to dealing with building our code and assets and figure out how are our app will consume these. We do not want to check in our generated assets, however Heroku deploys using a git mechanism.

Let’s start by creating some tasks to generate our css, then concatenate and ulgyfy our JS and we’ll finish by deploying these assets to Heroku as part of a successful build. We’ll also add some tasks to run these tasks on our local machine and have these re-generated when we save changes to the file.

git checkout -c generate-assets

Compile our less to css

Let’s tackle with our CSS files. The first thing we want to do is add the destination folder for our css content to the .gitignore list:

node_modules
.idea
bower_components
phantomjsdriver.log
app/css/

Under our source folder let’s create a less folder and create a main.less file in it. Here’s content of the file:

@import '../../bower_components/bootstrap/less/bootstrap';

Our build tasks will take this import directive and create us a nice main.css file that we can then use in our app.

npm install grunt-contrib-less --save-dev

And now let’s create a task to generate our css files by adding to our Gruntfile:

module.exports = function (grunt) {
    grunt.initConfig({
        express: {
            test: {
                options: {
                    script: './server.js'
                }
            }
        },
        cucumberjs: {
            src: 'tests/e2e/features/',
            options: {
                steps: 'tests/e2e/steps/'
            }
        },
        less: {
            production: {
                options: {
                    paths: ['app/css/'],
                    cleancss: true
                },
                files: {
                    'app/css/main.css': 'src/less/main.less'
                }
            }
        }
    });

    grunt.loadNpmTasks('grunt-express-server');
    grunt.loadNpmTasks('grunt-selenium-webdriver');
    grunt.loadNpmTasks('grunt-cucumber');
    grunt.loadNpmTasks('grunt-contrib-less');

    grunt.registerTask('generate', ['less:production']);

    grunt.registerTask('e2e', [
        'selenium_start',
        'express:test',
        'cucumberjs',
        'selenium_stop',
        'express:test:stop'
    ]);

};

When you run grunt generate, you should see the following output:

Running "less:production" (less) task
File app/css/main.css created: 131.45 kB → 108.43 kB

If you were to start up our server and browse to localhost:3000, our UI should have more of a Bootstrap feel to it!

Rendered HTML with generated css Now bootstrap also needs some fonts, so let’s move these across as part of the build.

npm install grunt-contrib-copy --save-dev

And add a simple task to copy our fonts across as well:

module.exports = function (grunt) {
    grunt.initConfig({
        express: {
            test: {
                options: {
                    script: './server.js'
                }
            }
        },
        cucumberjs: {
            src: 'tests/e2e/features/',
            options: {
                steps: 'tests/e2e/steps/'
            }
        },
        less: {
            production: {
                options: {
                    paths: ['app/css/'],
                    cleancss: true
                },
                files: {
                    'app/css/main.css': 'src/less/main.less'
                }
            }
        },
        copy: {
            fonts: {
                expand: true,
                src: ['bower_components/bootstrap-sass-official/vendor/assets/fonts/bootstrap/*'],
                dest: 'app/css/bootstrap/',
                filter: 'isFile',
                flatten: true
            }
        }
    });

    grunt.loadNpmTasks('grunt-express-server');
    grunt.loadNpmTasks('grunt-selenium-webdriver');
    grunt.loadNpmTasks('grunt-cucumber');
    grunt.loadNpmTasks('grunt-contrib-less');
    grunt.loadNpmTasks('grunt-contrib-copy');

    grunt.registerTask('generate', ['less:production', 'copy:fonts']);

    grunt.registerTask('e2e', [
        'selenium_start',
        'express:test',
        'cucumberjs',
        'selenium_stop',
        'express:test:stop'
    ]);

};

Running grunt generate task should now also copy our fonts across.

Fonts now included

This is great, but how do we get this to run as part of our successful build?

Heroku build packs

Heroku has a way to run commands after a build, these come in the form of build packs. Luckily for us someone has already gone through the effort of creating one to run Grunt after an install.

I have to say it’s not ideal, however given Heroku’s git based deployment approach, we have little choice but to generate these as part of the deployement. Typicall you would rely on the build to generate a package with all of the generated assets ready for consumption. This works though and does not force us to commit our generated assets into our repository.

So here’s how you go about installing our Build Pack for Grunt (be sure to replace --app lit-meadow-5649 with your actual heroku app name):

heroku login

heroku config:add BUILDPACK_URL=https://github.com/mbuchetics/heroku-buildpack-nodejs-grunt.git --app lit-meadow-5649

heroku config:set NODE_ENV=production --app lit-meadow-5649

Then we modify our Gruntfile to include a new heroku:production task, which basically references our build task:

module.exports = function (grunt) {
    grunt.initConfig({
        express: {
            test: {
                options: {
                    script: './server.js'
                }
            }
        },
        cucumberjs: {
            src: 'tests/e2e/features/',
            options: {
                steps: 'tests/e2e/steps/'
            }
        },
        less: {
            production: {
                options: {
                    paths: ['app/css/'],
                    cleancss: true
                },
                files: {
                    'app/css/main.css': 'src/less/main.less'
                }
            }
        },
        copy: {
            fonts: {
                expand: true,
                src: ['bower_components/bootstrap-sass-official/vendor/assets/fonts/bootstrap/*'],
                dest: 'app/css/bootstrap/',
                filter: 'isFile',
                flatten: true
            }
        }
    });

    grunt.loadNpmTasks('grunt-express-server');
    grunt.loadNpmTasks('grunt-selenium-webdriver');
    grunt.loadNpmTasks('grunt-cucumber');
    grunt.loadNpmTasks('grunt-contrib-less');
    grunt.loadNpmTasks('grunt-contrib-copy');

    grunt.registerTask('generate', ['less:production', 'copy:fonts']);

    grunt.registerTask('e2e', [
        'selenium_start',
        'express:test',
        'cucumberjs',
        'selenium_stop',
        'express:test:stop'
    ]);

    grunt.registerTask('unit', [
        'karma:unit'
    ]);

    grunt.registerTask('heroku:production', 'generate');
};

The final step involves re-jigging package.json to include those newly added grunt tasks as a general dependency:

    {
    "name": "weatherly",
    "version": "0.0.0",
    "description": "Building a web app guided by tests",
    "main": "index.js",
    "engines": {
        "node": "~0.10.28"
    },
    "scripts": {
        "test": "grunt test"
    },
    "repository": {
        "type": "git",
        "url": "https://github.com/gregstewart/weatherly.git"
    },
    "author": "Greg Stewart",
    "license": "MIT",
    "bugs": {
        "url": "https://github.com/gregstewart/weatherly/issues"
    },
    "homepage": "https://github.com/gregstewart/weatherly",
    "dependencies": {
        "express": "^4.4.5",
        "grunt-contrib-copy": "^0.5.0",
        "grunt-contrib-less": "^0.11.3"
    },
    "devDependencies": {
        "chai": "^1.9.1",
        "cucumber": "^0.4.0",
        "grunt": "^0.4.5",
        "grunt-contrib-copy": "^0.5.0",
        "grunt-contrib-less": "^0.11.3",
        "grunt-cucumber": "^0.2.3",
        "grunt-express-server": "^0.4.17",
        "grunt-selenium-webdriver": "^0.2.420",
        "webdriverjs": "^1.7.1"
    }
}

This is another thing about this approach that I am not a fan of, having to move what are essentially development dependencies into our production dependencies.

Now we are nearly ready to test this out, however there is one more task we need to add. Since we are using Bower for some of our front end components and we haven’t checked these into our repository, we’ll need to restore them from our bower.json file. Let’s first install a new grunt package to assist us:

npm install grunt-bower-task --save

And the edit our Gruntfile.js:

module.exports = function (grunt) {
    grunt.initConfig({
        express: {
            test: {
                options: {
                    script: './server.js'
                }
            }
        },
        cucumberjs: {
            src: 'tests/e2e/features/',
            options: {
                steps: 'tests/e2e/steps/'
            }
        },
        less: {
            production: {
                options: {
                    paths: ['app/css/'],
                    cleancss: true
                },
                files: {
                    'app/css/main.css': 'src/less/main.less'
                }
            }
        },
        copy: {
            fonts: {
                expand: true,
                src: ['bower_components/bootstrap-sass-official/vendor/assets/fonts/bootstrap/*'],
                dest: 'app/css/bootstrap/',
                filter: 'isFile',
                flatten: true
            }
        },
        bower: {
            install: {
                options: {
                    cleanTargetDir:false,
                    targetDir: './bower_components'
                }
            }
        }
    });

    grunt.loadNpmTasks('grunt-express-server');
    grunt.loadNpmTasks('grunt-selenium-webdriver');
    grunt.loadNpmTasks('grunt-cucumber');
    grunt.loadNpmTasks('grunt-contrib-less');
    grunt.loadNpmTasks('grunt-contrib-copy');
    grunt.loadNpmTasks('grunt-bower-task');

    grunt.registerTask('generate', ['less:production', 'copy:fonts']);
    grunt.registerTask('build', ['bower:install', 'generate']);

    grunt.registerTask('e2e', [
        'selenium_start',
        'express:test',
        'cucumberjs',
        'selenium_stop',
        'express:test:stop'
    ]);

    grunt.registerTask('heroku:production', 'build');
};

With that let’s push these changes and see if we can’t have a more nicely styled page appear on our Heroku app!

git add .
git commit -m "Generate less as part of the build and copy fonts to app folder"
git checkout master
git merge code-build
git push origin master

Concatenate and minify our JavaScript

Having generated our CSS at build time, it’s time to turn our attention to concatenating and minifying our JavaScript.

If you recall in our getting started section we set up our project and used Bower to manage our front end dependencies. For our code we will be Browserify and adopting a CommonJS approach to dealing with modules and dependencies.

To get started let’s first create a our source directory for our JavaScript, we’ll store our source under src/js and let’s create a file to test our build process called TodaysWeather.js and let’s save it under a sub folder called models:

var TodaysWeather = function () {
    console.log('test');
};

module.exports = TodaysWeather;

With that done let’s install a grunt task for Browserify

npm install grunt-browserify --save

The reason we have chosen a grunt task is that we will use this to export our source so that browsers can understand module.exports and use it to concatenate our code.

We’ll skip through a few steps below and edit our Gruntfile.js to include the task we just installed, define the steps to build our JavaScript and include it into our build task:

module.exports = function (grunt) {
    grunt.initConfig({
        express: {
            test: {
                options: {
                    script: './server.js'
                }
            }
        },
        cucumberjs: {
            src: 'tests/e2e/features/',
            options: {
                steps: 'tests/e2e/steps/'
            }
        },
        less: {
            production: {
                options: {
                    paths: ['app/css/'],
                    cleancss: true
                },
                files: {
                    'app/css/main.css': 'src/less/main.less'
                }
            }
        },
        copy: {
            fonts: {
                expand: true,
                src: ['bower_components/bootstrap-sass-official/vendor/assets/fonts/bootstrap/*'],
                dest: 'app/css/bootstrap/',
                filter: 'isFile',
                flatten: true
            }
        },
        bower: {
            install: {
                options: {
                    cleanTargetDir:false,
                    targetDir: './bower_components'
                }
            }
        },
        browserify: {
            dist: {
                files: {
                    'app/js/main.min.js': ['src/js/**/*.js']
                }
            }
        }
    });

    grunt.loadNpmTasks('grunt-express-server');
    grunt.loadNpmTasks('grunt-selenium-webdriver');
    grunt.loadNpmTasks('grunt-cucumber');
    grunt.loadNpmTasks('grunt-contrib-less');
    grunt.loadNpmTasks('grunt-contrib-copy');
    grunt.loadNpmTasks('grunt-bower-task');
    grunt.loadNpmTasks('grunt-browserify');

    grunt.registerTask('generate', ['less:production', 'copy:fonts', 'browserify']);
    grunt.registerTask('build', ['bower:install', 'generate']);

    grunt.registerTask('e2e', [
        'selenium_start',
        'express:test',
        'cucumberjs',
        'selenium_stop',
        'express:test:stop'
    ]);

    grunt.registerTask('heroku:production', 'build');
};

If we now run our generate task you should find a main.min.js file under app/js, which contains a bunch of Browserify and our test file. However you will notice that while it’s concatenated it’s not minified. Let’s fix this.

I chose to go with Uglifyify, as always let’s just install it:

npm install uglifyify --save

And then edit our Gruntfile.js by configuring our browserify task to use it is a transform:

module.exports = function (grunt) {
    grunt.initConfig({
        express: {
            test: {
                options: {
                    script: './server.js'
                }
            }
        },
        cucumberjs: {
            src: 'tests/e2e/features/',
            options: {
                steps: 'tests/e2e/steps/'
            }
        },
        less: {
            production: {
                options: {
                    paths: ['app/css/'],
                    cleancss: true
                },
                files: {
                    'app/css/main.css': 'src/less/main.less'
                }
            }
        },
        copy: {
            fonts: {
                expand: true,
                src: ['bower_components/bootstrap-sass-official/vendor/assets/fonts/bootstrap/*'],
                dest: 'app/css/bootstrap/',
                filter: 'isFile',
                flatten: true
            }
        },
        bower: {
            install: {
                options: {
                    cleanTargetDir:false,
                    targetDir: './bower_components'
                }
            }
        },
        browserify: {
            dist: {
                files: {
                    'app/js/main.min.js': ['src/js/**/*.js']
                }
            },
            options: {
                transform: ['uglifyify']
            }
        }
    });

    grunt.loadNpmTasks('grunt-express-server');
    grunt.loadNpmTasks('grunt-selenium-webdriver');
    grunt.loadNpmTasks('grunt-cucumber');
    grunt.loadNpmTasks('grunt-contrib-less');
    grunt.loadNpmTasks('grunt-contrib-copy');
    grunt.loadNpmTasks('grunt-bower-task');
    grunt.loadNpmTasks('grunt-browserify');

    grunt.registerTask('generate', ['less:production', 'copy:fonts', 'browserify']);
    grunt.registerTask('build', ['bower:install', 'generate']);

    grunt.registerTask('e2e', [
        'selenium_start',
        'express:test',
        'cucumberjs',
        'selenium_stop',
        'express:test:stop'
    ]);

    grunt.registerTask('heroku:production', 'build');
};

If you now run our generate task, the contents of main should be nicely minified. Now all that’s left to do is edit our index.html file and add our generated JavaScript file:

<!DOCTYPE html>
<!--[if lt IE 7]>      <html class="no-js lt-ie9 lt-ie8 lt-ie7"> <![endif]-->
<!--[if IE 7]>         <html class="no-js lt-ie9 lt-ie8"> <![endif]-->
<!--[if IE 8]>         <html class="no-js lt-ie9"> <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js"> <!--<![endif]-->
    <head>
        <meta charset="utf-8">
        <meta http-equiv="X-UA-Compatible" content="IE=edge">
        <title>Weatherly - Forecast for London</title>
        <meta name="description" content="">
        <meta name="viewport" content="width=device-width, initial-scale=1">

        <!-- Place favicon.ico and apple-touch-icon.png in the root directory -->

        <link rel="stylesheet" href="css/main.css">
    </head>
    <body>
        <!--[if lt IE 7]>
            <p class="browsehappy">You are using an <strong>outdated</strong> browser. Please <a href="http://browsehappy.com/">upgrade your browser</a> to improve your experience.</p>
        <![endif]-->

        <!-- Add your site or application content here -->
        <div class="container">
            <div class="header">
                <ul class="nav nav-pills pull-right">
                    <li class="active"><a href="#">Home</a></li>
                    <li><a href="#">About</a></li>
                    <li><a href="#">Contact</a></li>
                </ul>
            </div>

            <div class="jumbotron">
                <h1>London Right Now</h1>
                <p class="temperature">14 degrees</p>
                <p>Mostly cloudy - feels like 14 degrees</p>
            </div>

            <div class="row marketing">
                <div class="col-lg-6">
                    <h4>NEXT HOUR</h4>
                    <p>Mostly cloudy for the hour.</p>

                    <h4>NEXT 24 HOURS</h4>
                    <p>Mostly cloudy until tomorrow afternoon.</p>
                </div>
            </div>

            <div class="footer">
                <p><span class="glyphicon glyphicon-heart"></span> from Weatherly</p>
            </div>

        </div>
        <p></p>

        <script src="//ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
        <script>window.jQuery || document.write('<script src="js/vendor/jquery-1.10.2.min.js"><\/script>')</script>

        <!-- Google Analytics: change UA-XXXXX-X to be your site's ID. -->
        <script>
            (function(b,o,i,l,e,r){b.GoogleAnalyticsObject=l;b[l]||(b[l]=
                    function(){(b[l].q=b[l].q||[]).push(arguments)});b[l].l=+new Date;
                e=o.createElement(i);r=o.getElementsByTagName(i)[0];
                e.src='//www.google-analytics.com/analytics.js';
                r.parentNode.insertBefore(e,r)}(window,document,'script','ga'));
            ga('create','UA-XXXXX-X');ga('send','pageview');
        </script>
        <script src="js/main.min.js"></script>
    </body>
</html>

Before we commit our changes let’s edit our .gitignore file one more time and tell it not to include our generated JavaScript:

node_modules
.idea
bower_components
phantomjsdriver.log
app/css/
app/fonts/
app/js/

Let’s commit, merge and push to our remote repository:

git add .
git commit -m "JavaScript browserified and uglyfied"
git checkout master
git merge generate-assets
git push

You can read the full book over at GitBook or LeanPub. Updated content for this chapter can be found on GitHub.

Book Update

| Comments

It’s been some time since I announced that I was going to write a book on JavaScript and testing, but I do have a small-ish update to share. I have finished drafting the first section of the book, which covers getting up and running with continuous delivery. It’s very wrinkly and very much a stream of thoughts, so apologies for typos, omissions and bad grammar. However I would still like to encourage feedback! So if there’s anything you want to let me know about feel free to leave a comment or open up a Github issue.

I have been experimenting both with Gitbook and LeanPub to help with bringing this to the masses. You can view the Gitbook version here and the LeanPub one here. I’d be interested in hearing feedback about the two versions and which one you prefer. I really like the web version that Gitbook offer, however I much prefer LeanPub’s e-reader generated format.

I have also created a basic page with links to the publishers, a Facebook page and other bits and bobs.

That’s it, time to get back to some more writing!

Design Pattern: Parallel Change

| Comments

I recently read about this pattern and was curious to see who one would implement this in JavaScript. Below is the base class that will be refactored:

function Grid() {
    this.cells = [[]];
}

Grid.prototype.addCell = function (x, y, cell) {
    this.cells[x][y] = cell;
};

Grid.prototype.fetchCell = function (x, y) {
    return this.cells[x][y];
};

Grid.prototype.isEmpty = function (x, y) {
    return this.cells[x][y] == null;
};

In the example on the blog post, the x and y parameters were moved into a Coordinate object and the methods in question simply overloaded. However we don’t have this luxury in JavaScript. The simplest thing to do would be to check the arguments being passed into the individual methods, so here’s what that might look like:

function Grid() {
    this.cells = [[]];
    this.newCells = [];

    this.findCell = function (coordinate) {
        var foundCell;
        this.newCells.forEach(function (cell) {
            if (cell.coordinate.x === coordinate.x && cell.coordinate.y === coordinate.y) {
                foundCell = cell.cell;
            }
        });
        return foundCell;
    };
}

Grid.prototype.addCell = function (x, y, cell) {
    if(arguments.length > 2 && !isNaN(x)) {
        this.cells[x][y] = cell;
    } else {
        this.newCells.push({coordinate: x, cell: y});
    }
};

Grid.prototype.fetchCell = function (x, y) {
    if(arguments.length > 1 && !isNaN(x)) {
        return this.cells[x][y];
    } else {
        return this.findCell(x);
    }
};

Grid.prototype.isEmpty = function (x, y) {
    if(arguments.length > 1 && !isNaN(x)) {
        return this.cells[x][y] == null;
    } else {
        return this.findCell(x) == null;
    }
};

Not exactly elegant, but it works. While looking into options for method overloading I found this post by John Resig. Using his approach here’s what the original Grid class would look like:

// addMethod - By John Resig (MIT Licensed)
function addMethod(object, name, fn){
    var old = object[ name ];
    object[ name ] = function(){
        if ( fn.length == arguments.length )
            return fn.apply( this, arguments );
        else if ( typeof old == 'function' )
            return old.apply( this, arguments );
    };
}

function Grid() {
    this.cells = [[]];
}

addMethod(Grid.prototype, "addCell", function(x, y, cell){
    this.cells[x][y] = cell;
});

addMethod(Grid.prototype, "fetchCell", function(x, y){
    return this.cells[x][y];
});

addMethod(Grid.prototype, "isEmpty", function(x, y){
    return this.cells[x][y] == null;
});

So let’s go ahead and refactor the object to expand the interface and also take an object:

// addMethod - By John Resig (MIT Licensed)
function addMethod(object, name, fn){
    var old = object[ name ];
    object[ name ] = function(){
        if ( fn.length == arguments.length )
            return fn.apply( this, arguments );
        else if ( typeof old == 'function' )
            return old.apply( this, arguments );
    };
}

function Grid() {
    this.cells = [[]];
    this.newCells = [];

    this.findCell = function (coordinate) {
        var foundCell;
        this.newCells.forEach(function (cell) {
            if (cell.coordinate.x === coordinate.x && cell.coordinate.y === coordinate.y) {
                foundCell = cell.cell;
            }
        });
        return foundCell;
    };
}

addMethod(Grid.prototype, "addCell", function(x, y, cell){
    this.cells[x][y] = cell;
});

addMethod(Grid.prototype, "addCell", function(coordinates, cell){
    this.newCells.push({coordinate: coordinates, cell: cell});

});

addMethod(Grid.prototype, "fetchCell", function(x, y){
    return this.cells[x][y];
});

addMethod(Grid.prototype, "fetchCell", function(coordinates){
    return this.findCell(coordinates);
});

addMethod(Grid.prototype, "isEmpty", function(x, y){
    return this.cells[x][y] == null;
});

addMethod(Grid.prototype, "isEmpty", function(coordinates){
    return this.findCell(coordinates) == null;
});

For what’s it’s worth you can find the code and tests on my Github Design Pattern project