Grails: Cucumber with HTTPBuilder & RemoteControl

Here is (another) simple Book example using Cucumber with Grails.

Introduction

As an alternative to testing against the client (browser) ui (using geb) I will test against the server ui (or api). What’s interesting about this example? Here is a quick overview:

• it is using HTTPBuilder to talk to the server api
• it is using remote-control plugin to setup & cleanup test data
• it is using a cucumber ‘World’ object
• step definitions are minimal

This is also an update to my blog Automating Specification with Cucumber & Grails where I directly called grails controllers, services etc in the integration test style.

With Grails 2.3 and the new fork mode this style of testing is deprecated. Functional test code and application should run in separate processes to get a more production like environment. The old style will only work in non forked mode.

To prepare and clean up test data we have to choose another solution: The remote-control plugin. It will let us write closures (in our test code) that are executed in another process (the application) and we can use it to run test setup and tear down code using normal grails code.

Note: the remote-control plugin does not currently (grails 2.3.4 2.3.8) work with automatic reloading (reloading issue) in forked mode (serialVersionUID mismatch errror). We will have to run grails with the -noreloading option. This is not necessary if we use non forked mode.  Update (27.4.’14): Unfortunately I don’t know how to disable reloading for the forked jvm’s so we can only run this in non-forked mode for now.

Ok, so let us take a look at the code!

The code

Here is the list of the files. In comparision to the older examples there is a new subfolder world with a few new files that contain all the nice stuff I listed above.

test
  \-- functional
        \-- data
              Data.groovy 
        \-- hooks
              env.groovy
        \-- steps
              BookSteps.groovy
        \-- world
              Books.groovy
              Requests.groovy
              World.groovy
        ListBooks.feature
        NewBook.feature

The features & step definitions

Nothing new in my sample features:

ListBooks.feature

Feature: new book entry
    As a book owner
    I want to add books I own to the book tracker
    so that I do not have to remember them by myself

Scenario: new book
   Given I open the book tracker
    When I add "Specification by Example"
    Then I see "Specification by Example"s details

ListBooks.feature

Feature: list owned books
    As a book owner
    I want to list my books
    so that I can look up which books I own

Scenario: list existing books
   Given I have already added "Specification by Example"
    When I view the book list
    Then my book list contains "Specification by Example"

.. but the step definitions have changed:

steps/BookSteps.groovy

package steps

import static cucumber.api.groovy.EN.*


Given (~'^I open the book tracker$') { ->
    // nop
}

When (~'^I add "([^"]*)"$') { String bookTitle ->
    requestAddBook (bookTitle)
}

Then (~'^I see "([^"]*)"s details$') { String bookTitle ->
    assertAddBook (bookTitle)
}

Given (~'^I have already added "([^"]*)"$') { String bookTitle ->
    bookId = setupBook (bookTitle)
}

When (~'^I view the book list$') { ->
    requestAllBooks ()
}

Then (~'^my book list contains "([^"]*)"$') { String bookTitle ->
    assertAllBooksContains (bookId, bookTitle)
}

As you can see, there is not much code anymore. Of course it is still there, it just has moved to a better place. The nice thing about keeping the step definitions light is that it makes them really cheap and simple. After all it is called glue code. You won’t use a 1 cm layer of glue to stick two pieces together.

The big advantage is that you don’t need to care if you require a step with a slightly changed wording or if there is already a step that has the code you need. Simply create a new one and use that one liner to call your test api. We don’t need to care if there is a little bit of duplication because all the heavy lifting takes place in the test api.

The test api

Forcing ourself to move most of the code out of the steps has another advantage. In the more usual code environment (without the step definition “noise”) it is easier to follow our normal implementation habbits like removing duplication, creating clean code and so on. Hope this make sense to you. :-)

Here is the test api code for Book. I have moved setup, action and assertion methods together because I prefer grouping by topic (not necessarily in a single file of course but here it is small enough). If I want to know anything about the Books test api I just have to look here.

world/Books.groovy

package world

import data.Data
import grails.plugin.remotecontrol.RemoteControl
import static javax.servlet.http.HttpServletResponse.*


class Books {
    def booksRequestData

    def getBooksResponse () {
        booksRequestData.response
    }

    def getBooksResponseData () {
        booksRequestData.data
    }

    Long setupBook (String title) {
        def remote = new RemoteControl ()

        def book = Data.findByTitle (title)
        Long id = remote {
            ctx.bookService.add (book)?.id
        } as Long

        assert id
        id
    }

setupBook is the setup code used to prepare a single book for the list existing books scenario. It is using the remote-control plugin to create the test data.

It looks up the book details by the given title and then calls remote to execute the closure in the running grails application. The closure itself uses the BookService to add the book. The same service is used to add a book by the server api.

The ctx variable is provided by the remote-control plugin so we can get easily at the application artifacts. There is not much more to say about it. Take a look at its documentation for the rest. It is a a quick read.

    void requestAddBook (String title) {
        def newBook = Data.findByTitle (title)
        booksRequestData = post ('book/add', newBook)
        assert booksResponse.status == SC_OK
    }

requestAddBook adds a new book calling the server api. It is used in the new book scenario. It simply sends a normal post request to the application, calling the BookControllers add action, remembering the response information and checking that we got a 200 ok.

In this simple example we could have used it to setup the book as well. If we would need multiple books as test data though we would waste a lot of time running a single request for each book. Looping in the remote control closure will be a lot faster.

    void assertAddBook (String title) {
        def expected = Data.findByTitle (title)
        def actual = booksResponseData

        assert actual.id
        assert actual.title  == expected.title
        assert actual.author == expected.author
    }

This method simply checks that the response data correspond to the requested book.

    void requestAllBooks () {
        booksRequestData = getJson ('book/all')
        assert booksResponse.status == SC_OK
    }

This one is used to get the list of books as json calling the all action of BookController.

    void assertAllBooksContains (Long id, String title) {
        def expected = Data.findByTitle (title)
        def actual = booksResponseData.first ()

        assert actual.id     == id
        assert actual.title  == expected.title
        assert actual.author == expected.author
    }
}

Finally another assertion mehod that checks that the previously requested book list contains the expected book.

post & getJson request

The two network calls post and getJson used in the test api are implemented in the next file. There is no magic here, just two simple HTTPBuilder calls.

world/Requests.groovy

package world

import groovyx.net.http.ContentType
import groovyx.net.http.HTTPBuilder
import groovyx.net.http.Method


class Requests {

    def defaultSuccess = { response, data ->
        [response: response, data: data]
    }

    def defaultFailure = { response, data ->
        [response: response, data: data]
        assert false
    }

    def getJson (String targetUri, Closure success = null, Closure failure = null) {
        def http = new HTTPBuilder(binding.functionalBaseUrl)

        def result = http.request (Method.GET, ContentType.JSON) {
            uri.path = targetUri
//            headers.'X-Requested-With' = 'XMLHttpRequest'
//            headers.'Cookie' = cookies.join (';')
            response.success = success ?: defaultSuccess
            response.failure = failure ?: defaultFailure
        }
        result
    }

    def post (String targetUri, Map params, Closure success = null, Closure failure = null) {
        def http = new HTTPBuilder(binding.functionalBaseUrl)

        def result = http.request (Method.POST, ContentType.JSON) {
            uri.path = targetUri
//            headers.'X-Requested-With' = 'XMLHttpRequest'
//            headers.'Cookie' = cookies.join(';')
            requestContentType = ContentType.URLENC
            body = params
            response.success = success ?: defaultSuccess
            response.failure = failure ?: defaultFailure
        }
        result
    }
}

The only special thing is the line def http = new HTTPBuilder(binding.functionalBaseUrl) in both methods. functionalBaseUrl is the url the application is running on (you can also provide it via the baseUrl command line option) and is provided by grails.

Done? Not yet :-)

If you have read so far you may wonder how the step definitions can call the test api, how the test api finds the request methods and where the binding is comming from.

That is were the World comes into play…

The World, putting everything together

The World is simply an object we can use to to provide some additional stuff to the step definitions via cucumbers World hook. We can also use it share state between the steps of a single scenario. A new world is created for each running scenario.

world/World.groovy

package world

import grails.plugin.remotecontrol.RemoteControl
import static cucumber.api.groovy.Hooks.World


class BookWorld {
    def binding

    BookWorld (def binding) {
        this.binding = binding
    }

    void resetDatabase () {
        def remote = new RemoteControl ()

        boolean success = remote {
            ctx.databaseService.reset ()
            true
        }
        assert success
    }
}

World () {
    def world = new BookWorld (binding)
    world.metaClass.mixin Requests
    world.metaClass.mixin Books
    world
}

Here our World object is of type BookWorld with the Books code and Requests code mixed in. This is a groovy trick to add additional methods to the World object. Because thery are all on the World object they can call each other.

This file is a groovy script and when it is executed we pass the scripts binding to our world so we can use is later to get the functionalBaseUrl.

Reseting the Database

To run the scenarios independend of each other we have to reset our test data in the database. Both scenarios add the same book to the database and we would get a constraint violation if we do not clean up before the next scenario runs.

That what the resetDatabase method in the World is supposed to do. It is simply called from a Before hook like this:

hooks/env.groovy

package hooks

import static cucumber.api.groovy.Hooks.Before


Before () {
    resetDatabase ()
}

resetDatabase is using the remote-control plugin to call a special service that takes care of reseting the database by running a sql script. In this case it is very simple, it just clears the ‘book’ table (see below).

DatabaseService.groovy

package test

import groovy.sql.Sql

import javax.sql.DataSource


class DatabaseService {
    DataSource dataSource

    void reset () {
        def script = new File ('sql/reset.sql').text

        Sql sql = new Sql (dataSource)

        sql.withTransaction { def c ->
            sql.execute (script)
            sql.commit()
        }
        sql.close ()
    }
}

sql/reset.sql

-- clear tables for functional tests

delete from book;

That’s it.

The full code of this example is available on my github page here in the repository of the grails-cucumber plugin.

Thanks for reading :-)