Grails: Using fork mode with grails-cucumber plugin

Introduction

The good news is that we can finally run grails-cucumber using the forked mode introduced in grails 2.3 using the preferred testing style described in Grails: Cucumber with HTTPBuilder & RemoteControl. :-)

It was a long road making grails-cucumber compatible with forked mode. There were a few things that had to be changed in the plugin itself, then there were a couple of issues in grails functional testing setup and finally the remote-control plugin did not work in forked mode because of a reloading issue in spring-loaded.

To summarize, we will need at least the following versions to run the cucumber features with not forked & forked mode:

mode grails-cucumber remote-control grails
not forked >= 0.11.0-SNAPSHOT >= 1.4 >= 2.3.8
forked >= 0.11.0-SNAPSHOT >= 1.5 >= 2.4.0

the latest version of grails-cucumber is 1.0.0

Now let’s take a look at running the example from Grails: Cucumber with HTTPBuilder & RemoteControl in forked mode.

Running cucumber features in forked mode

What follows is more or less a generic description about running functional tests in forked mode and not specific to (grails-)cucumber.

There are two reasons why we would want to use the forked-mode

  • isolation of the the build path from the runtime/test paths
  • quicker roundtrips in development because we do not have to wait for jvm/grails startup

On my machine (late 2011, (surprise, sooo old already ;-) it takes about 25 seconds to run grails test-app functional:cucumber for the two scenarios in the example in non forked mode. Most of that time is used to startup the jvm and grails.

The first step to use the forked mode is to add the fork configuration into BuildConfig.groovy. If you created your project with grails 2.3 or above it will already exists.

grails.project.fork = [
    // configure settings for the run-app JVM
    run: [maxMemory: 768, minMemory: 64, debug: false, maxPerm: 256],
    // configure settings for the test-app JVM, uses the daemon by default
    test: [maxMemory: 768, minMemory: 64, debug: false, maxPerm: 256, daemon:true]
]

Next we run grails test to start the interactive grails console. I’m running the console in the test enviroment so that the remote-control plugin gets enabled in the running grails application (by default it will only be enabled in the test environment).

If you run ps -ef | grep java in another window you will see two grails java processes running.

We can run our application by simply entering run-app from the grails console:

grails> run-app
| Server running. Browse to http://localhost:8080/Books_remote
| Application loaded in interactive mode. Type 'stop-app' to shutdown.

Running ps again we see four(!) java processes. I expected to see three. Not sure why there are four.

To run the features we can now simply call

test-app functional:cucumber -baseUrl=http://localhost:8080/Books_remote/

.. just taking a few seconds now… since it does not have to start grails anymore.

Note that we have to pass the baseUrl parameter to test-app because test-app does not know where the application is running (baseUrl was broken in grails 2.3 until grails 2.3.8).

Make sure you have the slash at the end. In the test code I pass the url to HttpBuilder and without the slash it will drop the Books_remote from the url path.

and we receive the usual test-app output from grails and cucumber:

grails> test-app functional:cucumber --stacktrace --verbose -baseUrl=http://localhost:8080/Books_remote/
| Running 2 cucumber tests...
2 Scenarios (
2 passed
)
6 Steps (
6 passed
)
0m
0.185s
| Completed 2 cucumber tests, 0 failed in 0m 0s
| Tests PASSED - view reports in /Users/hauner/Development/Grails/grails-cucumber.git/test/projects/Books_remote/target/test-reports

As in development we can change the application code and the running grails application will reload the changes.

That’s it. Happy forking :)

Advertisements

Grails: Cucumber with HTTPBuilder & RemoteControl

Here is (another) simple Book example using Cucumber with Grails.

Introduction

As an alternative to testing against the client (browser) ui (using geb) I will test against the server ui (or api). What’s interesting about this example? Here is a quick overview:

  • it is using HTTPBuilder to talk to the server api
  • it is using remote-control plugin to setup & cleanup test data
  • it is using a cucumber ‘World’ object
  • step definitions are minimal

This is also an update to my blog Automating Specification with Cucumber & Grails where I directly called grails controllers, services etc in the integration test style.

With Grails 2.3 and the new fork mode this style of testing is deprecated. Functional test code and application should run in separate processes to get a more production like environment. The old style will only work in non forked mode.

To prepare and clean up test data we have to choose another solution: The remote-control plugin. It will let us write closures (in our test code) that are executed in another process (the application) and we can use it to run test setup and tear down code using normal grails code.

Note: the remote-control plugin does not currently (grails 2.3.4 2.3.8) work with automatic reloading (reloading issue) in forked mode (serialVersionUID mismatch errror). We will have to run grails with the -noreloading option. This is not necessary if we use non forked mode.  Update (27.4.’14): Unfortunately I don’t know how to disable reloading for the forked jvm’s so we can only run this in non-forked mode for now.

Ok, so let us take a look at the code!

The code

Here is the list of the files. In comparision to the older examples there is a new subfolder world with a few new files that contain all the nice stuff I listed above.

test
  \-- functional
        \-- data
              Data.groovy 
        \-- hooks
              env.groovy
        \-- steps
              BookSteps.groovy
        \-- world
              Books.groovy
              Requests.groovy
              World.groovy
        ListBooks.feature
        NewBook.feature

The features & step definitions

Nothing new in my sample features:

ListBooks.feature

Feature: new book entry
    As a book owner
    I want to add books I own to the book tracker
    so that I do not have to remember them by myself

Scenario: new book
   Given I open the book tracker
    When I add "Specification by Example"
    Then I see "Specification by Example"s details

ListBooks.feature

Feature: list owned books
    As a book owner
    I want to list my books
    so that I can look up which books I own

Scenario: list existing books
   Given I have already added "Specification by Example"
    When I view the book list
    Then my book list contains "Specification by Example"

.. but the step definitions have changed:

steps/BookSteps.groovy

package steps

import static cucumber.api.groovy.EN.*


Given (~'^I open the book tracker$') { ->
    // nop
}

When (~'^I add "([^"]*)"$') { String bookTitle ->
    requestAddBook (bookTitle)
}

Then (~'^I see "([^"]*)"s details$') { String bookTitle ->
    assertAddBook (bookTitle)
}

Given (~'^I have already added "([^"]*)"$') { String bookTitle ->
    bookId = setupBook (bookTitle)
}

When (~'^I view the book list$') { ->
    requestAllBooks ()
}

Then (~'^my book list contains "([^"]*)"$') { String bookTitle ->
    assertAllBooksContains (bookId, bookTitle)
}

As you can see, there is not much code anymore. Of course it is still there, it just has moved to a better place. The nice thing about keeping the step definitions light is that it makes them really cheap and simple. After all it is called glue code. You won’t use a 1 cm layer of glue to stick two pieces together.

The big advantage is that you don’t need to care if you require a step with a slightly changed wording or if there is already a step that has the code you need. Simply create a new one and use that one liner to call your test api. We don’t need to care if there is a little bit of duplication because all the heavy lifting takes place in the test api.

The test api

Forcing ourself to move most of the code out of the steps has another advantage. In the more usual code environment (without the step definition “noise”) it is easier to follow our normal implementation habbits like removing duplication, creating clean code and so on. Hope this make sense to you. :-)

Here is the test api code for Book. I have moved setup, action and assertion methods together because I prefer grouping by topic (not necessarily in a single file of course but here it is small enough). If I want to know anything about the Books test api I just have to look here.

world/Books.groovy

package world

import data.Data
import grails.plugin.remotecontrol.RemoteControl
import static javax.servlet.http.HttpServletResponse.*


class Books {
    def booksRequestData

    def getBooksResponse () {
        booksRequestData.response
    }

    def getBooksResponseData () {
        booksRequestData.data
    }

    Long setupBook (String title) {
        def remote = new RemoteControl ()

        def book = Data.findByTitle (title)
        Long id = remote {
            ctx.bookService.add (book)?.id
        } as Long

        assert id
        id
    }

setupBook is the setup code used to prepare a single book for the list existing books scenario. It is using the remote-control plugin to create the test data.

It looks up the book details by the given title and then calls remote to execute the closure in the running grails application. The closure itself uses the BookService to add the book. The same service is used to add a book by the server api.

The ctx variable is provided by the remote-control plugin so we can get easily at the application artifacts. There is not much more to say about it. Take a look at its documentation for the rest. It is a a quick read.

    void requestAddBook (String title) {
        def newBook = Data.findByTitle (title)
        booksRequestData = post ('book/add', newBook)
        assert booksResponse.status == SC_OK
    }

requestAddBook adds a new book calling the server api. It is used in the new book scenario. It simply sends a normal post request to the application, calling the BookControllers add action, remembering the response information and checking that we got a 200 ok.

In this simple example we could have used it to setup the book as well. If we would need multiple books as test data though we would waste a lot of time running a single request for each book. Looping in the remote control closure will be a lot faster.

    void assertAddBook (String title) {
        def expected = Data.findByTitle (title)
        def actual = booksResponseData

        assert actual.id
        assert actual.title  == expected.title
        assert actual.author == expected.author
    }

This method simply checks that the response data correspond to the requested book.

    void requestAllBooks () {
        booksRequestData = getJson ('book/all')
        assert booksResponse.status == SC_OK
    }

This one is used to get the list of books as json calling the all action of BookController.

    void assertAllBooksContains (Long id, String title) {
        def expected = Data.findByTitle (title)
        def actual = booksResponseData.first ()

        assert actual.id     == id
        assert actual.title  == expected.title
        assert actual.author == expected.author
    }
}

Finally another assertion mehod that checks that the previously requested book list contains the expected book.

post & getJson request

The two network calls post and getJson used in the test api are implemented in the next file. There is no magic here, just two simple HTTPBuilder calls.

world/Requests.groovy

package world

import groovyx.net.http.ContentType
import groovyx.net.http.HTTPBuilder
import groovyx.net.http.Method


class Requests {

    def defaultSuccess = { response, data ->
        [response: response, data: data]
    }

    def defaultFailure = { response, data ->
        [response: response, data: data]
        assert false
    }

    def getJson (String targetUri, Closure success = null, Closure failure = null) {
        def http = new HTTPBuilder(binding.functionalBaseUrl)

        def result = http.request (Method.GET, ContentType.JSON) {
            uri.path = targetUri
//            headers.'X-Requested-With' = 'XMLHttpRequest'
//            headers.'Cookie' = cookies.join (';')
            response.success = success ?: defaultSuccess
            response.failure = failure ?: defaultFailure
        }
        result
    }

    def post (String targetUri, Map params, Closure success = null, Closure failure = null) {
        def http = new HTTPBuilder(binding.functionalBaseUrl)

        def result = http.request (Method.POST, ContentType.JSON) {
            uri.path = targetUri
//            headers.'X-Requested-With' = 'XMLHttpRequest'
//            headers.'Cookie' = cookies.join(';')
            requestContentType = ContentType.URLENC
            body = params
            response.success = success ?: defaultSuccess
            response.failure = failure ?: defaultFailure
        }
        result
    }
}

The only special thing is the line def http = new HTTPBuilder(binding.functionalBaseUrl) in both methods. functionalBaseUrl is the url the application is running on (you can also provide it via the baseUrl command line option) and is provided by grails.

Done? Not yet :-)

If you have read so far you may wonder how the step definitions can call the test api, how the test api finds the request methods and where the binding is comming from.

That is were the World comes into play…

The World, putting everything together

The World is simply an object we can use to to provide some additional stuff to the step definitions via cucumbers World hook. We can also use it share state between the steps of a single scenario. A new world is created for each running scenario.

world/World.groovy

package world

import grails.plugin.remotecontrol.RemoteControl
import static cucumber.api.groovy.Hooks.World


class BookWorld {
    def binding

    BookWorld (def binding) {
        this.binding = binding
    }

    void resetDatabase () {
        def remote = new RemoteControl ()

        boolean success = remote {
            ctx.databaseService.reset ()
            true
        }
        assert success
    }
}

World () {
    def world = new BookWorld (binding)
    world.metaClass.mixin Requests
    world.metaClass.mixin Books
    world
}

Here our World object is of type BookWorld with the Books code and Requests code mixed in. This is a groovy trick to add additional methods to the World object. Because thery are all on the World object they can call each other.

This file is a groovy script and when it is executed we pass the scripts binding to our world so we can use is later to get the functionalBaseUrl.

Reseting the Database

To run the scenarios independend of each other we have to reset our test data in the database. Both scenarios add the same book to the database and we would get a constraint violation if we do not clean up before the next scenario runs.

That what the resetDatabase method in the World is supposed to do. It is simply called from a Before hook like this:

hooks/env.groovy

package hooks

import static cucumber.api.groovy.Hooks.Before


Before () {
    resetDatabase ()
}

resetDatabase is using the remote-control plugin to call a special service that takes care of reseting the database by running a sql script. In this case it is very simple, it just clears the ‘book’ table (see below).

DatabaseService.groovy

package test

import groovy.sql.Sql

import javax.sql.DataSource


class DatabaseService {
    DataSource dataSource

    void reset () {
        def script = new File ('sql/reset.sql').text

        Sql sql = new Sql (dataSource)

        sql.withTransaction { def c ->
            sql.execute (script)
            sql.commit()
        }
        sql.close ()
    }
}

sql/reset.sql

-- clear tables for functional tests

delete from book;

That’s it.

The full code of this example is available on my github page here in the repository of the grails-cucumber plugin.

Thanks for reading :-)

Intellij IDEA, Cucumber and German Spell Checking

Now that Intellij IDEA (12.1.3 in my case) supports auto completion & syntax highlighting for cucumber features not only in english but also in any other language that is available for gherkin it would be nice to have native spell checking as well.

To use your native language with cucumber you just have to place a language comment at the first line of your feature file. For example see this super useful feature description:

# language: de

Funktionalität: deutsche Feature-Beschreibungen
  Um cucumber in unserer Muttersprache zu benutzten
  möchte ich als Szenario-Schreiber
  die deutschen Schlüsselwörter benutzen

  Szenario: deutsche Schlüsselwörter & Steps
    Angenommen ich schreibe eine Feature-Beschreibung
    Wenn ich die deutschen Gherkin-Schlüsselwörter benutze
    Dann werden die deutschen Steps aufgerufen

To get spell checking for an additional language in IntelliJ we need to add a dictionary for that language. This is done in a few simple steps:

  • first, we need a dictionary for our language. This is a plain text file with a lot of words, each on a single line. I found a german dictionary on sourceforge.
  • second, we need to make sure it is encoded in utf–8. The german.dic file was encoded in latin–1. If it is not encode in utf–8 use your text editor of choice (e.g. Notepadd++ or TextWrangler or …) and convert it to utf–8 (no BOM).
  • third, create a folder (e.g. dictionaries) where you want to save the dic file
  • fourth, tell IntelliJ about the dictionary folder following the documentation, which is in short:
    1. open the Settings dialog
    2. type ‘spell’ into the search box and select Spelling
    3. switch to the Dictionaries tab
    4. and add the folder to the Custom Dictionaries Folder list

You should see now the dictionary under Dictionaries as a user dictionary and the checkbox enabled.

That’s it, no more typos in the features :-)

Grails: grails-cucumber and compiled steps

I am finally finishing the precompiled steps support in the cucumber grails plugin.

Currently if we run grails test-app functional:cucumber it will fire up the whole application and then run cucumber. Cucumber will load, parse and execute all our groovy step source files. If there is an import missing in one of the source files (we will look at an example in a minute) it will throw an error at us. At runtime.

Compiling the cucumber glue code has the advantage that we can catch this already at compile time before grails runs the application. Depending on the size of your app this can save you a number of time consuming round trips.

Let’s take the grails-cucumber Book example and remove the BookController import from BookSteps.groovy.

//disabled the import below...
//import books.BookController
import data.Data

import static cucumber.api.groovy.EN.*

...
When (~'^I add "([^"]*)"$') { String bookTitle ->
    bookController = new BookController ()
    bookController.params << Data.findByTitle (bookTitle)
    bookController.add ()
}
...

The When step create a new BookController. Now if we run it with the current version of grails-cucumber (0.8.0) we will get the following output:

Prompt:Books hauner$ grails test-app functional:cucumber
| Environment set to test.....
| Packaging Grails application..
| Packaging Grails application.....
| Server running. Browse to http://localhost:8080/Books
| Compiling 2 source files.
| Error Compilation error compiling [cucumber] tests: startup failed:
/Users/hauner/Development/Grails/grails-cucumber.git/test/projects/Books/test/functional/steps/BookSteps.groovy: 12: unable to resolve class BookController 
 @ line 12, column 22.
       bookController = new BookController ()
                        ^

/Users/hauner/Development/Grails/grails-cucumber.git/test/projects/Books/test/functional/steps/BookSteps.groovy: 33: unable to resolve class BookController 
 @ line 33, column 22.
       bookController = new BookController ()
                        ^

2 errors
 (Use --stacktrace to see the full trace)

We can see that grails is packaging the application and starts the server, then cucumber starts loading the step code and fails as expected.

Now using the new precompiled steps feature it will look like this:

Prompt:Books_compile hauner$ grails test-app functional:cucumber
| Environment set to test.....
| Compiling 1 source files.
| Error Compilation error compiling cucumber glue code:
            startup failed:
/Users/hauner/Development/Grails/grails-cucumber.git/test/projects/Books_compile/test/cucumber/steps/BookSteps.groovy: 14: unable to resolve class BookController 
 @ line 14, column 22.
       bookController = new BookController ()
                        ^

/Users/hauner/Development/Grails/grails-cucumber.git/test/projects/Books_compile/test/cucumber/steps/BookSteps.groovy: 35: unable to resolve class BookController 
 @ line 35, column 22.
       bookController = new BookController ()
                        ^

2 errors
 (Use --stacktrace to see the full trace)

No packaging and no server startup before it fails. Cool ;-)

To use precompiled steps we only need to modify the plugin configuration a little bit. To make this more interesting I have also moved the step code to another directory to separate the feature description from the implementation.

The default layout I use in the Book examples has features and step code in the functional folder.

test
  \-- functional
        \-- data
              Data.groovy 
        \-- hooks
              env.groovy
        \-- steps
              BookSteps.groovy
        ListBooks.feature
        NewBook.feature

I changed the layout for the new Book_compile example to:

test
  \-- cucumber
        \-- data
              Data.groovy 
        \-- hooks
              env.groovy
        \-- steps
              BookSteps.groovy
  \-- functional
        ListBooks.feature
        NewBook.feature

I choose cucumber for the source files but it can by anything you like. Apart from the source directories we have to tell cucumber were it will find the (compiled) steps.

This will look like this (CucumberConfig.groovy):

cucumber {
     // steps, hooks etc that will be compiled
    sources = ["test/cucumber"]

    // .. and where cucumber will find the compiled steps & hooks
    glue = ["classpath:steps", "classpath:hooks"]
}

With the (new) sources directive we set the source directories and with the (existing) glue directive we tell cucumber where it will find the step code in the classpath by prefix the entries with classpath:.

In my original example the cucumber source files do not have a package name. This is bad for compiling the steps because we would have to tell cucumber to start searching for them at the default package. This is not a good idea because cucumber would check all .class files in the whole classpath to find the glue code.

It is better to properly package the step code so we can make sure cucumber will only look at classes it cares about.

In the example I have used multiple top level packages to show that it is possible. Usually you should place it under the same parent package so that there is only a single classpath: required for the configuration.

Note that we did not add the data path to glue. It is just a helper class that does not contain cucumber code, so we do not need to tell cucumber about it. If we add it, cucumber will ignore it.

test/functional

if you want to keep the source below test/functional that is ok too. You do not need to set it as sources in CucumberConfig.groovy but you still have to adjust the glue settings.

that’s it…

I hope you will like this new feature of grails-cucumber. I still have some cleanup todo before I will release it, but it is on its way.. :-)

Escaping Fun: replaceAll (“\\\\”, “\\\\\\\\”)!

There is a small escaping bug in cucumber-jvm. The java generated groovy step snippets do not properly escape the escape character \ in the steps regular expression.

Currently it generates:

Given(~'^I have (\d+) cukes in my "([^"]*)" belly') { int arg1 ->
    // Express the Regexp above with the code you wish you had
    throw new PendingException()
}

which should be:

Given(~'^I have (\\d+) cukes in my "([^"]*)" belly$') { int arg1 ->
    // Express the Regexp above with the code you wish you had
    throw new PendingException()
}

Cucumber generates code snippets we have to escape the escape character in the snippet output too, i.e. the (\\d+). I have modified the groovy snippet generation before, so it should be an easy fix. Or so I thought. ;-)

It was not a big issue but it took me longer than expected to understand because escaping the escape characters is a bit confusing at first using replaceAll().

Escaping the escape character (\) gets interesting if it is a regular expression: it needs to be escaped again. All this endless escaping turns into this stupid piece of code:

public String escapePattern(String pattern) {
    return pattern.replaceAll ("\\\\", "\\\\\\\\");
}

The method above gets a regular expression string for a step as input (as I see it in the debugger):

"^I have (\\d+) cukes in my \"([^\"]*)\" belly$"

Actually the real string is just:

^I have (\d+) cukes in my "([^"]*)" belly$

And what we like to see as the final regular expression is:

^I have (\\d+) cukes in my "([^"]*)" belly$

We just want to replace \ with \\.

replaceAll takes a regular expression pattern (String) as the first parameter so we have to escape the \ twice to match it:

  • \ => \\ because \ is the escape character for regular expressions
  • \\ => \\\\ because \ is the escape character for Strings

Because \ is also a special character in the replacement (second) parameter of replaceAll. So we have to escape \ twice again:

  • \\ => \\\\ because \ is the escape character in the replacement parameter`
  • \\\\=> \\\\\\\\ because \ is the escape character for Strings

Which finally leads to this stupid line: pattern.replaceAll ("\\\\", "\\\\\\\\"); !

This can be simplified by using String.replace (CharSequence target, CharSequence replacement) (since 1.5). It does not use regular expressions which allows us to drop one level of escaping:

pattern.replace ("\\", "\\\\");

Which is a lot easier to understand. Which is also the final solution for the pull request :-)

Cucumber & Grails: Transaction Rollback

… the short version is: “the cucumber grails plugin will not provide automatic transaction rollback”.

After adding automatic rollback code in the cucumber grails plugin (version 0.5) and trying to use it I had to learn, that the automatic part is flawed and that it does not work in all cases.

I will try to explain why it does not work and how the cucumber plugin will deal with it. If you happen to know more about how Grails/Spring transactions work, please correct me where I am wrong. If you have to add something please do! :-)

First some background how Grails handles database transactions by default.

runtime

If we use Grails default settings, it will wrap any service call into its own transaction. If we make two service calls in a single controller action, each one will have its own transaction. We will take a look at a simple example to demonstrate the convention. See also the Services chapter in the grails documentation.

You can change the defaults, but changing them will not improve anything. It will just make the situation more complicated and therefore it does only supports the statement that we will not get automatic transaction rollback for functional testing. :-)

Reusing the plugins Book example and running grails (run-app) we can easily check this behaviour.

package books

class Book {
    String author
    String title

    static constraints = {
        title unique: true
    }
}

The service looks like this:

package books

class BookService {
    Book add (boolean error, Map params) {
        def newBook = new Book (params)
        newBook.save (failOnError: true)
        if (error) {
            throw new RuntimeException ("rollback!")
        }
        newBook
    }
}

To easily get an error when we want to, I have added a parameter to the add() method that tells the function to error.

The following controller code uses the service with code you should not write in real code. It is written like this to demonstrate grails transaction handling. You should use a single service call to make the action run with a single transaction.

Now when we run the multipleTransactions() action in the following controller:

package books

class BookController {
    def bookService

    def multipleTransactions () {
        params.author = "A1"
        params.title = "T1"
        bookService.add (false, params)

        params.author = "A2"
        params.title = "T2"
        bookService.add (true, params)
    }
}

… and look at the database (you can use the dbconsole to look into the h2 database) we will see a single row:

ID VERSION AUTHOR TITLE
1 0 A10 T10

After reading Services this should not be a big surprise. Both add() calls to the service get their own transaction. Because of the true parameter in the second call, it throws and error an grails will automatically rollback the transaction. But only the transaction of the second call. The transaction of the first call is already committed.

integration test

In integration tests grails does automatically wrap each test in a transaction and triggers a rollback when the test is finished (which by the way is the code I called to get the rollback in the plugin).

It just does not work in all cases.

It fails when we exercise the application with http requests. I assume because grails session/request code will run (which is not run in an integration test when we simply new the controller) using its own transaction code that is ignoring the transaction wrapped around the test (guessing here, I don’t yet understand what grails does in detail).

It also fails if we start to run code using withTransaction().

package books

class BookService {
    Book add (Map params) {
        def newBook
        Book.withNewTransaction {
            newBook = new Book (params)
            newBook.save (failOnError: true)
        }
        newBook     
    }
}

.. with the controller:

package books

class BookController {
    def bookService

    def transaction () {
        params.author = "A1"
        params.title = "T1"
        bookService.add (params)
    }
}

… and when we run two cucumber scenarios calling BookController.transaction () the second call will fail because of the unique constraint on the title. The withTransaction() part will not be rollbacked.

conclusion

So where does this information place us regarding rollback of database transactions in context of functional testing with cucumber? Although it is a play example it shows that transaction handling can already get complex in such a simple example. Now consider multiple request, service annotations that change the default behaviour or “manual” withTransaction() calls. It is not as easy as it looks.

Maybe it would work if we were able to intercept the creation of any transaction created by grails, always return the same transaction and rollback it when the test is finished.

Appart from not finding any information how this would work (maybe I didn’t use the correct search words) it does not look like a good idea. Although it sounds like a solution in the first moment, it will make grails behave differently in the test enviroment and the production environment. How big the difference is, is hard to say. It depends on how much you change grails default behaviour.

Anyway, it will behave differently and I think transactions are important enough that we do not want this when doing functional testing. When we run end to end, the transactions should be identical to production.

My conclusion is, that the transaction handling is complicated enough that the plugin can not handle this for us. Therefore it will no longer run grails transaction code automatically. We will have to take care of cleaning up the database by ourself. Either by completely dropping and re-creating the database schema before each test or by running some statements that will restore the database to its original state (I will write another article showing an example for both ways).

The plugin will still provide some simple configuration to enable the integration test rollback support. You can use it to get the automatic rollback when you build functional integration tests, e.g. you do not exercise the app using http requests and you don’t use withTransaction().

Automating Specification with Cucumber & Grails

Specification by Example is an interesting approach to build a specification in collaboration with a customer. One part of it is to automate the specification without changing it and to build a living documentation system that is always in sync with the software.

One possibility to automate the specification is to describe it using Cucumbers Given, When & Then style and writing some glue code to connect the steps with the system we build.

I will introduce you to a new cucumber grails plugin (grails.org/plugins, documentation) that make it a lot easier to automate cucumber specifications of a Grails application.

The plugin is a functional test plugin for grails and it is only available in grails functional test phase. It is interesting because it runs cucumber inside a running grails application which allows us to use the grails api in the cucumber step implementations. We can call GORM stuff like dynamic finders, services and controllers.

To make the magic work, the plugin is based on Cucumber-JVM the native JVM implementation of cucumber. Cucumber-JVM does support many JVM languages. Groovy is one of them which makes it a perfect fit for grails.

As we will see in the following introduction, the plugin integrates into grails standard test infrastructure which makes it quite easy to use. The full source code of the example and the plugin itself is online in my github repository (example source code, plugin source code).

We will walk through a very simple example that will use cucumber to test two scenarios against a grails web application. We will implement the steps under the skin of the application, i.e. below the ui.
I have written another article (here) that runs the same example against the ui using Geb in the steps to remote control a web browser (Note: because of my latest changes in the plugin the article is no longer 100% up to date).

Let us start with the setup…

setup

First, create a grails app with: grails create-app Books (I am using grails 2.0.3). Next, add the plugin dependency to grails-app/conf/BuildConfig.groovy in the plugins section:

plugins {
   test ":cucumber:0.4.0"
}

Done, setup complete.

.feature files

By convention, the plugin expects all cucumber files (features and steps) in test/functional (you can change it in the configuration, see the documentation).

Create the following two .feature files in test/functional which describe the functionality of the example application:

NewBook.feature:

Feature: new book entry
    As a book owner
    I want to add books I own to the book tracker
    so that I do not have to remember them by myself

Scenario: new book
   Given I open the book tracker
    When I add "Specification by Example"
    Then I see "Specification by Example"s details

ListBooks.feature:

Feature: list owned books
    As a book owner
    I want to list my books
    so that I can look up which books I own

@ignore
Scenario: list existing books
   Given I have already added "Specification by Example"
    When I view the book list
    Then my book list contains "Specification by Example"

Specification by Example: Website, amazon.com

running the features

I already mentioned that the plugin properly integrates into grails and that it is a functional test plugin. To run the cucumber features only we can use the usual grails test-app command using one of the following variations:

grails test-app functional:cucumber
grails test-app :cucumber
grails test-app functional:

Running it we will see the typical cucumber output for missing step implementations:

You can implement missing steps with the snippets below:

Given(~"^I have already added \"([^\"]*)\"$") { String arg1 ->
    // Express the Regexp above with the code you wish you had
    throw new PendingException()
}
When(~"^I view the book list$") { ->
    // Express the Regexp above with the code you wish you had
    throw new PendingException()
}
Then(~"^my book list contains \"([^\"]*)\"$") { String arg1 ->
    // Express the Regexp above with the code you wish you had
    throw new PendingException()
}
Given(~"^I open the book tracker$") { ->
    // Express the Regexp above with the code you wish you had
    throw new PendingException()
}
When(~"^I add \"([^\"]*)\"$") { String arg1 ->
    // Express the Regexp above with the code you wish you had
    throw new PendingException()
}
Then(~"^I see \"([^\"]*)\"s details$") { String arg1 ->
    // Express the Regexp above with the code you wish you had
    throw new PendingException()
}

tagging features

To concentrate on implementing the steps for the NewBook feature we will disable the second one by adding an @ignore tag to it and by telling the plugin to ignore the scenarios tagged with @ignore:

ListBooks.feature:

Feature: list owned books
    As a book owner
    I want to list my books
    so that I can look up which books I own

@ignore
Scenario: list existing books
   Given I have already added "Specification by Example"
    When I view the book list
    Then my book list contains "Specification by Example"

The plugin will pick up a couple of configuration options from grails-app/conf/CucumberConfig.groovy, so we create it and add a tags configuration like this:

cucumber {
    tags = ["~@ignore"]
}

tags is list of strings and each item corresponds to a standard cucumber --tags option.

Running grails test-app :cucumber again, we will only get the missing steps message for the NewBook feature:

You can implement missing steps with the snippets below:

Given(~"^I open the book tracker$") { ->
    // Express the Regexp above with the code you wish you had
    throw new PendingException()
}
When(~"^I add \"([^\"]*)\"$") { String arg1 ->
    // Express the Regexp above with the code you wish you had
    throw new PendingException()
}
Then(~"^I see \"([^\"]*)\"s details$") { String arg1 ->
    // Express the Regexp above with the code you wish you had
    throw new PendingException()
}

implementing steps, part one

Create a new file test/functional/steps/Book_steps.groovy and

  • copy the step templates into it
  • add an import for PendingException
  • mixin the EN language of Gherkin
  • escape the $ at the end of the regular expression because in groovy it is a special character in GStrings. Alternativly you can replace the double quotes with single quotes.

Finally it should look like this:

import cucumber.runtime.PendingException

this.metaClass.mixin (cucumber.runtime.groovy.EN)


Given (~"^I open the book tracker\$") { ->
    // Express the Regexp above with the code you wish you had
    throw new PendingException()
}

When (~"^I add \"([^\"]*)\"\$") { String arg1 ->
    // Express the Regexp above with the code you wish you had
    throw new PendingException()
}

Then (~"^I see \"([^\"]*)\"s details\$") { String arg1 ->
    // Express the Regexp above with the code you wish you had
    throw new PendingException()
}

Running again will throw a PendingException. To implement the steps we will need a domain class, a service and a controller. That is what we will do next, step by step…

Given (~‘^I open the book tracker$’)

This step is easy. We want to test under the skin ignoring the ui, so we simply do nothing in this step. In a real project we would probably check that the controller has an action that returns the entry page of our application.

Given (~"^I open the book tracker\$") { ->
    // nop
}

When (~‘^I add “([^"]*)”$’)

In this step we want to add a book. We need a controller with an add action that we will call in the step implementation.

Create a BookController with an add action method. It will get the book details we enter in the ui form and it will return the new Book as JSON that will be shown in the ui. We don’t care what the ui does with it, we will only check that we get the correct JSON.

As is usual in grails the controller will just delegate the work to a service and prepare the result for the ui. In this case rendering the Book as JSON. Here is the controller code:

package books

import grails.converters.JSON

class BookController {
    def bookService

    def add () {
        render bookService.add (params) as JSON
    }
}

The service:

package books

class BookService {
    Book add (Map params) {
        def newBook = new Book (params)
        newBook.save ()
        newBook
    }
}

And finally the Book domain class:

package books

class Book {
    String author
    String title
}

After creating the grails artifacts we can finally implement the step like this:

import books.BookController
import data.Data

// Scenario State
BookController bookController

When (~"^I add \"([^\"]*)\"\$") { String bookTitle ->
    bookController = new BookController ()
    bookController.params << Data.findByTitle (bookTitle)
    bookController.add ()
}

Note the additional imports and the bookController variable we will need to call the controller we create in the Then step.

If you are familiar with Grails you will probably understand most of it. We are simply using grails standard integration test logic for controllers.

Data is a small helper class that will lookup all the properties of a book identified by the bookTitle. This has the advantage that we do not have to list all properties of a book in the .feature file and the feature is more robust against changes like adding an additional property. That the additional property gets handled properly would be tested on the integration or unit test level.

Here is the source for test/functional/data/Data.groovy:

package data

import books.Book


class Data {
    static def books = [
        [title: "Specification by Example", author: "Gojko Adzic"]
    ]

    static public def findByTitle (String title) {
        books.find { book ->
            book.title == title
        }
    }

    static void clearBooks () {
        Book.findAll()*.delete (flush: true)
    }
}

Running the features now will still fail with a rather obscure java.lang.IllegalStateException exception (I guess at least for most of us, count me in). We are calling the controller outside of a real http request and to fix this we have to add a few simple lines of setup and tear down code.

Before & After

If we are building normal integration tests for a controller, grails will take care of setting up the test environment in a way that we will not get the IllegalStateException, that we have a mock request and response etc. and that we can set the request parameters with bookController.params = ... .

When we run cucumber we have to take care of this ourself. Let’s create a new file test/functional/hooks/env.grooy with the following content:

import org.codehaus.groovy.grails.test.support.GrailsTestRequestEnvironmentInterceptor

this.metaClass.mixin (cucumber.runtime.groovy.Hooks)


GrailsTestRequestEnvironmentInterceptor scenarioInterceptor

Before () {
    scenarioInterceptor = new GrailsTestRequestEnvironmentInterceptor (appCtx)
    scenarioInterceptor.init ()
}

After () {
    scenarioInterceptor.destroy ()
}

Running cucumber now will fail with a PendingException from our last step in this scenario. We still have to check that the result is correct to finish our first scenario.

Then (~‘^I see “([^"]*)”s details$’)

We need to assert that the returned JSON has an id (so we know it was save()d) and author and title with the same values we passed to the BookController:

Then (~"^I see \"([^\"]*)\"s details\$") { String bookTitle ->
    def expected = Data.findByTitle (bookTitle)
    def actual = bookController.response.json

    assert actual.id
    assert actual.title  == expected.title
    assert actual.author == expected.author
}

Next run and we see:

> grails test-app :cucumber --stacktrace
| Server running. Browse to http://localhost:8080/Books
| Running 1 cucumber test...
| Completed 1 cucumber test, 0 failed in 1623ms
| Server stopped
| Tests PASSED - view reports in target/test-reports

Great, the scenario passed! Now let’s finish our example application by implementing the steps for the ListBooks feature. There is a little bit more :-)

implementing steps, part two

First remove the @ignore tag from the list existing books scenario and run it again to get the templates for the missing steps and copy them to the BookSteps.groovy file (don’t forget to escape the ‘$’):

Given(~"^I have already added \"([^\"]*)\"$") { String arg1 ->
    // Express the Regexp above with the code you wish you had
    throw new PendingException()
}
When(~"^I view the book list$") { ->
    // Express the Regexp above with the code you wish you had
    throw new PendingException()
}
Then(~"^my book list contains \"([^\"]*)\"$") { String arg1 ->
    // Express the Regexp above with the code you wish you had
    throw new PendingException()
}

Given (~‘^I have already added “([^"]*)”$’)

We want to pre-populate our database. We will reuse our BookService to create a new Book in the database. Reusing our business code to setup test data looks like a good idea to make sure that the we get proper data into the database without duplicating the code for our tests.

Here is the code:

Given (~"^I have already added \"([^\"]*)\"\$") { String bookTitle ->
    def bookService = appCtx.getBean ("bookService")
    bookService.add (Data.findByTitle (bookTitle))
}

Now that we have two features running and interacting with the database we should also take care of cleaning up the database after each scenario.

more Before & After

Here is the additional code for test/functional/hooks/env.groovy:

import data.Data

Before () {
    ....
    Data.clearBooks ()
}

After () {
    ....
}

You can also add the the cleanup to the After hook. I have choosen to add it to the setup because if a scenario fails we can still look at its database state. Maybe it will help to diagnose the problem.

When (~‘^I view the book list$’)

Not much new here, we have to deliver the full list of books. We get it by adding a new all() action method to the controller that will return the list as JSON.

Here is the step code:

When (~"^I view the book list\$") { ->
    bookController = new BookController ()
    bookController.all ()
}

Here the new methods for the controller:

....    
class BookController {
    ....    
    def all () {
        def books = bookService.all ()
        render books as JSON
    }
}

.. and the service:

class BookService {
    ....
    List<Book> all () {
        Book.findAll ()
    }
}

Running again fails with the now common PendingException in the last step.

Then (~‘^my book list contains “([^"]*)”$’)

We check that the returned list contains the book we have pre-populated to the database in the Given step.

Here is the code:

Then (~"^my book list contains \"([^\"]*)\"\$") { String bookTitle ->
    def expected = Data.findByTitle (bookTitle)
    def all = bookController.response.json
    actual = all.getJSONObject (0)

    assert actual.id
    assert actual.title  == expected.title
    assert actual.author == expected.author
}

It is nearly the same as the previous Then check in the first feature, we just have to extract the object from the list first. We should probably refactor the assertions to a method and use that in both Then steps. We will keep that as an exercise for you ;-)

Running the test will report two passed cucumber features:

> grails test-app :cucumber --stacktrace
| Server running. Browse to http://localhost:8080/Books
| Running 2 cucumber tests...
| Completed 2 cucumber tests, 0 failed in 1612ms
| Server stopped
| Tests PASSED - view reports in target/test-reports

In case you have not noticed, we are done! ;-)

Summary

Congratulations, you have succesfully implemented two features for our example application! :-)

We have implemented a couple of cucumber steps using a grails domain class, a service, a controller and a few lines of code for the before and after hooks. We have seen were to put the feature files and the step implementations and how we can configure @tags.

This should cover the basics to get you started with cucumber and grails. :-)

Happy Cuking!