iOS: Setting Up Acceptance Testing

After setting up Jenkins/Hudson to build and run my unit tests and collecting code coverage in the last article I will give some guidance for setting up acceptance testing with Cucumber based on a new test framework called Frank.

This article will not make much sense if you do not know cucumber. To learn about cucumber check out the cucumber home page here or take a look at the Secret Ninja Cucumber Scrolls.

Frank embeds a http server into the test version of our app which will be run when the app gets started. Via this internal http server the cucumber part of frank will send commands to the app to remote control it (taps, button taps, filling texts and so on) or to look for ui elements I want to see on specific views.

The first step is create a “frankified” version of the app for testing by modifiying a copy of the iOS Apps target configuration in XCode:

• main gets replaced by a version that start the http server before it creates the usual UIApplicationMain
• it links to the Frank library
• and it includes a Frank resource bundle used by the http server

That is described in more detail in the Frank documentation so I will not repeat it here. After having a “frankified” app we can start writing cucumber scenarios.

Frank already comes with a number of usable cucumber steps, making it easier to get started and are also useful as a template for private steps.

Here is a list of the built-in steps:

Then /^I wait to see "([^\"]*)"$/
Then /^I wait to not see "([^\"]*)"$/
Then /^I wait to see a navigation bar titled "([^\"]*)"$/
Then /^I wait to not see a navigation bar titled "([^\"]*)"$/
Then /^I should see a "([^\"]*)" button$/
Then /^I should see "([^\"]*)"$/
Then /^I should not see "([^\"]*)"$/
Then /^I should see the following:/
Then /^I should see a navigation bar titled "([^\"]*)"$/

When /^I type "([^\"]*)" into the "([^\"]*)" text field$/
When /^I fill in "([^\"]*)" with "([^\"]*)"$/
When /^I fill in text fields as follows:$/

And /^I rotate to the "([^\"]*)"$/
When /^I touch "([^\"]*)"$/
Then /I touch the following:/
When /^I touch the button marked "([^\"]*)"$/
When /^I touch the "([^\"]*)" action sheet button$/
When /^I touch the (\d*)(?:st|nd|rd|th)? action sheet button$/

When /^I flip switch "([^\"]*)"$/
Then /^switch "([^\"]*)" should be on$/
Then /^switch "([^\"]*)" should be off$/

Then /^a pop\-over menu is displayed with the following:$/
Then /^I navigate back$/

Having quite a number of example steps it is not very difficult to create additional steps. All the built-in steps are just a few lines and easy to understand.

For example I created a step that looks like this:

Then /^I should see text fields for:$/ do |table|
  table.raw.each do |v|
    check_element_exists(%Q|textField accessibilityLabel:"#{v}"|);
  end
end

The interesting part is the check_element_exists. It is one of the basic calls used to implement the above steps. It takes care of all the communication with the http server in the “frankified” test app.

By default Frank uses the “Accessibility Label” to locate ui elements in the app using UISpec. The “Accessibility Label”s original use is to give a hint what a control is for to people with visual impairments. Frank uses it to mark ui elements with a “label” that we can use in our cucumber steps to refer to that ui element.

For example the cucumber scenario using my custom step looks like this:

Scenario:
  ask user to login at first startup

  Given the app runs for the first time
  Then I should be on the "Login" screen
  And I should see text fields for:
    | Login    |
    | Password |

The “And” step is using my custom step and it (i.e. UISpec triggered by a call to the embedded http server) will look for two text fields that are marked with the accessibility labels “Login” and “Password”.

I set the labels from Interface Builder by opening the Inspector, switching to the View Identity tab (it is the last tab) and setting the “Label” under the “Accessibility” options.

Note:
To make this work in the simulator I first had to enable “Accessibility” feature in MacOSX, i.e. enable “Enable access for assistive devices” in the “Universal Access” system preferences.

Really cool is, that the parameter to check_element_exists is a UISpec script expression.

This make it possible to check for other properties than the accessibilityLabel of an ui element. I can easily check that the “tag” property (it is on the View Attributes tab in the Interface Builder inspector) of an ui element has a specific value using the following expression:

check_element_exists("view tag:1")

Check out UISpec/UIScript to learn more about this.

To run the cucumber scenarios for my app I simply run the cucumber command from bash and watch frank running trough my acceptance tests.

Currently you will have to open XCode as frank uses apple script to run the app in the simulator from XCode. That is not the perfect way to run the cucumber test automatically from Jenkins/Hudson.

Summary: I have reached my next goal, running Cucumber acceptance tests against my iOS app using Frank. Apart from enabling the accessibility stuff, there were no complications to get it running.

In the next article iOS: Running Cucumber/Frank with Code Coverage in Jenkins/Hudson I will describe how I managed to run my “frankified” app headless from Jenkins/Hudson by using just a view lines of additional ruby code.

This is the 5th article of iOS: Setting Up a Test Environment is a Mess.