FixMyStreet has a large and hopefully comprehensive test suite that runs through all aspects of the codebase, checking everything is working. This makes it easier to change code and add new features, safe in the knowledge that any breakages will be quickly highlighted.

Speeding up the tests

Every time someone commits code to our GitHub repository, or opens a pull request, the tests are automatically run for us by Travis CI. We’re alerted to success or failure with little green ticks or red crosses on GitHub, and by notice in IRC.

The tests seemed to have slowed down considerably in recent times, but we couldn’t identify any changes at the FixMyStreet side which might have caused this.

However, there had recently been some spam scraping of Gaze, our web service that provides population density information to FixMyStreet (so that e.g. the maps can try and guess an appropriate zoom level, and so alerts can try and guess an appropriate radius), and rate limiting had been added to try and help combat it.

Dave spotted that this was being triggered by FixMyStreet test runs, leading to pauses as the suite waited for the rate limiting to ease. Thankfully, all Gaze calls were being routed through one function (that had been created in order to cope gracefully with a Gaze failure) and so it was a simple matter for this function to be stubbed out if being run as part of a test.

Before:

After:

There are many tests that still rely on the internet (e.g. for some MapIt lookups) and eventually it would be good to get to the point where they are all stubbed out and the test suite can run completely offline, probably even more quickly.

Multiple test running

When running the tests, the suite creates a test database (in PostgreSQL terms, it actually creates a temporary cluster) so that anything it does won’t affect your development database. Theoretically, this means you should be able to run the test suite multiple times simultaneously – perhaps it’s doing a full run, but you want to try and fix (and retest) the first error while it carries on. However, this was not working, and after some investigation it turned out that each run was creating (and overwriting) a test configuration .yml file, which meant the existing runs got all confused. Adding a process ID to the test configuration file meant that each run is independent and can successfully coexist with each other.

Keystroke saving

Lastly, you used to have to run the full suite with bin/run-tests t, but now if you run bin/run-tests, it will assume you meant t. A small thing, but it might save a few seconds over the years. ;-)

---

If you have any questions, or problems installing the code, please do get in touch, or post on our mailing list.