Testing

In the pyblosxom git repository, there are two big things that have test suites:

1. the PyBlosxom core code
2. the plugins that are in plugins/

These have two separate testing infrastructures, but both are based on unittest which comes with Python.

PyBlosxom core code testing

These tests are located in Pyblosxom/tests/ and test the PyBlosxom core functionality.

Tests are executed by:

python setup.py test

This uses the test_suite parameter to setup which is in both distribute and setuptools.

The test_suite is Pyblosxom.tests.testrunner.test_suite which is a function that goes through all the files in Pyblosxom/tests/ and loads tests in files where the filename starts with test_ and ends in .py.

There’s a helper module which has a UnitTestBase derived from unittest.TestCase which has some helper functions for building a blog setup, tearing it down, building a Request object, and comparing two dicts where one must be a subset the other.

Writing tests for the core is pretty easy:

1. create a file in Pyblosxom/tests/ with a filename that starts with test_ and ends with .py.
2. import UnitTestBase from Pyblosxom.tests.helpers
3. create a subclass of UnitTestBase
4. write some tests using pretty standard unittest stuff

There are a bunch of existing examples in the directory.

Testing plugins

Plugins are a lot harder because we want to make sure they remain modular and easy to install. “Easy to install” really means that a plugin is a single monolithic file. I didn’t want to have a ton of testing code in the modules, so I wrote the test harness to allow for plugins to have tests in the plugin as well as tests in a parallel tests directory.

To run all the plugin tests, I do something like this:

PYTHONPATH=. python plugins/testrunner.py

Tests in the plugin file

Testing inside a plugin is pretty limited because you can’t take advantage of the helper infrastructure–you’re limited to what you can load from Python and PyBlosxom core.

1. import unittest
2. create a class that extends unittest.TestCase
3. write your tests in that
4. create a function at the module level called get_test_suite that returns the tests loaded from the test class(es)

For an example, look at plugins/date/w3cdate.py.

Tests in the parallel directory

Writing tests in the parallel directory has the benefit of being able to use helper infrastructure, but the cost of being separate from the actual plugin. I think this is ok, but it’s something that we have to keep in mind when we make plugin fixes.

Tests are located in plugins/tests/ and are in a parallel directory structure to the plugins in plugins/ .

For example, the test_acronyms.py test file is in plugins/tests/text/test_acronyms.py and the plugin is in plugins/text/acronyms.py.

Test files should import PluginTest from plugins.test.test_base. This class has a bunch of helper functions for dealing with request objects, creating blog data, flavours, ...

1. import PluginTest from plugins.test.test_base
2. create a class that extends PluginTest
3. write your tests in that
4. edit plugins/tests/__init__.py and add a line to import things from your new test module

For examples, see files in plugins/test/.

Summary

I think that’s it. It definitely seems more complicated than it is. Generally speaking, the test infrastructure is ok, but not fantastically amazing. It’ll evolve over time as we need it to do more.

Speaking of doing more, there are parts of PyBlosxom that could use better testing. Help on this front would be fantastic.

Having said that, adding tests for the code we’re writing helps us a TON in the quality department. PyBlosxom 1.5 is ten times as good as previous versions because we’ve got a better testing infrastructure and we’re testing plugin functionality.

Many thanks to Ryan Barret for the awesome work he did a while back. The infrastructure we have now is based largely on his efforts.