I've written a small framework based on python's built in unit testing module.
I've included the source code here www.dropbox.com/.../src.zip
I'll put it on github at some point but I figured I should just put it out there for the moment and see if anyone has any feedback.
This is an example of a testbench to test two functions.
(qtest::runSuites
(qtest::TestSuite ((f qub:::join_lists))
(qtest::TestCase join_three_lists
(qtest::assertEqual (list "X" 2 3 4 5 6)
(f (list (list 1 2) (list 3 4) (list 5 6))))))
(qtest::TestSuite ((f qub::flatten_list))
(qtest::TestCase normal_use
(qtest::assertEqual (list 1 2 3 4 5 6 7)
(f (list 1 2 (list 3 (list 4 (list 5 6) 7))))))))
When the file is loaded, it would have the following output (I've intentionally edited the first one to fail):
(load "./code/src/std/test_lists.ils")
1 of 2 tests passed (1 failures) (0 skipped) (0 expected failures)
Test: join_three_lists
Result: Fail
Message: No msg.
Inputs: (list("X" 2 3 4 5 6) (f list(list(1 2) list(3 4) list(5 6))))
Evaluated Inputs: (("X" 2 3 4 5 6) (1 2 3 4 5 6))
Each test is represented by the TestCase macro. It creates a list containing the test's name and the test function object. This prevents the test from polluting the global namespace. Using a keyword arg you can also mark tests to be skipped or that you expect them to fail. Each test case must return the output of an assertion function.
The test cases are grouped up into suites which are typically used to evaluate a single object (usually a function). A suite is just a list to group the tests. Each suite is put into the body of a runSuites macro which flattens the the suites into a single list and runs them when the module is loaded. I've also included a function to recursively find and load all modules beginning with a test_ prefix.
The unit tests themselves have an automated test system to make sure that they're working. There are probably a few bugs hiding in there but it works for most common cases.
The code uses two namespaces, qtest and qub. It was written in IC 6.1.6.
If you've any suggestions I'd like to hear them.