Practical Common Lisp

test case has to boil down to a boolean expression, thumbs up or thumbs down.

Two First Tries

If you were doing ad hoc testing, you could enter these expressions at the REPL and check that they return T. But you want a framework that makes it easy to organize and run these test cases whenever you want. If you want to start with the simplest thing that could possibly work, you can just write a function that evaluates the test cases and ANDs the results together.

(defun test-+ ()
  (and
    (= (+ 1 2) 3)
    (= (+ 1 2 3) 6)
    (= (+ -1 -3) -4)))

Whenever you want to run this set of test cases, you can call test-+.

CL-USER> (test-+)
T

As long as it returns T, you know the test cases are passing. This way of organizing tests is also pleasantly concise—you don't have to write a bunch of test bookkeeping code. However, as you'll discover the first time a test case fails, the result reporting leaves something to be desired. When test-+ returns NIL, you'll know something failed, but you'll have no idea which test case it was.

So let's try another simple—even simpleminded—approach. To find out what happens to each test case, you could write something like this:

(defun test-+ ()
  (format t '~:[FAIL~;pass~] ... ~a~%' (= (+ 1 2) 3) '(= (+ 1 2) 3))
  (format t '~:[FAIL~;pass~] ... ~a~%' (= (+ 1 2 3) 6) '(= (+ 1 2 3) 6))
  (format t '~:[FAIL~;pass~] ... ~a~%' (= (+ -1 -3) -4) '(= (+ -1 -3) -4)))

Now each test case will be reported individually. The ~:[FAIL~;pass~] part of the FORMAT directive causes FORMAT to print 'FAIL' if the first format argument is false and 'pass' otherwise.[102] Then you label the result with the test expression itself. Now running test-+ shows you exactly what's going on.

CL-USER> (test-+)
pass ... (= (+ 1 2) 3)
pass ... (= (+ 1 2 3) 6)
pass ... (= (+ -1 -3) -4)
NIL

This time the result reporting is more like what you want, but the code itself is pretty gross. The repeated calls to FORMAT as well as the tedious duplication of the test expression cry out to be refactored. The duplication of the test expression is particularly grating because if you mistype it, the test results will be mislabeled.

Another problem is that you don't get a single indicator whether all the test cases passed. It's easy enough, with only three test cases, to scan the output looking for 'FAIL'; however, when you have hundreds of test cases, it'll be more of a hassle.

Refactoring

What you'd really like is a way to write test functions as streamlined as the first test-+ that return a single T or NIL value but that also report on the results of individual test cases like the second version. Since the second version is close to what you want in terms of functionality, your best bet is to see if you can factor out some of the annoying duplication.

The simplest way to get rid of the repeated similar calls to FORMAT is to create a new function.

(defun report-result (result form)
  (format t '~:[FAIL~;pass~] ... ~a~%' result form))

Now you can write test-+ with calls to report-result instead of FORMAT. It's not a huge improvement, but at least now if you decide to change the way you report results, there's only one place you have to change.

(defun test-+ ()
  (report-result (= (+ 1 2) 3) '(= (+ 1 2) 3))
  (report-result (= (+ 1 2 3) 6) '(= (+ 1 2 3) 6))
  (report-result (= (+ -1 -3) -4) '(= (+ -1 -3) -4)))

Next you need to get rid of the duplication of the test case expression, with its attendant risk of mislabeling of results. What you'd really like is to be able to treat the expression as both code (to get the result) and data (to use as the label). Whenever you want to treat code as data, that's a sure sign you need a macro. Or, to look at it another way, what you need is a way to automate writing the error-prone report-result calls. You'd like to be able to say something like this:

(check (= (+ 1 2) 3))

and have it mean the following:

(report-result (= (+ 1 2) 3) '(= (+ 1 2) 3))

Writing a macro to do this translation is trivial.

(defmacro check (form)
  `(report-result ,form ',form))

Now you can change test-+ to use check.

(defun test-+ ()
  (check (= (+ 1 2) 3))
  (check (= (+ 1 2 3) 6))
  (check (= (+ -1 -3) -4)))

Since you're on the hunt for duplication, why not get rid of those repeated calls to check? You can define check to take an arbitrary number of forms and wrap them each in a call to report-result.

(defmacro check (&body forms)
  `(progn
     ,@(loop for f in forms collect `(report-result ,f ',f))))

This definition uses a common macro idiom of wrapping a PROGN around a series of forms in order to turn them into a single form. Notice also how you can use ,@ to splice in the result of an expression that returns a list of expressions that are themselves generated with a backquote template.

With the new version of check you can write a new version of test-+ like this:

(defun test-+ ()
  (check
    (= (+ 1 2) 3)
    (= (+ 1 2 3) 6)