(= (+ -1 -3) -4)))
that is equivalent to the following code:
(defun test-+ ()
(progn
(report-result (= (+ 1 2) 3) '(= (+ 1 2) 3))
(report-result (= (+ 1 2 3) 6) '(= (+ 1 2 3) 6))
(report-result (= (+ -1 -3) -4) '(= (+ -1 -3) -4))))
Thanks to check, this version is as concise as the first version of test-+ but expands into code that does the same thing as the second version. And now any changes you want to make to how test-+ behaves, you can make by changing check.
You can start with fixing test-+ so its return value indicates whether all the test cases passed. Since check is responsible for generating the code that ultimately runs the test cases, you just need to change it to generate code that also keeps track of the results.
As a first step, you can make a small change to report-result so it returns the result of the test case it's reporting.
(defun report-result (result form)
(format t '~:[FAIL~;pass~] ... ~a~%' result form)
result)
Now that report-result returns the result of its test case, it might seem you could just change the PROGN to an AND to combine the results. Unfortunately, AND doesn't do quite what you want in this case because of its short-circuiting behavior: as soon as one test case fails, AND will skip the rest. On the other hand, if you had a construct that worked like AND without the short-circuiting, you could use it in the place of PROGN, and you'd be done. Common Lisp doesn't provide such a construct, but that's no reason you can't use it: it's a trivial matter to write a macro to provide it yourself.
Leaving test cases aside for a moment, what you want is a macro—let's call it combine- results—that will let you say this:
(combine-results
(foo)
(bar)
(baz))
and have it mean something like this:
(let ((result t))
(unless (foo) (setf result nil))
(unless (bar) (setf result nil))
(unless (baz) (setf result nil))
result)
The only tricky bit to writing this macro is that you need to introduce a variable—result in the previous code—in the expansion. As you saw in the previous chapter, using a literal name for variables in macro expansions can introduce a leak in your macro abstraction, so you'll need to create a unique name. This is a job for with-gensyms. You can define combine-results like this:
(defmacro combine-results (&body forms)
(with-gensyms (result)
`(let ((,result t))
,@(loop for f in forms collect `(unless ,f (setf ,result nil)))
,result)))
Now you can fix check by simply changing the expansion to use combine- results instead of PROGN.
(defmacro check (&body forms)
`(combine-results
,@(loop for f in forms collect `(report-result ,f ',f))))
With that version of check, test-+ should emit the results of its three test expressions and then return T to indicate that everything passed.[103]
CL-USER> (test-+)
pass ... (= (+ 1 2) 3)
pass ... (= (+ 1 2 3) 6)
pass ... (= (+ -1 -3) -4)
T
And if you change one of the test cases so it fails,[104] the final return value changes to NIL.
CL-USER> (test-+)
pass ... (= (+ 1 2) 3)
pass ... (= (+ 1 2 3) 6)
FAIL ... (= (+ -1 -3) -5)
NIL
As long as you have only one test function, the current result reporting is pretty clear. If a particular test case fails, all you have to do is find the test case in the check form and figure out why it's failing. But if you write a lot of tests, you'll probably want to organize them somehow, rather than shoving them all into one function. For instance, suppose you wanted to add some test cases for the * function. You might write a new test function.
(defun test-* ()
(check
(= (* 2 2) 4)
(= (* 3 5) 15)))
Now that you have two test functions, you'll probably want another function that runs all the tests. That's easy enough.
(defun test-arithmetic ()
(combine-results
(test-+)
(test-*)))
In this function you use combine-results instead of check since both test-+ and test-* will take care of reporting their own results. When you run test-arithmetic, you'll get the following results:
CL-USER> (test-arithmetic)
pass ... (= (+ 1 2) 3)
pass ... (= (+ 1 2 3) 6)
pass ... (= (+ -1 -3) -4)
