Writing unit tests is easy in Unison. You add your tests as specialwatch expressionsin yourscratch file,then add them to the codebase using theadd
orupdate
commands, then use thetest
to run the tests.
Note that unit tests can't access anyabilitiesthat would cause the test to give different results each time it's run. This means Unison can cache test results, which is completely safe to do. When you issue thetest
commandany tests that have been run before will simply report the cached results.
Basic unit tests
Okay, let's get started! Here's a simple test:
square : Nat -> Nat
square x = x * x
test> square.tests.ex1 = check (square 4 == 16)
Thetest>
line is called a "test watch expression." Like otherwatch expressionsin your scratch file, it will get evaluated (or looked up in the evaluation cache) on every file save. By convention, tests for a definition calledsquare
are placed in thesquare.tests
namespace.
By the way, you can write test watch expressions that span multiple lines. For instance, here's a test forList.reverse
:
use universal
test> List.reverse.tests.ex1 = check let
actual = reverse [1,2,3,4]
expected = [4,3,2,1]
expected === actual
Adding diagnostics for a failing test
When you have a test that's failing, like this one below, you often want to print out some information before it fails:
use universal
brokenReverse : [a] -> [a]
brokenReverse as = []
test> brokenReverse.tests.ex1 = check let
actual = brokenReverse [1,2,3,4]
expected = [4,3,2,1]
expected === actual
Notice we don't get any information about why it failed. Let's go ahead and fix that, by temporarily inserting a call to the functionbase.bug : a -> b
,which halts your program with an exception and prints out its argument, nicely formatted onto multiple lines if needed:
use universal
brokenReverse : [a] -> [a]
brokenReverse as = []
test> brokenReverse.tests.ex1 = check let
actual = brokenReverse [1,2,3,4]
expected = [4,3,2,1]
if not (expected === actual) then
bug ("Not equal!", expected, actual)
else
true
Here we're passing a tuple to thebase.bug
function, but we could pass it any value at all.
Adding tests to the codebase
Once you're happy with your tests, you can add them to the codebase and use thetest
command to see the test results in the current namespace. You can see test results for larger or smaller scopes just by moving around with thecd
command:
square : Nat -> Nat
square x = x * x
test> square.tests.ex1 = check (square 4 == 16)
test> List.reverse.tests.ex1 = check let
actual = reverse [1,2,3,4]
expected = [4,3,2,1]
expected == actual
.mystuff> add
.mystuff> test
But actually, it didn't need to run anything! All the tests had been run previously and cached according to their Unison hash. In a purely functional language like Unison, tests like these aredeterministicand can be cached and never run again. No more running the same tests over and over again!
Generating test cases with code
Unison's base library contains powerful utility functions for generating test cases with Unison code, which lets your tests cover a lot more cases than if you are writing test cases manually like we've done so far. (This style of testing is often calledproperty-based testing.)
The property-based testing support in Unison relies onan abilitycalledGen
(short for "generator"). If you haven't read aboutabilitiesyet, we suggest taking a detour to do so before continuing.
For instance, a'{Gen} Nat
is a computation that, when run, producesNat
values. You can sample from a'{Gen} a
multiple times to produce different values. Importantly, these arenot randomvalues. The sequence of values generated is entirely deterministic:
> sample 100 (natIn 0 10)
test> myTest = runs 100 '(expect (!(natIn 0 10) < 10))
When developing generators to use for testing, you'll often put those generators in a watch expression like this to make sure you understand what they are generating. Generators denote a set of values, and as the above shows, it is possible to exhaustively enumerate that set, at which pointGen.sample
will stop short. Above we asked it to generate 100 natural numbers in the range 0 to 10, but there's only 10 unique values, so it stops after that.
Combining generators
Where things get interesting is whencombininggenerators. There are a few ways of doing that. For a'{Gen} a
,you can use the!
operator to sample from it, and you can sample from multiple generators to build up a more complex generator. Let's have a look at an interesting example, which highlights something important about these generators:
As we can see,Gen
doesfairor "breadth-first" sampling from both of the generators involved, rather than exhausting one before moving on to the next.
Doing a breadth-first enumeration is the right move because as we build up more complex generators, where the space of possibilities is often so huge that it's only possible to sample a tiny fraction of it.
There are two other ways of combining generators. One islib.base.test.gen.pick
,which fairly samples from multiple generators in a breadth-first manner:
lib.base.test.gen.pick : ['{Gen} a] -> '{Gen} a
Here's an example:
The other iscost : Nat -> '{Gen} a -> '{Gen} a
,which assigns a "cost" to a generator. What does that mean? When a branch oflib.base.test.gen.pick
has a cost of5
for instance, the sampling process will take 5 samples from allotherbranches before switching to fairly sampling from both branches:
You may want to do some experimentation to get a feel for howGen
behaves. You can use thecost
to control which branches of the space of possibilities get explored first—a common use case will be to assign higher costs to "large" test cases that you wish to test less frequently.
Using generators to write property based tests
Once you've got your generators in good shape, you can combine these into property-based tests that verify some property forallgenerated test cases. For example, let's check that reversing a list twice gives back the original list:
test> List.reverse.tests.prop1 = runs 100 'let
original = !(listOf (natIn 0 100))
original' = List.reverse (List.reverse original)
expect (original === original')
> sample 10 (listOf (natIn 0 100))
Don't forget, if you encounter failures, you can usebase.bug
to view intermediate generated values that trigger the failure.
Because the test results are always deterministic and cached, you may want to crank up the number of samples taken before choosing toaddyour tests to the codebase.
.mystuff> add
.mystuff> test
Notice that all the test results are cached. If you laterupdate
the definitions being tested (likeList.reverse
in this example), the tests won't be found in the cache and will get re-run when you type thetest
command.
Other useful functions when writing property-based tests
The functionlib.base.test.runs
that we've been using has typelib.base.test.runs : Nat -> '{e, Gen} Test ->{e} [test.Result]
.To form a value of typeTest
,you can use the functionsexpect
(which we've seen) as well asok
,fail
,and others which we can locate usingtype-based search:
.base> find : Test
These functions are used to give different messages on success or failure. Feel free to try them out in your tests, and you may want to explore other functions in the testing package.
Lastly, thebase library is open for contributionsif you come up with some handy testing utility functions, or want to contribute better documentation (or tests) for existing definitions.
Thanks for reading!