Testing your Unison code

Writing unit tests is easy in Unison. You add your tests as specialwatch expressionsin yourscratch file,then add them to the codebase using theaddorupdatecommands, then use thetestto run the tests.

Note that unit tests can't access anyabilitiesthat would cause the test to give different results each time it's run. This means Unison can cache test results, which is completely safe to do. When you issue thetestcommandany tests that have been run before will simply report the cached results.

Basic unit tests

Okay, let's get started! Here's a simple test:

square : Nat -> Nat
square x = x * x

test> square.tests.ex1 = check (square 4 == 16)

Thetest>line is called a "test watch expression." Like otherwatch expressionsin your scratch file, it will get evaluated (or looked up in the evaluation cache) on every file save. By convention, tests for a definition calledsquareare placed in thesquare.testsnamespace.

🤓
Thecheckfunction has typeBoolean -> [Test.Result].Any test watch expression must have the type[Test.Result].Though we won't use this capability much, it's possible to have a single test produce multiple results, hence the[Test.Result]rather thanTest.Result.If you decide to write a different testing library, you just have to be able to produce a[Test.Result]in the end.

By the way, you can write test watch expressions that span multiple lines. For instance, here's a test forList.reverse:

use universal
test> List.reverse.tests.ex1 = check let
  actual = reverse [1,2,3,4]
  expected = [4,3,2,1]
  expected === actual
📕
Asdiscussed in the language reference,keyword-based constructs likeletbind tighter than function application, so you don't need any parentheses around theletblock which is used as the argument tocheck.

Adding diagnostics for a failing test

When you have a test that's failing, like this one below, you often want to print out some information before it fails:

use universal

brokenReverse : [a] -> [a]
brokenReverse as = []

test> brokenReverse.tests.ex1 = check let
  actual = brokenReverse [1,2,3,4]
  expected = [4,3,2,1]
  expected === actual

Notice we don't get any information about why it failed. Let's go ahead and fix that, by temporarily inserting a call to the functionbase.bug : a -> b,which halts your program with an exception and prints out its argument, nicely formatted onto multiple lines if needed:

use universal

brokenReverse : [a] -> [a]
brokenReverse as = []

test> brokenReverse.tests.ex1 = check let
  actual = brokenReverse [1,2,3,4]
  expected = [4,3,2,1]
  if not (expected === actual) then
    bug ("Not equal!", expected, actual)
  else
    true

Here we're passing a tuple to thebase.bugfunction, but we could pass it any value at all.

Adding tests to the codebase

Once you're happy with your tests, you can add them to the codebase and use thetestcommand to see the test results in the current namespace. You can see test results for larger or smaller scopes just by moving around withthecdcommand:

square : Nat -> Nat
square x = x * x

test> square.tests.ex1 = check (square 4 == 16)

test> List.reverse.tests.ex1 = check let
  actual = reverse [1,2,3,4]
  expected = [4,3,2,1]
  expected == actual
.mystuff> add
.mystuff> test

But actually, it didn't need to run anything! All the tests had been run previously and cached according to their Unison hash. In a purely functional language like Unison, tests like these aredeterministicand can be cached and never run again. No more running the same tests over and over again!

Generating test cases with code

Unison's base library contains powerful utility functions for generating test cases with Unison code, which lets your tests cover a lot more cases than if you are writing test cases manually like we've done so far. (This style of testing is often calledproperty-based testing.)

The property-based testing support in Unison relies onan abilitycalledGen(short for "generator"). If you haven't read aboutabilitiesyet, we suggest taking a detour to do so before continuing.

For instance, a'{Gen} Natis a computation that, when run, producesNatvalues. You can sample from a'{Gen} amultiple times to produce different values. Importantly, these arenot randomvalues. The sequence of values generated is entirely deterministic:

> sample 100 (natIn 0 10)

test> myTest = runs 100 '(expect (!(natIn 0 10) < 10))

When developing generators to use for testing, you'll often put those generators in a watch expression like this to make sure you understand what they are generating. Generators denote a set of values, and as the above shows, it is possible to exhaustively enumerate that set, at which pointGen.samplewill stop short. Above we asked it to generate 100 natural numbers in the range 0 to 10, but there's only 10 unique values, so it stops after that.

Combining generators

Where things get interesting is whencombininggenerators. There are a few ways of doing that. For a'{Gen} a,you can use the!operator to sample from it, and you can sample from multiple generators to build up a more complex generator. Let's have a look at an interesting example, which highlights something important about these generators:

test.sample 10 do
  use gen natIn
  n =
    use Nat +
    !(natIn 0 10) + 100
  m = !(natIn 0 100)
  (n, m)
⧨
[ (100, 0),
  (100, 1),
  (101, 0),
  (100, 2),
  (101, 1),
  (102, 0),
  (100, 3),
  (101, 2),
  (102, 1),
  (103, 0) ]
🎨
Syntax note:Thedokeyword (sometimes expressedas'let)is a common idiom to introduce a delayed computation which is a block. The precedence is such thatsample 10 do …followed by a newline is parsed likesample 10 (do …).

As we can see,Gendoesfairor "breadth-first" sampling from both of the generators involved, rather than exhausting one before moving on to the next.

Doing a breadth-first enumeration is the right move because as we build up more complex generators, where the space of possibilities is often so huge that it's only possible to sample a tiny fraction of it.

There are two other ways of combining generators. One isstubs.lib.base.test.gen.pick,which fairly samples from multiple generators in a breadth-first manner:

stubs.lib.base.test.gen.pick : ['{Gen} a] -> '{Gen} a

Here's an example:

test.sample 10
  <| stubs.lib.base.test.gen.pick [gen.natIn 0 10, gen.natIn 100 200]
⧨
[0, 100, 1, 101, 2, 102, 3, 103, 4, 104]

The other iscost : Nat -> '{Gen} a -> '{Gen} a,which assigns a "cost" to a generator. What does that mean? When a branch ofstubs.lib.base.test.gen.pickhas a cost of5for instance, the sampling process will take 5 samples from allotherbranches before switching to fairly sampling from both branches:

test.sample 10
  <| stubs.lib.base.test.gen.pick
       [cost 5 (gen.natIn 0 10), gen.natIn 100 200]
⧨
[100, 101, 102, 103, 104, 0, 105, 1, 106, 2]

You may want to do some experimentation to get a feel for howGenbehaves. You can use thecostto control which branches of the space of possibilities get explored first⁠—a common use case will be to assign higher costs to "large" test cases that you wish to test less frequently.

Using generators to write property based tests

Once you've got your generators in good shape, you can combine these into property-based tests that verify some property forallgenerated test cases. For example, let's check that reversing a list twice gives back the original list:

test> List.reverse.tests.prop1 = runs 100 'let
  original = !(listOf (natIn 0 100))
  original' = List.reverse (List.reverse original)
  expect (original === original')

> sample 10 (listOf (natIn 0 100))

Don't forget, if you encounter failures, you can usebase.bugto view intermediate generated values that trigger the failure.

Because the test results are always deterministic and cached, you may want to crank up the number of samples taken before choosing toaddyour tests to the codebase.

.mystuff> add
.mystuff> test

Notice that all the test results are cached. If you laterupdatethe definitions being tested (likeList.reversein this example), the tests won't be found in the cache and will get re-run when you type thetestcommand.

Other useful functions when writing property-based tests

The functionstubs.lib.base.test.runsthat we've been using has typestubs.lib.base.test.runs : Nat -> '{e, Gen} Test ->{e} [test.Result].To form a value of typeTest,you can use the functionsexpect(which we've seen) as well asok,fail,and others which we can locate usingtype-based search:

.base> find : Test

These functions are used to give different messages on success or failure. Feel free to try them out in your tests, and you may want to explore other functions in the testing package.

Lastly, thebase library is open for contributionsif you come up with some handy testing utility functions, or want to contribute better documentation (or tests) for existing definitions.

Thanks for reading!