Organizing your codebase (deprecated)

Seeprojects codebase organizationfor the latest documentation on organizing your codebase.

Though the Unison namespace tree can be organized however you like, this document suggests handy conventions to keep things tidy even with multiple concurrent workstreams, pull requests, library releases, and external dependencies being added and upgraded. We like conventions that can be followed without much thought and which make things easy, so you can direct your brainpower to actually writing Unison code, not figuring out how to organize it. 😀

We recommend using this exact same set of conventions for library maintainers, library contributors, and application developers. It makes it easy to follow the conventions regardless of which of these roles you are working in, and doesn't require reorganizing anything if you decide you want to publish your personal Unison repository as a library for others to build on.

🏗️
This documentation is still in draft form and these conventions are changing rapidly. Please help test out these conventions and let us know how they work for you. Also any general feedback or questions are welcome! Seethis ticketor start a threadin Slack #troubleshooting.

Here's what a namespace tree will look like that follows these conventions. This will be explained more below, and we'll also show how common workflows (like installing and upgrading libraries and opening pull requests) can be handled with a few UCM commands:

myProject
    main/
      Boolean/
      Nat/
      ...
      README : Doc
      releaseNotes : Doc
      lib/
        dependency1.v7/
        alice.mylib.v94/
        alice.mylib.v87/
        bob.somelib.main/

    latest/
      Boolean/
      Nat/
      README : Doc
      releaseNotes : Doc
      /lib
        dependency1.v7/
        ...

    releases/
      v1/
      v2/
      ...

    series/
      v1/
      v2/

    prs/
      myGreatPR
      anotherNewFeature
      ...
      ...

Directly under the namespace root, we have a project calledmyProjectwith amainnamespace for the project maintainers add their latest feature work in, alatest(sub-)namespace (for the latest stable version), areleasesnamespace containing released versions of main, aseriesnamespace (for releases branches, forked offmain),and aprsnamespace, containing work that will eventually get merged tomain.Additionally, if you contribute PRs to projects that aren't primarily your own, you might have a top-level.prsnamespace which contains copies of the libraries and namespaces for new features.

Its common to have multiple projects under the root of your codebase, representing the various libraries or applications you're writing in Unison.

Upgrading libraries to the latest version

There's no problem with having multiple versions of a library installed at once, though sometimes you want to upgrade to the latest version of a library and delete any references to the old version. There are a couple cases you can encounter here. Ifalice.mylib.v6is a superset ofalice.mylib.v5,this is as simple as just deleting thev5afterv6is installed.

😎 Since references to definitions are by hash, not by name, any renaming of definitions done betweenv5andv6of a library won't break any of your code. Though we recommend reading the release notes to discover these sorts of changes, or usingdiff.namespace somelib.v5 somelib.v6for an overview.

Ifv6replaced any definitions with new ones (using theupdatecommand orreplace.termorreplace.type),these are recorded in apatchwhich can be applied locally.

As a norm,alice.mylib.v6.releaseNoteswill cover how to apply patches to your local namespace. It will typically be commands like:

.myProject> branch main upgradeAliceLib
.myProject> patch external.mylib.v6.patches.v5_v6 upgradeAliceLib
.myProject> merge /upgradeAliceLib /main

Day-to-day development: creating and merging pull requests

Here's the basic workflow for drafting changes tomaininyour own project.It's not much different than a typical Git workflow, where you create branches, hack on them, then merge them tomainwhen done:

  1. myProject/main> branch main myCoolPR
  2. myProject/myCoolPR>Now hack away. Usemerge.preview main myCoolPRat any time to see what you've changed.
  3. myProject/myCoolPR> merge /myCoolPR /mainwhen you're done

To propose changes to another Unison codebase works similarly. We'll usethe Unison base libraryas an example:

  1. .> pull unison.public.base.main _baseyou can do this both for initial fetch and for subsequent updates
  2. .> fork _base .prs.base._mythingto create a copy of_base.You can create multiple forks in.prs.base,if you have multiple things you're working on.
  3. If you haven't already done so, set your default license for.prs.baseto match the license of the codebase you're contributing to. Forbaseit's the MIT license.
  4. Nowcd .prs.base._mythingand hack away as before. At any time review what's changed between your namespace and_basewithdiff.namespace ._base .prs. base._mything.
  5. Push your forked version somewhere public with yourUnison Share account.prs.base._mything> push myUser.public.prs.base._mything.No need to maintain a separate Git repo for every Unison library you want to contribute to.
  6. .prs.base._mything> pull-request.create unison.public.base.latest myUser.public.prs.base._mythingand this will create some output. Copy that output to your clipboard. We don't literally use the GitHub pull request mechanism for Unison repositories, we use GitHub issues instead.
  7. Use the GitHub issue comments for coordinating the review. Once merged, the maintainer will close the issue.
  8. Next, create a GitHub issue in the repo you're submitting the code to (that's right, anissue,nota GitHub PR). Many Unison repositories will havea GitHub issue templatefor this purpose. Make the issue title something descriptive, and for the issue body, paste in the output generated by
    pull-request.create` as well as some text describing the change, just like you would for any other pull request.
    
                                                                                                                                                                                                                                                                                                This workflow also works fine even if the source and destination repository are the same, so you might use the above PR process when suggesting changes to a library  that you maintain with other people.
    
                                                                                                                                                                                                                                                                                                ### Reviewing pull requests
    
                                                                                                                                                                                                                                                                                                We'll use [this issue as an example](https://github.com/unisonweb/base/issues/12). The issue created for the PR will have a
    pull-request.load''
    command to review the PR locally. We'll run that command in any empty namespace:
.review.pr12> pull-request.load unison.public.base.main pchiusano.public.unisoncode.prs.random2

If you.review.pr12> lsyou'll see three or four sub-namespaces:base(the original namespace),head(the new namespace),merged(the result ofmerge head base)and potentiallysquashed(the same content asmergedbut without intermediate history). The following commands can be performed against either themergedorsquashednamespace, depending on if preserving history is important to you:

  1. .review.pr12> diff.namespace base mergedto see what's changed. The numbered entries in the diff can be referenced by subsequent commands, sodiff.namespace base mergedmight be followed byview 1-7to view the first 7 definitions listed there.
  2. You can use comments on the GitHub issue to coordinate if there's any changes you'd like the contributor to make before accepting the PR. You can also make changes directly inmerged.
  3. .review.pr12> push unison.public.base.main mergedto pushmergedtomain
  4. .review.pr12> history mergedand copy the top namespace hash. That's the Unison namespace hash as of the merged PR. Then close the GitHub issue with a comment like "Merged to main in hash #pjdnqduc38" and thank the contributor. 😀 If you ever want to go back in time, you can say.> fork #pjdnqduc38 .pr12to give the#pjdnqduc38namespace hash the name.pr12in your tree.
  5. If you like,.> delete.namespace .review.pr12to tidy up afterwards.

Keeping in sync withmain

Periodically, you canpullthe latestmainusing:

.> pull git(https://gitub.com/unisonorama/myproject) main

If you have in-progress PRs that you want to sync with the latest, you can bring them up to date using.prs._myPR> merge .main.

Using unreleased library versions

Sometimes, you want to use some code that's only inmainof a library but hasn't made its way into a release. No problem. The install process looks the same, just do apull:

.> pull https://github.com/bob/mylib:.main .external.bob.mylib.main

As often as you like, you can re-issue the above command to fetch the latest version of the library. After doing so, you should then apply patches from the library to your local namespace. Check the project's README for information on how to do this, but typically, for a namespace,bob.mylib.main,there will just be a patch calledbob.mylib.main.patchwhich can be applied with:

.> fork main prs.upgradeBob
.> patch external.bob.mylib.main.patch prs.upgradeAliceLib

Assuming all is well after thatpatch,you can.> merge prs.upgradeBob main(and then sync that with any other PRs being drafted, as discussed in the previous section).

If you are feeling adventurous it's also possible to directly apply the patch to your in-progress PRs or evenmain.

How to create a release

Suppose you are creatingv12of a library. The process is basically toforka copy ofmain:

  1. Before getting started, we suggest reviewing the current patch inmainwith.> view.patch main.patch.The term and type replacements listed here should generally be bugfixes or critical upgrades that you expect users of your library to make as well. You can usedelete.term-replacementanddelete.type-replacementto remove any entries you don't want to force on library users. See below for more.
  2. Fork a copy ofmain:.> fork main series._v12
  3. The current dependencies in the release should be included in the library'slibnamespace. This convention ensures that anyone who obtains the library also receives its dependencies and the naming for those definitions at the time.
  4. Create or updateseries._v12.releaseNotes : Doc.You can include the current namespace hash ofseries._v12,a link to previous release notes, likereleases. _v11 releaseNotes,and if the release has a non-emptypatch,give some guidance on upgrading. Are all the edits type-preserving? If no, what sort of refactoring will users have to do?
  5. squash series._v12 releases._v12to create the release. This squashedreleases._v12will have no history and is more efficient for users topullinto their codebase.
    • Reset the patch and release notes inmain:.main> delete.patch patchand.main> delete.term releaseNotes.Anyone can upgrade from a past releasev3,by applying the patchesreleases._v4.patch,releases._v5.patchup throughreleases._v12.patch.
      • If desired, you can also produce cumulative patches for skipping multiple versions. These can be published on an ad hoc basis. If publishing these, just include them inreleases._v12.patches.v4_to_v12.
  6. Optional: you can add updated instructions for fetching the release to the the README on the repo's Unison Share page. You can also let folks know about the release via any other communication channels (Twitter, Slack, etc).

We don't recommend any fancy numbering scheme for versioning, just increment the version number. Use thereleaseNotesto convey metadata and additional nuance about each release.

Backporting fixes

Creating a bugfix release works the same way. Suppose thev12release has a bug. The bug has been fixed in the latesttrunkand you'd like to backport it. Just backport the fix to theseries._v12namespace and continue with the release steps as before, but this time createreleases._v12athen_v12b,_v12c,etc.

Patch management

Patches are collections of mappings from old hash to new hash (these entries are called "replacements"). We've seen above how these patches can be applied to upgrade a namespace using these replacements. The patches are built up viaupdateorreplace.termorreplace.typecommands which add replacements to a default patch (called "patch") that exists under each namespace in the tree. You can view this or any other patch usingview.patch:

.mylib.trunk> view.patch patch

  Edited Terms: List.frobnicate#92jajfh197 -> List.frobnicate
  Edited Terms: CinnamonRoll#93jg10ba -> SugarCookie

  Tip: To remove entries from a patch, use delete.term-replacement or
       delete.type-replacement, as appropriate.

🤓 Theupdateandreplace.termandreplace.typecommands also take an optional patch name, if you want to build up a patch somewhere other than the default patch. This is handy for keeping logically unrelated patches separate. You can alsomove.patchanddelete.patch.

Release patches

There are a lot of reasons you might add replacements to a patch during the course of development (see the development patches section below), but for patches published with a release, it's recommended to limit the replacements to cases where the old definition is invalid or would never be preferred the new version (generally just bugfixes for previously released definitions, performance improvements, and critical upgrades). Why do this? Because it makes things a lot easier for your users and avoids needless churn.

In other languages, where definitions are identified by name and the codebase is just mutated in-place, every change to the codebase amounts to a forced upgrade for users. We don't often think of it this way, but by updatingList.frobnicate,the old version ofList.frobnicateis effectivelydeletedand no one gets to reference the old definition ofList.frobnicate(unless they literally go through the Git history and bring it back somehow). Very often this is donenotbecause the old version is wrong or should never be used, but because we don't feel like coming up with another name for the new definition and decide to just repurpose an existing name.

When this name repurposing is done for definitions that have already been released, it generates work that often isn't necessary.

In Unison, the decision to repurpose a name iscompletely separatefrom the decision to force an upgrade on users. If you want to repurpose a name likeList.frobnicatebut the old definition is still valid (ask yourself "could someone reasonably still build on the old definition, or is the new definition always preferable?"), you can first delete the old definition (viadelete.term),or archive it by moving it to_archive. y2020_m03_d24_frobnicate,timestamped with the current date. Then create the new definition forList.frobnicate.

If you've already done anupdate,no problem. Just usedelete.term-replacementordelete.type-replacementto remove replacements from the patch before release.

Here are a few common types of updates that won't usually be part of a release patch, but will instead be part of a development patch, covered next:

  • Adding a parameter to a function to make it more generic: typically the less generic version is still perfectly valid
  • Changing the order of parameters: typically the previous argument order is still perfectly valid
  • Changing the type of the function: generally, when this is done, the function is actually doing something different

Release patches typically contain bugfixes, performance improvements, or critical updates (say some code depends on an external service, and that external service has a different interface now, invalidating the old code).

Development patches

During development, new, unreleased definitions may get updated or refactored multiple times before making it into a release, for instance:

  • An unreleased definition may have a bug that gets fixed before release. A patch records this bugfix.
  • An unreleased definition may be stubbed out (using thetodofunction) then the stubbed definition later filled in. A patch records this replacement.
  • Definitions may be refactored multiple times before being released, and these refactorings are recorded in a patch so all the developers on the release can easily stay up-to-date.

With some simple steps, you can make it easy to keep your release patch clean so when it comes time to make a release, the default patch in.main. patchis totally clean and just includes the replacements you want all users to make in their code:

  • When developing new definitions to be added tomain,do it in a separate namespace underprs,sayprs._myNewStuff.Within this namespace, useupdate,replace.termandreplace.typeas much as you like. When you're ready to merge, justdelete.patch prs._myNewStuff.patch.
  • For other development patches, like bugfixing an unreleased definition inmain,or replacing a stub, use a separate named development patch for it rather than the default patch that becomes the release patch during the release process. We recommendupdate .main.patches.devfor the default development patch. Anyone with in-progress pull requests can apply this patch to their work. It can be archived and reset periodically.
  • When repurposing the name of a released definition, use the repurposing names steps covered above rather than usingupdateon it.
  • For updates to released code that all users should make (like bugfixes or performance improvements), go ahead and do anupdateof the default patch. Note that if youfork trunk prs._somePR,then do your updates in_somePR,when that namespace is merged back into trunk, the updates you made to the default patch will arrive there as well.