Vagrant and Unison Without a Plugin

In this post I’ll describe how and why we use unison with Vagrant. This is a slightly manual way to do things (i.e. not a Vagrant plugin), but might suit you depending on your requirements.

Why Unison?

The first reason - in common with rsync - is that unison gives you “native” filesystem performance inside whichever VM you end up using (e.g. VirtualBox, VMWare, Digital Ocean, AWS EC2, etc). Mitchell Hashimoto detailed more about his testing here and later here when introducing rsync for Vagrant. Jeff Geerling also provided some useful analysis in NFS, rsync, and shared folder performance in Vagrant VMs.

It’s also relevant to note that this native filesystem approach then allows you to make use of inotify-based watch/build/reload approaches instead of polling, so I’d classify this as part of the “performance” justification too.

The second major reason - why you would choose unison over rsync in Vagrant - is that unison supports bidirectional sync. The fact that we haven’t seen bidirectional rsync (whatever that would look like) added to Vagrant doesn’t bode well for its chances - see the issue rsync-push and rsync-pull in the Vagrant repo that has been open for over a year now. I tried using the vagrant-rsync-back plugin and even if you got over the manual aspect of it, you still end up with problems if you’re using rsync with the --delete flag. For most development projects you’ll likely have problems if you omit the --delete flag and allow weeks/months/years of old files to keep piling up inside the VM.

Why Not the vagrant-unison Plugin?

A vagrant plugin called vagrant-unison has existed since March 2013, but unfortunately it hasn’t been updated since April 2013, and Pull Requests sit dormant. GitHub user @dmatora has valiantly forked a fork and updated it here, however I don’t think it was his attention to become the de facto maintainer for vagrant-unison and still consider this project to be essentially "unmaintained".

Installing Unison

A difficulty of unison is that you require the exact same versions on both ends of the connection, and they’re not backwards compatible. We settled on using version 2.48.3 because (a) it’s the latest, and (b) it supports the -repeat watch option which will in theory allow us to have efficient inotify-like capabilities on hosts too.

OS X

This is the easiest - if you’re using homebrew, then just run brew install unison.

Windows

Quick but a little hacky: download 2.48.3 from here and extract the exe into your path (e.g. c:\windows will work).

Ubuntu 14.04

This is what we use for our Vagrant VM. Unfortunately Ubuntu’s “stable” unison is old, and as mentioned previously - unison versions need to be exact. Fortunately, the Arch Linux binary for the same is compatible so we just needed a few lines inside our Vagrant provisioning script like this:

# Install unison 2.48.3
echo "Installing unison"
cd /
curl -sL http://<host>/unison-2.48.3-2-x86_64.pkg.tar.xz | tar Jx

Configuring Unison

Although it would be technically possible to automatically configure the required unison and ssh files during vagrant up, we’ve left this as a manual step for now. Here’s the contents of our bash script that can be run on the host:

#!/usr/bin/env bash

# create ssh-config file
ssh_config="$PWD/.vagrant/ssh-config"
vagrant ssh-config > "$ssh_config"

# create unison profile
profile="
root = .
root = ssh://default//app/
ignore = Name {.git,.vagrant}

prefer = .
repeat = 2
terse = true
dontchmod = true
perms = 0
sshargs = -F $ssh_config
"

# write profile

if [ -z ${USERPROFILE+x} ]; then
  UNISONDIR=$HOME
else
  UNISONDIR=$USERPROFILE
fi

cd $UNISONDIR
[ -d .unison ] || mkdir .unison
echo "$profile" > .unison/myproject.prf

Note: you may want to adjust the ignore = line to suit your project, for example ignoring node_modules if you don’t want to sync them back. Although it would be possible to guess the contents of the ssh-config file and embed it in the script like we did with the PRT file, we decided that it’s most fool-proof to call vagrant ssh-config and pipe that to a file. But for this reason, you must run the unison script after your “vagrant up” is complete.

One nice thing about the above is that it even works using the “Git shell” that ships with GitHub for Windows, so for Windows devs then there’s no extra software install required if they had already installed GitHub anyway. Just make sure to run the script with the bash prefix, e.g. bash bin/setup-unison.sh if the earlier bash script is saved as bin/setup-unison.sh.

Configuring Vagrant

You could almost describe this as "deconfiguring vagrant". What we want to do now is to tell Vagrant not to use the default sync method. This line will work:

config.vm.synced_folder '.', '/vagrant', disabled: true

Depending on how you performed Vagrant provisioning previously, this could have big repercussions though. If your shell provisioning scripts assumed that all your host files were available on the guest, this will no longer work. Previously our vagrant provisioning used to do an initial build of our project, but we realised that we really didn’t need that as part of provision. Instead, our Vagrant provisioning now is purely about provisioning the VM ready for our project, not setting up the project itself.

Running Unison

If you have run vagrant up, then your VM should be up, but without any of your project files sync’d. Then if you’ve run your host’s unison provisioning, then you should have the ssh and unison configuration files saved too. Now it’s time to start unison - run unison myproject and watch it sync. Open a separate terminal to then vagrant ssh and work on the project.

Resolving Inconsistent State

If you have done a vagrant destroy/up then you’ll probably get a unison error like this after you attempt to sync the new VM:

Fatal error: Warning: inconsistent state.
The archive file is missing on some hosts.
For safety, the remaining copies should be deleted.
  Archive arb9e6983e48796df04f206818b3ad1169 on host MBP should be DELETED
  Archive ara72f3c341f93537516da77373f7eb18c on host vagrant is MISSING
Please delete archive files as appropriate and try again
or invoke Unison with -ignorearchives flag.

In that case, just delete the file they tell you to. e.g.

rm ~/.unison/arb9e6983e48796df04f206818b3ad1169

Wrapping Up

This approach seems to work fine with Windows, OSX and Ubuntu hosts, plus with both VirtualBox and Digital Ocean hosts (both running Ubuntu 14.04). Plus the advantage is that it’s not a “black box” as it’s using vanilla unison that is well-documented.

To summarise:

  1. Install unison 2.48.3 on both host and Vagrant guest
  2. Provision host to save the ssh and unison configurations
  3. Keep unison myproject running
Rhys Arkins-

Rhys is the founder of Key Location