Make an opam local switch and dune init project work together

In the OCaml ecosystem, opam is the most popular package management tool, while dune is the foremost build tool.

Both have evolved independently. While they manage to work in tandem, there are a lot of irregularities to trip on where they meet.

Local switch in opam

opam's local switch feature can set up and isolate development environment and dependencies per project.

opam switch create creates a local switch if it is given the path to a directory. If the specified directory does not exist, then its is created.

opam switch create ./path/to/existing/dir

The above command creates an _opam directory inside the given path, inside which all of the opam-related artifacts for this project are stored.

A cool side-effect is that when a directory containing a local opam switch is visited, that switch is activated automatically. No need to opam switch set the switch explicitly.

dune init project

dune init project is the recommended way to bootstrap an OCaml project.

dune init project prj_name

The above command creates a directory named prj_name in the current directory, and populates it with project artifacts.

I found that dune cannot bootstrap a project in the current directory itself. So dune init project . fails to work.

The problem

I prefer to use opam's local switch feature to ensure the isolatation of all dependencies and tools related to an OCaml project. This means I also prefer to install dune per-project. So

1. I want to use dune to bootstrap a project
2. dune is installed through opam
3. opam needs a local switch for per-project installation
4. A local switch needs an existing empty project directory
5. dune cannot bootstrap a project in a current directory

The chicken and egg problem looks easy now, huh?

I started guessing a solution.

Attempt 1

Execute opam and dune commands from parent directory

I planned the following idealised sequence of commands:

# create an empty project dir
~/projects $ mkdir my_prj

# create a local switch
~/projects $ opam switch create ./my_prj

# install dune in our local switch
~/projects $ opam install dune

# bootstrap project
~/projects $ dune init project my_prj

I didn't make it past the 3rd command of installing dune because of the following (reasonable) error:

$ opam install dune
[ERROR] No switch is currently set. Please use 'opam switch'
to set or install a switch.

Ok, then. I tried to set the switch after creating it.

~/projects $ mkdir my_prj
~/projects $ opam switch create ./my_prj
~/projects $ opam switch set ./my_prj
[ERROR] Can not set external switch '/Users/jb/projects/my_prj' globally. To set
it in the current shell use:
eval $(opam env --switch=/Users/jb/projects/my_prj --set-switch)

The advice felt like a long-winded solution to me. I wanted a simpler solution.

So may be, first, I should enter the empty project directory to set up the project.

Attempt 2

Execute opam and dune commands from project directory

I carried out the following sequence of commands:

~/projects $ mkdir my_prj
~/projects $ cd my_prj 
~/projects/my_prj $ opam switch create . 
~/projects/my_prj $ opam install dune
~/projects/my_prj $ dune init project my_prj

The above sequence creates a my_prj/my_prj/ directory tree. That's not what I wanted.

Now what?

Solution

exploit opam's auto-activation of local switch

Now, consider the following sequence of commands:

# create an empty project dir
~/projects $ mkdir my_prj

~/projects $ cd my_prj 

# create a local switch, which is auto-activated after a successful creation
# because the switch is created in the current directory itself.
~/projects/my_prj $ opam switch create . 

~/projects/my_prj $ opam install dune

# go back to parent, with opam still pointing to the local switch env
# until the current session ends or until another switch is activated
~/projects $ cd .. 

# bootstrap our project
~/projects/my_prj $ dune init project my_prj

Note the 3rd command opam switch create .. After the command finishes execution, the switch is auto-activated because the local switch lives in the current directory . itself.

opam also sets an env var OPAM_SWITCH_PREFIX to the path of this activated switch.

$ echo $OPAM_SWITCH_PREFIX 
/Users/jb/projects/my_prj/_opam

This env var probably lives on until the current shell session ends even if the directory changes, or until another switch is activated.

That means that when the above command sequence goes one directory up to execute dune init project, opam is still pointing to the right local switch env.

Also dune does not mind if the directory my_prj already exists.

Case solved!

I can imagine a variation of the above sequence:

# create an empty project dir
~/projects $ mkdir my_prj

# create our local switch
~/projects $ opam switch create ./my_prj 

# activate our local switch
~/projects $ cd my_prj 

# go back to parent, with opam still pointing to the local switch env
# until the current session ends or until another switch is activated
~/projects/my_prj $ cd .. 

# install dune in our local switch
~/projects $ opam install dune

# bootstrap our project
~/projects $ dune init project my_prj

What next?

There is work under progress to wrap dune around opam, thus making dune as the unifying package management and build solution.