Performant tooling with Docker and macOS

Developers love performant tools, I mean, developers NEED performant tools!

I'm not going to talk about whether you should use macOS to develop.
Personally, I do, not gonna argument here.

Problem is, I use Docker, and it's sad that this sentence starts with "problem".
Docker has helped the dev community and I like this tools, I mean, I like the idea and the possibilities it offers.

But on macOS it's slow as hell.

To be honest, it's not slow in every situation but only some, unfortunately the one that are mostly the main usage of Docker in a development environment.
This is not new as you can see on this issue reported in 2016.

Issues are related to volume performance, specifically the way volumes are mounted on macOS.
And for applications which perform many I/O disk operations, it can become almost not usable.

Unfortunately, I'm in that case, being a web/symfony developer I handle projects with many dependencies/cache and thus many I/O.

I heard that Docker decided to have a look at this issue.
It took them far too much time to do so, but that's another topic.
Their effort seems to go somewhere for some people, not so much for me.

Of course, I tried every option before, and trust me on this one.
Being docker-sync (which was the best tradeoff), mutagen, VM with docker inside, and other fancy other attempts.

And today, I'm thrilled to tell you that I compiled the best of all those errands into a solution that's quite acceptable.
It is the solution I use for all my projects, and so far, it has been a pleasure again to use Docker on macOS.

This solution is heavily inspired by docker-sync and I basically removed a layer, and thus it gives more control on the whole process.
It can be resumed like this: don't use -v ever, instead make specific docker volumes and synchronize them with unison <3

Let's dive in, code is coming.

Part one: building

Installing unison local/docker

We want to install unison on our local machine, it will then connect to our container that has the mounted volume.

$ brew install unison

Then get the version, as it is important to have the same version on both the client (your local machine) and the server (the docker container with the volume).

$ unison -version
unison version 2.52.0 (ocaml 4.12.0)

Now we build a docker image with unison at the same version (both unison and ocaml version should match)

Dockerfile

FROM ubuntu:20.04

ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update && \
    apt-get install -y --no-install-recommends \
        curl wget build-essential coreutils openssh-server git bash \
        inotify-tools monit supervisor rsync ruby tzdata \
    && rm -rf /var/lib/apt/lists/* /var/cache/apt/*

# Install OCAML: adapt version number from your own local setup
RUN wget http://caml.inria.fr/pub/distrib/ocaml-4.12/ocaml-4.12.0.tar.gz \
    && tar xvf ocaml-4.12.0.tar.gz -C /tmp \
    && cd /tmp/ocaml-4.12.0 \
    && ./configure \
    && make world \
    && make opt \
    && umask 022 \
    && make install \
    && make clean \
    && rm -rf /tmp/ocaml-4.12.0 \
    && rm /ocaml-4.12.0.tar.gz

# Install Unison: adapt version number from your own local setup
RUN curl -L https://github.com/bcpierce00/unison/archive/v2.52.0.tar.gz | tar zxv -C /tmp \
    && cd /tmp/unison-2.52.0 \
    # needed for < 2.51.4 with OCALM 4.12 - see https://github.com/bcpierce00/unison/pull/480
    # and https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/unison.rb#L13
    # && curl https://github.com/bcpierce00/unison/commit/14b885316e0a4b41cb80fe3daef7950f88be5c8f.patch?full_index=1 -o patch.diff \
    # && ([[ "2.52.0" == "2.51.3" ]] && git apply patch.diff); \
    # rm patch.diff \
    && sed -i -e 's/GLIBC_SUPPORT_INOTIFY 0/GLIBC_SUPPORT_INOTIFY 1/' src/fsmonitor/linux/inotify_stubs.c \
    && make UISTYLE=text NATIVE=true STATIC=true \
    && cp src/unison src/unison-fsmonitor /usr/local/bin \
    && rm -rf /tmp/unison-2.52.0

# These can be overridden later
ENV TZ="Europe/Paris" \
    LANG="C.UTF-8" \
    UNISON_DIR="/data" \
    HOME="/root"

# Install SSH packages
RUN apt-get update && \
    apt-get install openssh-server -y --no-install-recommends && \
    rm -rf /var/lib/apt/lists/* /var/cache/apt/*

RUN mkdir -p /run/sshd
RUN mkdir -p /root/.ssh

# You should insert your public key in this file
COPY authorized_keys /root/.ssh/authorized_keys

# Editing the SSHD config file
RUN echo 'PermitRootLogin yes' >> /etc/ssh/sshd_config

# Make port 22 available to the world outside this container
EXPOSE 22

# Run SSHD when the container starts
CMD ["/usr/sbin/sshd", "-D"]

You may need to adapt a bit, see comments

Create the Docker volume and container

docker volume create --name your-project-name-volume

docker create -h your-project-name --name your-project-name-sync -it -p PORT:22 -v your-project-name-volume:/data your/image /bin/bash Replace PORT with some available port on your machine.

Part two: usage

Configuration in your docker-compose

Now, everywhere you need those project files, instead of linking the local filesystem with -v or volumes you reference your-project-name-volume instead!

docker-compose.yml

version: "3.3"
services:

    apache:
        build: docker/apache
        volumes:
            - "your-project-name-volume:/var/www"
# Instead of
#        volumes:
#            - "./:/var/www"

volumes:
    your-project-name-volume:
        external: true

Start the container and sync

docker start your-project-name-sync

cd PROJECT unison . ssh://[email protected]:PORT//data -repeat=watch -auto -batch -ignore 'Path .git' -ignore 'Path var' Replace PORT with some available port on your machine.

Of course, you can put that into a bash script:

#!/bin/bash

docker start your-project-name-sync && \
unison ./ ssh://[email protected]:PORT//data -repeat=watch -auto -batch -ignore 'Path .git' -ignore 'Path var'

Tradeoffs

This solution comes with some tradeoffs, mainly the fact that your code base is duplicated on a Docker volume, thus duplicating the disk space required. The first start will take some time too as it needs to synchronize the whole project.

I reckon, it's a bit of configuration to do, but to me that works so well that I consider those a detail.