Performant tooling with Docker and macOS
May 3, 2022
Developers love performant tools, I mean, developers NEED performant tools!
I'm not going to talk about whether you should use macOS to develop.
Personally, I do, not gonna argument here.
Problem is, I use Docker, and it's sad that this sentence starts with "problem".
Docker has helped the dev community and I like this tools, I mean, I like the idea and the possibilities it offers.
But on macOS it's slow as hell.
To be honest, it's not slow in every situation but only some, unfortunately the one that are mostly the main usage of Docker in a development environment.
This is not new as you can see on this issue reported in 2016.
Issues are related to volume performance, specifically the way volumes are mounted on macOS.
And for applications which perform many I/O disk operations, it can become almost not usable.
Unfortunately, I'm in that case, being a web/symfony developer I handle projects with many dependencies/cache and thus many I/O.
I heard that Docker decided to have a look at this issue.
It took them far too much time to do so, but that's another topic.
Their effort seems to go somewhere for some people, not so much for me.
Of course, I tried every option before, and trust me on this one.
Being docker-sync (which was the best tradeoff), mutagen, VM with docker inside, and other fancy other attempts.
And today, I'm thrilled to tell you that I compiled the best of all those errands into a solution that's quite acceptable.
It is the solution I use for all my projects, and so far, it has been a pleasure again to use Docker on macOS.
This solution is heavily inspired by docker-sync and I basically removed a layer, and thus it gives more control on the whole process.
It can be resumed like this: don't use
-v ever, instead make specific docker volumes and synchronize them with
Let's dive in, code is coming.
Part one: building
Installing unison local/docker
We want to install unison on our local machine, it will then connect to our container that has the mounted volume.
$ brew install unison
Then get the version, as it is important to have the same version on both the client (your local machine) and the server (the docker container with the volume).
$ unison -version unison version 2.52.0 (ocaml 4.12.0)
Now we build a docker image with
unison at the same version (both unison and ocaml version should match)
FROM ubuntu:20.04 ENV DEBIAN_FRONTEND=noninteractive RUN apt-get update && \ apt-get install -y --no-install-recommends \ curl wget build-essential coreutils openssh-server git bash \ inotify-tools monit supervisor rsync ruby tzdata \ && rm -rf /var/lib/apt/lists/* /var/cache/apt/* # Install OCAML: adapt version number from your own local setup RUN wget http://caml.inria.fr/pub/distrib/ocaml-4.12/ocaml-4.12.0.tar.gz \ && tar xvf ocaml-4.12.0.tar.gz -C /tmp \ && cd /tmp/ocaml-4.12.0 \ && ./configure \ && make world \ && make opt \ && umask 022 \ && make install \ && make clean \ && rm -rf /tmp/ocaml-4.12.0 \ && rm /ocaml-4.12.0.tar.gz # Install Unison: adapt version number from your own local setup RUN curl -L https://github.com/bcpierce00/unison/archive/v2.52.0.tar.gz | tar zxv -C /tmp \ && cd /tmp/unison-2.52.0 \ # needed for < 2.51.4 with OCALM 4.12 - see https://github.com/bcpierce00/unison/pull/480 # and https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/unison.rb#L13 # && curl https://github.com/bcpierce00/unison/commit/14b885316e0a4b41cb80fe3daef7950f88be5c8f.patch?full_index=1 -o patch.diff \ # && ([[ "2.52.0" == "2.51.3" ]] && git apply patch.diff); \ # rm patch.diff \ && sed -i -e 's/GLIBC_SUPPORT_INOTIFY 0/GLIBC_SUPPORT_INOTIFY 1/' src/fsmonitor/linux/inotify_stubs.c \ && make UISTYLE=text NATIVE=true STATIC=true \ && cp src/unison src/unison-fsmonitor /usr/local/bin \ && rm -rf /tmp/unison-2.52.0 # These can be overridden later ENV TZ="Europe/Paris" \ LANG="C.UTF-8" \ UNISON_DIR="/data" \ HOME="/root" # Install SSH packages RUN apt-get update && \ apt-get install openssh-server -y --no-install-recommends && \ rm -rf /var/lib/apt/lists/* /var/cache/apt/* RUN mkdir -p /run/sshd RUN mkdir -p /root/.ssh # You should insert your public key in this file COPY authorized_keys /root/.ssh/authorized_keys # Editing the SSHD config file RUN echo 'PermitRootLogin yes' >> /etc/ssh/sshd_config # Make port 22 available to the world outside this container EXPOSE 22 # Run SSHD when the container starts CMD ["/usr/sbin/sshd", "-D"]
You may need to adapt a bit, see comments
Create the Docker volume and container
docker volume create --name your-project-name-volume
docker create -h your-project-name --name your-project-name-sync -it -p PORT:22 -v your-project-name-volume:/data your/image /bin/bash
PORT with some available port on your machine.
Part two: usage
Configuration in your docker-compose
Now, everywhere you need those project files, instead of linking the local filesystem with
volumes you reference
version: "3.3" services: apache: build: docker/apache volumes: - "your-project-name-volume:/var/www" # Instead of # volumes: # - "./:/var/www" volumes: your-project-name-volume: external: true
Start the container and sync
docker start your-project-name-sync
unison . ssh://[email protected]:PORT//data -repeat=watch -auto -batch -ignore 'Path .git' -ignore 'Path var'
PORT with some available port on your machine.
Of course, you can put that into a bash script:
#!/bin/bash docker start your-project-name-sync && \ unison ./ ssh://[email protected]:PORT//data -repeat=watch -auto -batch -ignore 'Path .git' -ignore 'Path var'
This solution comes with some tradeoffs, mainly the fact that your code base is duplicated on a Docker volume, thus duplicating the disk space required. The first start will take some time too as it needs to synchronize the whole project.
I reckon, it's a bit of configuration to do, but to me that works so well that I consider those a detail.