We cover what our
docker-compose.yml
file is doing when we define and use a volume.We cover these parts of the
.yml
file:After running
docker-compose up -d
, we get all of our artifacts created. One of these is a named volumed, which we've named data
. We can inspect that volume:We can inspect the running containers using this volume as well to see it's mount points:
In the video, we switch over to an Ubuntu server running Docker, since we can't view the contents of that data volume directly on the Macintosh here. We can't see that since Docker runs in a layer of virtualization on our Mac or Windows computers.
DTR 2.1.x and newer. Docker Trusted Registry stores its data in a volume that corresponds with its replica id. For example, the dtr-registry-00 volume is where the DTR with replica id 00 stores its registry data when using the filesystem backend. To store your images in an NFS directory, specify the NFS URL at install time or switch to an NFS location after install with. Docker run --rm -it --name web001 volume:latest powershell At the same time, docker will create a folder with the Container’s name Inside the Volume folder. Below, you can see the volume I created In the DockerFile (Called data) and I’ll go ahead and create a file Inside it.
On the Ubuntu server, we spin up a similar MySQL container using a data volume configured in a
docker-compose.yml
file, anad then we inspect it to get the data directory. Finally we list out that directory to see the MySQL database data contents.We can see this matches the data found in the directory
/var/lib/mysql
from inside the container as well.Finally we spin down the MySQL docker container (destroying it!) and re-create it. After doing that, we can see that our MySQL database data has persisted - the volume was NOT destroyed when we ran destroyed the container.
Quick Overview of the Volumes Options
If option is marked Automatic, then you do not have to do anything for this option to be used.
If option is not Automatic, then you need to enable it manually on the global or project level to use.All non-automatic options are enabled by placing
DOCKSAL_VOLUMES='<value from volumes column>'
into the respectivedocksal.env
file or using fin config set
for the same, e.g., fin config set DOCKSAL_VOLUMES='NFS'
.Once you set a new volumes option, you must re-create cli
container. The easiest way is fin project reset
,but it will also remove all the data from db
volume. If you want to retain it, remove cli
container and startthe project again to recreate it: fin p remove cli; fin p start
OS | Docker | Volumes | Automatic | Comments |
---|---|---|---|---|
Linux | Native | bind | Yes | Direct project files access, native FS speed. |
macOS | VirtualBox | nfs | Yes | cli accesses project files from host via NFS. Pros: pretty fast, only 10-15% slower than native filesystem. Cons: does not support filesystem events (fsnotify). |
macOS | Docker for Mac | bind | Yes | Files from host are shared with VM via osxfs , then cli accesses them directly. Pros: supports filesystem events and is default for Docker for Mac. Cons: pretty slow, 40-50% slower than native filesystem. |
macOS | Docker for Mac | nfs | No | cli accesses project files from host via NFS. Pros: much faster than bind option. NFS is only 10-15% slower than native filesystem. Cons: does not support filesystem events (fsnotify). |
Windows | ANY | bind | Yes | Files from host are shared with VM via SMB, then cli accesses them directly. Pros: relatively fast, 20% overhead as compared to native FS. Cons: does not support filesystem events (fsnotify). |
ANY | ANY | unison | No | Files from host are shared with VM, but cli does not access them directly. Instead, cli filesystem is an independent volume, and files are synced from the VM to cli via an additional Unison container. Useful for huge projects where FS performance is a bottleneck. Pros: maximum cli filesystem performance. Cons: initial wait for files to sync into cli ; additional Docksal disk space use; sync delay when you switch git branches; higher CPU usage during files sync; sometimes Unison might ‘break.’ |
ANY | ANY | none | No | none option is like unison , but without the auto-sync. Useful for huge projects where FS performance is a bottleneck, but when unison does not work for you. Pros: maximum cli filesystem performance and no wait for the initial sync. Cons: you have to copy files manually or checkout and edit files inside cli container only. |
Full explanation on volumes in Docksal
Let’s go over volumes in Docker first.
In Docker, volumes are used primarily for persisting datagenerated by and used by Docker containers. Docker automatically creates and manages volumes, storing them in a speciallocation within the host machine’s filesystem. There are also different volume plugins, which add support forvarious other data storage backends for volumes.
Mounting an existing folder from the host machine can also be done via volumes usingbind mounts.
By default, Docksal uses the bind mount approach.
The VM layer used on macOS/Windows (through VirtualBox or Docker for Mac/Windows) adds some complexity to that, howeverthat’s not something you normally have to worry about. Both Docksal and Docker for Mac/Windows handle that automatically.
From the perspective of a container, a local Linux path is mounted regardless of the underlying host OS.On Mac, the host filesystem is mounted with NFS, on Windows - using SMB.
Let’s take a look at an example.
The host machine is a macOS and the codebase root directory (the “Projects” folder) is
/Users/username/Projects
.This directory is mounted with the same path inside the VM: /Users/username/Projects
. Any path within that directoryis exactly the same on the host and inside the VM.When a project stack is started, the project root directory (e.g.,
/Users/username/Projects/myproject
) is bind mountedinto /var/www
inside the containers. A corresponding line in docksal.yml
for this would be:${PROJECT_ROOT}
is automatically set to the project’s root directory on the host.The whole mount chain looks like this (drop the last part for Linux hosts).
Bind Volumes
Bind volumes mean native binding of files on host filesystem to Docker. But remember that on macOS and Windows“host filesystem” for Docker is actually a filesystem of a Virtual Machine it runs in. So while bind volumescan be used there it actually means that files from your real host has to be mapped into the VM first forthis option to work. On macOS it means mapping via
osxfs:cached
, while on Windows via SMB
.Instead of using a host path every time we want to mount a volume, we can give the volume a name and refer to it by name:
In the example above,
project_root
and docksal_ssh_agent
are “named volumes.” The first one is a project level one,while the second one is a global volume and is used by all projects.See stacks/volumes-bind.yml.
Defining volumes this way makes it much easier to override volume settings in one place (
volumes
section) vs multipleplaces in the yaml file. We can now swap bind mounting with something else. See below.osxfs:cached Mode with Docker for Mac
As explained above for bind volumes option the files are actually mapped to the VM that Docker runs invia
osxfs:cached
. Docksal automatically enables the osxfs:cached
mode on Docker for Mac.See stacks/overrides-osxfs.yml.
NFS Volumes
With this option
cli
container will map files directly from your real host, rather than mapping them fromthe Virtual Machine that Docker runs in.See stacks/volumes-nfs.yml.
This is what the file sharing chain looks like with a NFS volume.
As you can see, containers mount NFS via the host machine and not directly. This setup method only makes sense on macOSwith Docker for Mac, for testing and performance comparison purposes.
Using NFS Volumes
- Add
DOCKSAL_VOLUMES=nfs
either globally in$HOME/.docksal/docksal.env
or in.docksal/docksal.env
in a project fin project reset
Unison Volumes
We can also do more advanced and pretty interesting solutions, like using Unison to synchronize files between the hostand the
project_root
volume.See stacks/volumes-unison.yml.
Unison volumes make the most sense for Docker for Mac users as an alternative to the (still slow)
osxfs
file sharing.This is what the file sharing chain looks like with Unison over
osxfs
.project_root
is a named volume, PROJECT_ROOT
is a path on the host mounted into the same path in the VM via osxfs
.unison daemon
does a TWO WAY sync between PROJECT_ROOT
and project_root
.Unlike NFS or SMB,
osxfs
supports inotify
events, which makes it an ideal option for front-end developers relying onautomatic compilation tools and in-browser live reloading. In the chain above, inotify
events are not lost and arepropagated all the way from the macOS host to the container.The benefits of this setup:
- Full native container file system performance (reads and writes)
ionitify
event support- Low latency (~1s), two way file sync
The downsides:
- Initial sync can take time, especially on large codebases
- Higher disk space usage (double the size of the codebase)
- Additional load from the unison daemon, but nothing compared to the load
osxfs
produces.
Using Unison Volumes
Unprecedented, native-like FS speed on macOS and Windows (Linux is already native). See docksal/unison for details
- Add
DOCKSAL_VOLUMES=unison
in.docksal/docksal.env
in a project fin project reset
- Wait until initial sync finishes.
None Volumes
This method is similar to the Unison method, but without the actual sync happening at all.
Nothing is mounted from the host. An empty
Nothing is mounted from the host. An empty
project_root
volume is created and mounted inside containers.This can be used to provision completely blank environments and have all work (code checkout, etc.) done inside cli.
Provides THE BEST file system performance. Combined with Cloud9, can provide a way of provisioning instant blankdevelopment environments with the best performance and consistency for Mac and Windows (Linux has the best performancenaturally). The only added cost is having to stick with a web based IDE and terminal.
See stacks/volumes-none.yml.
When a project is created, there are a few volumes created for it. One of them is called
<project_name>_project_root
(where <project_name>
is the machine name of your project). This volume is mapped to a specific location on the host.If you need to move the location of a project on the host system, the project stack will first need to be removed (runfin project rm
inside the project), then you can move the project folder and start the project stack again.