Update - Docker build secrets
On February 24th, 2019 I published a follow-up post about using Docker build secrets instead of multi-stage builds. I recommend reading that post after this one!
Building Docker images with private npm packages
I recently completed a security audit of Docker images for a project. These images used
.npmrc files (npm config files) to download private npm packages. By default, an
.npmrc file contains a token with read/write access to your private npm packages. It looks like this:
Most blog posts, Stack Overflow answers, and documentation recommend you delete
.npmrc files from your
Dockerfile after installing private npm packages. Many of these guides don’t cover how to remove
.npmrc files from intermediate images or npm tokens from the image commit history though. In fairness, most of these resources date from before Docker shipped multi-stage builds in Docker v17.05 in May 2017.
Multi-stage builds allow us to securely use
.npmrc files in our Docker images. Only the intermediate images and commit history from the last build stage end up in our final Docker image. This enables us to
npm install our private packages in earlier build stages without leaking our tokens in the final image.
In this blog post, I’ll first describe the common ways people use
.npmrc files insecurely in Docker images. For each scenario, I’ll show how an attacker can exploit it to steal your npm access tokens. Finally, I’ll explain how multi-stage builds enable you to securely use
.npmrc files in your Docker images.
This blog post focuses on using Docker with Node.js and npm. The concepts I cover apply to any
Dockerfile that uses tokens, passwords, or other secrets though.
I also created a companion GitHub repository for this blog post so you can follow along with my examples. You can check it out at https://github.com/alulsh/docker-npmrc-security.
I have revoked all npm tokens featured in all screenshots.
#1 - Leaving
.npmrc files in Docker containers
If you fail to remove your
.npmrc file from your
Dockerfile, it will be saved in your Docker image. It will exist on the file system of any Docker container you create from that image.
Dockerfile where we create an
.npmrc file but fail to delete it after running
FROM node:8.11.3-alpine ARG NPM_TOKEN WORKDIR /private-app COPY . /private-app RUN echo "//registry.npmjs.org/:_authToken=$NPM_TOKEN" > .npmrc RUN npm install EXPOSE 3000 CMD ["node","index.js"]
Dockerfile creates the
.npmrc file using an
NPM_TOKEN environment variable that we pass in as a build argument (
ARG NPM_TOKEN). To build it locally, first clone the companion GitHub repository at https://github.com/alulsh/docker-npmrc-security then:
npm token create --read-onlyto create a read only npm token.
export NPM_TOKEN=<npm token>to set this token as an environment variable.
docker build . -f Dockerfile-insecure-1 -t insecure-app-1 --build-arg NPM_TOKEN=$NPM_TOKEN.
.npmrc files from Docker containers
If an attacker compromises your Docker container or manages to execute arbitrary commands on your application servers, they can steal your npm tokens by running
ls -al && cat .npmrc.
You can try it out yourself:
docker run -it insecure-app-1 ashto start the container. We need to use
bashsince we’re running Alpine Linux.
ls -al. You should see an
.npmrcfile in the
Stealing npm tokens from a running Docker container
#2 - Leaving
.npmrc files in Docker intermediate images
Most guides recommend deleting your
.npmrc file after running
npm install in your
Dockerfile. For example:
ARG NPM_TOKEN RUN echo "//registry.npmjs.org/:_authToken=$NPM_TOKEN" > .npmrc RUN npm install RUN rm -f .npmrc
Fortunately, this does remove the
.npmrc file from the top layer of your Docker image and from any containers. If an attacker compromised your Docker container they would not be able to steal your npm token.
RUN instruction creates a separate layer (also called intermediate image) in a Docker image. Multiple layers make up a Docker image. In the above
.npmrc file is stored in the layer created by the
RUN echo "//registry.npmjs.org/:_authToken=$NPM_TOKEN" > .npmrc instruction.
.npmrc files from intermediate images
To view the layers of your Docker image an attacker would need to either compromise your Docker daemon or obtain a copy of your Docker image. You likely won’t publish Docker images with private source code to public Docker registries. You might distribute these Docker images to contractors, consultants, and customers though. These third parties need your private source code but they do not need your npm tokens.
Here’s how an attacker or third-party could steal
.npmrc tokens from layers in your Docker images:
- Build the example Docker image with
docker build . -f Dockerfile-insecure-2 -t insecure-app-2 --build-arg NPM_TOKEN=$NPM_TOKEN.
docker save insecure-app-2 -o ~/insecure-app-2.tarto save the Docker image as a tarball.
mkdir ~/insecure-app-2 && tar xf ~/insecure-app-2.tar -C ~/insecure-app-2to untar to
cd ~/insecure-app-2. For fun, try running
cat manifest.jsonin this directory to view all of the layers.
Layers in a Docker image
for layer in */layer.tar; do tar -tf $layer | grep -w .npmrc && echo $layer; done. Credit goes to this StackOverflow answer for that one liner. You should see a list of layers with
tar xf <layer id>/layer.tar private-app/.npmrcto extract
private-app/.npmrcfrom the layer tarball. In my case, I needed to run
tar xf 1c3c8a7a05b2ffddbdbd9b1e93f662f68efae9246302122f756aae908e41676c/layer.tar private-app/.npmrc.
cat private-app/.npmrcto view the
.npmrcfile and npm token.
Stealing .npmrc files and npm tokens from Docker layers
#3 - Leaking npm tokens in the image commit history
More security conscious guides are aware of the Docker layer problem. They advocate creating and deleting the
.npmrc file in the same
RUN instruction or layer. Other guides recommend using the
--squash flag when running
docker build. Here’s a
Dockerfile where we create and delete our
.npmrc file in the same layer:
ARG NPM_TOKEN RUN echo "//registry.npmjs.org/:_authToken=$NPM_TOKEN" > .npmrc && \ npm install && \ rm -f .npmrc
Both approaches prevent
.npmrc files from being saved in layers. Unfortunately, the npm token is still visible in the commit history of the Docker image. The Docker image build process logs the plaintext values for build arguments (
ARG NPM_TOKEN) into the commit history of an image. For this reason, the official Docker documentation on Dockerfiles warns that you should not use build arguments for secrets.
Stealing npm tokens from Docker image commit histories
Similar to viewing Docker layers, to view your image commit history an attacker or third-party would need to compromise your Docker daemon or obtain a copy of your Docker image. This “hack” is easier though and only requires one command -
docker history. To try this yourself:
docker build . -f Dockerfile-insecure-3 -t insecure-app-3 --build-arg NPM_TOKEN=$NPM_TOKEN
docker history insecure-app-3
Stealing npm tokens from Docker image commit histories
Solution - Multi-stage builds
We can protect our build arguments from leaking with multi-stage builds. Only the final build in a multi-stage build will show up in the commit history of the final Docker image.
In the first stage of the build, we’ll use our
NPM_TOKEN build argument to
npm install our project. We can then copy our built Node application from the first stage to the second stage build using the
Most multi-stage build tutorials and examples show two different base images. You can use the same base image for multi-stage builds. In this example, I use
node:8.11.3-alpine as my base image for both stages.
# First build FROM node:8.11.3-alpine AS build ARG NPM_TOKEN WORKDIR /private-app COPY . /private-app RUN echo "//registry.npmjs.org/:_authToken=$NPM_TOKEN" > .npmrc && \ npm install --production && \ rm -f .npmrc # Second build FROM node:8.11.3-alpine WORKDIR /private-app EXPOSE 3000 COPY --from=build /private-app /private-app CMD ["node","index.js"]
To build this image, run
docker build . -f Dockerfile-secure -t secure-app --build-arg NPM_TOKEN=$NPM_TOKEN. To view the history, run
docker history secure-app.
Secure Docker commit history thanks to multi-stage builds
Our npm tokens no longer leak in the commit history!
Delete untagged images from multi-stage builds
Multi-stage builds create untagged images of earlier build stages in our local Docker daemon for the Docker build cache. The commit history of these images leaks plaintext build arguments. You should delete these images after you finish your builds.
docker history on these untagged images to steal npm tokens from them. Run
docker rmi <image id> to delete them.
Npm tokens in untagged images from multi-stage builds
If an attacker compromises your containers or Docker daemon, you’ll likely be dealing with bigger issues than your
.npmrc files. Removing npm tokens from your Docker images means you’ll be dealing with one less problem though.
Depending on your threat model, deleting
.npmrc files from your Docker images and containers may be all you need to do to mitigate most of your risk. If you distribute your Docker images to third party contractors, consultants, or customers you should use multi-stage builds to remove any secrets. This also applies if you publish images to Docker registries.
Multi-stage builds are easy to use and have little to no downsides. They can significantly improve the security, performance, and readability of your Docker images. If you use secrets to build your Docker images I recommend multi-stage builds even if you don’t distribute your Docker images and aren’t worried about the security of your Docker daemon.
Raising awareness of multi-stage builds
The Docker community is aware of the lack of a built-in way to manage secrets in Dockerfiles. They’re currently working on ways to improve using secrets in Docker. In the meantime multi-stage builds allow us to securely use
.npmrc files or other secrets in our Docker builds. They also improve the readability of our Dockerfiles and decrease the size of our images.
Most of the guides for using
.npmrc files in Docker images date from before multi-stage builds in May 2017. I’m currently drafting a pull request to the official npm documentation to update their guidance to use multi-stage builds. I’ll update this blog post with a link to the pull request as well as updates on its status!