How small can a docker image be?
I asked myself this question, when one of the teams I work with started a software project in golang recently. The target environment for this project is Kubernetes. The background of my question was kubernetes best practises that include small docker images.
We started with this first version of the Dockerfile:
FROM golang:1.11 RUN mkdir /app WORKDIR /app COPY hello.go . RUN go build -o hellogo . RUN groupadd -g 99 appuser && useradd -r -u 99 -g appuser appuser USER appuser CMD ["./hellogo"]
Note: all source code in this post is sample code to show case the approaches we considered.
No more time? Jump straight to conclusion.
Why are smaller docker image preferred?
I see two main reasons to strive for smaller docker images:
- the security attack surface is often smaller
- pulling and pushing docker images from and to remote docker registries is faster
Why is the security attack surface often smaller?
The smaller docker images are in terms of number of layers as well as size of layers, the smaller is the probability to inherit CVEs (cybersecurity vulnerabilities and exposures e.g. listed in https://cve.mitre.org/cve/) being shipped with these layers. The number of CVEs per docker image also depends on the number of CVEs attached to specific layers.
I’d rather consider striving for smaller docker images due to security aspects a recommendation than a rule.
To underscore this recommendation, I did compare two docker images’ CVEs in CVE appendix.
Why is pulling and pushing of smaller docker images from and to remote docker registries faster?
Well, the fewer data are sent over the network, the faster pulling and pushing of docker images can be performed. In the appendix about docker image sizes and times, I do list sample docker pull and push times from circleCI to docker hub.
How to shrink a docker image?
Based on the first version of the Dockerfile, we tested multi stage docker builds that follow the builder pattern.
Dockerfiles following the builder pattern are separated into two parts:
- part one creates a single binary that runs the application
- part two leverages the created binary file with a reduced docker image base layer
The above Dockerfile was changed to:
#part one FROM golang:1.11 as builder WORKDIR /go/src/github.com/lotharschulz/hellogodocker/ COPY hello.go . RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -ldflags '-w -extldflags "-static"' -o hellogodocker . # part two FROM alpine:latest RUN apk --no-cache add ca-certificates && \ addgroup -g 99 appuser && \ adduser -D -u 99 -G appuser appuser USER appuser WORKDIR /app COPY --from=builder /go/src/github.com/lotharschulz/hellogodocker/hellogodocker . CMD ["./hellogodocker"]
The first part contains the label builder. It is based on golang base docker image to create the binary file. The second part copies the binary from the builder. The resulting docker image is based on the smaller alpine docker base image and thus smaller overall.
The go build command includes
-ldflags '-w -extldflags "-static"' that makes the binary artifact statically linked.
Alpine base image
Since the second part of the aforementioned Dockerfile is also based on alpine, we tested if a Dockerfile based on alpine base image only would help to reduce the docker image.
The Dockerfile was changed to:
FROM alpine:latest RUN apk --no-cache add ca-certificates && \ addgroup -g 99 appuser && \ adduser -D -u 99 -G appuser appuser ADD hellogo / CMD ["/hellogo"]
While this Dockerfile is smaller it does not show significantly reduced docker image sizes as well as not significantly reduced docker image build, pull and push times. Please refer to Appendix Docker base image sizes & push and pull times as of circleCi hellogodocker build 91 for details.
From scratch base image
We tried to further shrink the image using from scratch as base image:
FROM scratch ADD ca-certificates.crt /etc/ssl/certs/ ADD hellogo / CMD ["/hellogo"]
This produces a rather minimal docker image. Actually it was the smallest we came up with.
However from time to time, we needed to debug the running app in a container.
We did experience issues in those situations with exec :
$ docker exec -it hellogodocker sh OCI runtime exec failed: exec failed: container_linux.go:348: starting container process caused "exec: \"sh\": executable file not found in $PATH": unknown
Based on these findings, the team decided to rely on alpine based image as docker image.
With smaller docker images, you do reduce the security attack surface of docker images and you reduce the docker image round trip time (build, push & pull).
4.8 MB (2 MB compressed) is the smallest docker image size produced based on the hellogodocker sample repository. It is ~0.6% of the initial uncompressed docker image size.
I consider this one indicator that reducing docker image sizes opens up great potential.
While docker image build times did not reduce so much, docker image pull times were reduces from ~20 seconds to ~ 1 seconds. Container start time will benefit from reduced docker pull times.
The best solution for the team is to use a docker image based on alpine because preinstalled utilities like sh are useful for the team.
Below are the build, pull and push times as well as docker image sizes based on the hellogodocker sample repository with alpine docker image:
- build time: 0.67 seconds
- push time: 3.803 seconds
- pull time 1.244 seconds
- image size 11.1MB (compressed 5MB)
The team experiences similar data and considers those acceptable for software development cycle times.
The appendix about docker image sizes and times contains more details about times and image sizes.
Docker base image sizes & push and pull times as of circleCi hellogodocker build 91
CVEs for docker images lotharschulz/hellogo:build.docker-min-compress–0.2.91 and lotharschulz/hellogo:build.docker-cache–0.2.91 scanned with Klar and ClairOS at 2018 09 20.
- lotharschulz/hellogo:build.docker-min-compress–0.2.91 (Dockerfile, make goal)
CLAIR_OUTPUT=High CLAIR_ADDR=http://192.168.99.100:30060 klar lotharschulz/hellogo:build.docker-min-compress--0.2.91 clair timeout 1m0s docker timeout: 1m0s no whitelist file Analysing 2 layers Got results from Clair API v1 Found 0 vulnerabilities
- lotharschulz/hellogo:build.docker-cache–0.2.91 (Dockerfile, make goal)
CLAIR_OUTPUT=High CLAIR_ADDR=http://192.168.99.100:30060 klar lotharschulz/hellogo:build.docker-cache--0.2.91 clair timeout 1m0s docker timeout: 1m0s no whitelist file Analysing 10 layers Got results from Clair API v1 Found 131 vulnerabilities Unknown: 16 Negligible: 28 Low: 47 Medium: 31 High: 9
How to scan docker images with Klar and ClairOS
- install klar e.g. from source code
go get github.com/optiopay/klar
- install minikube
git clone https://github.com/coreos/clair
git checkout -b release-2.0 origin/release-2.0
- adapt the clairOS config file in contrib/k8s/config.yaml as shown below:
# source: postgres://postgres:password@postgres:5432/postgres?sslmode=disable source: host=postgres port=5432 user=postgres password=password sslmode=disable
kubectl create secret generic clairsecret --from-file=./config.yaml kubectl create -f clair-kubernetes.yaml
- analyse docker images:
CLAIR_OUTPUT=High CLAIR_ADDR=http://192.168.99.100:30060 klar [docker file]
CLAIR_OUTPUT=High CLAIR_ADDR=http://192.168.99.100:30060 klar lotharschulz/hellogo:build.docker-cache--0.2.91
CLAIR_OUTPUT=High CLAIR_ADDR=http://192.168.99.100:30060 klar lotharschulz/hellogo:build.docker-min-compress--0.2.91
docker hub images sizes