Build Docker images on k8s nodes

In one of my previous posts, cross-compiling for Raspberry Pi with Docker, I wrote about and illustrated how the relatively new buildx command made it much easier to build and dump binaries on the host system.

Another useful feature of buildx is the ability to use different drivers when building images. As of today (10.10.2020), it supports 3 different drivers:

1. docker - uses the docker daemon built in builder (default). You’d use this if you want to build and use images locally.
2. docker-container - uses a BuildKit container spawned by docker. Useful when you want to build multi-platform images.
3. kubernetes - uses k8s pods with defined buildkit container images. Similar to the docker-container driver, but instead of building on the host, it builds on a k8s node

I find the kubernetes driver quite useful when you’re low on resources on your host or when you just want your builds to be fast; there’s probably other good reasons to use it (scalability?), but I’m not going to try to convince you. You should just try it and see if it fits your needs or not.

And the setup doesn’t take more than a minute, definitely less than the time it takes to read this post.

I’m going to assume you’ve already created a K8s cluster on GCP or AWS (or some other cloud provider), but to make sure, check that the current context is what you expect:

1

kubectl config current-context

Should, on a GKE cluster, show something similar to:

1

gke_<project name>_<region>_<cluster name>

Now, create a builder:

1

docker buildx create --name <builder name> --driver kubernetes --use

You can provide other options too; let’s say you want a couple of replicas:

1
2
3
4

docker buildx create --name <builder name> \
  --driver kubernetes \
  --driver-opt replicas=2\
  --use

And bootstrap it:

1

docker buildx inspect --bootstrap

That’s it, you’re ready to build images on your k8s nodes.