Home avatar

A collection of dev guides, tutorials and thoughts on various tech stacks, tools and programming languages.

Cross-compile Go with C dependencies for ARM

Go has some pretty good support/tools for cross-compiling binaries for multiple platforms/architectures. Just check the $GOOS/$GOARCH env vars listed in the official documentation.

But sometimes, you may need to use some lib/code written in C that hasn’t been ported to Go. For this reason, there’s cgo, which you can use to call C code from your Go code. And while it can be straightforward to use when you run the binaries on the same platform you used to build them, it can be a bit of a mess when cross-compiling for platforms/architectures that are different.

Build Docker images on k8s nodes

In one of my previous posts, cross-compiling for Raspberry Pi with Docker, I wrote about and illustrated how the relatively new buildx command made it much easier to build and dump binaries on the host system.

Another useful feature of buildx is the ability to use different drivers when building images. As of today (10.10.2020), it supports 3 different drivers:

  1. docker - uses the docker daemon built in builder (default). You’d use this if you want to build and use images locally.
  2. docker-container - uses a BuildKit container spawned by docker. Useful when you want to build multi-platform images.
  3. kubernetes - uses k8s pods with defined buildkit container images. Similar to the docker-container driver, but instead of building on the host, it builds on a k8s node

I find the kubernetes driver quite useful when you’re low on resources on your host or when you just want your builds to be fast; there’s probably other good reasons to use it (scalability?), but I’m not going to try to convince you. You should just try it and see if it fits your needs or not.

How to enable experimental features for Docker in Github's workflow for Ubuntu 18.04?

If you’re using Github’s workflows for CI/CD and you need to use some of Docker’s experimental features, or you want to use buildx or maybe you just want to use some of the new dockerfile experimental syntaxes then you need to enable the experimental features for the CLI and probably the daemon too.

When running natively on Linux or macOS, it’s pretty easy.

To enable the experimental features for the CLI, you just need to add the following to your ~/.docker/config.json config:

How to install and run QEMU on K8s nodes?

If you find yourself having to emulate different CPU architectures in a Kubernetes environment you’ll probably end up running some version of binfmt as an init container or maybe manually run it once.

That would probably be ok in most cases, but wouldn’t work if, for example, you’re running multiple pod replicas on the same node concurrently (say, when you setup a CI that spawns pods for every job in your pipeline) or when nodes are autoscaled.

How to access private Github repos when building Docker images?

If you landed on this page, it means you probably have the same issue I’ve stumbled upon as well: I need to access some private Github repos while building a docker image but I don’t want any ssh keys or credentials to end up in the final image.

While there’s already ways to achieve this, the recent additions of buildx (first release was back in April 2019) and experimental dockerfile syntaxes (available from v18.06 onwards) make it much easier to do it (or just more elegant).

Cross compile for Raspberry Pi with Docker

The Raspberry Pi is a pretty useful tool for quickly prototyping some IoT product, simulating an embedded environment, running a small Kubernetes cluster, etc; there are probably lots of reasons why you’d want to use one (or maybe not?).

As for developing software that can run on it, if you’re writing it in C or C++, there’s many ways to go about it:

  1. Sync the code changes you make on the Pi and build it natively on it (via rsync, scp, SSHFS, git, etc)
  2. Use crosstool-ng to setup a toolchain and use it to cross-compile your software; then copy the resulting binaries onto the Pi
  3. Use a precompiled toolchain to cross-compile the software; then copy the binaries onto the Pi
  4. Develop directly on the Pi through an SSH session
  5. Use a containerized OS such as Balena

And you can probably find many other creative ways developers came up with.

How to install cmake?

CMake is a cross-platform tool for building, testing and packaging software.

If you’ve been writing C or C++ (or maybe Fortran) you probably had to, at some point, use cmake to package your software.

If you’re a Linux user, you’d normally just need to run:

sudo apt-get install -y cmake

to install it. But sometimes, the version that you can install from the official repository is not the one you need.

How to install OpenSSL 1.1.1?

OpenSSL is a TLS/SSL and crypto library.

Most Linux distributions come packaged with some older version of OpenSSL, but if you need some of newest features (such as support for TLSv1.3), then you’ll need to manually install it.

These instructions should work for most Debian based distros.

Install the build dependencies:

sudo apt-get -y install build-essential checkinstall git zlib1g-dev

Clone OpenSSL: