If you run CI on cloud runners while your git server lives somewhere else - on-prem, another region, the far end of a slow VPN - you have probably watched a lot of time (and egress) disappear into cloning the same repositories over and over.

The setup I kept running into: an on-prem GitLab, an elastic fleet of runners autoscaling in the cloud, and a handful of repos in the tens of gigabytes. Multiply that by hundreds or thousands of jobs a day and every single one does a fresh clone across the WAN. The link is the bottleneck, the egress is not free, and the runners end up spending more time fetching code than running it.

The chan type in Go is a mechanism which can be used to send or receive data from one function to another. E.g:

func main() {
    c := make(chan string)
    go printHello(c)
    sayHello(c)
}

func sayHello(msgChan chan<- string) {
    msgChan <- "hello"
}

func printHello(msgChan <-chan string) {
    fmt.Println(<-msgChan)
}

Channels can be bidirectional (chan string) or directional (chan <- string sender/<-chan string receiver). And they operate like FIFO queues.

This story relates to The Streams API in JavaScript and Go which I wrote a while back.

To add some context, I was working on a simple app, which is similar to a router dashboard, for one of the products the company I work (Transcelestial) for is developing. And one of the main features of it was realtime metrics from the device.

As described in the prev post, I could have went with either WebSockets or the Streams API (long polling was not an option because we deal with modern browsers), but I chose to go with the streams API, because there was no need for bidirectional comm b/t the client and the server and all the APIs were already in place, so it was easier to just add the extra handler to stream the data.

It’s been some time since I have posted anything (I recently became a father, so I spend most of my free time with the 👶) and for the past few months (about 5 months), my blog has been down.

I’ve noticed some time in November that my blog was throwing a 500 and I just spent maybe 5 minutes to check the logs on the VM where it was running (GCP) to see why. Everything seemed ok (nginx was up and ghost showed no issues either).

You need to be running at least Raspbian OS with Buster or Raspberry Pi OS (might work with older OS versions too, but I haven’t tested it).

I’ve also updated the system, firmware, apps, etc to the latest version:

sudo apt update
sudo apt full-upgrade
sudo rpi-update

Install Go

We need Go to install runc and containerd.

If you ever had to build a realtime web app and you’ve built yourself a REST backend (or you need to use same legacy REST backend), you most likely stumbled upon a pretty common issue: how do I stream a bunch of data from the backend to make it seem it’s updated in realtime?

This is a continuation of the common design patterns I found in my old code series, which I started in a previous post.

The proxy pattern is a design pattern in which a class (proxy) acts as an interface to something else. The proxy could be an interface to anything: a network connection, another class, a file, etc.

A proxy can be useful in a variety of situations:

• A frontend for load balancing
• Hide private infrastructure
• Caching layer
• etc

A good example of a proxy that can be used as a load balancer (and other purposes) is nginx or net/http/httputil (the ReverseProxy).

This is a continuation of the common design patterns I found in my old code series, which I started in a previous post.

The observer pattern is a design pattern in which an object (a subject) keeps track of all of its dependents (observers) and notifies them of any state changes.

In Go, the closest example of this pattern are the builtin channels and the use of goroutines:

sub := make(chan interface{})

go func(c <-chan interface{}>) {
    for data := range c {
        fmt.Println(data)
    }
}(sub)

sub <- "Hey there"

Though, for multiple observers to be notified, you need to send the message once for each observer.

This is a continuation of the common design patterns I found in my old code series, which I started in a previous post.

A while back I was playing around with a Raspberry Pi 4 and some temperature sensors. Some of the sensors I was using were exposing data over the SPI communication interface and some were exposing it over the I2C interface.

But I didn’t want the client app that was reading the temperature to change every time I swapped a sensor, so I ended up making a simple wrapper that exposed a single method to read the temperature and which could be initialized with different protocols.

This is a continuation of the common design patterns I found in my old code series, which I started in a previous post.

The command pattern is a design pattern that encapsulates an action or request as an object that can be parameterized. And it’s commonly associated with terms like receiver, command, invoker and client.

Usually, the invoker doesn’t know anything about the implementation details of the command or receiver, it just knows the command interface and its only responsibility is to invoke the command and optionally do bookkeeping of commands.