My main contribution to the Squid Web Cache project is these days running the project’s infrastructure. A lot of it is the project’s CI/CD farm.
In order to run it, we rely on a very kind donation by DigitalOcean . We use a VM hosted there to run the main jenkins instance and part of the jobs for the x86-64 architecture. We are then using the jenkins digitalocean plugin to spin up instances (droplets) on demand when we need to have more throughput from our build jobs.
In order to maximise how we use our resources, we rely on docker to run all of our target linux userlands. This allows us to decouple the runtime environment from the machine that’s running it, and to ensure consistency across builds (also coming up: a proper staging system).
In this post I’ll focus on how we spin up these instances, the whole setup is a bit more convoluted.
The digitalocean plugin is quite well integrated and easy to use; TBH I haven’t tried plugins for EC2 or GCP, but my other reference point, jclouds, was much harder to configure and set up.
Given our prerequisites, we need ondemand instances to only contain the docker runtime and java, which is needed to run the jenkins slaves as unlike other setups I’ve found online, these run outside the docker containers.
In order to do that, we supply to the “User Data” section this config snippet:
#cloud-config
apt_upgrade: true
package_upgrade: true
packages:
- openjdk-11-jre-headless
- docker.io
users:
- name: name of the jenkins user on the executor machine
groups: docker
shell: /bin/bash
ssh-authorized-keys: ssh-rsa ssh public key of the user jenkins runs underThese actions will be run when the droplet is launched, and prep the executor for jenkins to ssh into it and run the test jobs. In order to give the droplet time to do that, we need to wait for it with this init script:
#!/bin/bash
echo "starting init script"
while ! cloud-init status|grep -qF 'done'
do
echo "waiting for cloud-init to be done"
sleep 10
doneThe next tricky bit is in the Droplet section, in the node Labels section we define a label for triggering the instance startup when needed. It can be anything, in our case docker-build-host, and an instance cap.
Referencing this label in the projects’ configuration matrix will trigger the spinup and imaging. Jenkins will then connect to the droplet via ssh and use docker run commands to test the various runtime environments
The Squid Web Proxy/Cache Blog 