blog/content/posts/2023/09/act-runner-image.md

154 lines
5.9 KiB
Markdown
Raw Normal View History

2023-09-18 17:04:07 +01:00
---
title: Creating a base OCI image for Nix flake builds within Gitea/Forgejo
date: 2023-09-18
tags: [nix, nixos, ci, docker, oci]
---
I've been moving more and more of my infrastructure to be self-hosted recently.
Part of that involves setting up CI jobs for testing and publishing artifacts, mostly rust crates but also this very blog.
I really wanted to re-use my existing Nix flakes for those projects, this way I know my [local dev env](https://git.cyplo.dev/cyplo/blog/src/branch/main/flake.nix#L15) would be the same env then [used on CI](https://git.cyplo.dev/cyplo/blog/src/branch/main/.gitea/workflows/build.yaml#L14).
I am [self-hosting](https://git.cyplo.dev/explore/) a Gitea instance (will probably be migrating to Forgejo) and it uses a [CI system](https://docs.gitea.io/en-us/usage/usage/actions/overview/) built to resemble [Github actions](https://github.com/actions) - basically you run your jobs as containers and within those you can run arbitrary commands. You can also take advantage of the existing ecosystem of `actions`.
I wanted a base image that would have on one hand `nix` with `flakes` enabled but on the other hand would be compatible with running popular actions from other authors. This meant having `nix`, `git` but also `nodejs` available amongts other things. I couldn't find one that would have both, so I built one !
I'm building on top of the [definitions](https://github.com/nix-community/docker-nixpkgs/blob/master/images/nix-flakes/default.nix) from [docker-nixpkgs](https://github.com/nix-community/docker-nixpkgs) and just tweaking them to add the things needed for `actions` and also for the definition itself to be a flake, for an added flavour. This allows me to add multiple image definitions in the same repo and the build them independently when needed.
Here's the whole definition in all of its glory, defining 2 images - `hello` and `flakes-action` - the `hello` being a test image for testing the process itself and `flakes-action` is the one I'm using on CI currently.
```nix
{
description = "docker base images";
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
flake-utils.url = "github:numtide/flake-utils";
};
outputs = { self, nixpkgs, flake-utils }:
flake-utils.lib.eachDefaultSystem
(system:
let
pkgs = import nixpkgs { inherit system; };
pkgsStatic = pkgs.pkgsStatic;
lib = pkgs.lib;
in
{
packages = {
hello = pkgs.dockerTools.buildImage {
name = "hello-docker";
config = {
Cmd = [ "${pkgs.hello}/bin/hello" ];
};
};
flakes-action = pkgs.dockerTools.buildImageWithNixDb {
name = "flakes-action";
contents = with pkgs; [
./root
bash
coreutils
curl
gawk
gitFull
git-lfs
gnused
nodejs
wget
sudo
nixFlakes
cacert
gnutar
gzip
openssh
xz
(pkgs.writeTextFile {
name = "nix.conf";
destination = "/etc/nix/nix.conf";
text = ''
accept-flake-config = true
experimental-features = nix-command flakes
'';
})
];
extraCommands = ''
# for /usr/bin/env
mkdir usr
ln -s ../bin usr/bin
# make sure /tmp exists
mkdir -m 1777 tmp
# need a HOME
mkdir -vp root
'';
config = {
Cmd = [ "/bin/bash" ];
Env = [
"LANG=en_GB.UTF-8"
"ENV=/etc/profile.d/nix.sh"
"BASH_ENV=/etc/profile.d/nix.sh"
"NIX_BUILD_SHELL=/bin/bash"
"NIX_PATH=nixpkgs=${./fake_nixpkgs}"
"PAGER=cat"
"PATH=/usr/bin:/bin"
"SSL_CERT_FILE=${pkgs.cacert}/etc/ssl/certs/ca-bundle.crt"
"USER=root"
];
};
};
};
});
}
```
If you want to build this yourself you can:
```bash
git clone https://git.cyplo.dev/cyplo/base-images.git
cd base-images
nix build '.#flakes-action'
docker load < result # this took me so much time, to realise I need `load` and not `import`...
docker tag [image id] yourimage.repo/base-images/flakes-action:latest
docker push yourimage.repo/base-images/flakes-action:latest
```
Then to use on CI, an example of a Gitea CI config:
```yaml
on: push
jobs:
Publish:
runs-on: flakes-action
steps:
- uses: actions/checkout@v3
name: Checkout
- name: Build
run: |
nix develop -c hugo --gc --minify
```
It uses the image pushed and both a custom build script but also a well-known `checkout` action.
You need to teach your Gitea runner about the image first btw; if you use NixOS for the runner definition, it could look like this:
```nix
services.gitea-actions-runner = {
instances.boltyone = {
enable = true;
url = "https://yourgitea.domain";
tokenFile = config.sops.secrets."gitea-runner-token".path;
name = "bolty one";
labels = [
"flakes-action:docker://yourimage.repo/base-images/flakes-action:latest"
"ubuntu-kinetic:docker://ubuntu:kinetic"
"linux_amd64:host"
];
};
};
```
P.S. shoutout to [nixery](https://nixery.dev/) that I tried first and the resulting images were just a bit off as it was not easy to get them to support flakes. I think it's an amazing tool in its own right though and you should try it, you can do things like `docker run -ti nixery.dev/shell/git/htop bash` and it will happily just give you an image with those arbitrary nixpkgs included !
Happy hacking !