blog/content/posts/2023/09/act-runner-image.md

title	date	tags
Creating a base OCI image for Nix flake builds within Gitea/Forgejo	2023-09-18	nix nixos ci docker oci

I've been moving more and more of my infrastructure to be self-hosted recently. Part of that involves setting up CI jobs for testing and publishing artifacts, mostly rust crates but also this very blog.

I really wanted to re-use my existing Nix flakes for those projects, this way I know my local dev env would be the same env then used on CI.

I am self-hosting a Gitea instance (will probably be migrating to Forgejo) and it uses a CI system built to resemble Github actions - basically you run your jobs as containers and within those you can run arbitrary commands. You can also take advantage of the existing ecosystem of actions.

I wanted a base image that would have on one hand nix with flakes enabled but on the other hand would be compatible with running popular actions from other authors. This meant having nix, git but also nodejs available amongts other things. I couldn't find one that would have both, so I built one !

I'm building on top of the definitions from docker-nixpkgs and just tweaking them to add the things needed for actions and also for the definition itself to be a flake, for an added flavour. This allows me to add multiple image definitions in the same repo and the build them independently when needed.

Here's the whole definition in all of its glory, defining 2 images - hello and flakes-action - the hello being a test image for testing the process itself and flakes-action is the one I'm using on CI currently.

{
  description = "docker base images";

  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
    flake-utils.url = "github:numtide/flake-utils";
  };

  outputs = { self, nixpkgs, flake-utils }:
    flake-utils.lib.eachDefaultSystem
      (system:
        let
          pkgs = import nixpkgs { inherit system; };
          pkgsStatic = pkgs.pkgsStatic;
          lib = pkgs.lib;

        in
        {
          packages = {
            hello = pkgs.dockerTools.buildImage {
              name = "hello-docker";
              config = {
                Cmd = [ "${pkgs.hello}/bin/hello" ];
              };
            };
            flakes-action = pkgs.dockerTools.buildImageWithNixDb {
              name = "flakes-action";
              contents = with pkgs; [
                ./root
                bash
                coreutils
                curl
                gawk
                gitFull
                git-lfs
                gnused
                nodejs
                wget
                sudo
                nixFlakes
                cacert
                gnutar
                gzip
                openssh
                xz
                (pkgs.writeTextFile {
                  name = "nix.conf";
                  destination = "/etc/nix/nix.conf";
                  text = ''
                    accept-flake-config = true
                    experimental-features = nix-command flakes
                  '';
                })
              ];

              extraCommands = ''
                # for /usr/bin/env
                mkdir usr
                ln -s ../bin usr/bin

                # make sure /tmp exists
                mkdir -m 1777 tmp

                # need a HOME
                mkdir -vp root
              '';
              config = {
                Cmd = [ "/bin/bash" ];
                Env = [
                  "LANG=en_GB.UTF-8"
                  "ENV=/etc/profile.d/nix.sh"
                  "BASH_ENV=/etc/profile.d/nix.sh"
                  "NIX_BUILD_SHELL=/bin/bash"
                  "NIX_PATH=nixpkgs=${./fake_nixpkgs}"
                  "PAGER=cat"
                  "PATH=/usr/bin:/bin"
                  "SSL_CERT_FILE=${pkgs.cacert}/etc/ssl/certs/ca-bundle.crt"
                  "USER=root"
                ];
              };
            };
          };
        });
}

If you want to build this yourself you can:

git clone https://git.cyplo.dev/cyplo/base-images.git
cd base-images
nix build '.#flakes-action'
docker load < result # this took me so much time, to realise I need `load` and not `import`...
docker tag [image id] yourimage.repo/base-images/flakes-action:latest
docker push yourimage.repo/base-images/flakes-action:latest

Then to use on CI, an example of a Gitea CI config:

on: push
jobs:
  Publish:
    runs-on: flakes-action
    steps:
      - uses: actions/checkout@v3
        name: Checkout
      - name: Build 
        run: |
          nix develop -c hugo --gc --minify          

It uses the image pushed and both a custom build script but also a well-known checkout action.

You need to teach your Gitea runner about the image first btw; if you use NixOS for the runner definition, it could look like this:

services.gitea-actions-runner = {
  instances.boltyone = {
    enable = true;
    url = "https://yourgitea.domain";
    tokenFile = config.sops.secrets."gitea-runner-token".path;
    name = "bolty one";
    labels = [
      "flakes-action:docker://yourimage.repo/base-images/flakes-action:latest"
      "ubuntu-kinetic:docker://ubuntu:kinetic"
      "linux_amd64:host"
    ];
  };
};

P.S. shoutout to nixery that I tried first and the resulting images were just a bit off as it was not easy to get them to support flakes. I think it's an amazing tool in its own right though and you should try it, you can do things like docker run -ti nixery.dev/shell/git/htop bash and it will happily just give you an image with those arbitrary nixpkgs included !

Happy hacking !