https://git.cyplo.dev/cyplo/dotfiles

My dotfiles - including my vim, terminal and font config. My current setup consists of multiple machines running NixOS. This is using flakes for reproducibility and home manager for setting up user-specific things.

generating ssh keys

ssh-keygen -t ed25519

Workstations are set up by running sudo nixos-rebuild switch --flake '.#' and servers are deployed using deploy '.#hostname' (or deploy to deploy all servers). I don't use home manager the program, everything is referenced from the top flake.

adding new workstation to sops

As workstations can be both sources (changing sops-encrypted data) so they need to be able to encrypt stuff; and targets (needing to decrypt a secret in runtime) we need to add both keys to sops;

• add the source key nix-shell -p ssh-to-age --run "ssh-to-age < ~/.ssh/id_ed25519.pub"
• add the target key sudo cat /var/lib/sops-nix/key.txt
• reencrypt relevant files find . -name "*.yaml" -exec sops updatekeys {} \;

Setting up a new server

• use nixos-anywhere + disko

infra setup

graph TB
    subgraph "Workstations"
        thinky["thinky<br/>ThinkPad<br/>Linux/GNOME"]
        foryog["foryog<br/>Yoga<br/>Linux/GNOME"]
        airy["airy<br/>macOS (aarch64)"]
        cushy["cushy<br/>macOS (aarch64)"]
    end

    subgraph "Servers"
        bolty["bolty<br/>NAS/HomeServer<br/>10.0.0.0/24 advertiser"]
        mb1["mb1<br/>VPS"]
        cupsnet["cupsnet<br/>VPS<br/>DNS: BIND (pni.frl)"]
        amshed["amshed<br/>VPS<br/>Headscale control plane"]
    end

    subgraph "DNS Servers"
        bind["BIND DNS<br/>(on cupsnet)<br/>Authoritative for pni.frl"]
        quad9["Quad9<br/>IPv6: 2620:fe::fe, 2620:fe::9<br/>IPv4: 9.9.9.9, 149.112.112.112"]
    end

    subgraph "Tailscale Network (pni.frl)"
        headscale["Headscale<br/>Control plane: pni.frl<br/>Magic DNS: magic.pni.frl<br/>(on amshed)"]
        tailnet["Tailnet<br/>100.64.0.0/10<br/>DNS: 100.100.100.100 (MagicDNS)"]
    end

    %% Tailscale connections
    thinky -.->|Tailscale| tailnet
    foryog -.->|Tailscale| tailnet
    airy -.->|Tailscale<br/>via app| tailnet
    cushy -.->|Tailscale<br/>via app| tailnet
    bolty -.->|Tailscale<br/>Route Server| tailnet
    mb1 -.->|Tailscale| tailnet
    cupsnet -.->|Tailscale| tailnet
    amshed -.->|Hosts| headscale
    headscale -->|Controls| tailnet

    %% DNS relationships
    tailnet -->|Auth DNS only<br/>pni.frl zone| bind
    tailnet -->|MagicDNS<br/>100.100.100.100| headscale
    tailnet -->|Primary DNS<br/>IPv6 preferred| quad9
    bind -->|No recursion<br/>Auth only| quad9
    cupsnet -->|Hosts| bind
    headscale -->|Fallback DNS| quad9

    %% Special routes
    bolty ==>|Advertises<br/>10.0.0.0/24| tailnet

    style tailnet fill:#e1f5fe
    style headscale fill:#bbdefb
    style bind fill:#fff3e0
    style quad9 fill:#f3e5f5
    style bolty fill:#c8e6c9
    style cupsnet fill:#c8e6c9
    style mb1 fill:#c8e6c9
    style amshed fill:#c8e6c9

Anubis Protection for Web Services

This configuration uses Anubis to protect web services from AI scrapers and bots using proof-of-work challenges.

Architecture

graph LR
    subgraph "Internet"
        user["User/Bot"]
    end

    subgraph "Edge Layer"
        nginx["Nginx<br/>(HTTPS/TLS)"]
    end

    subgraph "Protection Layer"
        anubis["Anubis<br/>(PoW Challenge)"]
    end

    subgraph "Service Layer"
        forgejo["Forgejo<br/>:8083"]
        blog["Blog Nginx<br/>:3923"]
    end

    user -->|HTTPS| nginx
    nginx -->|Unix Socket| anubis
    anubis -->|HTTP| forgejo
    anubis -->|HTTP| blog

    style anubis fill:#ff6b6b
    style nginx fill:#4ecdc4
    style forgejo fill:#45b7d1
    style blog fill:#96ceb4

How It Works

1. Initial Request: Users access the service through the public domain (e.g., git.cyplo.dev)
2. Nginx Frontend: Handles TLS termination and proxies to Anubis via Unix socket
3. Anubis Challenge:
   • First-time visitors receive a JavaScript proof-of-work challenge
   • Must calculate SHA256 hashes with 4 leading zeros (configurable difficulty)
   • Valid solutions earn a cookie (48-hour duration)
4. Service Access: After solving, requests are proxied to the actual service

Configuration

Services are configured using the services.custom.webserver module:

services.custom.webserver = {
  enable = true;
  name = "myservice";           # Unique identifier
  domains = [ "example.com" ];  # List of domains
  port = 8080;                  # Port where your service listens
};

Each service must provide its own HTTP server on the specified port. For static sites, use nginx:

services.nginx.virtualHosts."mysite-internal" = {
  listen = [{ addr = "127.0.0.1"; port = 8080; }];
  locations."/" = {
    root = "/var/www/mysite";
    extraConfig = "try_files $uri $uri/ =404;";
  };
};

Monitoring & Metrics

This configuration uses VictoriaMetrics for metrics collection and storage, replacing Prometheus and Grafana.

Architecture

graph TB
    subgraph "Remote Hosts - Tailnet"
        workframe["workframe<br/>Node Exporter :9100<br/>LLM Servers :8101<br/>LiteLLM :8102<br/>SearXNG :8089"]
        cupsnet["cupsnet<br/>Node Exporter :9100<br/>Forgejo :8083<br/>Mastodon :3000"]
        foryog["foryog<br/>Node Exporter :9100"]
    end

    subgraph "Remote Hosts - Wireguard"
        mb1["mb1<br/>Node Exporter :9100<br/>wg-tunnel"]
    end

    subgraph "bolty - Metrics Server"
        vmagent_local["vmagent<br/>Local scraper"]
        vmagent_remote["vmagent<br/>(remote hosts)<br/>Metrics collectors"]
        victoriametrics["VictoriaMetrics<br/>:8428<br/>Time series DB"]
        vmui["VMUI<br/>victoriametrics.pni.frl/vmui<br/>Built-in web UI"]
    end

    subgraph "Local Sources on bolty"
        node["Node Exporter :9100"]
        growatt["Growatt 10.0.0.122:80"]
        servarr["Servarr Suite<br/>Sonarr, Radarr, Prowlarr<br/>Lidarr, Bazarr"]
        media["Media Services<br/>Jellyfin, Navidrome"]
        download["Download Clients<br/>qBittorrent, Aria2"]
        homeauto["Home Automation<br/>Zigbee2MQTT"]
        backup["Restic Server :8000"]
        self["Self-Monitoring<br/>VictoriaMetrics :8428<br/>VictoriaLogs :9428"]
    end

    subgraph "Network Paths"
        tailnet["Tailnet<br/>victoriametrics.pni.frl"]
        wg_tunnel["Wireguard<br/>192.168.254.1 → 192.168.254.2"]
    end

    %% Local collection
    node -->|scrape| vmagent_local
    growatt -->|scrape| vmagent_local
    servarr -->|scrape| vmagent_local
    media -->|scrape| vmagent_local
    download -->|scrape| vmagent_local
    homeauto -->|scrape| vmagent_local
    backup -->|scrape| vmagent_local
    self -->|scrape| vmagent_local
    vmagent_local -->|remote_write<br/>localhost| victoriametrics

    %% Remote hosts via tailnet
    workframe -->|vmagent<br/>remote_write| tailnet
    cupsnet -->|vmagent<br/>remote_write| tailnet
    foryog -->|vmagent<br/>remote_write| tailnet
    tailnet -->|http://victoriametrics.pni.frl:8428| victoriametrics

    %% Remote host via wireguard
    mb1 -->|vmagent<br/>remote_write| wg_tunnel
    wg_tunnel -->|http://192.168.254.2:8428| victoriametrics

    %% Visualization
    victoriametrics -->|query| vmui

    style vmagent_local fill:#4ecdc4
    style vmagent_remote fill:#4ecdc4
    style victoriametrics fill:#45b7d1
    style vmui fill:#96ceb4
    style tailnet fill:#e1f5fe
    style wg_tunnel fill:#fff3e0
    style node fill:#fff3e0
    style growatt fill:#fff3e0
    style servarr fill:#fff3e0
    style media fill:#fff3e0
    style download fill:#fff3e0
    style homeauto fill:#fff3e0
    style backup fill:#fff3e0
    style self fill:#fff3e0

How It Works

1. Metrics Collection: vmagent scrapes metrics from various exporters and services
2. Storage: Metrics are sent via remote_write to VictoriaMetrics for efficient storage
3. Querying: VictoriaMetrics provides a Prometheus-compatible API and built-in VMUI for visualization
4. Access: VMUI is accessible via tailnet at https://victoriametrics.pni.frl/vmui

Configuration

• Metrics Module: Shared module at nixos/modules/metrics.nix that configures node exporter and vmagent
• VictoriaMetrics: Single-node storage, configured in nixos/boxes/bolty/victoriametrics.nix
• vmagent: Replaces Prometheus for scraping, configured via the metrics module
• Node Exporter: System metrics exporter, enabled by the metrics module

Scraped Services

bolty (local server):

• System metrics (node exporter)
• Growatt solar inverter
• VictoriaMetrics self-metrics
• VictoriaLogs self-metrics
• Servarr suite: Sonarr, Radarr, Prowlarr, Lidarr, Bazarr
• Media services: Jellyfin, Navidrome
• Download clients: qBittorrent, Aria2
• Home automation: Zigbee2MQTT
• Backup: Restic server

cupsnet:

• System metrics (node exporter)
• Forgejo (Git server)
• Mastodon (social media)

workframe:

• System metrics (node exporter)
• llama-cpp-fast (LLM server)
• LiteLLM proxy
• SearXNG (search engine)

mb1:

• System metrics (node exporter)

foryog:

• System metrics (node exporter)