Builder Images

This repository holds container images. Pipelines use these images to build seanfarm infrastructure components.

Images

An image is onboarded when its directory has an images/<name>/VERSION file. The dockerfile-set pipeline then builds the image. The list below does not show every image. Most images have their own version file. Examples: pipeline-tools, envtest-tester, hetzner-builder, cast-render, ibmi-builder, zig-builder, zdns-builder, pipelines-ci, linux-kernel, zfs-userspace, and node-bootc. See images/ for the full list.

Image	Purpose
zot	Patched Zot registry runtime for AMD64 and ARM64, built from the pinned seanfarm fork
binjovi-buildkit-client	Fixed rootless BuildKit worker and exact-revision image discovery on AlmaLinux 10
binjovi-node-runtime	Native node helper runtime with util-linux namespace entry and no interpreter
binjovi-gitops-validator	Fixed, shell-safe GitOps validation worker for Binjovi builds on AlmaLinux 10
binjovi-flux-deployer	Fixed Flux publication worker and package-complete runtime base for the native Binjovi agent
binjovi-argocd-deployer	Fixed, trusted Argo CD convergence observer that terminates an older operation that still runs when a newer release starts observing, with the current HTTP, DNS, TCP, and registry availability probe for Binjovi releases on AlmaLinux 10
binjovi-package-tester-debian	Credential-free DEB verifier that installs exact packages from a signed private APT repository
binjovi-package-tester-alma	Credential-free RPM verifier that installs exact packages from a signed private DNF repository on AlmaLinux 10
binjovi-package-publisher	Fixed repository stager, reader, candidate finalizer, production signer, verifier, and Forgejo Release asset publisher
kernel-builder	Linux kernel + OpenZFS build environment (multi-arch cross-toolchains, sccache+ccache, mc, nfpm) on AlmaLinux 10
elixir-builder	Elixir 1.20 and Erlang/OTP 29 toolchain for Binjovi Sol on AlmaLinux 10
elixir-tester	elixir-builder plus a baked PostgreSQL 16.14 server for repository gates on AlmaLinux 10
beam-runtime	Runtime base for Elixir releases: five pinned libraries and the uid 914 beam user on AlmaLinux 10
c-builder	Baked gcc and make for C validation gates on AlmaLinux 10
pkg-host	Server-rendered, read-only MinIO package edge on AlmaLinux; replaces nginx + custom browser JavaScript
static-web	Read-only Go static-file server on AlmaLinux for ConfigMap-backed sites; replaces generic nginx/alpine utility containers
availability-probe	Host-network HTTP, DNS, TCP, registry, and bounded CLI artifact release availability gate with a zero-failure observation window and strict OCI-or-Git release identity
envtest-tester	Fixed Go and Kubernetes envtest toolchain for isolated native operator test stages
ibmi-analyst	Ghidra 11.1.2 with the PowerAS IBM i processor extension, the pinned sean/rpg-demo recovery toolkit, its build-time tool lock, and zelnet on AlmaLinux 10; the fleet's only Java and Python carve-out
sysadmin	One Image To Rule Them All — cluster admin CLIs + arbitrary OS image construction + OpenZFS userspace + grub2/xorriso + skopeo/buildah/oras/cosign + qemu + DNS/network diags (Nix flake; the one deliberate non-Alma exception)

The availability probe binds every result to one source revision and one release identity. OCI releases supply digest; GitOps releases supply the 64-character release_id. A configuration that supplies both fields, or neither field, fails before the first network request. This keeps availability evidence attributable to one immutable release.

The pipeline-tools image also provides binjovi-oci-publish --proof. Proof mode copies one quarantine image by its expected digest to the fixed build/binjovi-publisher-proof repository. Its destination tag is derived from the image name and digest. An existing tag with another digest fails closed. The command writes the verified destination reference and digest as JSON. It does not publish a release tag and it cannot select another repository.

The pipeline-tools image also provides the trusted Binjovi Go operator publisher. It copies one exact candidate digest to one project-matched release tag. It rejects tag collisions, signs the immutable digest with the OpenBao cosign-ec key, and verifies that signature with an exported public key before it returns release evidence.

The go-builder and envtest-tester images supply toolchains. Native recipe Dockerfiles own source checks, compilation, and tests. The former Git-cloning Go wrapper commands are retired.

Building

The binjovi-dockerfile-set command uses the shared binjovi-gitops-select binary to fetch only missing base and source commits. Each task gets a separate working tree. The repository cache lock covers fetch and clone setup, then releases before source checks run. Retained refs keep borrowed Git objects available to concurrent tasks. Set BINJOVI_DOCKERFILE_CACHE to select the cache root; the default is /tmp/binjovi-dockerfile-cache.

Binjovi builds these images natively in the seanfarm cluster. It compiles the image set into a fanout graph of durable tasks that the build agents claim, and publishes each item as its own durable task. The ci-builder-images-pipeline Argo Workflow that used to do this is gone: it lived in sean/pipelines, which is archived, and the Argo Workflows engine was removed from the cluster on 2026-09-08.

Publishing a version is only half of shipping it

Several of these images are not run from this repository. sean/binjovi copies tools out of them at build time, pinned by tag and digest, so a new version here reaches nothing until that pin is bumped there. Nothing fails in the meantime: this repository builds and publishes exactly as it should, and the consumer keeps running the version it was pinned to.

That gap has bitten once, and it hid well. binjovi-buildkit-client v7 fixed binjovi-dockerfile-set to fetch +refs/pull/*/head, without which the tool cannot resolve any commit created by an agit push -- every pull request in this fleet. v7 was built, published and signed on 2026-09-08; sean/binjovi stayed pinned to v4 for a day. Trunk builds kept passing the whole time, because a plain clone can see a trunk commit, so the lane looked healthy while no pull request in this repository could build at all.

So: after publishing a new version of an image sean/binjovi consumes, bump the matching FROM in its Dockerfile in the same sitting. container_contract_test.exs there pins the minimum binjovi-buildkit-client version with the reason, so that one cannot silently go backwards.

Successful Builds keep candidate images in the quarantine repository. A Release copies the exact candidate digests to registry.sean.farm/build/<image>. Zot gives the builder-images publisher read, create, and update access only to the listed image repositories. It does not give the publisher delete access.

The binjovi-package-publisher image build also runs a fixed repository round trip. It creates real DEB and RPM fixtures, stages and signs a repository, reads it back through the object-store boundary, and verifies every digest and signature. Each member record binds the package name, media type, byte size, and digest. A mismatch between the stage and fetch byte contracts stops the image build before publication. Release publication uses a read-only source key for private candidates and a separate promote-only key for the public package repository.

Server-package Builds use a different typed path. The project builder runs only the repository-owned ci/binjovi-package.sh file. A fixed task creates APT and DNF repositories, signs their metadata with a new ephemeral key, and writes an immutable private staging manifest last. A credential-bearing init container verifies and copies that repository into an emptyDir. The Debian and Alma test containers have no Secret. They use apt-get and dnf to install exact package versions from the signed local repository. Distribution dependencies come through the in-cluster package cache.

After both repository tests pass, a fixed finalizer records the exact repository as the candidate. The Release publisher copies the tested package bytes and unsigned metadata without regeneration. It replaces only the ephemeral signatures with production signatures. A separate pod downloads and verifies all public bytes before the Release can succeed. Forgejo asset publication uses a repository-bounded token and downloads the same public members by digest. For a desktop Release, the finalizer can also accept MP4 and JPG reel pairs. It stores these files outside the tested package repository and adds them to the Release manifest. Thus, the repository digest does not change, but the Release manifest binds each media file by name, media type, size, and digest. A missing half of a reel pair stops the Release. The disposable install-test pods run as root only because dpkg and dnf must write the test filesystem. They have no package-store, signing, Forgejo, or platform credential.

Trigger

Open a feature pull request against trunk. Binjovi validates the onboarded image set. Release publishes the exact candidate digests from the selected Build.

Native pipeline timing

Native discovery and build tasks produce the candidate image set. Release publishes and verifies each selected image digest. This project has no Deploy target; consumers update their image pins separately.

Open the project dashboard and expand an execution attempt for the task timeline and available CPU, off-CPU, and syscall profiles. Cached BuildKit steps still appear as cached work; a missing sample does not prove that a task used no CPU.

Select an exact successful Build with binjovictl release builder-images --build BUILD_ID --wait. Then use binjovictl status builder-images to check its release and any configured deployment. Telemetry explains time spent; immutable artifact and revision evidence establish completion.

The GitOps validator keeps an immutable ref for each cached commit. It uses a shared checkout while the BuildKit cache is locked, so Git cleanup cannot remove the source or base commit during validation.