feat(shipwright): sharpen the dedicated build cache #120

Merged

binjovi-bot merged 1 commit from feat/buildkit-cache-flags into trunk

2026-09-01 08:04:09 +00:00

sean commented

2026-09-01 08:02:26 +00:00

Owner

Round two of the build-cache work, driven by tonight's measurements against the shipped registry cache (binjovi v0.1.122 / seanfarm PR #110):

Cache blobs transfer as gzip: 81.5s to pull cached dependency layers, 16-44s to export, every build. The export line gains compression=zstd,force-compression=true.
The cache manifest uses Docker media types. zot-v2's GC treating cache blobs as unreferenced is the same failure shape as the pre-migration wedge, so the export also gains image-manifest=true,oci-mediatypes=true.
The shared tag decays: mode=max only exports blobs a build materialized, so each warm build overwrites the complete cache with a partial one (measured: a 101.7s deps.get re-run after one such shed). New parameter cache-export (default true, existing consumers unchanged) lets a consumer make admission builds import-only, with only trunk verification builds - which run everything and therefore always hold the full set - writing the tag.
Cache traffic transits the external edge; both of tonight's warm-measurement build failures were edge 503s during the zot cutover. New parameter cache-ref-insecure (default empty follows the output setting) lets the cache ref live on zot.registry.svc.cluster.local:5000 while the output image stays on the TLS edge, and the binjovi buildkit-world profile gains registry:5000 egress (same shape as its athens and pkg-cache rules).

All defaults preserve current behavior; the strategy change is inert until binjovi's executor passes the new parameters (companion binjovi PR follows). tests/buildkit-registry/run-all.sh passes; shipwright and binjovi kustomizations build clean.

https://claude.ai/code/session_01LeVatedQ4sCA1u6LHjo7B5

Round two of the build-cache work, driven by tonight's measurements against the shipped registry cache (binjovi v0.1.122 / seanfarm PR #110): - Cache blobs transfer as gzip: 81.5s to pull cached dependency layers, 16-44s to export, every build. The export line gains compression=zstd,force-compression=true. - The cache manifest uses Docker media types. zot-v2's GC treating cache blobs as unreferenced is the same failure shape as the pre-migration wedge, so the export also gains image-manifest=true,oci-mediatypes=true. - The shared tag decays: mode=max only exports blobs a build materialized, so each warm build overwrites the complete cache with a partial one (measured: a 101.7s deps.get re-run after one such shed). New parameter cache-export (default true, existing consumers unchanged) lets a consumer make admission builds import-only, with only trunk verification builds - which run everything and therefore always hold the full set - writing the tag. - Cache traffic transits the external edge; both of tonight's warm-measurement build failures were edge 503s during the zot cutover. New parameter cache-ref-insecure (default empty follows the output setting) lets the cache ref live on zot.registry.svc.cluster.local:5000 while the output image stays on the TLS edge, and the binjovi buildkit-world profile gains registry:5000 egress (same shape as its athens and pkg-cache rules). All defaults preserve current behavior; the strategy change is inert until binjovi's executor passes the new parameters (companion binjovi PR follows). tests/buildkit-registry/run-all.sh passes; shipwright and binjovi kustomizations build clean. https://claude.ai/code/session_01LeVatedQ4sCA1u6LHjo7B5

sean added 1 commit

2026-09-01 08:02:26 +00:00

feat(shipwright): sharpen the dedicated build cache

binjovi/ci Binjovi completed the frozen plan

Details

949250d0ef

The cache-ref export gains zstd with force-compression (measured: 81.5s
pulling cached dependency layers and 16-44s exporting, both gzip) and
the OCI image-manifest form, which a strict registry's garbage collector
recognizes as referenced.

Two backward-compatible parameters follow from tonight's decay
measurement: mode=max only exports blobs a build materialized, so a warm
build that skips stages sheds layers from the shared tag over time.
cache-export=false lets such builds import without overwriting the
complete cache. cache-ref-insecure lets the cache reference live on the
in-cluster HTTP registry endpoint while the output image stays on the
TLS edge; the binjovi buildkit-world profile gains registry:5000 egress
for that path.

Claude-Session: https://claude.ai/code/session_01LeVatedQ4sCA1u6LHjo7B5