fix: retire the whole per-lane BuildKit object set, not just the claims #429
Loading…
Reference in a new issue
No description provided.
Delete branch "fix/per-lane-buildkit-tombstone-v2"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
#426 left work undone. Its tombstone deleted claims only, and refused four of the five because their Deployments were still running — which is what it is designed to do, and which surfaced the real problem: the Binjovi Kustomization is
prune: false, so deleting a manifest stops Flux managing an object without deleting it. Four daemons kept running with nothing in the tree declaring them, and the claims they held could not go while they ran.It is not only the Deployments. 42 objects were orphaned across two namespaces and eight kinds: Deployments, Services, ConfigMaps, claims, cert-manager Issuers and Certificates, CiliumNetworkPolicies, and one cluster-scoped ClusterBuildStrategy. The Certificates are the part that is not merely untidy — cert-manager keeps renewing them for daemons that no longer exist.
Deployments go first and the Job waits for their pods, because a running pod holds the
pvc-protectionfinalizer and a claim delete would hang to the deadline rather than fail.The object list and the per-namespace RBAC are generated from the deleted manifests rather than transcribed, and every rule is
resourceNames-scoped to the exact objects.Tested against a stubbed API before shipping
That found two defects a syntax check could not:
IFS=| readkeeps that whitespace — so every field carried leading spaces, no comparison matched, and the deployments phase silently deleted nothing;[ "$kind" = deployments ] && continuereturns non-zero on the false branch, and underset -ethat kills the subshell mid-loop.The stub run now issues exactly 42 deletes — 12 certificates, 8 issuers, 5 network policies, 4 each of deployments/services/configmaps/claims, 1 ClusterBuildStrategy — in the right order, exit 0.
https://claude.ai/code/session_01KZoQin34jeyt6nDGqvJA76
The first tombstone deleted claims only, and refused four of the five because their Deployments were still running -- which is what it is designed to do, and which surfaced the real problem: the Binjovi Kustomization is prune: false, so deleting a manifest stops Flux managing an object without deleting it. Four daemons kept running with nothing in the tree declaring them, and the claims they held could not go while they ran. It is not only the Deployments. 42 objects were orphaned across two namespaces and eight kinds: Deployments, Services, ConfigMaps, claims, cert-manager Issuers and Certificates, CiliumNetworkPolicies, and one cluster-scoped ClusterBuildStrategy. The Certificates are the part that is not merely untidy -- cert-manager keeps renewing them for daemons that no longer exist. Deployments are deleted first and the Job waits for their pods, because a running pod holds the kubernetes.io/pvc-protection finalizer and a claim delete would hang to the deadline rather than fail. The object list and the per-namespace RBAC are generated from the deleted manifests rather than transcribed, and every rule is resourceNames-scoped to the exact objects. TESTED AGAINST A STUBBED API BEFORE SHIPPING, which found two defects a syntax check could not: * the object table is indented for readability, and `IFS='|' read` keeps that whitespace, so every field carried leading spaces, no comparison matched, and the deployments phase silently deleted nothing; * `[ "$kind" = deployments ] && continue` returns non-zero on the false branch, and under `set -e` that kills the subshell mid-loop. The stub run now issues exactly 42 deletes -- 12 certificates, 8 issuers, 5 network policies, 4 each of deployments, services, configmaps and claims, and 1 ClusterBuildStrategy -- in the right order, and exits 0. Claude-Session: https://claude.ai/code/session_01KZoQin34jeyt6nDGqvJA76