fix(tests): make 55 refutations across ten guards able to fail #465
Loading…
Reference in a new issue
No description provided.
Delete branch "sean/inert-refutations"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
'! grep ...' written as a bare statement NEVER fails: bash documents that set -e does not exit if a return value is inverted with '!'. Fifty-five assertions were written that way, including no forgejo-admin-token in three execution manifests, no toEntities: [world] in two policies, no BEGIN OPENSSH PRIVATE KEY in the deploy secret, and no wildcard in the database grant. Before converting anything, all 55 were mechanically rewritten to hard-fail and run: exactly one fired, and it was mis-anchored rather than wrong. That is what makes the conversion safe in one pass. The correct form, '! grep ... || fail ...', consumes the status and is left alone (ten remain). The helper lives in tests/refute.sh rather than tests/lib.sh, which is the cluster test library with its own fail. An unreadable input is treated as a failure, not an absence. tests/refutations-are-effective.sh exercises the helper three ways and refuses any guard that goes back to a bare '! grep', with a non-vacuous scan and three negative controls. Stacks on #464. https://claude.ai/code/session_01KZoQin34jeyt6nDGqvJA76
RELEASE THIS AFTER the controllers are gone, not with them. Both tombstones here refuse until they are, but the ordering is real: a CRD deleted while its controller still watches it wedges on a finalizer. ELEVEN CRDs. Three from Argo Events, eight from Argo Workflows. The three ArgoCD CRDs share the argoproj.io group and stay -- 28 live Applications need them. The `crds` Flux Kustomization is prune: false and says why in its own comment ("CRITICAL: Never prune CRDs to prevent data loss"), so removing the files stops Flux re-applying them and nothing more; a tombstone Job does the delete, with the eleven names pinned in `resourceNames` so RBAC is the fence whatever the script says. Removing workflows.argoproj.io garbage-collects the 64 finished Workflow records and their pods, on purpose. FOURTEEN RBAC OBJECTS. Three ServiceAccounts with no running pod, four `*-executor` RoleBindings bound to a Role the Argo cascade deletes, Role/RoleBinding binjovi-deploy-flux in ns flux-system -- a standing write grant on Flux objects held by an account nothing uses, while the native flux agents have their own -- and five binjovi-argo-* CiliumNetworkPolicies that select isolation labels NO RUNNING POD carries. THE ASYMMETRY IS THE RISK. binjovi-deploy-argocd was created for Argo like all of them and shares their naming, but the live native binjovi-argocd-agent pod runs as it. It stays; its `-executor` RoleBinding does not. The tombstone cannot reach it -- it is absent from every resourceNames list -- and the Job asserts it is present before and after. `binjovi-control-workflows` keeps its pod and log reads and loses its `argoproj.io/workflows` rule, which is standing privilege on a kind that no longer exists. The file is renamed to what it now is: native-deploy-rbac.yaml. GUARDS. `tests/binjovi-argo-execution.sh` was two guards in one. Its second half pins the live Kyverno credential policy that protects the package and RPG lanes, so it survives as tests/binjovi-untrusted-credential-guard.sh; the first half is replaced by assertions that the retired names cannot return. While rewriting it: every `! grep` refutation in that guard was INERT. Bash documents that `set -e` does not exit "if the command's return value is being inverted with !", so a dozen refutations reported nothing and passed whatever the file said. They now go through a `refute` helper that exits. It caught a mistake in this very commit on its first run -- an over-broad refutation of `apiGroups: [argoproj.io]`, which ArgoCD Applications also need. tests/argo-crds-retired.sh and tests/binjovi-argo-rbac-retired.sh pin the rest, each with negative controls: an ArgoCD CRD inside the delete grant, a removed controller pre-check, a deleted ArgoCD CRD file, a grant widened to the live agent's ServiceAccount, an unpinned rule, and a dropped post-condition each turn a guard red on its own. `tests/argo-workflows-crds/run-all.sh` pinned the v4.0.8 CRD schemas by digest and goes with them, along with its entries in run-all.sh and every-guard-has-a-caller.sh. Claude-Session: https://claude.ai/code/session_01KZoQin34jeyt6nDGqvJA76feadb696dcb634d1e448