feat(rustfs): key every declared identity and let Binjovi converge the tenant

sean commented

2026-09-09 09:53:24 +00:00

Owner

Stage 2 cluster half. Four identities (binjovi-logs, forgejo, loki, zot) generate and freeze a RustFS key; the tenant root credential reaches namespace binjovi alone, pinned by resourceNames; the converge agent gets one Secret with a key per identity. tests/rustfs-identities.sh is rewritten for the new split -- a file that still converges keeps every converge rule, a key-only file must generate, name and FREEZE its key -- and gains a both-directions check that every generated key is delivered to the converge and nothing else is. Five new negative controls, all confirmed red. Also adds the object-store agent OAuth client, and denies untrusted workloads the rustfs namespace as MinIO already is. Nothing converges yet. https://claude.ai/code/session_01KZoQin34jeyt6nDGqvJA76

sean added 3 commits

2026-09-09 09:53:24 +00:00

feat(rustfs): key every declared identity and let Binjovi converge the tenant 651c869768

Stage 2 of the MinIO retirement. `sean/binjovi-plans` declares the RustFS
tenant in `object-store.json`, and Binjovi now converges it: buckets, lifecycle
rules, policies, users, attachments, and a proof that each key reaches its own
buckets and is refused the ones it must never reach. This is the cluster half.

THE SPLIT, AND WHY IT IS WHERE IT IS.

External Secrets generates each key, freezes it, and delivers it to the
consuming namespace. Binjovi owns the tenant's shape. Binjovi is arbitrary code
execution as a service, so it must not hold the right to write a Secret into a
consumer namespace; `object-store.json` records each identity's destination
namespace, Secret name and key names so the two halves can be checked against
each other, and gives Binjovi no way to write them.

The tenant ROOT credential leaves namespace rustfs exactly once, to namespace
binjovi, through a ClusterSecretStore that admits that one namespace. Namespace
binjovi holds the control plane. Untrusted project code runs in binjovi-builds
and is refused by infrastructure/binjovi/execution-platform.yaml. The reader
Role is pinned by resourceNames to the root credential and the five identity
keys, so it cannot hand over anything else in the namespace.

FOUR NEW IDENTITIES, KEY ONLY.

binjovi-logs, forgejo, loki and zot each generate and freeze a key. They do not
create their bucket, policy or user -- that is the converge's job now -- and
they do not yet deliver to their consumer. Delivery lands in Stage 3, in the
commit that repoints the consumer, because the consumer's key names travel with
it. Delivering now would create a Secret nothing reads and a namespace grant
nothing uses.

tempo.yaml keeps its own converge Job. It is the one that proved the mechanism
against RustFS rc.5 on 2026-09-08, it is live, and it retires when Stage 3
repoints olly.

THE GUARD CHANGES WITH THE DESIGN.

tests/rustfs-identities.sh required a converge Job in every identity file. That
is now wrong: a key-only file has none. It holds a file that still converges to
every rule that Job ever had -- ClusterIP endpoint, both halves of the isolation
proof, unconditional `user add`, `ilm rule import` and never `add` -- and holds
a key-only file to generating, naming and FREEZING its key.

It also gains the check the new split needs, in both directions: every
generated key must be delivered to the converge and inside its reader scope,
and the converge must be given no key that no file here generates. A mismatch
would make that identity fail with "no generated key for identity" -- the
intended failure, and loud, but it is a spelling mistake and belongs here.

Five new negative controls, all confirmed to turn the guard red: an identity
key never delivered, a reader Role narrowed past a key it must read, the root
key exposed beyond namespace binjovi, a ClusterSecretStore with its conditions
removed, and a generated key left unfrozen. The six existing controls still
trip.

Nothing converges yet: Binjovi's own object-store lane and the catalog entry
that runs it are separate changes, and the catalog entry lands last.

Claude-Session: https://claude.ai/code/session_01KZoQin34jeyt6nDGqvJA76

fix(cilium): deny untrusted workloads the RustFS namespace as MinIO already is 6099b32025

Two egress deny-lists fence untrusted workloads away from the credential and
state stores: the `binjovi-project-jobs` policy over project code in
binjovi-builds, and the buildkit-bench daemons. Both deny `minio`. Neither
denied `rustfs`, because the namespace arrived after they were written.

So for the length of this migration an untrusted Job has been denied the old
object store and allowed the new one, and the gap widens with every bucket that
moves. RustFS holds Tempo's traces today; Stage 3 adds loki, binjovi-logs,
forgejo and pkg, and Stage 4 adds the whole registry.

The buildkitd policy is deliberately NOT changed. It stopped denying `minio` on
2026-09-07 when the Zelnet package and IBM i RPG lanes joined that daemon:
their builds fetch and push artifacts over the object store, and Cilium
resolves deny before allow, so a deny there breaks those lanes at build time
with a connection timeout and nothing pointing at the cause. That asymmetry is
recorded in both places now, so the next reader does not "fix" one to match the
other.

Claude-Session: https://claude.ai/code/session_01KZoQin34jeyt6nDGqvJA76

feat(binjovi): register the object-store agent's OAuth identity

binjovi/ci Binjovi completed the frozen plan

Details

48bb2d1f50

The lane that converges the RustFS tenant needs a client credential like every
other native agent. Same shape as the eleven that already exist: a generated
password, frozen once, and a Job that upserts the Hydra client with the lane's
own scope.

Two things differ, both on purpose.

The scope is `binjovi.agent.object-store`, which no other client holds. The
control plane derives an agent's lane from its client id, so this credential
can lease object-store work and nothing else, and no other credential can lease
object-store work.

The Secret is mounted straight from its source rather than copied out through
ClusterSecretStore/binjovi-agent-oauth. The agents in namespaces workflows and
binjovi-builds need that copy because they are in another namespace. This agent
runs in namespace binjovi, beside the control plane, because that is the one
namespace the tenant root credential is admitted to. Adding the hop would mean
either widening that store to namespace binjovi -- exposing all twenty-seven
agent client secrets there -- or a second store carrying one Secret across zero
namespaces.

Claude-Session: https://claude.ai/code/session_01KZoQin34jeyt6nDGqvJA76

sean added 1 commit

2026-09-09 10:18:47 +00:00

fix(authz): declare pipeline-cosign-signer, which four places only ever used

binjovi/ci Binjovi completed the frozen plan

Details

7af70b39ce

The identity that cosign-signs the kernel chain was referenced in four places
and declared in none: the kernel-chain 5-sign Job runs as it, a RoleBinding in
namespace kyverno grants it, an OpenBao auth role binds it, and two Kyverno
guards name it in their allow-lists. Something in the retired Pipelines stack
created it, so it went when that stack did.

WHAT THAT LOOKED LIKE, WHICH IS THE POINT.

Nothing said "serviceaccount not found" anywhere a person would look. The Job
was created; the Job CONTROLLER then refused to create its pod, so the object
sat `Running 0/1` for two and a half days with the reason only in an event and
2,663 identical FailedCreate warnings behind it. `wait: true` on
Kustomization/shipwright-kernel-sign then waited out its full twenty-minute
health check, latched Progressing, and the seanfarm Flux deploy failed on it --
which is what blocked the release train for this repository.

The signed artifacts are fine. The last real signing run completed before the
identity vanished, and re-signing at the same digest is idempotent, so what has
been failing since is a re-run. But the NEXT kernel tag could not have been
signed at all, and the in-place roll's stager refuses an unsigned image -- so
this sat directly on the roll path.

It is declared here, beside its own RoleBinding and OpenBao role, so the
identity and its grants retire together next time.

A GUARD FOR THE CLASS, NOT THE CASE.

tests/service-account-references.sh refuses any pod reference to a
ServiceAccount this repository does not declare. It reads PARSED documents
rather than text, because every CRD schema describing a pod template contains
the literal `serviceAccountName:` as a property name. 75 references, 121
declarations, all resolved. Two negative controls: removing this declaration
again, and misspelling a live reference.

This is the third time a reference has outlived its declaration in this
retirement -- a ServiceAccount and two Forgejo tokens in namespace workflows
were the first two -- and the first time anything catches it before the cluster
does.

RECORDED, NOT FIXED: the same OpenBao role still binds
binjovi-release-publisher, which this migration's own RBAC tombstone deleted
yesterday. The bind now grants nothing. Removing it also means retiring that
name from two Kyverno credential guards and the lines that pin them, and those
describe which credentials a registry-publish pod may mount. That is a security
boundary and its own change.

Claude-Session: https://claude.ai/code/session_01KZoQin34jeyt6nDGqvJA76

binjovi-bot merged commit 7af70b39ce into trunk

2026-09-09 10:35:18 +00:00

Rows
Columns

feat(rustfs): key every declared identity and let Binjovi converge the tenant #472