feat(object-store): declare the RustFS tenant's desired state #132
Loading…
Reference in a new issue
No description provided.
Delete branch "sean/object-store-desired-state"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Stage 2 of the MinIO retirement, declaration half. Seven buckets and five identities -- binjovi-logs, forgejo, loki, tempo, zot -- each with its exact policy document, the buckets it may reach, the buckets it must be denied, and the namespace/Secret/key names its credential is delivered under. Every policy is copied verbatim from what is live (tenantbucket/ and rustfs-identities/tempo.yaml); the twelve package and RPG identities exist only inside MinIO and are added as each is read out during Stage 3. The guard carries the two lessons already paid for: a wildcard is refused in the BUCKET position (splitting olly-platform's platform-* is the largest security improvement in this migration) while s3:* as an ACTION over an identity's own bucket stays legal, because forgejo and zot hold exactly that; and the endpoint is pinned to the ClusterIP Service with an explicit :9000, because the headless Service caused two HTTP 499 failures during the 2026-09-02 node roll and RustFS has no port-80 listener. Eleven negative controls run on every invocation. Nothing reads the file yet. https://claude.ai/code/session_01KZoQin34jeyt6nDGqvJA76
Stage 2 of the MinIO retirement: Binjovi provisions the object store instead of the `minio-resource-operator` (16 CRDs, 3 CRs, zero MinioUser objects -- every user today is an imperative `mc admin user add`) and the `tenantbucket/` converge Jobs. This is the declaration half. The reconciler reads it; nothing reads it yet. WHAT IS DECLARED. Seven buckets and five identities: `binjovi-logs`, `forgejo`, `loki`, `tempo` and `zot`. Each identity carries the exact policy document, the buckets it may reach, the buckets it must be DENIED, and the namespace, Secret name and key names its credential is delivered under. EVERY POLICY HERE IS SOURCED, NOT INVENTED. `binjovi-logs`, `forgejo`, `zot` and the tempo policy are copied verbatim from what is live -- `tenantbucket/{binjovi-logs,service-identities}.yaml` and `rustfs-identities/tempo.yaml`. `loki` is the tempo shape over its own two buckets. The twelve package and RPG identities are NOT here: their policies exist only inside MinIO, created imperatively, so they are added as each is read out of the live tenant during Stage 3. THE KEY NAMES ARE THE CONSUMERS'. Three conventions are already in use -- `minio_access_key`/`minio_secret_key` for olly, forgejo and zot, `access_key`/`secret_key` for binjovi-logs, and `accesskey`/`secretkey` for the build agents. They are recorded per identity rather than normalised, so repointing a consumer stays an endpoint change and nothing else. THE GUARD CARRIES THE TWO LESSONS THIS MIGRATION HAS ALREADY PAID FOR. The WILDCARD, refused in the bucket position only. MinIO's widest credential is `olly-platform`: `platform-*` over bucket names, plus `workflows` and `pkg`, held by five different Secrets. Splitting it is the largest security improvement available here, so a wildcard bucket is refused -- while `s3:*` as an ACTION over an identity's own bucket stays legal, because forgejo and zot hold exactly that today. Both halves are proved: `s3:*` is accepted, `platform-*` and a bare `*` are refused. The ENDPOINT, pinned to the ClusterIP Service with an explicit `:9000`. The headless Service hands out individual server addresses and an upload that selects one during a node roll fails with HTTP 499 -- twice on 2026-09-02. And RustFS has no port-80 listener, so an implicit port means a repointed consumer talks to nothing. The headless name, a missing port and `:80` are each refused. The rest is hygiene, and all of it is enforced: the policy's resources must equal the identity's declared bucket list, `denied` must be non-empty and name declared buckets it cannot reach, the canary must itself be a declared bucket (a negative check against a bucket that does not exist passes for the wrong reason), and no two identities may write the same Secret. Eleven negative controls run on every invocation: the check mutates its own input eleven ways and requires each to go red. Claude-Session: https://claude.ai/code/session_01KZoQin34jeyt6nDGqvJA76