fix(forgejo): retire the bucket-init Job, orphaned since the storage cutover #257

Merged

binjovi-bot merged 1 commit from fix/retire-forgejo-bucket-init into trunk

2026-09-04 16:59:57 +00:00

sean commented

2026-09-04 16:56:51 +00:00

Owner

forgejo-bucket-init ran mc mb --ignore-existing s3/alice-forgejo and has not succeeded once since 2026-08-06. That afternoon the storage cutover moved Forgejo to the bare forgejo bucket, then deleted the alice-forgejo bucket outright along with MinIO user alice and alice-policy, and narrowed this Job1s own credential to s3:* on arn:aws:s3:::forgejo and nothing else.

So it asked a deliberately scoped-out key to create a deliberately destroyed bucket. The Access Denied was the boundary working.

Not cosmetic. backoffLimit 20 under restartPolicy OnFailure = ~1.6h of CrashLoopBackOff per day, then a TTL reap and a fresh Job 24h later. The kernel roll green gate fails on ANY crashlooping pod cluster-wide with no exemption, and this was the only pod matching it. It intermittently blocked kernel rolls.

Verified from both sides, not from commit subjects (which are unreliable here: d2583f4f claims forgejo and zot were the only consumers, and this Job was a third). In git alice-forgejo is live config on exactly one line and no guard references it. Live, MinIO holds 11 buckets and none is alice-forgejo. The composition is retired so nothing regenerates it.

Only the second document goes. forgejo-admin-token-issuer stays, byte-identical.

Merging is not sufficient on its own — forgejo-app runs prune: false, so the live Job needs one manual delete. Noted in the header for the next reader.

https://claude.ai/code/session_01MdSbMhzabSbpG8TtP9Ur3H

`forgejo-bucket-init` ran `mc mb --ignore-existing s3/alice-forgejo` and has not succeeded once since 2026-08-06. That afternoon the storage cutover moved Forgejo to the bare `forgejo` bucket, then **deleted the alice-forgejo bucket outright** along with MinIO user `alice` and `alice-policy`, and narrowed this Job1s own credential to `s3:*` on `arn:aws:s3:::forgejo` and nothing else. So it asked a deliberately scoped-out key to create a deliberately destroyed bucket. The Access Denied was the boundary working. **Not cosmetic.** backoffLimit 20 under restartPolicy OnFailure = ~1.6h of CrashLoopBackOff per day, then a TTL reap and a fresh Job 24h later. The kernel roll green gate fails on ANY crashlooping pod cluster-wide with no exemption, and this was the only pod matching it. It intermittently blocked kernel rolls. **Verified from both sides**, not from commit subjects (which are unreliable here: `d2583f4f` claims forgejo and zot were the only consumers, and this Job was a third). In git `alice-forgejo` is live config on exactly one line and no guard references it. Live, MinIO holds 11 buckets and none is alice-forgejo. The composition is retired so nothing regenerates it. Only the second document goes. `forgejo-admin-token-issuer` stays, byte-identical. **Merging is not sufficient on its own** — `forgejo-app` runs `prune: false`, so the live Job needs one manual delete. Noted in the header for the next reader. https://claude.ai/code/session_01MdSbMhzabSbpG8TtP9Ur3H

sean added 1 commit

2026-09-04 16:56:51 +00:00

fix(forgejo): retire the bucket-init Job, orphaned since the storage cutover

binjovi/ci Binjovi completed the frozen plan

Details

58a4824a9e

forgejo-bucket-init ran `mc mb --ignore-existing s3/alice-forgejo` and had not
succeeded once since 2026-08-06. Three things that afternoon made it impossible:

  f53699e8 22:30 pointed Forgejo's storage at the bare `forgejo` bucket after a
           936/936-object pre-stage, and repointed forgejo-s3-credentials at the
           forgejo-scoped MinIO key.
  d2583f4f 22:40 deleted the alice-forgejo bucket outright, along with MinIO user
           `alice` and alice-policy, and narrowed the surviving policy to
           `s3:*` on arn:aws:s3:::forgejo and nothing else.
  5c087ed6 23:43 retired the shared seanfarm key.

So the Job asked a deliberately scoped-out credential to create a deliberately
destroyed bucket. The Access Denied it returned was the boundary working, not a
fault -- the same credential authenticates fine and serves Forgejo's live 987
objects in the `forgejo` bucket.

It was overlooked rather than kept. This file has exactly one commit, the 16:45
flatten, and the three retirement commits landed at 22:30, 22:40 and 23:43 the
same day, one of them editing a sibling file in this directory without noticing
it. d2583f4f DID delete a bucket-init Job -- the XTenantBucket/alice one in ns
minio -- and missed this second copy flattened into forgejo-app.

It was not cosmetic. backoffLimit 20 under restartPolicy OnFailure meant about
1.6h of CrashLoopBackOff a day, then BackoffLimitExceeded, then a TTL reap and a
fresh Job 24h later. The kernel roll's green gate fails on ANY crashlooping pod
cluster-wide, with no namespace, kind or label exemption, so this intermittently
blocked kernel rolls: it was the only pod in the cluster matching that predicate.

Verified vestigial from both sides rather than from the commit subjects, which
cannot be trusted here -- d2583f4f asserts forgejo and zot were "the only two
consumers that ever existed" and this Job was a third. In git, `alice-forgejo`
appears as live configuration on exactly one line, and no test or guard mentions
the Job, the bucket or the policy. Live, MinIO holds 11 buckets and none is
alice-forgejo. The composition it was flattened from is retired, so nothing can
regenerate it.

Only the second document goes. forgejo-admin-token-issuer stays and is unchanged
byte for byte: it mints the PAT that ForgejoRepository/Organization CRs and
`make forgejo-cred` consume. Both ExternalSecrets are unchanged too.

The upstream inventory count for minio/mc drops 5 -> 4 with the reason recorded,
which is the counted allowlist doing its job rather than a bump.

REMOVING THIS FROM GIT IS NOT ENOUGH ON ITS OWN. forgejo-app runs prune: false
during the adoption soak, so the live Job must be deleted by hand once or it
keeps cycling. That is called out in the header comment for the next reader.

Claude-Session: https://claude.ai/code/session_01MdSbMhzabSbpG8TtP9Ur3H

binjovi-bot merged commit 58a4824a9e into trunk

2026-09-04 16:59:57 +00:00

No reviewers

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

sean/seanfarm!257

No description provided.

Rows
Columns