feat(kernel): exit-handler cordon sweep — never leave a Ready node cordoned #2072

Merged

pipeline-bot merged 2 commits from feat/cordon-safety into trunk 2026-08-19 08:18:52 +00:00

Conversation 0 Commits 2 Files changed 4 +53 -2

sean commented 2026-08-19 07:52:39 +00:00

Owner

Copy link

Kaizen improvement 2 of run 3 (the sf38 left-cordoned degradation class). Ready→uncordon loudly; NotReady stays, louder. Guard P8 + contract M12.

Kaizen improvement 2 of run 3 (the sf38 left-cordoned degradation class). Ready→uncordon loudly; NotReady stays, louder. Guard P8 + contract M12.

sean added 1 commit 2026-08-19 07:52:39 +00:00

feat(kernel): exit-handler cordon sweep — a roll never leaves a Ready node cordoned ... 7b9818d1b2

The sf38 incident's worst consequence was not the false Failed: the
node LEFT CORDONED degraded cnpg to 3/5, etcd to 4/5 and 2 minio pods,
so every later preflight went red for confusing reasons, and only a
human uncordon healed it. The roll is the only actor that cordons
workers (the kernel-deploy-node mutex serializes rolls), so the
always-run reap-agents exit handler now sweeps: a cordoned Ready node
is uncordoned with a loud cordon-sweep line; a NotReady node stays
cordoned, louder — uncordoning a sick node would schedule workloads
onto it. Guard P8 + contract M12 (11/11).

sean added 1 commit 2026-08-19 08:17:02 +00:00

fix(kernel): final tail after a BuildRun succeeds — the ccache stats ... 4b8923c456

must reach the archive

The sf40 bake proved the v0.1.670 stream works (5051 compile lines
captured) and exposed its gap: the follow's 600s bound expired at the
exact moment the ccache -s block printed, the build finished inside
the re-attach window, and the stats were lost again. After the
Succeeded condition, one bounded NON-follow --tail=200 fetch lands the
stats in the archived log without replaying the build.

pipeline-bot merged commit 4b8923c456 into trunk 2026-08-19 08:18:52 +00:00

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

sean/pipelines!2072

No description provided.