feat(node-bootc): give the whole disk tail to ZFS, drop the hugepage reservation #198

Merged

binjovi-bot merged 1 commit from chore/drop-mayastor-carve into trunk

2026-09-04 08:05:22 +00:00

sean commented

2026-09-04 08:04:40 +00:00

Owner

Mayastor was removed from the cluster on 2026-09-04. Two things in this image existed only to serve it.

`hugepages=1024`

Pre-allocated 2 GiB of 2 MiB hugepages at boot, because Mayastor's SPDK io-engine reserved them at process start and a post-boot sysctl raced it. With Mayastor gone that is 2 GiB of RAM reserved and unusable on every node — the master never ran an io-engine at all and had been holding it for 79 days.

The disk carve

seanfarm-disk-init split the free tail two-thirds mayastor, one-third tank. On a cpx52 that gave ZFS 143G and Mayastor 288.8G.

By the end Mayastor stored nothing, while the ZFS pools ran 57-75% full at 55-66% fragmentation — and one of them would not accept a new 1Gi volume at all. tank now takes the whole tail, about 433G on a cpx52.

sgdisk -n 4:0:0 -t 4:BF00 -c 4:tank "$DEV"   # p4 tank, all remaining space

The early-exit guard changes with it. It tested for mayastor and tank present; with mayastor never created again that could never be true, so a rebuilt node would carve a second time. It now tests tank alone. The MARKER check above it already covered this, but a guard that cannot fire is worse than no guard.

Scope

This changes newly created nodes only. Every running node keeps its layout, because /var/lib/seanfarm/disk-init.done exists on all of them (dated 2026-06-16) and disk-init exits on it. Repartitioning the running fleet is separate, deliberate work.

Checks

All heredocs balanced (DISKINIT, ZFSPOOL, SCRIPT ×5, EOF ×9). Both embedded scripts extracted and bash -n clean. scripts/lint-kernel-cache.sh passes, including its assertion that node-bootc still masks bootc-fetch-apply-updates.

https://claude.ai/code/session_01MdSbMhzabSbpG8TtP9Ur3H

Mayastor was removed from the cluster on 2026-09-04. Two things in this image existed only to serve it. ## `hugepages=1024` Pre-allocated 2 GiB of 2 MiB hugepages at boot, because Mayastor's SPDK io-engine reserved them at process start and a post-boot sysctl raced it. With Mayastor gone that is **2 GiB of RAM reserved and unusable on every node** — the master never ran an io-engine at all and had been holding it for 79 days. ## The disk carve `seanfarm-disk-init` split the free tail two-thirds mayastor, one-third tank. On a cpx52 that gave ZFS **143G** and Mayastor **288.8G**. By the end Mayastor stored nothing, while the ZFS pools ran 57-75% full at 55-66% fragmentation — and one of them would not accept a new 1Gi volume at all. `tank` now takes the whole tail, **about 433G on a cpx52**. ``` sgdisk -n 4:0:0 -t 4:BF00 -c 4:tank "$DEV" # p4 tank, all remaining space ``` The early-exit guard changes with it. It tested for `mayastor` **and** `tank` present; with mayastor never created again that could never be true, so a rebuilt node would carve a second time. It now tests `tank` alone. The `MARKER` check above it already covered this, but a guard that cannot fire is worse than no guard. ## Scope **This changes newly created nodes only.** Every running node keeps its layout, because `/var/lib/seanfarm/disk-init.done` exists on all of them (dated 2026-06-16) and `disk-init` exits on it. Repartitioning the running fleet is separate, deliberate work. ## Checks All heredocs balanced (DISKINIT, ZFSPOOL, SCRIPT ×5, EOF ×9). Both embedded scripts extracted and `bash -n` clean. `scripts/lint-kernel-cache.sh` passes, including its assertion that node-bootc still masks `bootc-fetch-apply-updates`. https://claude.ai/code/session_01MdSbMhzabSbpG8TtP9Ur3H

sean added 1 commit

2026-09-04 08:04:40 +00:00

feat(node-bootc): give the whole disk tail to ZFS, drop the hugepage reservation

binjovi/ci Binjovi completed the frozen plan

Details

9529ecd53e

Mayastor was removed from the cluster on 2026-09-04. Two things in this
image existed only to serve it.

hugepages=1024 pre-allocated 2 GiB of 2 MiB hugepages at boot, because
Mayastor's SPDK io-engine reserved them at process start and a post-boot
sysctl raced it. With Mayastor gone that is 2 GiB of RAM reserved and
unusable on every node. The master never ran an io-engine at all and had
been holding it for 79 days.

seanfarm-disk-init split the free tail two-thirds mayastor, one-third tank.
On a cpx52 that gave ZFS 143G and Mayastor 288.8G, and by the end Mayastor
stored nothing while the ZFS pools ran 57-75% full at 55-66% fragmentation.
One of them would not accept a new 1Gi volume at all. tank now takes the
whole tail: about 433G on a cpx52.

The early-exit guard changes with it. It tested for mayastor AND tank
present; with mayastor never created again that could never be true, so a
rebuilt node would carve a second time. It now tests tank alone. The
MARKER check on the line above already covered this, but a guard that
cannot fire is worse than no guard.

SCOPE. This changes NEWLY CREATED nodes only. Every running node keeps its
layout, because /var/lib/seanfarm/disk-init.done exists on all of them
(dated 2026-06-16) and disk-init exits on it. Repartitioning the running
fleet is separate, deliberate work.

Verified: all heredocs balanced, and both embedded scripts extracted and
syntax-checked. scripts/lint-kernel-cache.sh passes.

Claude-Session: https://claude.ai/code/session_01MdSbMhzabSbpG8TtP9Ur3H

binjovi-bot merged commit 9529ecd53e into trunk

2026-09-04 08:05:22 +00:00

No reviewers

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

sean/builder-images!198

No description provided.

Rows
Columns