sean/seanfarm
Watch
1
0
Fork
You've already forked seanfarm
 0
Code Issues Pull requests 1 Projects Releases 423 Packages Wiki Activity

docs(rustfs): measure the erasure ratio instead of assuming it #447

Merged
binjovi-bot merged 1 commit from docs/rustfs-erasure-measured into trunk 2026-09-08 08:46:53 +00:00
Conversation 0 Commits 1 Files changed 2 +32 -2
sean commented 2026-09-08 08:40:59 +00:00
Owner
Copy link

Every capacity number in the MinIO migration rests on RustFS storing 2 raw bytes per logical byte. That was an assumption, and the manifest carried an instruction to go measure it before the zot move relied on 220 GiB.

Measured now, against the resized cluster:
object written 1.000 GiB (1073741824 bytes)
landed on rustfs-0 .. rustfs-3 0.50 GiB each, evenly
raw total 2.004 GiB
ratio 2.004x

So 4 x 110Gi = 440 GiB raw is ~220 GiB usable. MinIO holds 144 GiB, leaving roughly 76 GiB of headroom — enough for the 136 GiB zot bucket, which is the step that would otherwise have failed.

Method

The payload came from /dev/urandom, not /dev/zero. The ZFS datasets under these claims have compression on, so a compressible payload measures the compressor and reports a flattering ratio that real object data would never reproduce. 1 GiB is also large enough that per-object metadata is noise.

Also recorded: deletes reclaim asynchronously

Removing the probe bucket returned the drives from 525568 KiB to 256-384 KiB over about four minutes, not immediately. Anything that reads free space straight after a delete gets a stale number — relevant when Phase 5 reclaims MinIO's 550 GiB, and when a mirror is retried.

No behaviour change: comment and documentation only.

https://claude.ai/code/session_01KZoQin34jeyt6nDGqvJA76

Every capacity number in the MinIO migration rests on RustFS storing 2 raw bytes per logical byte. That was an assumption, and the manifest carried an instruction to go measure it before the zot move relied on 220 GiB. Measured now, against the resized cluster: | | | |---|---| | object written | 1.000 GiB (1073741824 bytes) | | landed on `rustfs-0` .. `rustfs-3` | 0.50 GiB each, evenly | | raw total | 2.004 GiB | | **ratio** | **2.004x** | So 4 x 110Gi = 440 GiB raw is **~220 GiB usable**. MinIO holds 144 GiB, leaving roughly 76 GiB of headroom — enough for the 136 GiB `zot` bucket, which is the step that would otherwise have failed. ## Method The payload came from `/dev/urandom`, **not** `/dev/zero`. The ZFS datasets under these claims have compression on, so a compressible payload measures the compressor and reports a flattering ratio that real object data would never reproduce. 1 GiB is also large enough that per-object metadata is noise. ## Also recorded: deletes reclaim asynchronously Removing the probe bucket returned the drives from 525568 KiB to 256-384 KiB over about **four minutes**, not immediately. Anything that reads free space straight after a delete gets a stale number — relevant when Phase 5 reclaims MinIO's 550 GiB, and when a mirror is retried. No behaviour change: comment and documentation only. https://claude.ai/code/session_01KZoQin34jeyt6nDGqvJA76
docs(rustfs): measure the erasure ratio instead of assuming it
All checks were successful
binjovi/ci Binjovi completed the frozen plan
Details
484e07da98
 
Every capacity number in the MinIO migration rests on RustFS storing 2 raw bytes
per logical byte. That was an assumption, and the manifest carried an
instruction to go measure it before the zot move relied on 220 GiB. Measured
now, against the resized cluster:

  object written                1.000 GiB (1073741824 bytes)
  landed on rustfs-0..3         0.50 GiB each, evenly
  raw total                     2.004 GiB
  ratio                         2.004x

So 4 x 110Gi = 440 GiB raw is about 220 GiB usable. MinIO holds 144 GiB, leaving
roughly 76 GiB of headroom -- enough for the 136 GiB zot bucket, which is the
step that would otherwise have failed.

The payload came from /dev/urandom, not /dev/zero. The ZFS datasets under these
claims have compression on, so a compressible payload measures the compressor
and reports a flattering ratio that real object data would never reproduce.

Also recorded: deletes reclaim asynchronously, not immediately. Removing the
probe bucket returned the drives from 525568 KiB to 256-384 KiB over about four
minutes. Anything that reads free space straight after a delete gets a stale
number, which matters when Phase 5 reclaims MinIO's 550 GiB and when a mirror is
retried.

No behaviour change: comment and documentation only.

Claude-Session: https://claude.ai/code/session_01KZoQin34jeyt6nDGqvJA76
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
sean/seanfarm!447
No description provided.