RAIDZ expansion: why old blocks keep their parity cost

16 October 2026 / Source review and reproducible accounting

Adding a disk can spread existing RAIDZ blocks across more devices without making those blocks more space-efficient. Physical placement and data-to-parity ratio are different properties. A subsequent scrub does not turn the old blocks into newly encoded, wider stripes.

What expansion changes

RAIDZ is ZFS's distributed-parity layout, with one, two or three parity fragments depending on the level. Blocks need not use an identical stripe width. The full-width approximation used below is therefore a teaching model, not its allocation algorithm. [1]

OpenZFS 2.3 describes expansion as reading allocated space and redistributing it over the enlarged group. Old blocks keep their data-to-parity ratio; new blocks use the wider geometry after completion. RAIDZ2 remains double parity, not triple parity. [2]

The implementation calls this time-dependent geometry: it records expansion boundaries and uses a block's birth time to recover its logical width. Sectors that once shared a physical column can end up on different disks. The placement changes while the old logical parity group remains interpretable. [3, Time Dependent Geometry]

This is not a replacement-device resilver. The operation increases an existing group's width; it does not add an independent top-level vdev either. See the separate rebuild and resilver analysis for recovery semantics.

Six TiB of payload, two parity geometries

Assume full-width, equal-size fragments with no compression, padding, metadata or retained snapshots. Hold the live payload at 6 TiB, where 1 TiB = 2⁴⁰ bytes. Compare an original width of four with a final width of five; both have two parity fragments per full row.

Old rows contain 2 data + 2 parity, so 6 TiB of payload requires 12 TiB including parity. New rows contain 3 data + 2 parity, so the same payload requires 10 TiB. The difference is 2 TiB of modeled physical allocation, not 2 TiB of missing files or a measured capacity loss.

Raw allocation for the same 6 TiB payload

Model, not measurement. Horizontal scale: 0-12 TiB including parity. Each row changes only the fraction of payload retaining old geometry.

0% old10 TiB

25% old10.5 TiB

50% old11 TiB

75% old11.5 TiB

100% old12 TiB

Five synthetic mixtures at fixed payload, not stages timed during expansion.
Payload at old ratio	Old / new payload (TiB)	Raw allocation (TiB)	Extra vs all-new (TiB)
0%	0 / 6	10	0
25%	1.5 / 4.5	10.5	0.5
50%	3 / 3	11	1
75%	4.5 / 1.5	11.5	1.5
100%	6 / 0	12	2

Let D be payload, f the fraction of those payload bytes at the old geometry, N the old width, M the new width, and P the unchanged parity count. Then:

raw = D * f * N / (N - P) + D * (1 - f) * M / (M - P)
extra = D * f * P * (M - N) / ((N - P) * (M - P))

The denominator is payload bytes, not number of files or fraction of total pool capacity. The 50% row means 3 TiB in each cohort. A real repeatedly expanded pool can contain several old widths; one old/new mixture is insufficient to describe all of them.

Download CSV, provenance and runnable model. This chart and table execute the same dependency-free module. Tests compare its arithmetic with full-row sector counts for all three parity levels. There is no sampling uncertainty or confidence interval: these are deterministic assumptions, with substantial real-world applicability limits.

Allocation is not the displayed free-space estimate

The 2.3 attach manual separately warns that the vdev's assumed parity ratio remains unchanged for space reporting. Consequently, tools such as zfs list can report less space than expected for newly written blocks. Do not attribute every discrepancy to retained old parity, and do not use this model to predict AVAIL. [2]

To investigate a real pool, record the software version, expansion history, block sizes, compression, reservations, snapshots and which command produced each number. A raw-device total, a dataset quota and a parity-adjusted estimate answer different questions.

Why a scrub is not a space-reclamation rewrite

A scrub reads stored data to verify checksums and repairs detected damage where redundancy permits. Its purpose is not to reallocate every healthy block using a new layout. Expansion completion and the verification pass are separate milestones. [4]

OpenZFS starts a pool-wide scrub after expansion by default; the pinned implementation exposes zfs_scrub_after_expand. The copy pass itself is not a complete checksum-verification pass. Check the actual completion and error status instead of treating reflow completion as evidence that all copied data was verified. [3]

The zfs rewrite command is documented in the pinned 2.3.4 source. It rewrites selected file blocks at new locations, potentially applying changed properties. It is not equivalent to a scrub. Rewriting snapshot-referenced or cloned blocks may increase pool space use; rewritten blocks also enter subsequent incremental send streams. [5]

Our 12-to-10 TiB comparison assumes old allocations disappear. If all 12 TiB remain referenced while a fresh 10 TiB copy is allocated, the same ideal arithmetic instead gives 22 TiB, before metadata. This is a counterexample to a blanket savings promise, not a prediction for a particular rewrite. Do not delete snapshots or rewrite a nearly full pool on the strength of this calculation.

What RAID-scaling research does, and does not, establish

Zheng and Zhang's FastScale (FAST 2011) studies RAID-0 expansion, not RAIDZ. Its simulator combines SimPy and DiskSim with IBM Ultrastar 36Z15 parameters and TPC-C, financial and web traces. The financial baseline adds two disks to four modeled 4 GB disks with 32 KB stripe units. It reports 952 seconds versus 6,830 for SLAS; Figure 13 plots maxima within 100-second windows, not a latency percentile. [6, sections 4.1-4.3]

FastScale moves data only from old disks to new disks and uses aggregation and lazy metadata checkpoints. Its conclusion explicitly excludes parity. That design helps explain why movement volume and migration ordering matter, but its speedup cannot be transferred to OpenZFS reflow, modern drives or this capacity model. We did not reproduce its experiments. [6, sections 3 and 6]

Operational boundaries

The expansion feature is not read-only compatible. Recovery and import environments must support an active raidz_expansion feature; test compatibility before changing a pool. Enabling or activating pool features is outside this analysis. [7]

We ran arithmetic and publication tests only. No storage devices were attached, no ZFS pool was created, and no migration duration, failure probability or usable capacity was measured. Padding, small blocks, metadata, compression, deduplication, free-space reserves and snapshot sharing can invalidate a direct comparison with the table.

Continue with stripe width versus repair placement and snapshot recovery boundaries. Corpus: RAIDZ and RAIDZ expansion.

Sources

Accessed 16 October 2026. The 2.3 documentation branch may evolve; implementation details are pinned to zfs-2.3.0 and rewrite behavior to zfs-2.3.4, not claimed as universal across versions.