Research notes / Storage integrity
HDD write verification: completion, persistence and comparison
/ Primary-source review
A completed write, a persistence guarantee and a comparison against expected bytes are different evidence. Even a command named VERIFY does not necessarily compare against the data you intended.
Start with the property being checked
Linux distinguishes REQ_PREFLUSH, which makes earlier completed writes non-volatile before the dependent I/O begins, from REQ_FUA, which delays completion of that write until its data is non-volatile. These are persistence semantics, not requests to compare the stored bytes with an independent reference. [4]
Hesela's evidence checklist, not an ordering of safety levels. Each row needs its own successful completion and scope.
| Evidence | Question addressed | Still not established |
|---|---|---|
| Write completion | Did this layer report success? | Which lower-layer obligations were included? |
| Flush / write FUA [4] | Was the required data made non-volatile? | A separate read-back comparison. |
| Medium verification [2] | Did the device's selected checks pass? | Agreement with host-supplied expected bytes. |
| Byte comparison [2] | Did the checked data match the supplied reference? | Whether that reference is correct and current. |
Consider an intentionally constructed example: the expected new payload is B, but the old payload A remains fully readable with valid error-correction information. Readability alone cannot decide whether the update happened. Comparing against an independently retained B can expose the mismatch. This is a logical example, not an observed drive failure.
IRAW: move verification, not the completion boundary
Riska and Riedel's 2008 IRAW proposal retains completed-write data in cache and compares it with disk contents during idle periods. Cache eviction and non-preemptible seeks constrain coverage and foreground latency. Their study uses five enterprise traces and DiskSim 2.0; it is not a deployed-fleet reliability measurement. Power-failure consistency is outside its scope. [1, sections 2-5]
The methodological trap is throughput: arrivals are fixed in this open-loop simulation. Small IOPS differences therefore do not establish unchanged saturation capacity. Verification coverage is a fraction of writes checked, not a fraction of future failures prevented. [1, section 5.2]
Our inference: a coverage result needs a denominator, delay distribution and eviction policy before it can inform an integrity objective. A check performed later must not be silently credited to the earlier acknowledgement. We did not rerun the simulator or establish implementation in any current model.
SCSI: VERIFY is not always a byte comparison
Seagate's February 2006 command reference distinguishes BYTCHK=0 medium verification without data comparison from BYTCHK=1 comparison with supplied data. WRITE AND VERIFY writes first, then verifies; it is not a harmless scan of existing contents. Verification criteria also depend on mode pages or vendor behavior. [2, sections 3.49 and 3.58]
Do not apply a historical one-bit field description to every later command revision. The sg3_utils v1.48 manual records the expansion to two bits in SBC-3 revision 34. Its --ndo option supplies comparison bytes; a mismatch returns exit status 14. DPO lowers cache-retention priority: it is not a universal cache-bypass switch. [3]
Nor should the name VERIFY alone be used as proof of a fresh physical read. Gilbert's SG driver discussion explicitly considers prefetching data into the target drive's cache before verification. Check the exact command, cache behavior and implementation when physical-media access is the question. [5]
No commands here were issued to a storage device. Device support, bridges, logical-block size and protection-information settings must be verified before translating these distinctions into a test procedure.
Make a verification report auditable
Our proposed reporting fields separate observation from inference:
- Device model, firmware, transport and software version.
- Exact operation, comparison mode, block range and excluded regions.
- Origin and version of the expected data, if a comparison uses it.
- Start, completion, status and errors, not merely a scan request.
- Cache and persistence assumptions, plus intervening writes or repairs.
A matching old reference can validate old data. A reference corrupted before comparison can validate the wrong payload. Verification therefore complements the separate questions of identity, freshness and recovery; it does not replace them. See checksums and lost writes, scrub scope and flush and FUA boundaries.
Corpus definitions: read-after-write verification, IRAW and SCSI VERIFY.
Primary sources and limits
- Alma Riska and Erik Riedel (Seagate Research). Idle Read After Write - IRAW. USENIX ATC 2008, pp. 43-56; sections 2-5.3.
- Seagate. SCSI Commands Reference Manual, 100293068 Rev. A, February 2006; sections 3.49 and 3.58, printed pp. 175-176 and 198.
- sg3_utils v1.48: sg_verify manual, BYTCHK, DPO and exit status (manual header retains 1.45).
- Linux kernel documentation: explicit volatile write back cache control.
- Douglas Gilbert. Linux SG driver version 4.0: request-sharing verification and prefetch discussion.
The paper and command manual are historical, manufacturer-affiliated sources. Software documentation is version-scoped where possible. This review provides no hardware benchmark, failure-rate estimate or certification.