NFS stable writes: what COMMIT, flush and FUA actually promise

Read the achieved stability, not just the request

NFSv3 WRITE includes a requested stability level and returns the achieved level in committed. The server may provide a stronger level than requested. These are protocol guarantees, not observations of a particular appliance. [1, section 3.3.7]

Stable storage need not mean the final disk surface. RFC 1813 permits protected intermediate storage if the required recovery guarantee holds; failure of the storage medium itself is outside that guarantee. Treat persistence as a failure-model claim, not a synonym for a specific cache technology. [1, section 4.8]

A stable short write is still a short write

Suppose a request carries 8,192 bytes but a successful FILE_SYNC reply reports only 4,096 bytes accepted. The accepted prefix is stable; the remaining 4,096 bytes still need a WRITE. COMMIT cannot supply bytes the server never accepted. This is a constructed example, not a measured transfer. RFC 1813 explicitly allows short writes and directs the client to send the remainder. [1, section 3.3.7]

A successful COMMIT can still require a rewrite

COMMIT makes buffered writes and metadata stable for a file range. A zero count extends from offset to end of file. Its returned write verifier must agree with the verifier associated with the outstanding writes. An unexpected verifier requires retransmission of uncommitted buffers. It is not a content checksum. [1, section 3.3.21]

1. WRITE: the server accepts the full range, replies UNSTABLE and returns verifier A.
2. Failure: the server loses that volatile state and restarts with verifier B.
3. COMMIT: it can return NFS3_OK and B even though the earlier buffered bytes were lost.
4. Recovery: the client detects A versus B, retransmits the uncommitted data and establishes stable completion again.

This is an illustrative protocol trace, not an observed incident or a client implementation. It assumes one writer and no permanent media failure. The single-file range interface does not establish a transaction across several files; that conclusion follows from the operation's scope, not from a performance test.

PREFLUSH and FUA cover different boundaries

In the Linux block layer, REQ_PREFLUSH makes earlier completed writes persistent before the accompanying I/O starts. It can also be used without a data payload. REQ_FUA makes the flagged write persistent before that request completes. Both may be combined; neither flag should be silently substituted for the other. Linux can implement FUA with a later flush when native support is absent. [2]

For example, suppose payload write D has completed and commit-record write C follows. FUA on C alone supplies C's persistence boundary, not a guarantee that D became persistent before C started. PREFLUSH supplies that ordering boundary; FUA supplies C's completion boundary. This conceptual example is not a complete filesystem or database commit algorithm: atomicity, checksums and recovery still need their own design.

A block request cannot flush bytes that an application has not submitted. Do not map each NFS COMMIT to exactly one hardware flush: the server's filesystem, controller and cache stack implement the higher-level promise. This is a scope distinction, not a measurement of command counts.

Write-through is relative to a layer

Linux dm-cache documents write-through completion after the write reaches both cache and origin devices. In write-back mode, a cached block can instead be updated only in the cache and marked dirty. Those destinations can themselves have volatile caches. The policy name therefore does not certify the entire persistence path. [3] Device-level cache handling is a separate contract. [2]

The dm-cache documentation separately describes metadata commits triggered by FLUSH or FUA requests and warns about recent-write loss on power failure. Cache coherence, persistence and application consistency are distinct properties. [3]

Configuration can weaken the promise

The Linux NFS server's async export option is not merely another spelling of a client's UNSTABLE WRITE. The exports manual explicitly warns that it permits replies before stable storage in violation of the protocol and can lose or corrupt data after an unclean restart. Record effective export options when evaluating durability; do not infer them from the client's request. [4, async and sync]

A proposed validation should record client/server versions, negotiated NFS version, export options, write reply counts, achieved stability, verifier changes, sync errors and cache protection. Test only disposable data in an isolated environment, separating client failure, server failure and storage power loss. Check recovered acknowledged records against an independent expected log. No such fault-injection experiment was performed for this review.

Client options answer different questions

noac disables attribute caching and makes application writes synchronous, but does not mean all data caching disappears. Close-to-open consistency uses checks at open and writeback at close for sequential sharing; it is not a guarantee for simultaneous writers. [5, cache consistency]

O_DIRECT on NFS bypasses only the client's page cache. The flag is not sent to the server, which may still cache data. The client requests synchronous handling; this does not prove that a misconfigured server honors it. [6, O_DIRECT]

soft and softerr change timeout recovery. The Linux manual warns of silent corruption in some cases; a hard mount keeps retrying and can leave callers waiting. These are availability and error-handling choices, not a replacement for checking stable completion. [5]

Nor does persistence serialize concurrent appenders: Linux documents a race when multiple processes use O_APPEND on NFS because append must be simulated. Record the application's coordination strategy as well as its cache flags. [6, O_APPEND]

Why the client/server distinction is older than NFSv3

Sandberg and colleagues' 1985 paper describes server-side stable completion alongside client-side buffering and writeback at close. There is no contradiction: buffering before an RPC is different from acknowledging that RPC before persistence. The implementation discussion explains why unwritten server blocks would otherwise become client recovery state. [7, pp. 121, 124, 127-128]

That paper evaluates a Sun-2 implementation using application workloads and transfer tests, then discusses cache and RPC optimizations. It is historical evidence for the design tradeoff, not a benchmark of current NAS appliances. NFSv3 later formalized safe asynchronous server writes through WRITE and COMMIT. [1, section 3.3.7] The normative v3 contract remains the RFC; the historical paper does not specify the later verifier mechanism.

Evidence, limits and related definitions

We reviewed NFSv3's specification and the linked Linux documentation. NFSv4 has its own state and recovery rules; this article does not transfer the complete NFSv3 recovery procedure to it. A correct protocol trace cannot prove that firmware, controllers or an appliance honor the contract.

For application-level error handling, see failed fsync and retry. For uncertain outcomes, see storage timeouts.

Corpus: NFS, NFS COMMIT, NFS write verifier, cache flush, Force Unit Access, write-back cache, write-through cache, and Direct I/O.

Primary sources

1. B. Callaghan, B. Pawlowski and P. Staubach. NFS Version 3 Protocol Specification. RFC 1813, June 1995, sections 3.3.7, 3.3.21 and 4.8.
2. Linux kernel documentation. Explicit volatile write back cache control: REQ_PREFLUSH and REQ_FUA.
3. Linux kernel documentation. Device-mapper cache: operating modes and on-disk metadata.
4. nfs-utils exports(5), Debian trixie manual: async and sync export options.
5. nfs-utils nfs(5), Debian trixie: client timeout options and data/metadata coherence.
6. Linux man-pages open(2), Debian trixie: NFS O_DIRECT scope and O_APPEND races.
7. R. Sandberg, D. Goldberg, S. Kleiman, D. Walsh and B. Lyon. Design and Implementation of the Sun Network Filesystem. Summer USENIX 1985, pp. 119-130 (scan).

Reviewed 12 October 2026. No copied figures or new hardware measurements.