Research notes / Database recovery
RocksDB: a WAL sync is not a memtable flush
14 October 2026 / Source review / API reference: RocksDB v10.10.1
An atomic batch, a synchronized log and a completed memtable flush answer different questions. With manual WAL buffering, even a successful SyncWAL() does not imply that the buffer was written to the WAL file. [4]
Atomic visibility is not durable acknowledgment
A WriteBatch applies a group of edits atomically. The write's synchronization policy is separate: ordinary asynchronous acknowledgment can precede persistent storage, while a synchronous write waits for synchronization. Neither operation requires every changed entry to have reached an SST file. [2]
RocksDB column families partition one database into separately configurable key spaces. They have separate memtables and table files but share the WAL; a batch can span families. They are not independent databases or replicas. [1]
Four calls, three distinct boundaries
| Operation | Boundary crossed | Not implied |
|---|---|---|
FlushWAL(false) | Internal WAL buffer to its file. | Power-loss durability or SST creation. |
SyncWAL() | WAL file synchronization. | Draining a manually buffered WAL. |
FlushWAL(true) | Buffer drain followed by WAL synchronization. | Memtable flush or a backup. |
Flush(options, families) | Selected memtables to SST files. | Atomic grouping unless enabled. |
With manual_wal_flush=true and non-sync writes, records can remain in RocksDB's own memory. Thus the familiar process-crash argument based on surviving OS buffers no longer applies to those buffered records. [3]
The pinned API recommends FlushWAL(true) for the combined operation. It also warns that a later SyncWAL() is not identical to writing with sync=true: visibility can precede the later synchronization. [4]
The no-WAL cross-family trap
Illustrative reasoning, not an observed crash: one batch inserts an object in a data family and its lookup entry in an index family, with WAL disabled. If only the data family reaches persistent tables before a crash, recovery lacks the matching index update. Atomic application of the batch did not establish atomic persistence across independently flushed families. [6]
Atomic flush groups the selected families' installation in MANIFEST. Automatic atomic flush covers all families; manual flush requires the application to name the participants. It is not needed for this cross-family recovery property when all relevant writes are WAL-protected. [6]
Do not turn that into a universal transaction guarantee. A single-family flush still targets one family. A successful asynchronous request is not completed work: FlushOptions::wait controls waiting. The options header also cautions that later WAL-enabled writes can replay independently of the atomic flush. [4] [5]
MANIFEST tracks the database state, not another copy of every value
The MANIFEST records version edits describing database state. CURRENT names the active manifest log. For an atomic group, recovery waits for the complete set of version edits before applying it; it does not install a partial group. [7]
Hesela's inference: finding an SST file in a directory is insufficient to establish that recovery includes it in the database. User-data logging and metadata installation solve different problems. Do not rename, delete or reconstruct recovery files by guessing from filenames; use supported backup and recovery tooling.
A quiet family can keep a shared WAL alive
A WAL can still be required after a busy family flushes because it contains unflushed entries from another family. RocksDB uses max_total_wal_size to trigger flushing that can release older logs. A large WAL footprint alone therefore does not prove that compaction is failing. [8]
Keep the diagnostic questions separate: which family's recovery still needs the log, which memtables await flushing, and which SST runs await compaction? This is a source-derived troubleshooting distinction, not a measured performance improvement.
Why the FAST paper permits different WAL policies
Dong, Kryczka, Jin and Stumm's FAST 2021 experience paper describes RocksDB's architecture and operational lessons, including synchronous, buffered and disabled WAL modes. A surrounding distributed system may already maintain a recovery log, making duplicate logging avoidable. That context does not show that an arbitrary standalone database can safely disable its WAL. [9, sections 2.2 and 4]
This is an engineering experience report, not a randomized comparison of crash probabilities across those modes. We reviewed the architecture and WAL-treatment discussion, not a reproducible power-failure dataset. The article's inference is to compare performance only after matching the acknowledgment and recovery contracts.
A recovery review needs a precise boundary
Record the engine version, WAL policy, manual-buffer setting, selected families, batch boundary, waiting policy, returned errors and the external log, if any. Test process exit separately from host failure. A completed call cannot establish that hardware honored its persistence contract.
Continue with WAL and checkpoint guarantees, deletion versus reclamation and LSM read and compaction costs. The glossary defines a memtable separately from an SSTable.
Sources and limits
Primary-source comparison of RocksDB project documentation, v10.10.1 public API contracts and the FAST 2021 experience paper, with explicit failure-model distinctions. No RocksDB engine experiment, power-loss test, benchmark or paper reproduction was performed. Pinned headers describe v10.10.1, not a claim about the newest release; storage must honor synchronization.
- RocksDB project. Column Families: atomic writes and shared WAL.
- RocksDB project. Basic Operations: atomic updates and synchronous writes.
- RocksDB project. WAL Performance: non-sync and manually buffered modes.
- RocksDB v10.10.1. db.h: Flush, FlushWAL and SyncWAL contracts.
- RocksDB v10.10.1. options.h: manual_wal_flush, atomic_flush and FlushOptions.
- RocksDB project. Atomic flush: grouped persistence and explicit column-family selection.
- RocksDB project. MANIFEST: CURRENT, VersionEdits and atomic groups.
- RocksDB project. Write Ahead Log: lifecycle and flush triggers.
- Siying Dong, Andrew Kryczka, Yanqin Jin and Michael Stumm. Evolution of Development Priorities in Key-value Stores Serving Large-scale Applications: The RocksDB Experience. FAST 2021, pp. 33-49; sections 2.2 and 4.
Sources checked 14 October 2026. The pinned release is an inspection target, not a latest-version claim.