LTFS performance: when does the write job finish?

Placement changes the next operation's cost

IBM's documented LTFS placement sequence writes eligible files to the data partition first and retains their bytes in memory. A full buffer or unmount then triggers index writing and placement in the index partition. Nearby files can be faster to access immediately after mounting; that advantage is not a permanent random-access guarantee. [IBM, section 3.5.4]

The placement policy therefore deserves a write-side assessment as well as a read-side one. Two locations on the same cartridge are not independent backups.

What the 2015 scheduler experiment measured

Villa Real, Richmond, Biskeborn and Pease evaluated Unified on IBM LTO-5 hardware using synthetic workloads. Section VI-E writes 1,000 files of 10 MiB; ten qualify for the index partition. Each file is fsynced before close. [NAS 2015, sections V and VI]

The larger cache deferred all index-partition writes until unmount. The section does not report a separate unmount duration or clearly establish an end-to-end completion time. It also leaves drive start/stop time uncontrolled. These results are not a current-drive speed forecast.

A separate streaming experiment averages ten runs after a warm-up transfer and excludes tape turnarounds. Its timing protocol must not be silently assigned to the cache experiment. The paper also finds software prefetching can add overhead: more buffering is not automatically better for every read pattern.

Incremental indexes solve a different problem

SNIA LTFS 2.5.1 permits incremental metadata in the data partition, reducing repeated full-index recording. Compliant implementations must read these increments but need not generate them. Clean unmount still requires full indexes. [SNIA, sections 5 and 9; Annex H]

Incremental does not mean a byte-level patch: when extentinfo orextendedattributes appears, its complete contents are required. Recovery applies the preceding full index and the intervening increments. Annex H's suggested chain limit is guidance, not a universal mandatory value.

Engineering inference: avoiding repeated full metadata is distinct from reducing file fragmentation or partition switches. An optimization in one category does not demonstrate improvement in the others.

Measure a complete archival job

For a controlled comparison, preserve the same input files and verify the restored bytes. Record these boundaries separately rather than reporting one unlabeled throughput number:

1. Application writes, including all close and synchronization errors.
2. Metadata synchronization and successful unmount.
3. A fresh mount and retrieval of both selected and ordinary files.

Keep drive, firmware, software version, cache budget, compression, placement rules, access order and index policy fixed or explicitly varied. Report repeated-run variation and include positioning and turnaround time when they belong to the job. This is a proposed evaluation protocol, not an experiment performed here.

IBM documents timer-, close- and unmount-triggered index policies. Changing that policy changes the recovery/performance tradeoff; it should not be a hidden benchmark setting. [IBM, mounting options] Read the LTO compatibility and recovery reference before treating a successful mount as proof of recoverability.

Scope and sources

Source review, not a hardware benchmark, crash test or recovery guarantee. No modern-drive speedup is inferred, and the historical paper's unmount timing remains unresolved. Machine-readable context: LTFS index and data placement policy.

1. Villa Real et al. An I/O Scheduler for Dual-Partitioned Tapes. NAS 2015, sections V-VI (author-hosted full paper)
2. SNIA LTFS Format Specification 2.5.1, sections 5, 9 and Annex H
3. IBM Spectrum Archive SDE and LE, SG24-8090-04 (2018), mounting options and section 3.5.4