Restoring file names: when matching hashes hide a lost entry

A passing content check can hide a collapsed namespace

Our 48 disposable copy cases included 20 deliberately colliding transformations. In 10 of those cases, both source-content checks passed through the transformed names even though only one output entry existed. This is a counterexample to a verification rule, not a measured rate of backup failure.

We created two real files per case, transformed each name in the application, and copied to a fresh directory using ordinary overwrite mode. The underlying mount preserved the original byte names in every identity control. We did not configure a case-insensitive filesystem or emulate NTFS, APFS or ext4 casefold.

For distinct contents, reversing the copy order changed which payload survived each collision. For equal contents, the surviving bytes matched both expectations. Counting and comparing the original byte-name inventory detected the loss in both situations. Content deduplication does not justify dropping names: separate paths can carry separate roles even when their bytes happen to match.

Normalization, case folding and decoding answer different questions

NFC and NFD give canonical equivalents a common representation; NFKC and NFKD also merge compatibility distinctions. A circled digit and an ordinary digit remain different under NFC but coincide under NFKC. Normalization is therefore not a universally lossless filename cleanup operation. [1]

Case folding is separate from normalization and can expand a character: Python folds the German sharp s to two ASCII s characters. Our policy labels specify the exact operation order. The label nfkc-casefold means NFKC, Python casefold, then NFKC; it is not an implementation of Unicode's separately defined NFKC_Casefold operation. [5]

Invalid UTF-8 introduces another choice. Our strict policies reject the two invalid byte names. Decoding with replacement turns both into the same replacement-character name. Python's filesystem interfaces can preserve Unix names using bytes or the filesystem encoding/error handler; an exported JSON display string is not automatically a reversible filename identifier. The observations retain hexadecimal original bytes. [6]

The destination directory, not the OS label, defines the contract

Linux ext4 documents per-directory case-insensitive lookup and name preservation on disk. Its comparison representation can differ from its stored spelling; non-strict invalid-encoding handling can fall back to opaque bytes. The documented encoding version is contextual, not a promise about every installed kernel. [2] Microsoft likewise documents directory-level case-sensitivity flags. [3]

A generic lowercase or Unicode transform cannot certify a particular target. Record the actual filesystem, directory settings, client/API behavior and Unicode rules. For a nested tree, every parent component matters. A share or application may apply its own comparison rules before the filesystem sees the name.

The overlooked connection is identity and access policy

Basu, Sampson, Qian and Jaeger's 2022 preprint studies case-sensitivity collisions in Linux copying utilities. Sections 5 and 6 generate resource-type pairs, observe filesystem operations, and compare resulting contents and metadata. Its package-script survey is a usage sample, not a field incident denominator. Section 8 explains why an archive-only collision scan can miss pre-existing target entries or disagree with the destination's rules. [4]

That work connects filename equality to metadata and authorization, not just display spelling. We do not reproduce its exploit cases or claim that today's versions retain its reported behavior. Our narrower contribution isolates a verifier that accepts equal contents after losing an entry. Combine this check with permission restoration and archive-entry selection; none substitutes for the others.

What a restore acceptance test must preserve

1. Keep original names losslessly, with an explicit encoding or byte representation. Store display labels separately.
2. Define the intended source-to-destination mapping and verify that distinct required entries do not collapse under the actual target rules. Include pre-existing entries and parent components.
3. Reject unsupported names or collisions before committing output, or use a documented reversible renaming plan. Do not silently normalize and overwrite.
4. Compare the restored entry inventory and mapping, then verify type, contents, links, permissions and application behavior as required by the recovery contract.

Our fixture guard rejected all 20 colliding plans before creating an output file; the 28 noncolliding controls retained both entries. This guard uses known transforms on fixed single-component names. It is not a secure extractor, an atomic restore protocol or protection against concurrent path changes.

See the corpus definition of filename collision and the existing backup verification framework.

Reproduce the observations

Download the standard-library Python probe and raw observations. Run on a writable POSIX mount that can preserve the fixture byte names; GNU stat is required. The output path must not already exist.

python3 filename-identity.py --directory . --output filename-results.json

The recorded run used Python 3.12.3, Unicode database 15.0.0 and Linux 6.8.0-146-generic. The probe records the mount-type label reported by stat, its own SHA-256, original/output inventories and complete content-match results. The eight-policy comparison matrix is separate from the 48 materialized copy cases.

Inputs are constructed examples, not user data. No archive extractor, network share, live backup product, nested path, link, concurrent writer, permission restoration or power-loss survival was tested. The Unicode reference explains the concepts; the computation uses the recorded Python Unicode database, not the newer reference's entire character repertoire.

Primary sources

1. Unicode UAX #15, revision 57 (Unicode 17.0): canonical and compatibility normalization. Accessed 2026-10-20.
2. Linux kernel documentation: ext4 case-insensitive file name lookups. Accessed 2026-10-20.
3. Microsoft Learn: per-directory case sensitivity. Accessed 2026-10-20.
4. Basu, Sampson, Qian and Jaeger (2022): Unsafe at Any Copy, arXiv:2211.16735v1. Accessed 2026-10-20.
5. Python 3.12: str.casefold. Accessed 2026-10-20.
6. Python 3.12: file names, filesystem encoding and error handler. Accessed 2026-10-20.