FITS archive checks: reordered pixels can preserve both checksums

Nine cases, with the actual pixels checked

FITS stores images and other scientific data in header/data units (HDUs). The checksum convention supplies a data-only check, DATASUM, and an entire-HDU check, CHECKSUM. NASA records its first use in 1994 and registration in 2007. It is an established archive mechanism, not a cryptographic authentication format. [1] [2]

The table reports checks before image-array access. Our starting image contains four signed 32-bit pixels: 10, 20, 30 and 40. Every single-unit fixture has one 2880-byte header and one padded 2880-byte data record. We alter disposable copies and then check raw bytes, decoded values and each HDU separately.

The one-bit control changes 10 to 11 and fails both checks. The pixel swap changes the sequence to 20, 10, 30, 40 and passes both. Its whole-file SHA-256 differs from the separately retained original, while the embedded checksum cards are byte-for-byte unchanged. The padding-only control fails both checks while returning the original pixels. This contrast matters when diagnosing a mismatch. The pixel swap is an intentional checksum collision, not the valid-old-record case in our identity and freshness analysis.

The sum loses the order of its words

The convention adds big-endian 32-bit words using one's-complement arithmetic: overflow is folded back into the low end. Addition does not preserve word order. Exchanging complete, aligned words within the data records therefore preserves their sum. With an unchanged header, it preserves the whole-HDU check too. The proposal explicitly warns about word rearrangement in its introduction; this experiment reproduces that known boundary. [2, introduction and sections 2-3]

Our independent accumulator obtains a data sum of 100 before and after the swap. The whole-HDU sum remains 0xffffffff, the convention's negative zero. A regression test checks all 24 permutations of the four words. These selected transformations do not estimate how often storage systems reorder data or how frequently other corruption escapes detection.

Metadata can change the scientific values

Image interpretation follows physical value = BSCALE * stored value + BZERO. The scale and offset live in the header; a data-only checksum cannot cover them. [5]

Changing only BSCALE from 1 to 2 leaves the complete padded data record and its SHA-256 identical, while the decoded values become 20, 40, 60 and 80. DATASUM passes and CHECKSUM fails. After deliberately regenerating both checks, both pass for the changed interpretation. Regenerating checksums can accompany a legitimate processing step, but cannot establish that the step was scientifically correct.

For storage verification, we disable automatic scaling during checksum calls and decode the image through a separate read-only open. Astropy can change its in-memory header representation when image values are scaled. Keeping those operations separate makes clear which representation was checked. [5]

Reading the pixels changes which bytes get checked

Our ninth fixture flips one bit at file offset 2899, inside data-record padding and beyond the four pixels. The padded record's sum changes from 100 to 101. Both embedded checks fail on a fresh open, although the decoded pixels remain 10, 20, 30 and 40.

In a separate read-only open with scaling disabled, we call both verification methods, access hdu.data, then call them again. The return pair changes from (0, 0) to (1, 1). The complete file hash is identical before and after reading: no bytes were repaired or checksums regenerated.

The pinned image implementation explains the difference. Before loading, the checksum path reads the padded on-disk data span. Once the image array is loaded, it accumulates the array bytes, which exclude the altered padding. This is a version-scoped observation about the checked representation, not a claim that pixel data were silently changed. [4] [7]

For archive-byte verification, check a stable file before accessing image arrays and retain those verdicts. Reading all pixels and checking afterward does not establish the same coverage. Conversely, a checksum mismatch need not mean the decoded pixels differ: padding and headers are part of the protected record too. The other eight fixtures provide controls, with unchanged verification pairs across this unscaled data-access step.

A returned file handle is not the verdict

All nine files opened and passed structural verify("exception"). The bit-flipped image also opened without a warning under default options. With checksum=True, the library returned the file and issued checksum warnings. With both cards absent, it returned without a checksum warning. Those are measured outcomes under the normal warning policy, not exceptions being caught and ignored.

Astropy documents optional verification and warning-based mismatch reporting. Its explicit methods distinguish three states. Compare them by value, not truthiness: in Python, bool(2) is true. [3] [4]

The CFITSIO row is taken from its C API documentation, not a second execution of this experiment. DATAOK and HDUOK are output verdicts, separate from the routine's operational status. A nonzero test would also misclassify its mismatch value. [6]

Check the expected inventory as well as each unit

The two-HDU corruption fixture verifies its primary HDU and fails its second HDU. Checking only the first would miss the injected fault. The removal fixture instead drops the entire second HDU: the remaining file is structurally readable and its surviving checks pass. Comparing the observed one-HDU inventory with the expected two-HDU inventory detects the omission; the per-HDU sums cannot supply that expectation.

For an archive ingest or restore, keep an independently protected manifest of expected files, digests and HDU identities. Explicitly distinguish absent, failed and valid checks for every expected unit. Preserve scaling and other interpretation metadata, and validate application-level expectations such as dimensions and expected extensions. A count alone is not an identity check.

Verify a stable, read-only copy before any checksum regeneration or transformation. A digest whose reference can be replaced alongside the file does not authenticate an archive. This complements the different read-coverage problems demonstrated for Parquet and the external interpretation dependencies demonstrated for HDF5 virtual datasets.

Run the experiment

Download the Python harness, pinned dependencies and complete JSON observations. The harness creates its own temporary files, accepts no input-data path and refuses to overwrite an existing output. It records raw hashes, per-HDU sums, return codes, warnings and decoded arrays.

python3 -m venv .venv
.venv/bin/pip install -r requirements.txt
.venv/bin/python observe.py --output my-results.json

Recorded with Python 3.12.3, Astropy 8.0.2 and NumPy 2.3.4 on 2026-10-18. Fixtures and hashes reproduce exactly under the recorded dependencies; generation time and platform metadata describe each run separately.

These are constructed, uncompressed image cases, not telescope measurements or a sampled incident population. The experiment does not test compressed images, tables, concurrent modification, remote storage, physical-media faults or power-loss behavior. Its contribution is a reproducible connection between the checksum algebra, application API and archive acceptance policy, not a new failure-rate estimate.

Primary sources

1. NASA FITS registry: Checksum Keyword Convention
2. Seaman, Pence and Rots: FITS Checksum Proposal (23 May 2002), introduction and sections 2-3
3. Astropy: verification using the FITS checksum keyword convention
4. Astropy v8.0.2: verify_datasum, verify_checksum and warning handling
5. Astropy: FITS image scaling with BSCALE and BZERO
6. CFITSIO C user's guide: File Checksum Routines and DATAOK/HDUOK
7. Astropy v8.0.2 image.py: loaded-image checksum path