Enterprise SSD replacements ran at 0.22% a year across 1.4 million NetApp drives, with firmware the biggest swing

The answer. Across the whole NetApp population the annual replacement rate was 0.22%, and it varied by model from 0.07% to nearly 1.2%. The authors say these numbers are significantly lower than those reported for data center SSDs and common numbers for hard disks. Even for models with similar specifications the rate can differ a lot: 0.53% for II-G 15TB drives against 1.13% for II-C 15.3TB drives. Google's own flash study found that most models had about 5% of drives permanently removed within 4 years and the worst about 10%, which is a different measure with a similar direction: flash replacements are rare compared with hard disks, but not zero.

Why drives were replaced. The most common single reason was a SCSI-layer error, about 33% (32.78%) of replacements and one of the severe types. A drive becoming completely unresponsive accounted for only 0.60%. About one third of replacements were predictive (category D) and so the drive was still operating when it was swapped out. Lost writes and aborted or timed-out commands were about equally common. The wear limit was not the problem: the typical drive used less than 2% of its rated program-erase cycles, and the 99th and 99.9th percentile drives had used 15% and 33%.

Early life and firmware. Instead of the textbook bathtub curve, failure rates rose for 12 to 15 months, then fell slowly for another 6 to 12 months before levelling out, so drives spend 20% to 40% of a 5-year life in infant mortality, with rates 2 to 3 times larger than later in life. For eMLC drives, those with less than 1% of rated life used were 1.25 times more likely to be replaced than drives with more. Firmware mattered a lot: 70% of drives stayed on one firmware version across snapshots, and the earliest versions could have an order of magnitude higher ARR than later versions, such as a factor of 8 for family II-A (FV2 to FV3) and more than 10 for II-F, and more than 2 for I-B. The effect persisted when only drives that never changed firmware were compared. Families such as II-J also showed a later version with a higher rate, so a newer version is not always better.

Warning signs and RAID groups. The defect list (g-list) was empty for 99.04% of drives, and drives with at least one entry were replaced significantly more often, including for reasons other than prediction. This is the same direction as the reallocated-sector result for hard disks in Ma et al., and Google likewise reports that previous errors predict later uncorrectable errors. Inside a RAID group, the chance of another replacement within a week was 9.39% after a replacement, against 0.0504% in a random week, a factor of more than 180. Of the consecutive replacements in one group, 46% were at most one day apart and 52% within a week. Larger groups had more replacements, but no clear rise in the share of groups with a follow-up failure, so the authors find no evidence that group size drives multiple failures.