Friday, May 15, 2020

HDD Failure Rates Based on 129,764 HDDs Running

As of March 31, 2020, Backblaze had 132,339 spinning hard drives in our cloud storage ecosystem spread across four data centers. Of that number, there were 2,380 boot drives and 129,959 data drives. This review looks at the Q1 2020 and lifetime hard drive failure rates of the data drive models currently in operation in our data centers and provides a handful of insights and observations along the way. In addition, near the end of the post, we review a few 2019 predictions we posed a year ago. As always, we look forward to your comments.

Hard Drive Failure Stats for 1Q20
At the end of 1Q20, Backblaze was using 129,959 hard drives to store customer data. For our evaluation we remove from consideration those drives that were used for testing purposes and those drive models for which we did not have at least 60 drives (see why below). This leaves us with 129,764 hard drives. The table below covers what happened in 1Q20.

Notes and Observations
The Annualized Failure Rate (AFR) for 1Q20 was 1.07%. That is the lowest AFR for any quarter since we started keeping track in 2013. In addition, the 1Q20 AFR is significantly lower than the 1Q19 AFR which was 1.56%.

During this quarter 4 drive models, from 3 manufacturers, had 0 drive failures. None of the Toshiba 4TB and Seagate 16TB drives failed in 1Q20, but both drives had less than 10,000 drive days during the quarter. As a consequence, the AFR can range widely from a small change in drive failures. For example, if just one Seagate 16TB drive had failed, the AFR would be 7.25% for the quarter. Similarly, the Toshiba 4TB drive AFR would be 4.05% with just one failure in the quarter.
On the contrary, both of the HGST drives with 0 failure in the quarter have a reasonable number of drive days, so the AFR is less volatile. If the 8TB model had 1 failure in the quarter, the AFR would only be 0.40% and the 12TB model would have an AFR of just 0.26% with 1 failure for the quarter. In both cases, the 0% AFR for the quarter is impressive.
There were 195 drives (129,959 minus 129,764) that were not included in the list above because they were used as testing drives or we did not have at least 60 drives of a given model. For example, we have: 20 Toshiba 16TB drives (model: MG08ACA16TA), 20 HGST 10TB drives (model: HUH721010ALE600), and 20 Toshiba 8TB drives (model: HDWF180). When we report quarterly, yearly, or lifetime drive statistics, those models with less than 60 drives are not included in the calculations or graphs. We use 60 drives as a minimum as there are 60 drives in all newly deployed Storage Pods.