Gartner never said 71% of tape restores fail

How many times have you read that Gartner said 71% of tape restores fail?  Google it.  You’ll find dozens of references to this Gartner “statistic.”  It was cited again recently in an article by Highly Reliable Systems, along with a bunch of other stats about how tape sucks. I saw Dave Russell of Gartner last week and asked him about this statistic.  He said he had never heard it, but that he would look into it.  It turns out that the only way he could find it was to Google it.  He searched Gartner’s entire archive and could find no paper that ever suggested at 71% failure rate for tape restores.

He said, “I am somewhere between annoyed and pretty darn angry about what I believe are continued misquotes re. Gartner and tape failure rates.  I’ve been the lead analyst for backup and recovery technologies since 2005, and none of what’s out there have been published during my watch.”  The only report that referenced tape and the number 71% was a report David did in March of 2006.  Here is what it said:

New, and less-expensive, disk options make the use of disk for faster recovery a more viable option than backup to tape. In a poll of 252 attendees at the 2005 Gartner PlanetStorage conference, 26 percent reported that half or more of their recoveries were currently done from disk. That number jumped to 62 percent when the time frame was extended to 2007. As they look five years into the future to 2010, 71 percent expect that tape will be used mostly for archiving and disaster recovery.

I did a bunch of web searches for “Gartner 71% tape restores fail,” and found that if I search for those words prior to March of 2006, I don’t find much.  I do find an article from Jon Toigo in 2005 that says he hears IT people quoting a 10% failure rate from Gartner, but he believes that number is fictitous (which it probably was.)  I also find a whitepaper from Exabyte that refers to a 2002 article from Adam Couture of Gartner Group.  I just asked David Russell to see if he can find that article.  I also found another whitepaper from Tandberg citing similar numbers and the same paper.  Maybe that one has some basis in reality.  Most interestingly, I did find this page which claims to be the text of a Feb 2003 article from Computer Technology Review that says that “A recent study [it doesn’t cite the study] found that while tape backups are used extensively, restoring data from a tape backup system fails an astounding 70 percent of the time. The reasons for such an alarming rate of failure range significantly–and may vary from bad tapes or tape drives to the inability to find the backup tapes or careless processing by IT staff.”  (My experience has been it’s been far more careless processing by IT staff than bad tapes.)

The important thing is that prior to March of 2006, a Google search shows no references to Gartner thinking that 71% of tape restores fail.  Then David Russell wrote his report in March of 2006 that said that “71 percent expect that tape will be used mostly for archiving and disaster recovery.”  If you change your Google search to the year after his paper came out, you find a bunch of quotes to the 71%, the first of which comes from this DPM Datasheet from Microsoft — promoting DPM.  Then all of the sudden, the floodgates are open and everyone is quoting this number — no one (including Microsoft) actually giving their source, other than simply saying “Gartner said it.”  Most of them also seem to quote the Yankee Group (saying 42%) and Strategic Research (54%).  I wonder if they ever said what these articles say they said. 

Another quote I’ve seen is this: “according to Ben Matheson, group product manager for Microsoft’s “Data Protection Manager” Division,  42% of attempted recoveries from tape backups in the past year have failed.”  (BTW, please note that this is the same number as the Yankee Group number above, so maybe he was just quoting that number) I saw this in an article updated last week. According to LinkedIn, Ben Matheson hasn’t worked for Microsoft since February of 2006, so that quote can’t be correct either.  But once you’ve got a great quote, why let it go?  Wait, I may have found our Gartner quote culprit. Let’s see, Ben Matheson leaves Microsoft as its DPM product manager in February of 2006.  The new person took over shortly thereafter.  The next month a Gartner paper is written, and within two months we have the Microsoft DPM product group citing it incorrectly.  Could it have been a new gung-ho product manager misquoting Gartner?  Then everyone else starts quoting Gartner by quoting this Microsoft paper.  Next thing you know it, it’s real!  (This is just conjecture, of course. Don’t sue me, person who took over from Ben Matheson.)

So what?

We all know tape backups and restores, fail, right?  Who cares if no one at Gartner said it?  The first reason is truth.  This statistic is cited so often that it has been accepted as truth, and it isn’t.

The second reason is that you can’t debate the truth of a fake report. If it was a real report, we could check the stats behind the stat, and see how many of these “tape restore failures” were caused by human error and had nothing to do with the fact that they were using tape.   But since there never was any report, we can’t do such a thing.

Please, people.  Don’t quote third parties like that if you can’t cite the source.  It’s too easy to misquote.


Written by W. Curtis Preston (@wcpreston), four-time O'Reilly author, and host of The Backup Wrap-up podcast. I am now the Technology Evangelist at Sullivan Strickler, which helps companies manage their legacy data

5 comments
  • It is the responsibility of anyone who reports statistics to verify the source. As a reporter myself, if I hear an interesting number, my first task is to track down the original report from which the statistic came. No original source, no quote.

    The only exception might be if I get a number from a legitimate source, perhaps a vendor I am talking to for a story, in which case I might say something like, “Jones, quoting Gartner data, said XXXXX.” However, “Jones” in this case is someone I trust and who may have access to data not available to the public.

    Joe Kovar, CRN

  • Hi Curtis. FYI (Thanks for the info)
    I have done hundreds of DR Tests and invocations using tape and disk over the past 12 Years.
    I can comfortably say that most restores do fail yes, but tape cannot be blamed. The Primary reason for all failures on restores are people and software… My Own stat will say that up to 85% of all 1st DR Tests and actual recoveries fail due to mismanagement of data and backups. Any Organization needs to test their backups regularly, but they don’t.
    The secret is all in the testing.
    Restores and full recoveries are less likely to fail if your backup selections are correctly placed and tested. Some of the biggest companies overlook this problem and ownership and policies for “Backup” just doesn’t exist.

    So there is some truth in this hype, but its not accurate…No-One seems to ask WHY ?

    Hope this could contribute.

    Thanks
    C

  • For over 20 years experiences working with backups, I have only seen:
    – 3 media failures
    — 1 of them where caused by software overwritten the label (beginning of tape), no data loss
    — 1 of them caused by SUN’s exporer checking status on all devices (mt -f st0 rewind); data loss, but no one asked for the restore
    — 1 of them where a damaged tape media, bad production according to vendor, we replaced all of this tape ranges to avoid data loss, we picked older volume, and recover using archive logs, no data loss.

    On the other hand, there are multiple tracks on a tape, and if you use RAID on disks, this will be aprox similare.
    The differences is that you never get noticed if one track is bad, but you get noticed if one disk in the RAID is broken.

    We have successfully implemented Disk to Disk backups (no VTL) for open system.
    We use both server de-duplication and client de-duplication with compression.
    We have implemented VTL for iSeries.

    This was a need to improve restore for customers, and at same time create a solution that suits different smaller office locations.

    We are protecting aprox: 3PiB
    We have 10 different bigger locations, and up to 200 different smaller locaitons.

    Regards Tomas

  • Isn’t this all beside the point? 69% of the 70%, is sysadmins evolving storage to no longer being in lock step with backup. Blaming MEDIA, was and is a completely specious argument, when the truth is that the admin model for backup is just inadequate in most cases. As soon as disk backup gets older, and thus more neglected, the failure rates will match.

  • This is/was the most quoted totally made up statistic in the backup industry, and the fact that it was made up I think is very much the point. As to the second part of your claim, since failure rates never were that high, I’d have to disagree.

    In addition, I do totally acknowledge that backups to disk succeed more often than backups to tape. This is why I don’t advise sending backups directly to tape; send them to disk instead. I advise tape only for a cheap offsite copy (if that’s all you can afford) and long term retention.

    So, no, I don’t think the backups to disk will get as bad as the failure rates on tape — even though they were never as bad as people said they were.