Tape more reliable than disk for long term storage

Tape is inherently a more stable magnetic medium than disk when used to store data for long periods of time.  This is simply "recording physics 101," according to Joe Jurneke of Applied Engineering Science, Inc. 

I had heard rumblings of this before, but it was Joe that finally explained it in almost plain English in a post to this thread from hell on LinkedIn.  Here's the core of his argument:

By the way, the time dependent change in magnetization of any magnetic recording is exponentially related to a term known as KuV/kt. This relates the "blocking energy" (KuV) which attempts to keep magnetization stable, driven by particle volume (V) and particle anisotropy (Ku) to the destabilizing force (kt) the temperature in degrees kelvin (t) and Boltzmans constant (k).  Modern disk systems have KuV/kt ratios of approximately 45-60. Modern production tape systems have ratios between 80 and 150. As stated earlier, it is exponentially related. The higher the ratio, the longer the magnetization is stable, and the more difficult it is to switch state…..Recording Physics 101….

I had to call him to get more information.  He explained how this came about.  Disk drives have been pushed for greater and greater densities, which caused their vendors to create a much tighter "areal density."  Tape, on the other hand, mainly got longer and fatter to accomodate more data in the same physical space.  (Yes, it increased areal density, too, but nowhere near as much as the disk drive folks did.)  The result is that the tape folks have more room to play, allowing them to use magnetic particles with a bigger particle volume (the V in the equation).  The bigger the particle volume, the more stable the magnetism is, according to the KuV/kt equation.  In addition, tapes are generally stored outside of the drive, which means their temperature is lower than disk drives.  That means they have a lower k volume (degrees kelvin), which is one of the "bad" numbers in the KuV/kt equation.  Having a higher V value and a lower t value is what translates into tape systems having ratios of 80-150, vs disk systems that have ratios of approximately 45-60. While I don't have an exact cite to point to in order to show these exact values, what he's describing makes perfect sense to me.
 

Add to this the fact that tape drives also have a lower bit error rate than disk.  SATA disk is 1:10^14, FC disk is 1:10^15, LTO is 1:10^16, and IBM 3xx0 and Oracle T10000s are 1:10^17.

Add to this the fact that tape drives always do a read after write, where disk drives do not always do this.

Sooo…

Tape drives:

  1. Write data more reliably than disk
  2. Read it after they've written it to make sure they did (where disks often don't do that)
  3. Have significantly less "bit rot" or "bit flip" than disk drives over time.

Like I said in a previous post, I think we've put these guys out to pasture a little too soon.


Written by W. Curtis Preston (@wcpreston), four-time O'Reilly author, and host of The Backup Wrap-up podcast. I am now the Technology Evangelist at Sullivan Strickler, which helps companies manage their legacy data

26 comments
  • Good stuff.

    But, I’m curious what is taught in Physics 301 – since this is the 101 course šŸ™‚

    Basically, the hermetic seal of traditional rotational media has to come into play at some point. In my opinion, this increases HDD’s viability; and decreases that of tape.

    I’ll share two of my personal experiences that made me swear to disk for primary storage of a backup target. First of all, these were egregious examples; but nonetheless illustrate my point.

    First one was a tire manufacturer in central Mexico. My material handling solution was installed near the final area of tire processing to provide QA functions, due to connectivity requirements – the computer system had to be within 200 feet of the equipment. “LUCKY ME” as my system was installed directly above the whitewall side buffer. That is where the whitewall or lettering of tires is painted on. So, my tape drive was littered with paint, and failed in less than a year on writes – we never had to restore via tape, scared to have tried it.

    Second point prover for me was a large merchandiser in Madrid, Spain. Tapes and tape drives kept failing, when in the comptuer room. At the time (not sure if this is still the case), they smoked in the computer room. Seems that cigarette smoke doesn’t help tape operation.

    While these are atypical situations, it “set the tone” for my thoughts.

    Curious if spindown makes any impact in the physics above.

    Cheers. Let’s talk.

  • @Rick

    And if you store disk drives in an oven, they’re not very reliable either. Tape drives are meant to be used in a clean data center environment. Use them outside of that and you will not get the desired result.

  • @BRSDude

    Can’t really comment on that, but are you sure that what you’re seeing isn’t DESIGNS for disks w/thermal ratios > 60? That’s not quite the same as actual disks. BTW, how much > 60 did you find?

    I’m really going to defer to Joe on this one, as he’s the one w/38 years of experience in designing magnetic media. Hopefully he’ll chime in.

  • Yes, I know that Curtis – but having to protect data in harsh environments tape wasn’t an option. I could keep the air cool, but not clean.

    My examples were egregious – I know but nonetheless, I still think the sealing has to impact this stuff also.

  • The point of the article is about long term storage of backups. Even if you assume some “dirt” in the air (such as a datacenter I worked in that had people and printers in the datacenter), none of that is going to affect a tape sitting in a box/drawer somewhere for long periods of time. It will only affect the viability of the tape drive in long-term use situations.

    In a day-to-day situation, I’ll agree that a dirty environment will introduce contaminants into the tape & drive, causing the tape drive to fail. But it doesn’t cause the MEDIA to fail, in my experience. You put that tape in a different drive, or replace that drive, and it will read just fine. It’s another advantage of tape, that the media is separate from the drive.

    All the same: don’t use tapes in very dirty environments. Bad combination.

  • Sometimes the technology needs to be in ‘dirty’ locations. I don’t mean to have an Edward Haletky moment here, but we can’t always ensure that everything is in the datacenter.

  • And in those situations, I would use disk for backups. And for archives, I would archive to tape and then get those tapes out of their lickity split. šŸ˜‰

  • If I could have back in the day, I’d have done some sort of cloud solution. But it was 2005 baby.

  • Gentlemen,

    A couple of points:

    1. Hermetically sealed disk drives are still exposed to environment – especially thermal changes. KuV/kt still is acting on the disk magnetization at elevated temperature.

    2. Spinning down a disk drive, powering it off would tend to reduce it’s operating temperature. All drive manufacturers specify the drive operating temperature and in many cases, the thermal rise above ambient. Lowering the temperature is beneficial to thermal stability (KuV/kt)

    3. The indistry has been working on getting higher thermal stability. Difficult to achieve at the high areal densities which requires extremely fine grain disk media. If tape were operating at the same areal density, it too would have the same problem. As long as tape remains at a lower areal density, it should be capable of supporting excellent thermal stability.

    4. Debris and contaminants can be an issue. With tapes in a darkened data center, they are pretty well protected. With the use of tape cartridges, the shells protect against a tremendous amount of user transgressions. Just dont put your tape drives next to the high speed laser printers!

    Best regards,

    Joe

  • (Curtis, whatever you’re doing for page formatting is causing the text to go off the right margin where it can’t be seen, at least on my iPhone. No amount of pinch zoom appears to make a difference. The default zoom level on Chrome on XP also has content going off screen.)

    The whole “disk must replace tape” meme is starting to sound like a bit out of Catch-22.

  • Is that better? I had some tags leftover from when I cut and pasted Joe’s comment from LinkedIn.

  • Why would you pass up this perfect opportunity to blame it on Steve Jobs?

  • Perhaps I missed something in the discussion but would the obsolescence factor come into play here?
    We have tapes that we simply wouldn’t be able to read with our current devices. Most organizations wouldn’t go to the trouble to migrate from one generation tape device to the next.
    We have extended retention requirements (> 50 year) so tape isn’t a reality for us. I would suggest D2D is more effective and with the usual refresh of 5 years on spinning rust a migration occurs each time the disk is refreshed.
    Hope this made sense. Comments?

  • With 50 year retention requirements, you will be required to migrate data no matter which medium you choose. Disk should be migrated every 3 years, and usually is in practice. Tape can be migrated much less frequently. You must migrate tape if (a) your retention requirements are greater than its expected retention abilities or (b) you want to change drive formats.

    I prefer to use formats that have backward-read compatibility. LTO, for example, can usually read two generations back. Assuming you upgrading the second every new LTO drive came out, and wanted to throw out the previous drives, you would migrate data about every five years. This is only if you wanted to throw out the old drives. What most people do is keep around a few of the old drives and ignore the migration issue. But at some point you really must migrate. The older drives are no longer being made and you can’t get new versions of them or parts for the old ones. But the point of my post is that you don’t have to migrate data that often to keep the tapes (and the data on them viable) viable.

    For your data retention, though, (50 yrs) you’re going to be forced to migrate, no matter what device you choose. How many times do you want to migrate? With disk, most would say AT LEAST 10 times, probably closer to 15, with the typical every-three-years migration that is recommended for disk. With tape, you could conceivably plan things so you do it only 5 times.

  • Hey Curtis, I read the original FB post and responses here. Lots of interesting perspective and mix of opinions.

    Iā€™ve seen disk based backups come onto the scene and thereā€™s no dispute that it has a critical place in OR, DR, and BC.
    Yes tapes have a shelf life that seems to be longer than disk; I get the KuV/kt ratio discussion, but this assumes that these are the only variables; temperature, polarity, and potential difference… I did a tape handling assessment of an environment that had over 140,000 tapes in circulation. They had a number of operational issues with true tape archive (primary copy to tape) that bit them several times over. I know we say test your tapes yearly, but in many/most small and large shops this doesnā€™t happen for archive tapes. And when there are media errors, rarely does this cause people to rethink their archive strategy until itā€™s too late.

    Let me caveat the following story with the understanding that Iā€™m sure there is a level of probability from one shop to the next to consider. But in the few discovery projects Iā€™ve been on where we were pulling backup tapes from years of Exchange backups, Iā€™ve seen my share of media errors that left data unrecoverable. Here are some of the highlights of my findings from one project.

    -Well known outsourced tape handler (go ahead and guess, Iā€™ll bet youā€™re right) mixed their tapes with another customerā€™s tapes and said, ā€œoh those recalled tapes arenā€™t yours? Just toss them and weā€™ll find yoursā€!!! They never did and that guy was fired.

    -The cases they put tapes in were lined with foam that over 3-5 years started to deteriorate into dust, fowling the tapes for good!!!

    -Newer LTO tapes with built in electronic chips used to hold metadata info, are more susceptible to damage from being dropped. The CTO shared a story with me where they said the tape handler dumped out a tote full of tapes on the floor right in front of him, while he waited for the elevator. This happened amidst a string of tape drop media error issues the storage team was dealing with.
    So the fact that people are keeping archive copies of data for several years within backup is usually for one of two reasons.

    1. They think the data is important but are too lazy to do the work and justify a real archive solution, or not.
    2. They know the data is important but donā€™t understand the implications of where they store data.

    I know that disk is susceptible to corruption, but tape is susceptible to the human element, usually untrained and detached from the technology (outsourced tape handlers). I submit that there are people in the air gap with tape archive copy. I guess another probability calculation to consider. In the end I think itā€™s the litigation risk/cost that drives the financial decision to go with disk or tape for archive. Without a compelling risk/cost justification, the cost of disk even with dedupe, canā€™t touch tape.

    Most of these posts ignore the uncertainty of the human element with tapes, which in my experience isnā€™t so negligible. I think the value of the data needs to match the resiliency of the media that it resides on and Iā€™m not sold on tape being better than disk unless the tape stays out of human hands. What do you think about the human element and tape?

    -James

  • [message and URL deleted by admin]

    Hey, Norm. If you want advertising on Backup Central, we’d be happy to sell you some banner ads.

  • Norm, don’t go cheap with just a banner: add few bucks more and you can get also a very positive blog on your stuff!

  • @Stefano

    You must have me confused with someone else. I am not a blogger for hire. The only compensation this website provides is banner ads, and there is no quid-pro-quo. They are simple advertising, as there would be in any other publication.

    As to the comment to which you responded, this person put a blatant ad to his product in a comment, with a link. While I’m not a pay-to-play blogger, I don’t appreciate people trying to use my platform (which I pay quite a bit of money to maintain) to advertise their products without compensation.

  • All the science aside, you still have to handle tapes and there is the rub… Stuff happens. Compare this to disk based solutions which are not physically touched and have software to deal with bit rot.

  • “All the science aside?” Isn’t that another way of saying “ignoring the science?” Because you can’t. Science shows that data lasts on tape longer than it lasts on disk.

    You do not have to handle tapes in a properly sized tape storage system any more than you have to handle disk system. The fact that people undersize their tape libraries (and thus require tape handlind) is not a ding against tape; it’s a ding against the designers.

    “[disk systems] have software to deal with bit rot.” Tape systems do too. What’s your point?

  • @James

    I totally forgot to respond to your comment on line after our two hour phone con about it. šŸ˜‰

    If will boil your arguments down to:
    1. Tapes do indeed fail; you have seen it
    2. Tapes get mishandled

    Tapes do fail; so do disks. But for some reason a lot of people only make one copy of their important archive or backup on tape. They would never do that on disk, but they do it on tape.

    I concede your point. Tapes fail. That is why anything worth having on one tape is worth having on two.

    Tapes get mishandled. I will address this in two parts.

    1. Tapes get mishandled onsite

    I said this in my previous comment. A properly designed onsite tape system should not have any tape handling, other than to put new tapes in. Problem solved.

    2. Tapes get mishandled going offsite

    There is no question that tapes being sent offsite are a risk in a number of ways. This risk has to be managed in light of the cost/risk of either not sending this data offsite at all, or the cost/risk of going to a dedupe-enabled, replication-based offsite delivery mechanism.

    Let’s say that you go to a completely disk-based replication system for your daily backups and to get your archive data offsite as well. The point of this article is that the long term (many years) copy of that data on both sides is better off on tape than it is on disk — assuming my caveat on point 1 is taken care of.

  • No argument with tape vs disk from a hardware standpoint. Yes it’s cheaper, lasts longer etc etc.
    Here’s the rub. People are involved in this process and they fail. It’s life. Therefore the more people need to do to deploy and maintain a process the more errors there will be. That includes tape transportation (no not all of us can afford automated tape systems), media errors and not migrating from old to new technology when needed. Not to mention that test recovery from tape is one of the first things to go by the wayside when the pressure is on.
    When we can solve those issues, I’ll vote for tape as well.

  • @Bob

    What else would you vote for? It can’t be disk. Because if you can’t afford a tape system sufficiently large enough to handle your storage needs without human intervention, then you definitely can’t afford a DISK system to do that. The acquisition cost of the latter tends to be 3-4 times the former.

    If cost wasn’t an issue, then disk would totally be better. You can solve its long term viability (discussed in this article) by automating data movement ever 2 years. You wouldn’t worry about the incredible expense of powering & cooling all of those disks when they’re just storing data no one wants to read (or you’d pay for spin-down technologies). You wouldn’t pay for a “dude in a truck” service, you’d pay for unlimited bandwidth to replicate all data to two backup sites.

    But cost is always a factor. It’s the reason people think they can’t afford separate backup and archive packages (although they couldn’t be more wrong). It’s the reason people don’t buy big enough tape libraries. It’s the reason people keep plodding along with a really inferior backup product.

    And as to tapes getting lost/damaged during transit, all of that risk can totally be mitigated IF SOMEONE WANTED TO PAY FOR IT. But they don’t, and then they blame the tape.

    No one would think of storing any important data on non-RAIDed disks these days, but we put data on tape without having multiple copies and then ship it around all the time. Why are we so hard on tape and nice to disk?

  • [quote name=W. Curtis Preston]@Bob
    No one would think of storing any important data on non-RAIDed disks these days, but we put data on tape without having multiple copies and then ship it around all the time. [/quote]
    RAID for tape is usually RAID1, which is more expensive than RAID5 or RAID6 from a media perspective. What is the current availability of RAID5/RAID6 (or RAIT) technology for tape? It has the obvious disadvantage of requiring as many tape drives as the stripe width, which has probably discouraged its use, but perhaps some sites would use it if it works well.

  • @Lance

    They tried RAIT once, it didn’t take. Among other things, it makes tape’s number 1 problem (streaming) worse. It also makes tape management a nightmare.

    WRT your point about cost, I’ll take “RAID1” on tape vs RAID5 on disk, since tape media is at least 50 times cheaper than tape.