Maybe you were like me when you first read about NetBackup’s Open Storage Option (OST): you were underwhelmed. You also may have been waiting for vendors to jump on it. (It was announced two years ago.) Things have definitely changed since then. In fact, at a recent very large customer that was considering purchasing a dedupe target, they chose only to look at vendors that supported NetBackup’s OST. Click Read More to see why this option is so important for NetBackup customers, and why other backup software vendors better come up with something like it real-quick-like.
Like others (especially VTL vendors) I did initially see Symantec’s Open Storage Option as an attempt by Symantec to kill the VTL market and replace all of them with regular disk behind PureDisk servers. I remember saying to one of their product managers (in a not-so-friendly manner), “VTLs are not going away, and the sooner you stop ignoring that fact, the better!”
Then I found out that OST wasn’t anti-VTL. It’s one of the most common misconceptions about OST (promulgated on Scott Waterhouse’s The Backup Blog). The truth is that OST doesn’t care how the target storage device stores its backups, which allows each disk vendor to store them how they want to store them. If they want to store them on a filesystem, great. If they want to store them on virtual tapes, fine. Just be able to accept a backup by name and store it and retrieve it by name. I liken the API to RMAN. RMAN gives you a blob of data and calls it “database_string_date_timestamp_etc.” and asks you to store it. It doesn’t care HOW or where you store it, just that you do, and that you give it back when they ask for it. That’s exactly how OST is.
The Open Storage Option (which, for some reason is referred to by Symantec folk as OST. I’ve been told it’s Open STorage. Alright…) is an API put forth by Symantec for writing to disk. When NBU wants to send a job to disk, it tells the storage device via the API that it has a backup for it to store, and it gives that backup a name. The storage device then accepts the backup and stores it in a manner of its choosing. After that point, NBU can do one of three things: expire it, restore from it, or copy/duplicate it. If the storage device is told by NBU (via the API) to expire the image, then it is supposed to delete it from storage. When NBU wants to restore from an image, it tells the storage device via the API what image it wants to restore from. The storage device then delivers the image to NBU for restore. (Again, both of these are very similar to the way RMAN talks to backup apps.)
True coolness comes when NBU wants to duplicate a backup. OST supports what it calls “optimized duplication.” In order for this to work, you need to have registered two OST-capable storage devices from the same vendor. NBU then tells device A to perform an “optimized duplication” to device B. The two devices then communicate and figure out which new data segments have to be replicated from device A to device B in order for device B to be able to say it has this image. (The number of segments replicated will vary greatly based on deduplication factors and how unique this image is in comparison to previous images already stored on the target device.) When the replication/optimized duplication is done, it reports that to NBU via the API, and all of this shows up in the Activity Monitor. Obviously, if there are problems with the copy, the storage device can report that via the API as well, and that will be reflected in the Activity Monitor. When the job is completed successfully, NBU will now know about both copies of the image. You can connect a media server to the replicated device B and can use those images just like you would any other images, which includes copying those images to tape.
This capability is 100% in line with what I’ve been describing for years as the Nirvana of dedupe. Backup to one device, replicate to another, have the backup app know about the replicated backup, then copy the replicated backup to tape. Now I have onsite backups, offsite backups, and a long-term-retention copy on tape – and I never have to move tapes around. No more guys in trucks! No more chances that one of them will go in Starbucks while the truck is stolen, or drop your tape along the way – no more chance your company will have to go on CNN explaining why you lost your tape. Your only tapes are in a locked tape library inside a locked cage inside a locked data center inside a secure building – and they never move. Suuuwheet.
So that’s my feeling about OST. I think it’s the greatest thing since sliced bread. Other vendors need something like it, but I’ll do another post about what I think they should do.
----- Signature and Disclaimer -----
Written by W. Curtis Preston (@wcpreston). For those of you unfamiliar with my work, I've specialized in backup & recovery since 1993. I've written the O'Reilly books on backup and have worked with a number of native and commercial tools. I am now Chief Technical Evangelist at Druva, the leading provider of cloud-based data protection and data management tools for endpoints, infrastructure, and cloud applications. These posts reflect my own opinion and are not necessarily the opinion of my employer.