NBU 6.5 new disk features

There are some new disk features in NBU 6.5, and there seems to be some confusion around them.  Hopefully I can clear that up.

First, neither of these options should be confused with the current way that integrated VTLs copy virtual tapes to physical tapes, which is thebarcoding-matching method that was discussed in other posts. These options are COMPLETELY DIFFERENT.

Option 1: OpenStorage API

Symantec working with "Intelligent Disk Targets" (their term for disk targets that have a file system interface and do cool stuff like de-dupe). The idea is for the IDTs to code to this API. Once they've  done that, Symantec will do a whole bunch of cool stuff via this APIthat they can't do via a VTL interface. This will include the things that are available via the Advanced Disk Storage Unit in 6.5, and will also include other things like optimized de-duplication, direct to tape copy, etc. (I don't think the direct to tape copy is currently in the code. That's a future.)

AS OF TODAY, the API is in 6.5, but no Symantec partners have released products that use it. Some have coded and are testing, but none have made it available via GA. (My guess would be that the early adopters  here are likely to be vendors like Data Domain whose primary interface to NBU has been a NAS interface.)

Here's the page on this (although right now the link to the data sheet is broken).
NetBackup 6.5 Options

Option 2: NDMP Direct Copy

I've been asking for this for a while, and I'm really excited it's out, well, almost out…

Symantec didn't/doesn't like the current Integrated VTL model of copying virtual to physical tape. (They like the idea of having the VTL do its own copying, but not how it's currently done. It's a copy process that they can't control and/or report on. )

This option, that they've been working on for 1.5 years or so, is going to give us our cake and let us eat it too. This will allow VTLs thatcan talk to tape to do so and let NetBackup control and report on the process. NetBackup will use NDMP as a control protocol (the original coders of NDMP would be so proud to know of this use of their protocol that was never envisioned) to tell the VTL to copy image A to another tape. Via this protocol a tape will be loaded into a drive, NetBackup
will label it and get it ready to send the image, and then NetBackup will tell the VTL to send the image, much in the way that it tells a filer today to send an NDMP dump directly to tape. As far as NBU is concerned, they're not involved once that's done. The VTL will tell it, "done," and NBU will record which tape it got copied to. The VTL could also, of course, say things like "failed," or "tape is full, I need another tape," etc. (It supports spanning tapes.)

(The fact that this is using NDMP has nothing to do with the format of the backup. It will be able to copy regular backups and NDMP backups.)

What I'm looking forward to is a VTL that uses this interface NOT to copy to real tape, but to copy to another VTL, using de-dupe & replication in the background to copy the image instantly and magicly, while still allowing NetBackup to control the process. Just think. If you were use de-dupe and replicating VTL a to VTL b, VTL a could "copy" an image to VTL B without actually making the copy, just by moving bits around. This would rock, but it's definitely just a dream at this point.

AS OF TODAY, the API is in 6.5, but none of the partners have gone GA with their product that supports this feature. They are in testing and it should be out soon (my guess is sooner than the Open Storage API).  (Again, my guess here would be that the early adopters of this are likely to be Integrated VTL vendors like Falconstor, Quantum/ADIC, & NetApp. There's no reason that all VTL vendors shouldn't be able to do this, though.)

You can read about how to configure this option in the NetBackup NDMP manual.

Written by W. Curtis Preston (@wcpreston), four-time O'Reilly author, and host of The Backup Wrap-up podcast. I am now the Technology Evangelist at Sullivan Strickler, which helps companies manage their legacy data

5 comments
  • Unless I’m misreading between the lines here, what you’re saying here is that I *still* can’t write backups with NetBackup (or TSM, or NetWorker, or… well, hell, let’s go out on a limb and call out Bacula, given that Amanda’s about that undead already) to a VTL and end up with a meaningful volser in my voldb?

    Give me a break. This is not a hard problem to solve. For example, if I’m NetBackup and you’re the VTL, just pass me the updated media ID for a given image back, and I’ll update it in the image DB, after you’ve duped it off disk to tape. These are trivial API additions (not not not changes) that don’t break existing API in any way. What is the damn problem with these people?

  • はい。はい私は日本語のマニュアルを読む。

    According to Babelfish, that says "Yes I do. I do read manuals in Japanese." Although, when I translate it back into English, it says, "It is. It is, as for me the Japanese manual? ? You read." Alright, you caught me. I don’t.

    I have NO idea what happened there. I have corrected the link to point to the right document now.

  • What I said is that you still can’t (although soon will be able to) have the VTL perform the copy from virtual tape to physical tape, while having NetBackup (or anyone else) control, manage, and report on the process.

    That doesn’t mean:
    1. You can’t have NBU/NW/TSM/BE/whatever make the copy of virtual to physical. (It’s just that the bits will move through the media server in the usual way.) Basically just use the VTL like a physical tape library. To suggest that this alone doesn’t lend a tremendous amount of value to the system is ludicrous.

    2. You can’t use the old method of barcode matching (which works for a lot of people). There are several downsides, but like I said. It works for people.

    And, BTW, criticizing the VTL industry for this problem is really unfair. Many VTL vendors have been pushing Symantec and other companies to do this for years, and they’re just now getting around to doing it.

    And it’s not as simple as you seem to say. For example, you have to handle tapes of different size while fully utilizing both tapes’ capacities, backups that span tapes, tapes that fail halfway through the process, etc. It’s not rocket science, per se, but it’s not as easy as you are purporting. In addition, you’ve got several VTL companies that all have a different idea of how to do this, and several ISVs each of which has a different idea how to do this. Symantec was able to use their position in the market place to get the VTL/IDT vendors to do it their way. The other ISVs still aren’t sure what they’re going to do. (Surely the VTL vendors would like to do this with everybody, but there is no plan for everybody. There’s just the plan for this one vendor at this point.)

  • 1. I’m not sure I agree that there’s a “huge” amount of value to a VTL that’s essentially a more-tape-like D[S]SU (in NBU parlance), but I see your point.

    2. I haven’t ever thought barcode matching wasn’t hokey: it’s doing something for ourselves that the computer should obviously be doing force (tracking data location).

    I’m aware of the complications you mention, but I still think it’s absurd that we still don’t have this interaction.

    And, btw, I absolutely do NOT blame the VTL vendors for this problem: I blame Symantas (et al) for failing to produce a useful API by which the VTL vendors could query and update the backup software’s image and media DBs. I know the market reasons they don’t, but that doesn’t mean I don’t think they’re a bunch of jerks.