Why are the major backup products slow to fully support vSphere?

Reading announcements from companies like EMC, Symantec, CommVault, and CA, you’d think they were in the forefront of “proper” support for EMC’s vSphere.  When you read things like “NetBackup & Backup Exec certified for vSphere 4.0,” you would think that means that it supports vSphere’s new backup features.  You would be incorrect on both counts, and the same is true for EMC, CommVault, CA, and (oddly enough) VizionCore!

The problem with backing up VMware has always been I/O.  You can back up the VMs as if they were PMs, and you’ll run into the laws of physics pretty quickly — although this is what something like 90% of people do.  It limites many people’s use of VMware by limiting the number of VMs they can put in one ESX server.

Along came VCB, which I think stands for Very Complicated Backup.  It attempted to solve the problem by installing a physical proxy server that could see the ESX server’s storage, and having the proxy server move the data.  Although there were two modes, the proper mode that allows for recovery of files AND the VM (what I call the image method) requires copying the entire VMDK file to the proxy server, and then copying it to your backup system.  It also requires a two-step copy during restores as well.  Some products would use the image method for the full, and then use the virtual copy method (which isn’t a two-step copy process) for the incremental.  That’s not too bad.  But some vendors actually copy and/or read the entire VMDK file every night to figure out which blocks have changed.  That’s the I/O equivalent of a full every night!  Now that’s what I call bad design.

Enter vSphere and its change-block-tracking API.  (VCB is officially gone, although the framework is still there for now for legacy apps.)  Backup applications that fully support vSphere can ask VMware what blocks have changed for a given VMDK and copy just those blocks.  There’s no need to read the whole file to figure that out; just ask VMware.  (For the Oracle folks among us, this is exactly equivalent to what Oracle did with RMAN in 10g.  Prior to 10g, doing an incremental was actually worse than doing the full because of all the I/O going on to figure out which blocks had changed!)  And the new vSphere API doesn’t require a two-step backup or a two-step restore!

Sounds great, right?  But guess what?  None of the “biggies” (EMC, Symantec, CA, CommVault, Vizioncore) are actually using all of the new API. When they say they support vSphere, what they’re usually saying is that they support it the way they supported ESX 3.5, which was usually VCB.  They all say that full support is “coming soon,” usually “next quarter.”  But I think it’s important to point out who actually supports it now.

I can find only four products that support vSphere’s new backup API.

  • VMware’s VMware Data Recovery
  • PHDVIrtual’s ESXpress
  • Veeam Backup & Replication

If you want full VMware vSphere support, with incremental backups that don’t take forever, and one-step backups and restores, these are the products to look at today. I’m not saying the other products are garbage. (In fact, I hear a lot of good things about other areas of those other products. But in this area they are behind — and I think this area is really important.)

Update: CA’s ARCserve supports the single step backup and restore, but they do not support the change-block tracking feature.

I’m just surprised.  Aren’t these companies partners?  Don’t they get the software early? Can’t they see the value of supporting new stuff as soon as it comes out?  Why is it only the lesser-known products are the ones stepping out front here?

Written by W. Curtis Preston (@wcpreston), four-time O'Reilly author, and host of The Backup Wrap-up podcast. I am now the Technology Evangelist at Sullivan Strickler, which helps companies manage their legacy data

7 comments
  • We have an agreement. This is the future. Microsoft has the same technology in Windows 2008. If the recovery application vendors (come on Legato) will adapt this technology and expand it to support Linux and other UNIX flavors, it will redefine backup and in large extent recovery. Using disk as a target, with this approach, will make it even more beautiful.
    But I have to ask, once this is done, what is the future of de-duplication ?

    The Doctor

  • People still do repeated fulls using the technology I’m discussing. It’s not a CDP or near-CDP solution; it’s just a way to get the incremental blocks way faster.

    BTW, where did you go? I sent you an invite for the podcast and never heard from you.

  • was at VMWorld all of last week ๐Ÿ™‚

    Anyways, why should one pay the premium of de-duplication once block level incrementals are available, doing a synthetic full on a block level makes much more sense.

    The Doctor

  • Boy you sure have a bone to pick with dedupe. It really does add a lot to the picture — and to the recovery picture. I’m sorry you had some bad luck with one of the vendors, but your experience does not match the rest of our experiences.

    But since you brought it up, as long as you’re doing FULLS, synthetic or otherwise, dedupe adds value. Block-level incrementals reduce the dedupe ratio for sure, but not to nothing. The only thing left then is to debate how much premium you’re going to pay.

    You can still reply to my email and I can see what we can get setup over there.

  • I’m not surprised the big companies are lagging. They always do. VSphere will be midway in the life cycle before they are all full featured. I mean sheesh it’s taken TSM how long to be able to do individual email restores for Exchange?

    I AM surprised at VisionCore specifically. I figured they would have reacted quicker but then their overall portfolio is vastly more diversified than Veeam (which is an admittedly weak excuse).

  • I configured Vmware Data Recovery Yesterday and got a test working. I’m still in the nascent testing phases. Great idea, the dedupe. One thing bothered me, how can I get the result on tape? It’s lovely to have this much more sophisticated back up doing it’s thing, but when my san goes into a giant pit in our next earthquake, it won’t do me a whole lot of good.
    I have vcb 1.5 working in Networker 7.5.1 although the LNIM (Legato Networker Integration Module) for VCB 1.5 hasn’t been officially released by EMC for Networker.
    We have about 50 vm’s. Emc says to break them up into groups of 5 or 6. That means about 10 networker saveset groups for us to manage with about 5 in each group. All that stuff has to fit on the proxy server mount point and you have to hope that the last job runs successfully and deletes the snaps before the next one runs and starts dumping more in. It seems to be slow too. I have to admit, I would just rather back them all up the way I used to, over the network on backup exec using clients. It’s less hassle. The server goes straight to tape instead of all this snapping,writing, deleting, snapping, writing deleting.

    Does anyone know how to write Vmware Data Recovery to tape from the disk it writes to first?

    Pat Page
    Contra Costa County Health Services
    Network Admin, Martinez, CA

  • It’s really a backup app for those who don’t have a backup app, and for those who don’t want much, like copying to tape. ๐Ÿ˜‰