VMware backup follow-up

Last month I wrote a blog post about how I thought that Hyper-V was ahead of VMware in the backup race.  If you haven’t read, you should read it before reading this follow-up.  Before you read that post, however, you should probably know about what’s in this post that explains Windows VSS and why you should care.

Now that we’re all caught up, let’s summarize, shall we?  Although the vStorage API fixes most of the problems with VCB, it still has one gaping hole to be fixed.  It does not fully leverage Windows VSS to the point that it is not possible to get an application-consistent backup of Windows 2008 guests without putting some kind of agent in the VM.  You can get app-consistent backups of Windows 2003 apps, but the apps won’t know you backed them up because VMware uses VSS_COPY and not VSS_FULL_BACKUP when it creates the snapshot. And that’s why we’re here.

I made a number of recommendations in that post.  I listed a few storage systems and backup products that can allow you to work around this little problem, but I left some out.  That’s what this post is about.  The following is a more comprehensive list of products that have worked around this problem.

  • Storage Systems
    • NetApp FAS Arrays & the SnapManager line
    • FalconStor IPStor
  • Backup Software
    • Backup Exec
    • ESXPress
    • Veeam

All of these products have a similar approach.  They each put some kind of agent in the guest that coordinates the backups.  it doesn’t work the same way a typical host-based backup agent does; it’s only purpose is to do what VMware should be doing — putting the apps and filesystems into backup mode at the appropriate time.

Until VMware addresses this limitation, I think these products are your best bet for getting application-consistent backups without having to run guest-level backups.

Update 3/8: As pointed out in a comment, there are other ways of backing up VMs that are better than putting traditional agents in the guest.  Both source-dedupe and CDP help with the I/O problem that traditional backup agents in the guest cause because both of them significantly reduce the amount of I/O the guest-level backup causes.  The point of this post, however, was how you could do proper backups of a VM without having to do any guest-level backups — even very small ones.  But I thought a clarification was in order.

Written by W. Curtis Preston (@wcpreston), four-time O'Reilly author, and host of The Backup Wrap-up podcast. I am now the Technology Evangelist at Sullivan Strickler, which helps companies manage their legacy data

9 comments
  • … just your final sentance ‘Until VMware addresses this limitation, I think these products are your best bet for getting application-consistent backups without having to run host-level backups.’

    I’m sure you meant to write ‘*guest-level* backups’.

    Adam

  • Another best way for backing up VMs (I think) is to put a good agent into the guest, agent with source dedup which dramatically reduce resource consumption.
    And if you could have a VM proxy to use the vStorage API, it’s even better, because, vStorage API presents only new blocks, and your agent (with source dedup), backs up with dedup the new blocks, so a very small amount of data.

  • The point of the post was how to do proper backups of a VM without any guest-level backups, but your point is well taken. There are other ways to solve the I/O problem. I updated my post.

  • Curtis:
    Great topic, and a nice blog about Hyper-V.
    I would like to mention a product out there that will backup ESX virtual machines , thru the use of a VCB proxy and you never have to touch the virtual systems.Sound to good to be true? Welcome the SSVCBAgent.

    The VMware Consolidated Backup (VCB) with the TSM
    Interoperability Module (TSMIM) allows file-level backup for virtual machines running any version of Microsoft Windows that is supported by VMware ESX Server version 3.0. It also enables full virtual machine (full VM) backup for all virtual machines running on VMware ESX Server version 3.0.
    This method requires a VCB proxy server to be configured along with the TSMIM and the TSM backup/archive client for Windows. The TSM client is registered with the TSM Server and added to a schedule. Using the TSM scheduler pre and post schedule commands and the command scripts provided by the TSMIM module (pre-command.bat and post-command. bat), you can specify a list of virtual machines to be backed up and whether to perform a file level backup or a full VM backup.
    Backed up data is saved on the proxy server in a unique directory structure for each virtual machine. Both file level data and full VM backups are saved in these directories so it is possible to selectively restore individual files and to perform rapid full system restores.

    The STORServer Agent for VMware Consolidated Backup controls the backup and
    restore for each virtual machine by integrating the VMware Consolidated Backup
    utilities and IBM Tivoli Storage Manager (TSM) backup-archive client and
    SAN agent.
    The STORServer Agent integrates with the Consolidated Backup proxy
    utilities to freeze the virtual machine, create a snapshot, and thaw the
    machine. The Agent mounts the snapshot to the VCB proxy server (running
    on Microsoft® Windows®) and calls the TSM backup-archive client to
    perform either an incremental file level backup (for virtual machines running
    the Windows operating system) or a full image backup (for Windows and
    non-Windows virtual machines). The data is stored in the TSM Server
    storage pools.
    The virtual machine continues to operate without interruption while the
    backups take place. Changes to the virtual machine while in snapshot
    mode are written to a redo log stored within the virtual machine.
    When the backup is complete, the Agent dismounts and deletes the
    snapshot. Changes in the redo log are then written to the virtual machine.
    The restoration process is just as easy as the backup process.
    You can add vm’s to your environment and schedule them for immediate backups without needing to install or configure anything on the VM itself.

    John Stephens

  • Did you read the post to which you’re replying? Specifically the part that says “it is not possible to get an application-consistent backup of Windows 2008 guests without putting some kind of agent in the VM.” Your post, which amounts to a commercial for your product, says that your tool “integrates with the Consolidated Backup proxy utilities to freeze the virtual machine, create a snapshot, and thaw the machine.” Those would be the very same VMware utilities that I just got done say are not up to the job yet. In addition, it’s also worth nothing that you mention nothing about the vSphere storage API and talk all about VCB, which is now deprecated and will disappear in the next version of vSphere.

  • Curtis:
    Yes, the post was read, and yes I can backup windows 2008 guest wihtout putting any agent on the vm. I am sorry if you suspect the information of good working solutions amount to commercial’s, that is not the intent. I suspect many of the blogs posted here and everwhere else that express working soultions to everyday backup needs could be seen as commercials, as you mention.
    Now back to what matters, The current release of this solution works with VCB only. It does not use the vSphere storage API. However, that does not mean that it cant work in a vSphere V4.0 environment. vSphere V4.0 supports the use of VCB 1.5-U1 (and U2, which just came out). So it uses the VCB framework on the proxy and whether it is a vSphere V4.0 or older V3.5 environment is irrelevant.

    The next major release of this product will use the vSphere V4.0 API and will not require the VCB framework, or an agent on the guest as it works presently.

    I see in your original post your quote," I listed a few storage systems and backup products that can allow you to work around this little problem, but I left some out. That’s what this post is about."

    Is that what this post is still about?

  • The reason I called it a commercial is that you obviously cut and pasted something from a brochure or your website.

    Please read the post again and read the posts to which it links. I explain why it is not possible to do what you’re saying you are doing. You may be able to back up Windows 2008 guests without putting an agent on them, but it is not possible to do it correctly. Given the state of VMware tools today, is only possible to do a crash consistent backup. And crash consistent backups aren’t good enough.