Login Form






Lost Password?
No account yet? Register (FREE)

Search Backup Central

Curtis
Who is This Guy?

Disclaimer

The opinions contained within this website, it's blog(s), forums, and Wikis, are those of the original poster and do not represent the position of my (or any other) employer. This blog is not owned by my employer nor does it officially represent any company.
Forget NDMP & use Synthetic Full backups PDF Print E-mail
Written by W. Curtis Preston   
Tuesday, 20 March 2007
NDMP is a great way to get faster backups for your filers by connecting tape drives directly to them.  But combining Synthetic Full Backups and NFS/CIFS access to your filers might be a lot better.

NDMP (the Network Data Management Protocol) is designed to allow you to properly back up your filers.  The main feature of NDMP is that it allows you to directly connect your tape drives to your filer and back them up without having to send your backup data across the network.  If you're using NDMP to back up across the LAN (AKA three-way NDMP) you're missing the main benefit of NDMP; backing up via NDMP isn't usually any faster than backing up via NFS or CIFS if you're backing up across the LAN.

NDMP also allows the filer vendor to resolve the ACL/permissions issues surrounding the fact that filers support multiple operating systems.  Without NDMP, you need to back up CIFS data via CIFS and NFS data via NFS.  NDMP allows the filer vendor to write a single backup format that can back up both types of data.

NDMP also has limitations, though.  The main limitation being that it does not support cross-filer recovery.  You cannot back up a NetApp and restore it to a BlueArc filer -- even though both support NDMP.  This is because NDMP is a communication protocol only; it does not specify the backup format.  Each filer vendor therefore creates their own format, making NDMP restores between filer vendors not possible.

NDMP backups (like any other traditional backup method) also place a load on the filer and can take a really long time. Finally, your backup product will probably charge you to use NDMP.

Synthetic backups, if your backup product supports them, are a great way to get a good backup of your filer without these limitations. Synthetic backups use the most recent full and incremental backups to create another full backup without having to transfer any data from the client; all the work is done on the backup server.  Some products can also create synthetic differentials (AKA cumulative incremental or level 1) backups that contain all files that have changed since the last full backup.

First, the caveats. 

  1. Your product must support this feature.  NetBackup, NetWorker, & CommVault support this feature. TSM's backup sets are synthetic fulls, but they are designed to be used outside of TSM and can't be used for regular TSM restores.
  2. You will have to backup your data via the correct protocol.  Unix-style data can be backed up via NFS, and Windows-style data can be backed up via CIFS. 
  3. NDMP users will only see a benefit for typical user data.  This won't help much when backing up database data.  The idea is that you only have to perform an incremental backup; if that backup is the equivalent of a full backup, this idea won't help.

So, here's the idea.

  1. Use NFS or CIFS to mount your data to your backup server.  (Please don't mount it to a client.  That sends the data across the network twice - from the filer to the client and from the client to the server.)
  2. Use your backup server to create a full backup of that network drive.
  3. Time passes.
  4. Perform an incremental backup of the filer via the same NFS/CIFS path.
  5. Time passes.
  6. Perform another incremental backup of the filer via the same NFS/CIFS path.
  7. Time passes.
  8. When you think you need another full backup, perform a synthetic full backup.  It will merge the latest full backup and the incrementals to create a new full backup that looks just like a "real" full backup.  It's just that the data will all move directly from one tape to another, without having to bother the filer. 

This way, you get all the benefits of occasional full backups (mainly faster restores) without having to actually make them.

Comments? 

Comments
Add NewSearchRSS
hga   | Registered | 2007-03-21 15:12:31
One is making a bet that your vendor will properly construct a synthetic full backup. Bugs in that might be a bit subtle to detect, and it probably would be a good idea to keep the real full backup and incrementals until you make another real full backup.

Otherwise, if time to restore from a total failure is important, it sounds like a great idea.

- Harold
cpreston   | Super Administrator | 2007-03-21 15:27:57
Any backup can have bugs. Are you saying that you'd prefer the known bugs and hiccups of dump to unknown (possible) bugs in synthetic full?

Have you ever read or heard what Linux Torvalds is saying these days about dump? (http://lwn.net/2001/0503/a/lt-dump.php3 Here's a few excerpts:

"Dump was a stupid program in the first place. Leave it behind."

"Right now, the cpio/tar/xxx solutions are definitely the best ones ... Whatever
problems they have, they are still better than the _guaranteed_(*) data
corruptions of dump."

(Dump is what NetApp and a few other NAS vendors use underneath NDMP, BTW.)
hga   | Registered | 2007-03-22 08:41:52
No, not at all. I have no love for dump, and I don't think I've used it since 1987 or so (4.2 or .3 BSD on MicroVAX IIs).

As for the comments by Linus, which I think you mention in your newest book, I'm not at all impressed: in essence what he's saying is that it was decided not to carry dump along with Linux by the time of the 2.4 kernel (perhaps a good decision, but "cpio/tar", while good archivers, are no substitute for the real thing), and that no replacement was even envisioned as of 2001, and from your book, none has been made.

For someone who worked with systems like MULTICS, history is repeating itself as a farce, where we are developing systems that are pale shadows of what we once did....

Anyway, in all cases I'm assuming the same utility was used to collect the original backups, and that the synthetic full is made to look like it was made with the same utility.

I'm a programmer who sometimes wears a sysadmin hat, and who knows how easy it is to create bugs, and how few organizations really test the software they ship. Hopefully commercial backup vendors do an above average job, but you would know that much better than I.

Me, I'm with Jacques Clouseau in I think the Return of the Pink Panther: "I suspect everyone, and I suspect no one."

- Harold
cpreston   | Super Administrator | 2007-03-22 11:00:05
You're exactly right that the same utility /format will be used to create the synthetic backup. However, if you're going to be doing synthetic fulls, you're not going to use NDMP; hence, you're not going to use dump.

So here are your two choices:
* dump with regular occasional "real" fulls
* NFS/CIFS -> server -> some commercial utility, followed by occasional synthetic fulls

And I'm saying that the latter has the following advantages:
* It doesn't use dump (which has had problems since it was designed)
* It allows you to perform fulls whenever you want, even during the day
* It places no load on the filer or LAN when you're doing synthetic fulls

So I'm saying give it a shot!
cpreston - Just throwing out an idea   | Super Administrator | 2007-04-09 17:21:49
Just threw out an idea to get us talking. Thanks for the feedback.

I agree with pretty much everything you said. (I'm not sure if you can use All with NDMP either, depending on your backup software, but I get your point.)

I've backed up some of the biggest NAS environments out there and have done some pretty "kooky" things that others said wouldn't scale, and they did, so I have a little more faith than the average joe about what will work in a large environment. You might be surprised as to how well this will scale.

Summary: Just think about my idea. If it works for you, then cool!
cpreston - VTLs?   | Super Administrator | 2007-05-01 11:24:32
This actually came from ag100. Had a problem with his post..

Hey - It goes without saying, but I think this really depends on individual environments and requirements... From my experience, larger environments with a mix of NFS, CIFS and multi-protocol shares, this has a hard time scaling. It's also added complexity and removed some key benefits seen using NDMP (not that I'm completely thrilled with NDMP, either).

Here's why:

* You generally (perhaps not in all cases) have to specify save set names manually, instead of using an "All", when backing up remote file systems. If you have a lot of file systems (or a growing environment), this increases the possibility of someone forgetting to add a file system and increases the inherant the risks associated with human interaction (typos, etc...). Now that some backup software no longer requires you to specify each file system individually, (assuming your volumes are sized reasonably, etc...) using "All" can be a nice feature, which also reduces risk.

Along the same lines, if your filer has a lot of save sets, you can run into limitations in your backup software... For example, I believe NetWorker only supports ~900 bytes in a save set listing and 1024 bytes per client profile. If you run over that, you're stuck managing multiple client profiles per client, or using scripts to issues save commands manually.


* Another item that must be considered, depending on requirements, is the network. Local NDMP backups, which takes advtantage of disk (via VTL) or modern tape technology perform can perform relatively well. If you had a requirement to backup file systems with a high data change rate (pst's, databases, etc...), you might find that performing even daily incremental backups can put considerable strain on backup resources.

* If you have both NFS and CIFS shares, you may have to have UNIX and Windows tape servers available. That can be painful, especially if you don't have both in your environment now.

* Lastly, we've been receiving an increasing number of requests for multi-protocol shares. As you mentioned, due to permissions requirements, this is a known limitation of network based backups NAS backups.

I'm not bashing the idea, and for some environments, it's a perfect fit, it just hasn't scaled well for me in the past.

Thoughts?

-Aaron
yaron   | Registered | 2007-05-04 16:05:42
Hi,

From my exprience, LAN based NDMP is 2-3 times faster than NFS based backup. In my setup, the Networker server and the library are located at a DR room, over a kilometer away, so pulling fibers just for connecting drives to the filer is not an option.

There are a few problems with your argument about dump. Dump handles correctly the issue of not restoring files which were DELETED. tar/cpio cannot do that. Linus, when speaking of dump's data corruption, is probably refering to dumps of a mounted file system. This is something you shouldn't do in the first place. As for NDMP usage of dump, you might recall that when running dump of a NetApp, the filer takes a snapshot and backs it up. This should guarantee that the dump will not be corrupted.
Only registered users can write comments!

Copyright (C) 2007 Alain Georgette / Copyright (C) 2006 Frantisek Hliva. All rights reserved.
 
< Prev   Next >
[ Back ]

Sponsored Links