Welcome! » Log In » Create A New Profile

NDMP Backup of NetApp (CIFS) with Very Large Number of Files

Posted by joe1871 
NDMP Backup of NetApp (CIFS) with Very Large Number of Files
June 27, 2014 01:10PM
Hi,

I use Netbackup 7.5 to back up a Netapp filer. I have certain volumes on that filer that have grown exceptionally large - approaching a terabyte per vol, and one volume holds more than 5 million files, a second has > 2.0 million. These are CIFS volumes, with NFS shares as well. We back them up over fiber using NDMP. This has been getting progressively slower, no surprise.

NDMP already does take a snapshot on the filer and uses that to back up the volume. That is perfect. However my backup of the larger of these two volumes is taking 26 hours plus, and is growing. Do I have any resource in the short term? Is there anything I can do to improve the time this is taking?

Thanks in advance. I did look this question up, but I did not see any answers where the user was employing NDMP, hence my question.

JB
NDMP Backup of NetApp (CIFS) with Very Large Number of Files
June 30, 2014 06:52AM
JB,

We backup filers with NDMP. We have several volumes over 1 TB, including 1 volume that is about 8 TB. We run 2 backup streams per filer, and basically give each volume its own backup stream. We are mostly successful, but the 8 TB volume takes a bit over 3 days to backup. One failure and you miss that week's backup - no time to retry. But, NDMP backups are just like file system backups performance-wise, millions of tiny files kills performance.

The only things that can help performance is fast enough tape drives and filer head cpu. Most filer heads I have dealt with cannot push much more than 150 MB/s via NDMP. (We don't have the highest performance heads, so take that with a grain of salt.) We are using LTO4 tape drives, so running 2 streams at 70-80 MB's does not tax the tape drives at all.

The last time we did a head upgrade, we definitely saw an uptick in performance. But, NDMP has its limitations.

BTW, if you are using something like NetApp SIS are any kind of "dedupe" on the filer, the filer has to re-hydrate the data as it sends it out via NDMP. I'm sure that doesn't help performance, but I've never stopped to measure it. But that is also something you really need to be aware of if you have to restore!

Bryan

-----Original Message-----
From: veritas-bu-bounces < at > mailman.eng.auburn.edu [mailto] On Behalf Of joe1871
Sent: Friday, June 27, 2014 3:11 PM
To: VERITAS-BU < at > MAILMAN.ENG.AUBURN.EDU
Subject: [Veritas-bu] NDMP Backup of NetApp (CIFS) with Very Large Number of Files

Hi,

I use Netbackup 7.5 to back up a Netapp filer. I have certain volumes on that filer that have grown exceptionally large - approaching a terabyte per vol, and one volume holds more than 5 million files, a second has > 2.0 million. These are CIFS volumes, with NFS shares as well. We back them up over fiber using NDMP. This has been getting progressively slower, no surprise.

NDMP already does take a snapshot on the filer and uses that to back up the volume. That is perfect. However my backup of the larger of these two volumes is taking 26 hours plus, and is growing. Do I have any resource in the short term? Is there anything I can do to improve the time this is taking?

Thanks in advance. I did look this question up, but I did not see any answers where the user was employing NDMP, hence my question.

JB

+----------------------------------------------------------------------
|This was sent by joeb < at > toolwire.com via Backup Central.
|Forward SPAM to abuse < at > backupcentral.com.
+----------------------------------------------------------------------

_______________________________________________
Veritas-bu maillist - Veritas-bu < at > mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu

DTCC DISCLAIMER: This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error, please notify us immediately and delete the email and any attachments from your system. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email.

_______________________________________________
Veritas-bu maillist - Veritas-bu < at > mailman.eng.auburn.edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Sorry, only registered users may post in this forum.

Click here to login