Backup Central

I am currently tasked with locating and correcting bottlenecks in our backup environment. After quite a few hours spent on google, I really cannot come up with good procedures for doing so.

What are some techniques you have for identifying bottlenecks?

LAN infrastructure is 100mb/s (blah!), we have 6 LTO-3 tape drives, and use Netbackup.

The normal rates we are seeing are between 10mb/s to 30mb/s, does this sound right?

On our SAN we see between 40mb/s to 60mb/s, what are you seeing?

Doing a large, 'normal' copy/ftp from a test system to the backup server - while running iostat/vmstat (or similar) on both ends - should give you a good start on performance testing and determining where the problems may be.

[Amusingly, a great test for us is the Sun EBS (Networker) 7.4.4-Sparc patch file - its 500 meg.]


Those LAN backup numbers are correct for a 100-Full network environment (from personal experience).

The 40-60 sounds good for SAN; You did not mention what FC speed you've negotiated at between server and drive, but I would assume 1 Gb from my own experience with LTO-2. I think LTO-3 can do at least 2 Gb, so you should strive for that if possible.

In order for your LAN numbers to match the SAN ones, at minimum your backup server(s) need bumping to Gigabit network (our own backup server was the first to make the switch, too). The 'client' systems do not need this, but you'll want to keep multiple hosts streaming to each tape drive in order to compensate.


Once the network is cleared, run your tests again and see if something else shows up. I can't get very specific about hardware, since our backup environment is Sparc-based and we do things a bit odd here.

--TSK

I guess it would be good to mention this is an all windows network

Not sure about a windows equiv to iostat

Also, one huge thing would be to have a way to monitor tape drive statistics. To tell how much data/sec is being transferred and if "shoe shining" is occuring.

There is an add for perfmon for this on the tapedrives that netbackup provides, but i swear perfmon interupts the drives during it's queries. With perfmon running, my job will go down to 1-5mb/s. Without it running I'll jump back to 15-20mb/s...

Read up on the NetBackup Tuning Guide posted in the support site.
Just pick Netbackup, 6.5, and search using the word "tuning."

But you most definitely need to use the NBU buffer tweaks, upgrade to Gb LAN links, and use more streams if possible.


A good test as mentioned was FTP, the Windows CLI FTP client is handy as when it's done it shows the avg speed per sec, and total byte count.

Or you can use Perfmon...