Backup Central

Hi,

I would like to use bacula for backup, but the observed perfomance is
not good enough for our use case.
The backup-server running Director, SD and FD (version 1.38.11 on=20
Debian unstable) is an AMD Athlon(tm) XP 1500+ with 1 GB of ram.
It reads the files via Gbit-ethernet from an NFS-share.
The storage device is an autochanger-library with an HP Ultrium-3=20
tape drive, which is attached to the server via an Adaptec=20
AIC-7892A U160/m-SCSI-card.

The problem is, that Bacula in a test run gives a rate of upto 18MB/s=20
with a single large file served from NFS. Backing up approx. 7,5=20
million files with a total size of 920GB brings down the rate to
approx. 6MB/s (md5 signatures and no spooling).
I can hardly estimate if this is good or bad.

Well, I know, that a rate of 6 MB/s is to slow by a factor of 3 to 4=20
for our goal of backing up 1-1.5TB in less than 20hours.

I'm sure, that I should upgrade the hardware to get more performance,
but before doing so, I would like to hear from others,=20
what hardware they use for backup, how their setup is and which=20
backup rates they achieve.

Bye,
Sven=0A=0A=0A=0A"Jetzt Handykosten senken mit klarmobil - 14 Ct./Min.! Hier=
klicken"=0Ahttp://www.klarmobil.de/index.html?pid=3D73025=0A

On 3/20/07, amo001 < at > ju... <amo001 < at > ju...> wrote:
Hi,

I would like to use bacula for backup, but the observed perfomance is
not good enough for our use case.
The backup-server running Director, SD and FD (version 1.38.11 on
Debian unstable) is an AMD Athlon(tm) XP 1500+ with 1 GB of ram.
It reads the files via Gbit-ethernet from an NFS-share.
The storage device is an autochanger-library with an HP Ultrium-3
tape drive, which is attached to the server via an Adaptec
AIC-7892A U160/m-SCSI-card.

The problem is, that Bacula in a test run gives a rate of upto 18MB/s
with a single large file served from NFS. Backing up approx. 7,5
million files with a total size of 920GB brings down the rate to
approx. 6MB/s (md5 signatures and no spooling).
I can hardly estimate if this is good or bad.

Well, I know, that a rate of 6 MB/s is to slow by a factor of 3 to 4
for our goal of backing up 1-1.5TB in less than 20hours.

I'm sure, that I should upgrade the hardware to get more performance,
but before doing so, I would like to hear from others,
what hardware they use for backup, how their setup is and which
backup rates they achieve.


We have a dual processor opteron 246 with 4 GB of memory serving as
the director (main file server with 3TB of raid 6) and a second dual
processor opteron 246 with 4GB of memory that is the bacula-sd
connected to a 2 drive LTO2 autochanger. Our postgresql database is on
a third box Athlon64 3200 with 2GB of memory. We use spooling on all
jovs that do not originate from the director. Our backup rates vary
depending of the type and the source but 18MB/s is typical for a full
backup from the director that gets a compression rate of around 1.5:1.
All the servers and nearly all the clinents are gigabit. The bacula-sd
and teh data base servers also have multiple roles as the bacula-sd
has over 1TB of raid 5 and runs 4 virtual machines and the data base
server also runs an ldap server.

John

Hi,

On 3/20/2007 3:54 PM, amo001 < at > ju... wrote:
Hi,

I would like to use bacula for backup, but the observed perfomance is
not good enough for our use case.
The backup-server running Director, SD and FD (version 1.38.11 on
Debian unstable) is an AMD Athlon(tm) XP 1500+ with 1 GB of ram.
It reads the files via Gbit-ethernet from an NFS-share.
The storage device is an autochanger-library with an HP Ultrium-3
tape drive, which is attached to the server via an Adaptec
AIC-7892A U160/m-SCSI-card.

I'd do some tests: Measure the network throughput, disk throughput (in
case you want to use spooling, which I recommend) and tape throughput.

I suspect that, given your hardware, you will find that neither the
network nor the disk is fast enough to allow continous streaming to tape.

The problem is, that Bacula in a test run gives a rate of upto 18MB/s
with a single large file served from NFS. Backing up approx. 7,5
million files with a total size of 920GB brings down the rate to
approx. 6MB/s (md5 signatures and no spooling).

Measuring NFS throughput with your actual fileset will probably result
in a rather unimpressive result... it might be better to have an FD
running on your NFS server, if possible. If you've got some sort of
small NAS system that will not be possible, and these things are usually
not known for their exceptional performance.

I can hardly estimate if this is good or bad.

For backup purposes, that's not good. Depending on your hardware, it
might be the best you can achieve.

If you're really stuck with that hardware, I'd strongly suggest to use
spooling through an extra disk, preferrably a fast one on a dedicated
controller. PCI throughput will limit what you can achieve anyway.

Well, I know, that a rate of 6 MB/s is to slow by a factor of 3 to 4
for our goal of backing up 1-1.5TB in less than 20hours.

I'm sure, that I should upgrade the hardware to get more performance,
but before doing so, I would like to hear from others,
what hardware they use for backup, how their setup is and which
backup rates they achieve.

You don't want to know about my office / testing setup but, to
saturate an LTO-3 drive, you'll need really good bus throughput (PCI-X
or PCIe) and a fast spooling disk. I'd recommend a fast stripe set as a
spooling disk.

Dual Xeon or Opteron systems are a good starting point today, because
when using a local catalog database your database doesn't block the CPU
for Bacula, at least.

Arno


Bye,
Sven



"Jetzt Handykosten senken mit klarmobil - 14 Ct./Min.! Hier klicken"
http://www.klarmobil.de/index.html?pid=73025


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Bacula-users mailing list
Bacula-users < at > li...
https://lists.sourceforge.net/lists/listinfo/bacula-users

--
IT-Service Lehmann al < at > it...
Arno Lehmann http://www.its-lehmann.de

Hi Sven,
I wouldn't nfs mount anything on my backup server. I would just backup
the NFS share on the server-side, i.e. run bacula-fd on the NFS server
and add the NFS shared dir to the NFS server's FileSet.

Btw, last night my clients got the following Rates in MB/s:
4,7,14, 21,10
These rates vary due to different network connectivity, i.e. WAN vs LAN,
quality of uplinks between switches, backplane bandwidth of switches...

Fyi, my backup server is:
Dell PE 2650 (1 x 73 GB RAID 1, 1 x 450 GB RAID 0 , 4 GB RAM, 2 GB swap)
Bacula 2.0.1 (400 GB of Data-Spooling configured, Current Pools=Weekly,
Monthly, Scratch, Migrate, Archive)
RHEL 4 AS
Quantum PX502 (LTO-3, FC, 1-Drive)

Mike

amo001 < at > ju... wrote:
Hi,

I would like to use bacula for backup, but the observed perfomance is
not good enough for our use case.
The backup-server running Director, SD and FD (version 1.38.11 on
Debian unstable) is an AMD Athlon(tm) XP 1500+ with 1 GB of ram.
It reads the files via Gbit-ethernet from an NFS-share.
The storage device is an autochanger-library with an HP Ultrium-3
tape drive, which is attached to the server via an Adaptec
AIC-7892A U160/m-SCSI-card.

The problem is, that Bacula in a test run gives a rate of upto 18MB/s
with a single large file served from NFS. Backing up approx. 7,5
million files with a total size of 920GB brings down the rate to
approx. 6MB/s (md5 signatures and no spooling).
I can hardly estimate if this is good or bad.

Well, I know, that a rate of 6 MB/s is to slow by a factor of 3 to 4
for our goal of backing up 1-1.5TB in less than 20hours.

I'm sure, that I should upgrade the hardware to get more performance,
but before doing so, I would like to hear from others,
what hardware they use for backup, how their setup is and which
backup rates they achieve.


Bye,
Sven



"Jetzt Handykosten senken mit klarmobil - 14 Ct./Min.! Hier klicken"
http://www.klarmobil.de/index.html?pid=73025


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Bacula-users mailing list
Bacula-users < at > li...
https://lists.sourceforge.net/lists/listinfo/bacula-users

amo001 < at > ju... wrote:
Hi,

I would like to use bacula for backup, but the observed perfomance is
not good enough for our use case.
The backup-server running Director, SD and FD (version 1.38.11 on
Debian unstable) is an AMD Athlon(tm) XP 1500+ with 1 GB of ram.
It reads the files via Gbit-ethernet from an NFS-share.
The storage device is an autochanger-library with an HP Ultrium-3
tape drive, which is attached to the server via an Adaptec
AIC-7892A U160/m-SCSI-card.

The problem is, that Bacula in a test run gives a rate of upto 18MB/s
with a single large file served from NFS. Backing up approx. 7,5
million files with a total size of 920GB brings down the rate to
approx. 6MB/s (md5 signatures and no spooling).
I can hardly estimate if this is good or bad.

Well, I know, that a rate of 6 MB/s is to slow by a factor of 3 to 4
for our goal of backing up 1-1.5TB in less than 20hours.

I'm sure, that I should upgrade the hardware to get more performance,
but before doing so, I would like to hear from others,
what hardware they use for backup, how their setup is and which
backup rates they achieve.

So, you have all this stuff on another server (doesn't appear to be
described) and is being mounted via NFS which is then backed up.

Just for kicks, try simply using tar to glom that share onto tape and
see what rate you get.

I'm just wondering if perhaps it might be NFS performance that is
killing your rate. But you generally want to isolate components for
comparison. This would cut Bacula out of the loop and thus eliminate it
as a cause for the slow rates.

Someone else will undoubtedly have some other ideas.

Just a few comments about NFS. It has a reputation for being touchy. It
can be set up as over UDP or over TCP. If it is over UDP, even just a
few errors can cause it's performance to become abysmal. You could look
at your error rates. If that is a problem, and you can't put a backup
client on your NFS server (say, because it is a NAS), then you might
actually get better performance by using a samba connection, simply
because samba will be over TCP. I'd still be inclined to set it up with
a fast holding disk, because if you fail to drive your tape at a fast
enough clip, the tape drive itself may become a speed problem, slowing
things down even more.

---------------

Chris Hoogendyk

-
O__ ---- Systems Administrator
c/ /'_ --- Biology & Geology Departments
(*) \(*) -- 140 Morrill Science Center
~~~~~~~~~~ - University of Massachusetts, Amherst

<hoogendyk < at > bi...>

---------------

Erdös 4

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

amo001 < at > ju... wrote:
Hi,

I would like to use bacula for backup, but the observed perfomance is
not good enough for our use case.
The backup-server running Director, SD and FD (version 1.38.11 on
Debian unstable) is an AMD Athlon(tm) XP 1500+ with 1 GB of ram.
It reads the files via Gbit-ethernet from an NFS-share.

You should probably include the NFS version. I would assume you are
running V3, but it is possible you are using either an older or newer
version, which I'm pretty sure will make a difference in terms of speed
and other factors.

That said, I wouldn't back up over NFS unless I absolutely had to (and I
have not yet, and have a fairly diverse setup). I believe there may be
some information about this in the manual, but I couldn't swear to it.

- --
---- _ _ _ _ ___ _ _ _
|Y#| | | |\/| | \ |\ | | |Ryan Novosielski - Systems Programmer III
|$&| |__| | | |__/ | \| _| |novosirj < at > um... - 973/972.0922 (2-0922)
\__/ Univ. of Med. and Dent.|IST/AST - NJMS Medical Science Bldg - C630
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGABbVmb+gadEcsb4RAvNaAKCFS1+gcTCl0u5EAaShbwuZ1cADKQCguf7s
PVFc+kMbZoDgZnsVcPXblSA=
=HQHi
-----END PGP SIGNATURE-----