I have been trying to improve my backup system by using the new (5.0) feature of Maximum concurrent Jobs on storage devices, and would appreciate some hints on how to get good performance with my configuration. I'm using bacula 5.0.3 on CentOS 5. The concurrent configuration works beautifully, it is just unexpectedly slow (for those who came across this post by Google, some hints are below; I had to guess on some configurations since I didn't find all the details for this feature in the documentation).
My problem is that the performance is poor. For testing, I did a full backup of two Linux servers with around 2.5-3.5 GB of data each. Backed up sequentially, each backup took about 6-10 minutes. When backing them up concurrently, the same backup took well over 1 hour!
I am backing up to file volumes. The file volumes are located on an NFS server connected with Gigabit Ethernet. The NFS server is a FreeNAS 7 box, which is usually blazingly fast.
The two Linux boxes I am backing up are actually virtual machines on VMWare ESXi. Director, FD and SD are all on the same physical hardware.
So how would I go about tracking down what slows down the performance so dramatically?
And, unrelated to my question, here are the hints to make "maximum concurrent jobs on device" work:
You need to make changes to both the bacula-sd.conf and the bacula-dir.conf file, as follows:
In bacula-sd.conf:
- add more devices. They need to all be identical except for the name and the device itself. Each device should be configured with Maximum concurrent jobs=1. For instance, for a file-based device, I use this as the first device (for the second and third, I just changed Name and Archive Device):
Device {
Name = BACKUP1-dir1
Media Type = File
Device Type = File
Archive Device = /misc/BACKUP1/bacula1
LabelMedia = yes; # lets Bacula label unlabeled media
Random Access = Yes;
AutomaticMount = yes; # when device opened, read it
RemovableMedia = no;
AlwaysOpen = yes;
Maximum Concurrent Jobs = 1;
}
The storage resource should have
Maximum Concurrent Jobs=3
This should match the number of devices you use exactly.
What had tripped me up initially was that in bacula-dir.conf, you need to modify the storage resource, as well. It must list all three devices. Also, again set Maximum Concurrent Jobs to exactly the number of devices (if you set it to a higher value, your jobs will be blocked by the SD, or worse, by the device itself. You don't want that since it may cause timeouts if you have Run Before scripts).
Storage {
Name = Disk1
# Do not use "localhost" here
Address = baculasd.mydomain # N.B. Use a fully qualified name here
SDPort = 9103
Password = "XXXXXXX"
Device = BACKUP1-dir1
Device = BACKUP1-dir2
Device = BACKUP1-dir3
Media Type = File
Maximum Concurrent Jobs = 3
}
------------------------------------------------------------------------------
Learn Windows Azure Live! Tuesday, Dec 13, 2011
Microsoft is holding a special Learn Windows Azure training event for
developers. It will provide a great way to learn Windows Azure and what it
provides. You can attend the event by watching it streamed LIVE online.
Learn more at http://p.sf.net/sfu/ms-windowsazure
_______________________________________________
Bacula-users mailing list
Bacula-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
