SearchFAQMemberlist Log in
Reply to topic Page 1 of 1
Disk backup strategy advice / help
Author Message
Post Disk backup strategy advice / help 
I have Bacula 5.2.3 up and running and need some advice with the following

I used this tutorial
http://bacula.org/fr/dev-manual/Automated_Disk_Backup.html as a starting
point and it's working well and I have backups appearing as follows...

460M Dec 29 23:19 Full-0001
26.3G Dec 30 23:52 Full-0003

702M Dec 24 23:05 Inc-0001
10.0G Dec 30 01:54 Inc-0002
2.3G Dec 31 00:06 Inc-0004
3.1G Dec 31 00:56 Inc-0005
611M Dec 31 00:56 Inc-0006


Now that the backups seems to be working I need to figure out how to
implement an offsite strategy, I want to use a combination of removable
disks and rsync to do this.

I want to use removable disks to take a backup offsite either
weekly or forthnightly. (Just copy that most recent Full-XXXX file to
the removable disk)
I would like to rsync the daily INC & Weekly Diff backups offsite
if possible (100MB Link).

My plan is that for complete recovery I will use a combination of the
full backup that i will get from the Removable disks and the Rsync'd INC
/ Diff

If I add a new server to be backed up to Bacula midweek it does a full
backup in the INC pool. This might be a big backup and screw-up my Rsync
job.

Does this seem like a good idea and goes anyone know how keep Full
backups out of the INC or DIFF pool

Thanks
Keith







------------------------------------------------------------------------------
Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
infrastructure or vast IT resources to deliver seamless, secure access to
virtual desktops. With this all-in-one solution, easily deploy virtual
desktops for less than the cost of PCs and save 60% on VDI infrastructure
costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
_______________________________________________
Bacula-users mailing list
Bacula-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

View user's profile Send private message
Post Disk backup strategy advice / help 
On Wed, Jan 04, 2012 at 12:16:55PM +0000, keith wrote:
460M Dec 29 23:19 Full-0001
26.3G Dec 30 23:52 Full-0003
702M Dec 24 23:05 Inc-0001
10.0G Dec 30 01:54 Inc-0002
2.3G Dec 31 00:06 Inc-0004
3.1G Dec 31 00:56 Inc-0005
611M Dec 31 00:56 Inc-0006

Is the 10G on 24th mostly additional, changed or moved data? Are these
compressed backups?

Now that the backups seems to be working I need to figure out how to
implement an offsite strategy, I want to use a combination of removable
disks and rsync to do this.

I think bacula is not the ideal tool for running additional offsite
backups. And very likely rsync is not a good way if you use bacula.

I have 3 possibilities in mind:

1. If you are not talking about windows clients, I'd consider using rsync
(e.g. via rsnapshot) to run the complete offsite backup unrelated to
bacula. Run one rsync/rsnapshot job per client and the 'new' client will
just run longer, independent of the others except the shared bandwidth.
Via rsnapshot you only need to do 1 full backup per client, changed
files just lead to new full backupsets, but only the difference needs
to be transferred. We do that on several locations and wrote a wrapper
round rsnapshot (which is a wrapper round rsync), debian packages are
available at
deb http://ftp.lihas.de/debian stable main
package rsnapshot-backup.
If you add some file unification tool, you get away with far less used
diskspace.
+ only changes need to be transferred
+ initial backup can easily be transferred on external media to save bandwidth
- no bacula, no bacula indexes
- no backup of windows clients / anything that doesn't have rsync

2. Alternatively you can use the normal bacula backup + a copy job.
As copy jobs only work on same bacula-sd, you could e.g. NFS-mount some
external server and store the target pools there. The copy full pool is
to local disks on individual mountpoints. Move the volumes to the remote
location and replace it with links to remote NFS.
+ works with all clients
- regularily transporting volumes offsite is required

3. Run a complete seperate job instance to the remote site using a
bacula-sd installed there. Use virtual full backups to create the fulls
from the full/diff/inc backups. Initially a full backup has to pass the
remote connection.
+ works with all clients
0 initial full might be expensive in bandwidth

Currently I use 1. and 2. myself. With 3. I ran into trouble selecting
the correct pools in my environment and virtual full in general
including a tape changer with a single drive.

If I add a new server to be backed up to Bacula midweek it does a full
backup in the INC pool. This might be a big backup and screw-up my Rsync
job.
Does this seem like a good idea and goes anyone know how keep Full
backups out of the INC or DIFF pool

Just do a manual initial full backup on the new client. As I assume they
don't appear magically in your backup setup.

Regards,
Adrian
--
LiHAS - Adrian Reyer - Hessenwiesenstraße 10 - D-70565 Stuttgart
Fon: +49 (7 11) 78 28 50 90 - Fax: +49 (7 11) 78 28 50 91
Mail: lihas < at > lihas.de - Web: http://lihas.de
Linux, Netzwerke, Consulting & Support - USt-ID: DE 227 816 626 Stuttgart

------------------------------------------------------------------------------
Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
infrastructure or vast IT resources to deliver seamless, secure access to
virtual desktops. With this all-in-one solution, easily deploy virtual
desktops for less than the cost of PCs and save 60% on VDI infrastructure
costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
_______________________________________________
Bacula-users mailing list
Bacula-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Post Disk backup strategy advice / help 
On 04/01/2012 21:52, Adrian Reyer wrote:
On Wed, Jan 04, 2012 at 12:16:55PM +0000, keith wrote:
460M Dec 29 23:19 Full-0001
26.3G Dec 30 23:52 Full-0003
702M Dec 24 23:05 Inc-0001
10.0G Dec 30 01:54 Inc-0002
2.3G Dec 31 00:06 Inc-0004
3.1G Dec 31 00:56 Inc-0005
611M Dec 31 00:56 Inc-0006
Is the 10G on 24th mostly additional, changed or moved data? Are these
compressed backups?
Hi Adrian, The 10G was just a new additional client being introduced
into the Bacula.

Now that the backups seems to be working I need to figure out how to
implement an offsite strategy, I want to use a combination of removable
disks and rsync to do this.
I think bacula is not the ideal tool for running additional offsite
backups. And very likely rsync is not a good way if you use bacula.
Oh ok....

I have 3 possibilities in mind:

1. If you are not talking about windows clients, I'd consider using rsync
(e.g. via rsnapshot) to run the complete offsite backup unrelated to
bacula. Run one rsync/rsnapshot job per client and the 'new' client will
just run longer, independent of the others except the shared bandwidth.
Via rsnapshot you only need to do 1 full backup per client, changed
files just lead to new full backupsets, but only the difference needs
to be transferred. We do that on several locations and wrote a wrapper
round rsnapshot (which is a wrapper round rsync), debian packages are
available at
deb http://ftp.lihas.de/debian stable main
package rsnapshot-backup.
If you add some file unification tool, you get away with far less used
diskspace.
+ only changes need to be transferred
+ initial backup can easily be transferred on external media to save bandwidth
- no bacula, no bacula indexes
- no backup of windows clients / anything that doesn't have rsync
We have some Unix servers but the bulk of our servers are Windows.
Our old/current backup strategy is to do full backups nightly and these
are about 450G Compressed.

If I could I would like to do some type of copy jobs where I copy the
Incremental files to another place on the server and then get rsync to
down these files knowing that they were just the incrementals.


2. Alternatively you can use the normal bacula backup + a copy job.
As copy jobs only work on same bacula-sd, you could e.g. NFS-mount some
external server and store the target pools there. The copy full pool is
to local disks on individual mountpoints. Move the volumes to the remote
location and replace it with links to remote NFS.
+ works with all clients
- regularily transporting volumes offsite is required

Only just read about "Migration / Copy" jobs last night (I am slowly
getting through the Bacula manual) and will probably try to get one of
these jobs working later today. I plan to have one dedicated bacula-fd
server and have also planned to put the removable disks (Offsite Backup)
into this server. If I can get Bacula manage my offsite disks and to
also know whats on these disk will be great.


3. Run a complete seperate job instance to the remote site using a
bacula-sd installed there. Use virtual full backups to create the fulls
from the full/diff/inc backups. Initially a full backup has to pass the
remote connection.
+ works with all clients
0 initial full might be expensive in bandwidth

Currently I use 1. and 2. myself. With 3. I ran into trouble selecting
the correct pools in my environment and virtual full in general
including a tape changer with a single drive.
There are network / firewall issue outwith my control that would make remote Bacula backups an issue.


If I add a new server to be backed up to Bacula midweek it does a full
backup in the INC pool. This might be a big backup and screw-up my Rsync
job.
Does this seem like a good idea and goes anyone know how keep Full
backups out of the INC or DIFF pool
Just do a manual initial full backup on the new client. As I assume they
don't appear magically in your backup setup.

Regards,
Adrian

Your right, they shouldn't just be appearing but they are while I play
with Bacula :>) But in the future when adding new clients in then it
makes sense to manually kick off a full backup.


Adrian, thanks for the detailed answers. I give the copy jobs a try and
see how I get on.

Cheers
Keith.



------------------------------------------------------------------------------
Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
infrastructure or vast IT resources to deliver seamless, secure access to
virtual desktops. With this all-in-one solution, easily deploy virtual
desktops for less than the cost of PCs and save 60% on VDI infrastructure
costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
_______________________________________________
Bacula-users mailing list
Bacula-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

View user's profile Send private message
Post Disk backup strategy advice / help 
Hello,

I have managed my offsite backup setup so that only bacula volumes get
rsync-ed and it works fine for half a year. Total backup size is about
500Gb, nightly amount of data rsync-ed is between 1 and 10Gb (so my home
adsl connection with 10Mbit/s downstream is OK to keep offsite backups
at home).
This is in an assumption that you need offsite backup only when you've
totally lost onsite one. In this case bacula server has to be
restored/reinstalled first (or better in advance, to be able to test
restoration) and database has to be restored before using offsite backup.
Additionally I encrypt the bacula volumes on the fly while remotely
rsync-ing them (on the fly by means of fuse encfs).

Regards,

Alex Ehrlich


------------------------------------------------------------------------------
Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
infrastructure or vast IT resources to deliver seamless, secure access to
virtual desktops. With this all-in-one solution, easily deploy virtual
desktops for less than the cost of PCs and save 60% on VDI infrastructure
costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
_______________________________________________
Bacula-users mailing list
Bacula-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Post Disk backup strategy advice / help 
On Wed, Jan 4, 2012 at 22:52, Adrian Reyer <bacula-lists < at > lihas.de> wrote:

Hi Adrian

I think bacula is not the ideal tool for running additional offsite
backups. And very likely rsync is not a good way if you use bacula.

I rsync data, catalog and bsr files on external disks and I would know
what it's not a goold solution.



--
Sebastien Douche <sdouche < at > gmail.com>
Twitter: < at > sdouche / G+: +sdouche

------------------------------------------------------------------------------
RSA(R) Conference 2012
Mar 27 - Feb 2
Save $400 by Jan. 27
Register now!
http://p.sf.net/sfu/rsa-sfdev2dev2
_______________________________________________
Bacula-users mailing list
Bacula-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Post Disk backup strategy advice / help 
Hi Sebastian,

On Mon, Jan 16, 2012 at 11:38:50AM +0100, Sebastien Douche wrote:
I think bacula is not the ideal tool for running additional offsite
backups. And very likely rsync is not a good way if you use bacula.
I rsync data, catalog and bsr files on external disks and I would know
what it's not a goold solution.

Well, as you copy the matching catalog as well it should just be fine.
It depends all on what you want to be your backup be for. In my case I
want to be save from loosing single/some backup media while the backup
server is still fine as I run redundant servers anyway. You obviously
plan for a complete backup system breakdown on the expense of a harder
time while restoring single lost media. On the other hand, if I
experience a complete backup server outage I have to bscan the offsite
tapes.
Benefits of both ways you get by e.g.
- copyjob for media offsite, if some medium fail, the copy job gets
active if you delete it.
- sql-server replication offsite, alternatively dump&restore
- having all configuration files ready offsite. As I run
'Linux-VServers' I would just rsync the backup-server itself.

Regards,
Adrian
--
LiHAS - Adrian Reyer - Hessenwiesenstraße 10 - D-70565 Stuttgart
Fon: +49 (7 11) 78 28 50 90 - Fax: +49 (7 11) 78 28 50 91
Mail: lihas < at > lihas.de - Web: http://lihas.de
Linux, Netzwerke, Consulting & Support - USt-ID: DE 227 816 626 Stuttgart

------------------------------------------------------------------------------
RSA(R) Conference 2012
Mar 27 - Feb 2
Save $400 by Jan. 27
Register now!
http://p.sf.net/sfu/rsa-sfdev2dev2
_______________________________________________
Bacula-users mailing list
Bacula-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Post Disk backup strategy advice / help 
On Mon, Jan 16, 2012 at 19:15, Adrian Reyer <bacula-lists < at > lihas.de> wrote:
Well, as you copy the matching catalog as well it should just be fine.
It depends all on what you want to be your backup be for. In my case I
want to be save from loosing single/some backup media while the backup
server is still fine as I run redundant servers anyway. You obviously
plan for a complete backup system breakdown on the expense of a harder
time while restoring single lost media. On the other hand, if I
experience a complete backup server outage I have to bscan the offsite
tapes.
Benefits of both ways you get by e.g.
- copyjob for media offsite, if some medium fail, the copy job gets
 active if you delete it.
- sql-server replication offsite, alternatively dump&restore
- having all configuration files ready offsite. As I run
 'Linux-VServers' I would just rsync the backup-server itself.

Thank you Adrian for your explanation.

--
Sebastien Douche <sdouche < at > gmail.com>
Twitter: < at > sdouche / G+: +sdouche

------------------------------------------------------------------------------
Keep Your Developer Skills Current with LearnDevNow!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-d2d
_______________________________________________
Bacula-users mailing list
Bacula-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Display posts from previous:
Reply to topic Page 1 of 1
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
  


Magic SEO URL for phpBB