Backup Central

On Sat, Dec 10, 2011 at 7:45 PM, member horvath
<member < at > thehorvaths.co.uk> wrote:
I've have considered the archive function however I wasn't aware that
the changes would be rsync'd.
I thought it would create a tar archive of the most recent backup then
xfer that to the archive host.
Am I wrong in thinking this?
I also need to ensure data integrity by check suming the remote copy
with the onsite.

The archive function just creates the tar snapshot of the hosts you
specify. You can point that wherever you like, but making it local is
faster. Once its complete your script should then transfer it, however
and wherever you like.

Rsync just happens to be what most people would use for this, and that
protocol includes a checksum verification.

If you use a different transport mechanism, then you many need to
include a checksum routine in your script.

Regarding your scheduling requirements, the best procedure is to get
to know the BackupPC way and work within that, rather than imposing
rules that probably came previous systems. Note that within the BPC
data store, there actually is no physical difference between
incremental and full backup sets - the only difference is the
algorithm use to determine whether or not files have been updated, and
that depends on the transport protocol.

You can configure things so that you have for example " at least this
many " and be conservative.

Also note the archive tar snapshots will not have the space savings
inherent in BPC's data storage deduplication, so it's likely using
this method will result in your offsite storage requirements being
many times larger than the BPC server onsite. Not necessarily a
problem as diskspace is cheap these days, but something to plan for. .
.

------------------------------------------------------------------------------
Learn Windows Azure Live! Tuesday, Dec 13, 2011
Microsoft is holding a special Learn Windows Azure training event for
developers. It will provide a great way to learn Windows Azure and what it
provides. You can attend the event by watching it streamed LIVE online.
Learn more at http://p.sf.net/sfu/ms-windowsazure
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Thanks very much for the info.
my backups are many terabytes in size so making local copies over and
above the onsite backup is not practical.
To remind you I need a 30day/6month onsite and only the most recent offsite.
Once the initial offsite is performed (This will be very large) The
ongoing "incremental s" will still be several GBs.
I need the archive option in backuppc to rsync the most recent copy
offsite on a daily basis.

On 10 December 2011 23:54, <hansbkk < at > gmail.com> wrote:
On Sat, Dec 10, 2011 at 7:45 PM, member horvath
<member < at > thehorvaths.co.uk> wrote:
I've have considered the archive function however I wasn't aware that
the changes would be rsync'd.
I thought it would create a tar archive of the most recent backup then
xfer that to the archive host.
Am I wrong in thinking this?
I also need to ensure data integrity by check suming the remote copy
with the onsite.

The archive function just creates the tar snapshot of the hosts you
specify. You can point that wherever you like, but making it local is
faster. Once its complete your script should then transfer it, however
and wherever you like.

Rsync just happens to be what most people would use for this, and that
protocol includes a checksum verification.

If you use a different transport mechanism, then you many need to
include a checksum routine in your script.

Regarding your scheduling requirements, the best procedure is to get
to know the BackupPC way and work within that, rather than imposing
rules that probably came previous systems. Note that within the BPC
data store, there actually is no physical difference between
incremental and full backup sets - the only difference is the
algorithm use to determine whether or not files have been updated, and
that depends on the transport protocol.

You can configure things so that you have for example " at least this
many " and be conservative.

Also note the archive tar snapshots will not have the space savings
inherent in BPC's data storage deduplication, so it's likely using
this method will result in your offsite storage requirements being
many times larger than the BPC server onsite. Not necessarily a
problem as diskspace is cheap these days, but something to plan for. .
.

------------------------------------------------------------------------------
Learn Windows Azure Live!  Tuesday, Dec 13, 2011
Microsoft is holding a special Learn Windows Azure training event for
developers. It will provide a great way to learn Windows Azure and what it
provides. You can attend the event by watching it streamed LIVE online.
Learn more at http://p.sf.net/sfu/ms-windowsazure
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

------------------------------------------------------------------------------
Learn Windows Azure Live! Tuesday, Dec 13, 2011
Microsoft is holding a special Learn Windows Azure training event for
developers. It will provide a great way to learn Windows Azure and what it
provides. You can attend the event by watching it streamed LIVE online.
Learn more at http://p.sf.net/sfu/ms-windowsazure
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

On Sun, Dec 11, 2011 at 1:03 PM, member horvath
<member < at > thehorvaths.co.uk> wrote:
Thanks very much for the info.
my backups are many terabytes in size so making local copies over and
above the onsite backup is not practical.
To remind you I need a 30day/6month onsite and only the most recent offsite.
Once the initial offsite is performed (This will be very large) The
ongoing "incremental s" will still be several GBs.
I need the archive option in backuppc to rsync the most recent copy
offsite on a daily basis.

The safest/easiest approach is to simply run a 2nd instance of
backuppc from an offsite location, perhaps through a vpn. This can be
practical even for large systems if you make the initial fulls with
the server on-site, then move it, using rsync for the transfers. That
way you don't have a single point of failure as you do if you make one
backup and copy it. And unless you have a lot of duplicate files on
your systems, the amount of data you transfer daily will be about the
same anyway.

Some people have reported success with rsync'ing their whole archive
directory to a remote system, but it is generally not practical due to
the time it takes to traverse/recreate the huge number of hardlinks.
You can image-copy the whole fileystem, but even that will take too
much time to do daily and it has to be done with the filesystem
unmounted - or at least not changing.

Next best might be to extract a tar image of the latest run as soon as
it completes, split into reasonable sized chunks in a directory, then
rsync that to your offsite copy. However I wouldn't expect rsync to
be very efficient in that scenario.

--
Les Mikesell
lesmikesell < at > gmail.com

------------------------------------------------------------------------------
Learn Windows Azure Live! Tuesday, Dec 13, 2011
Microsoft is holding a special Learn Windows Azure training event for
developers. It will provide a great way to learn Windows Azure and what it
provides. You can attend the event by watching it streamed LIVE online.
Learn more at http://p.sf.net/sfu/ms-windowsazure
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

On Mon, Dec 12, 2011 at 2:03 AM, member horvath
<member < at > thehorvaths.co.uk> wrote:
Thanks very much for the info.
my backups are many terabytes in size so making local copies over and above the onsite backup is not practical.
To remind you I need a 30day/6month onsite and only the most recent offsite.
Once the initial offsite is performed (This will be very large) The ongoing "incremental s" will still be several GBs.
I need the archive option in backuppc to rsync the most recent copy offsite on a daily basis.

I think you need to research and understand a bit more about how BPC
works in order to plan your strategy effectively.

The "archive option" in backuppc is the same things as "local copies
over and above the onsite backup", it was designed to allow people to
take full snapshots off-site, usually using tape media. In your case
the resulting huge monolithic full-set files are impractical to send
over the WAN.

The only way to get incremental functionality over the wire using
BackupPC is to run another BPC server at the remote location on a
separate schedule, and rsync is the way to go if your target hosts
allow.

Les is da man, follow his advice!

Another option is to physically rotate your backup filesystems off-site

- Never underestimate the bandwidth of a station wagon full of tapes
hurtling down the highway. — Andrew Tanenbaum

------------------------------------------------------------------------------
Learn Windows Azure Live! Tuesday, Dec 13, 2011
Microsoft is holding a special Learn Windows Azure training event for
developers. It will provide a great way to learn Windows Azure and what it
provides. You can attend the event by watching it streamed LIVE online.
Learn more at http://p.sf.net/sfu/ms-windowsazure
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/