Backup Central

Hi folks,

Any thoughts on how I might optimise the tree copy as noted in my email
below?

Thanks in advance.

--
Regards,

Miah


-----Original Message-----
From: Miah Gregory <mace < at > darksilence.net>
To: rsnapshot-discuss < at > lists.sourceforge.net
Subject: [rsnapshot-discuss] Synchronising/backing up an rsnapshot tree
Date: Thu, 21 Jul 2011 13:08:21 +0100
Mailer: Evolution 2.30.3

Hi folks,

I have been reading through the earlier thread entitled 'accelerating
"cp -al" under nfs', and although it's not quite our use case, it does
discuss the same issue which we are facing: speed.

In brief, our set up involves a backup machine running rsnapshot pulling
backups together from various other machines into one place. We have a
post-exec script which then rsyncs the whole rsnapshot backup tree to an
external, encrypted RDX cartridge.

The problem is that rsync of the whole tree is quite time consuming
(rsnapshot part of the backup takes ~ 1 hour in total, rsync to
encrypted media, ~ 7 hours).

My initial thoughts were to effectively replay the rsnapshot commands
(cp/mv/rm) to the RDX cartridge, followed by a final rsync of just
daily.0, which should then be somewhat faster.

However, short of parsing the logfile, there doesn't appear to be a way
to do this. All of the internal commands can be wrapped in scripts,
except there is no cmd_mv option, so going the wrapper route is out.

Another possibility might be to have the external cartridge managed by a
separate rsnapshot process which backs up from daily.0? Would this be
reliable and give good data consistency? If so, would this work well for
remote/off-site backups of the primary backup tree too?

Thanks in advance!



------------------------------------------------------------------------------
Get a FREE DOWNLOAD! and learn more about uberSVN rich system,
user administration capabilities and model configuration. Take
the hassle out of deploying and managing Subversion and the
tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

On Thu, Jul 21, 2011 at 5:08 AM, Miah Gregory <mace < at > darksilence.net> wrote:
Another possibility might be to have the external cartridge managed by a
separate rsnapshot process which backs up from daily.0? Would this be
reliable and give good data consistency? If so, would this work well for
remote/off-site backups of the primary backup tree too?

This is a pretty reasonable option, I think. You might have to
experiment a bit to get just the right settings (like maybe you'll
have to backup the directories within daily.0/, rather than daily.0/
itself).

My system involves rotating USB mirrors of my rsnapshot root offsite.
My mirror script is a hacked-together Perl script which does:

- Find the LVM2 volume for the mirror and mount it.
- Compare the rsnapshot root and mirror directory timestamps, rename
as appropriate.
- Walk the remainder oldest to newest using rsync to copy things over.
- Unmount and offline the LVM2 volume for the mirror.

The script is 200 lines of Perl, and while it's not truly nasty, it is
pretty specific to my install, and once it was working I never
bothered to clean it up - which is why I'm not going to post it here,
either . Just suffice to say that doing it this way is also pretty
reasonable. The reason I went with this rather than a second
rsnapshot was because I do my rotation once a month, I swap two
offsite disks, and I wanted them to be as identical as possible for
ease of manual verification (a second rsnapshot instance would result
in differing timestamps). Targeting a specific working install of
rsnapshot also simplifies things a bit (my script makes many
assumptions on where things are, and dies with an error message rather
than trying to fix anything).

-scott

------------------------------------------------------------------------------
Get a FREE DOWNLOAD! and learn more about uberSVN rich system,
user administration capabilities and model configuration. Take
the hassle out of deploying and managing Subversion and the
tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

On Sun, Aug 21, 2011 at 09:57:22AM +0100, Miah Gregory wrote:

Another possibility might be to have the external cartridge managed by a
separate rsnapshot process which backs up from daily.0? Would this be
reliable and give good data consistency?

Yes, it's what I recommend for maintaining multiple copies of backups.
Just make sure that the second process doesn't try to run at the same
time as the first one! You can do this by making them both use the same
lock file.

--
David Cantrell | http://www.cantrell.org.uk/david

You may now start misinterpreting what I just
wrote, and attacking that misinterpretation.

------------------------------------------------------------------------------
uberSVN's rich system and user administration capabilities and model
configuration take the hassle out of deploying and managing Subversion and
the tools developers use with it. Learn more about uberSVN and get a free
download at: http://p.sf.net/sfu/wandisco-dev2dev
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss