Backup Central

Hello kind rsnapshot list,

I am seeking advice on how to best address an issue with cp -al on a
large filesystem.

Background: I have a fairly large (~10TB) filesystem that I have started
backing up with rsnapshot. So far almost everything seems to be running
smoothly (using sync_first), until I attempt the first level of periodic
rotations (in my case, weekly). This spawns a cp -al process that uses
all 4GB of physical memory on the machine (and based on the performance,
seems to be going into swap, though not thrashing swap yet).

My first idea, which might be a bit naive, is to comment out cmd_cp from
rsnapshot.conf, to let rsnapshot use its builtin copy instead. From
reading the man page, I'm not losing anything by doing this--as far as I
know, this filesystem should have no device files, and if there are any
FIFOs or sockets I don't really care about them.

Is there any other benefit I lose from using the builtin copy instead of
GNU cp? In my one test so far, memory usage seems much more reasonable
using the builtin copy, so I'm very much hoping to stay with it if I can.
Is there a better way of working around memory usage of cp on such a
large filesystem?

--keith


--
kkeller < at > wombat.san-francisco.ca.us



------------------------------------------------------------------------------
RSA(R) Conference 2012
Save $700 by Nov 18
Register now
http://p.sf.net/sfu/rsa-sfdev2dev1
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

On Tue, Nov 08, 2011 at 03:34:10PM -0800, Keith Keller (kkeller < at > wombat.san-francisco.ca.us) wrote:

a cp -al process that uses all 4GB of physical memory on the machine

My first idea, which might be a bit naive, is to comment out cmd_cp from
rsnapshot.conf, to let rsnapshot use its builtin copy instead.

That should work with the caveats you've already noted,
but as an alternative, you might consider limiting cp
(or entire rsnapshot) memory usage with ulimit.
Gnu cp should adapt to whatever reasonable amount of
memory you give it; finding the size for optimal
performance may take some experimenting.

--
Tapani Tarvainen

------------------------------------------------------------------------------
RSA(R) Conference 2012
Save $700 by Nov 18
Register now
http://p.sf.net/sfu/rsa-sfdev2dev1
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

On 2011-11-09, Tapani Tarvainen <rsnapshot < at > tapanitarvainen.fi> wrote:

That should work with the caveats you've already noted,
but as an alternative, you might consider limiting cp
(or entire rsnapshot) memory usage with ulimit.
Gnu cp should adapt to whatever reasonable amount of
memory you give it; finding the size for optimal
performance may take some experimenting.

Great, I will give that a try when I get some time.

What would the main advantage be to sticking with cp given the lack of a
need to back up special files? Is it mostly just avoiding the extra
rsync step after the internal copy, or are there other factors I'm not
seeing? Mostly it has a bearing on how hard I try to make the time; if
there are other significant disadvantages to using the internal copy, I
may choose to make that time sooner rather than later.

Thanks,

--keith

--
kkeller < at > wombat.san-francisco.ca.us


------------------------------------------------------------------------------
RSA(R) Conference 2012
Save $700 by Nov 18
Register now
http://p.sf.net/sfu/rsa-sfdev2dev1
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

On 2011-11-10, Chris Dennis <cgdennis < at > btinternet.com> wrote:
On 11/08/2011 11:34 PM, Keith Keller wrote:
Hello kind rsnapshot list,

I am seeking advice on how to best address an issue with cp -al on a
large filesystem.

Or you could use rsnapshot with the link_dest option, which avoids the
copy phase entirely.

Ah, I knew I was forgetting to mention something!

I did mention that I am using sync_first, but I forgot to mention that I
tried (only one time) to use link_dest. It seems like in that
configuration, rsnapshot weekly (my smallest interval) wants to mv .sync
weekly.0, which means that the next rsnapshot sync would have to recopy
all 10TB of data, which I certainly want to avoid.

But perhaps I don't know all how hard_link influences rsnapshot
behavior. If link_dest=1, will rsnapshot sync use the interval
directory instead of the .sync directory? That step I didn't get to
test; I simply guessed that rsnapshot sync would try to recreate the
.sync directory.

Thanks,

--keith

--
kkeller < at > wombat.san-francisco.ca.us



------------------------------------------------------------------------------
RSA(R) Conference 2012
Save $700 by Nov 18
Register now
http://p.sf.net/sfu/rsa-sfdev2dev1
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

On Thu, Nov 10, 2011 at 03:51:14PM +0000, Chris Dennis wrote:
On 11/10/2011 03:35 PM, Keith Keller wrote:
On 2011-11-10, Chris Dennis<cgdennis < at > btinternet.com> wrote:
On 11/08/2011 11:34 PM, Keith Keller wrote:
Hello kind rsnapshot list,

I am seeking advice on how to best address an issue with cp -al on a
large filesystem.

Or you could use rsnapshot with the link_dest option, which avoids the
copy phase entirely.

Ah, I knew I was forgetting to mention something!

I did mention that I am using sync_first, but I forgot to mention that I
tried (only one time) to use link_dest. It seems like in that
configuration, rsnapshot weekly (my smallest interval) wants to mv .sync
weekly.0, which means that the next rsnapshot sync would have to recopy
all 10TB of data, which I certainly want to avoid.

With link_dest it is fine to start with an empty destination directory, and
use link_dest from a previous snapshot (in keith's case weekly.0) to handle
unchanged files.

But please be very careful about using sync_first and link_dest together
if your rsnapshot version is less than 1.3.1. See the rsnapshot FAQ
http://rsnapshot.org/faq.html for details.

--
___________________________________________________________________________
David Keegel <djk < at > cyber.com.au> Cyber IT Solutions Pty. Ltd.
http://www.cyber.com.au/~djk/ Linux & Unix Systems Administration


------------------------------------------------------------------------------
RSA(R) Conference 2012
Save $700 by Nov 18
Register now
http://p.sf.net/sfu/rsa-sfdev2dev1
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

On 2011-11-10, David Keegel <djk < at > cyber.com.au> wrote:
On 11/08/2011 11:34 PM, Keith Keller wrote:

I did mention that I am using sync_first, but I forgot to mention that I
tried (only one time) to use link_dest. It seems like in that
configuration, rsnapshot weekly (my smallest interval) wants to mv .sync
weekly.0, which means that the next rsnapshot sync would have to recopy
all 10TB of data, which I certainly want to avoid.

With link_dest it is fine to start with an empty destination directory, and
use link_dest from a previous snapshot (in keith's case weekly.0) to handle
unchanged files.

So, in my scenario, if I use sync_first and link_dest, and rsnapshot weekly
moves the .sync directory out of the way, will rsnapshot sync then try to
sync to weekly.0? Or will it make a new .sync directory and recopy
everything?

From reading more in rsnapshot, and in the rsync man page, it seems like
link-dest is almost like a double rsync: it copies from DEST if the file
has changed since LINKDESTDIR, otherwise does a hard link. Is that an
accurate statement? If so, then my above quoted statement is actually not
accurate; it'll still make a new .sync directory, but would actually copy
only the changes.

On the other side of things, from reading man rsync, if you use
--link-dest, rsync prefers to work in an empty directory. But the way I
currently have my snapshots, I take a sync daily, and do only a weekly
rotation, which (if I'm reading the docs correctly) might actually munge
the previous snapshots. So for now I will probably stick with using the
rsnapshot internal copy but not link_dest, and test the suggestion of
using ulimit along with GNU cp at a later date.

But please be very careful about using sync_first and link_dest together
if your rsnapshot version is less than 1.3.1. See the rsnapshot FAQ
http://rsnapshot.org/faq.html for details.

Yes, I am definitely using 1.3.1; I'd seen that FAQ entry and verified
my version before testing both parameters together.

Thanks to everyone for your suggestions!

--keith

--
kkeller < at > wombat.san-francisco.ca.us


------------------------------------------------------------------------------
RSA(R) Conference 2012
Save $700 by Nov 18
Register now
http://p.sf.net/sfu/rsa-sfdev2dev1
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

Hallo Keith Keller,

Am Donnerstag, 10. November 2011 23:03 schrieb Keith Keller:
......
On the other side of things, from reading man rsync, if you use
--link-dest, rsync prefers to work in an empty directory. But
the way I currently have my snapshots, I take a sync daily, and
do only a weekly rotation, which (if I'm reading the docs
correctly) might actually munge the previous snapshots. So for

IMHO you schould do a daily rotation after each sync, too.

--
Herzliche Grüße!
Rolf Muth
Meine Adressen duerfen nicht fuer Werbung verwendet werden!
PGP Public Key:
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xF8DC41935544C89A

------------------------------------------------------------------------------
RSA(R) Conference 2012
Save $700 by Nov 18
Register now
http://p.sf.net/sfu/rsa-sfdev2dev1
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

Hi Rolf,

On 2011-11-11, Rolf Muth <rolf.muth < at > web.de> wrote:

Am Donnerstag, 10. November 2011 23:03 schrieb Keith Keller:
......
On the other side of things, from reading man rsync, if you use=20
--link-dest, rsync prefers to work in an empty directory. But
the way I currently have my snapshots, I take a sync daily, and
do only a weekly rotation, which (if I'm reading the docs
correctly) might actually munge the previous snapshots. So for

IMHO you schould do a daily rotation after each sync, too.

Unfortunately, on such a large dataset the rotation takes at least eight
hours, and I would like to avoid so much disk writing (even if it's only
creating directories) if possible. I realize that this creates a
backup hole, but I'd rather have the latest sync on the snapshot server
and give up daily snapshots (plus doing a daily sync reduces the sync
time that precedes the weekly snapshot).

I haven't fully committed to this; I may end up changing my mind and
doing the daily snapshots anyway. I need to do more testing once all
the other issues on this machine are settled (I still haven't fully
tested weekly snapshots, for example).

--keith

--
kkeller < at > wombat.san-francisco.ca.us


------------------------------------------------------------------------------
RSA(R) Conference 2012
Save $700 by Nov 18
Register now
http://p.sf.net/sfu/rsa-sfdev2dev1
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

Hallo Keith Keller,

Am Freitag, 11. November 2011 18:12 schrieb Keith Keller:
Hi Rolf,

On 2011-11-11, Rolf Muth <rolf.muth < at > web.de> wrote:
Am Donnerstag, 10. November 2011 23:03 schrieb Keith Keller:
......
On the other side of things, from reading man rsync, if you
use=20 --link-dest, rsync prefers to work in an empty
directory. But the way I currently have my snapshots, I take
a sync daily, and do only a weekly rotation, which (if I'm
reading the docs correctly) might actually munge the previous
snapshots. So for

Normally not, with --delete all deleted files should be deleted and
changed files should be changed in the next backup.

You could test that with two files to be backuped:
The first one you delete after a sync, it should vanish in the next
sync.
The second you change after the first sync, and only the last
version should be in the next sync.

IMHO you schould do a daily rotation after each sync, too.

Unfortunately, on such a large dataset the rotation takes at
least eight hours, and I would like to avoid so much disk writing
(even if it's only creating directories) if possible. I realize
that this creates a backup hole, but I'd rather have the latest
sync on the snapshot server and give up daily snapshots (plus
doing a daily sync reduces the sync time that precedes the weekly
snapshot).

I haven't fully committed to this; I may end up changing my mind
and doing the daily snapshots anyway. I need to do more testing
once all the other issues on this machine are settled (I still
haven't fully tested weekly snapshots, for example).

--keith

--
Herzliche Grüße!
Rolf Muth
Meine Adressen duerfen nicht fuer Werbung verwendet werden!
PGP Public Key:
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xF8DC41935544C89A

------------------------------------------------------------------------------
RSA(R) Conference 2012
Save $700 by Nov 18
Register now
http://p.sf.net/sfu/rsa-sfdev2dev1
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss