Backup Central

The rsnapshot HOWTO recommends that the snapshots should be automated using cron. On my system, like most home systems, the natural tool for this is actually anacron.

In order for rsnapshot to run properly, it appears that the various kinds of backup need to be invoked in reverse order of frequency. From the HOWTO:
It is usually a good idea to schedule the larger intervals to run a bit before the lower ones. For example,
in the crontab above, notice that daily runs 30 minutes before hourly. This helps prevent race conditions
where the daily would try to run before the hourly job had finished.
However my anacrontab, as it comes from the box, runs jobs with shorter intervals first. From my anacrontab:
#period in days delay in minutes job-identifier command
1 5 cron.daily nice run-parts /etc/cron.daily
7 25 cron.weekly nice run-parts /etc/cron.weekly
< at > monthly 45 cron.monthly nice run-parts /etc/cron.monthly
Anacron is invoked with the "-s" argument, so that these jobs run sequentially. A little inspection of the logs shows that, in fact, the jobs are run in this order: daily, weekly, monthly.

Some questions:

• Will anacron run properly if jobs are run in order of frequency? I think it wouldn't.
• Is there some way (arguments, etc) to make it run properly?
• Is there an harm that would come from reversing the order of commands in anacrontab?

Also, is there any easy way to search the mailing list archive? There doesn't seem to be a search box on the archive page.

Thanks - jon

2011/8/2 Jonathan Ryshpan <jonrysh < at > pacbell.net>:
However my anacrontab, as it comes from the box, runs jobs with shorter
intervals first.  From my anacrontab:

#period in days   delay in minutes   job-identifier   command
1       5       cron.daily              nice run-parts /etc/cron.daily
7       25      cron.weekly             nice run-parts /etc/cron.weekly
< at > monthly 45     cron.monthly            nice run-parts /etc/cron.monthly

I have this:
START_HOURS_RANGE=10-22
< at > daily 50 backup.daily df -k /backup; nice rsnapshot
-v -c /etc/rsnapshot.dream.conf daily; df -k /backup
< at > weekly 30 backup.weekly df -k /backup; nice rsnapshot
-v -c /etc/rsnapshot.dream.conf weekly; df -k /backup
< at > monthly 15 backup.monthly df -k /backup; nice rsnapshot
-v -c /etc/rsnapshot.dream.conf monthly; df -k /backup

Best
Martin

------------------------------------------------------------------------------
BlackBerry&reg; DevCon Americas, Oct. 18-20, San Francisco, CA
The must-attend event for mobile developers. Connect with experts.
Get tools for creating Super Apps. See the latest technologies.
Sessions, hands-on labs, demos & much more. Register early & save!
http://p.sf.net/sfu/rim-blackberry-1
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

On Tue, 2011-08-02 at 19:21 +0200, Martin Schröder wrote:
2011/8/2 Jonathan Ryshpan <jonrysh < at > pacbell.net ([email]jonrysh < at > pacbell.net[/email])>:
However my anacrontab, as it comes from the box, runs jobs with shorter
intervals first.  From my anacrontab:

#period in days   delay in minutes   job-identifier   command
1       5       cron.daily              nice run-parts /etc/cron.daily
7       25      cron.weekly             nice run-parts /etc/cron.weekly
< at > monthly 45     cron.monthly            nice run-parts /etc/cron.monthly



I have this:
START_HOURS_RANGE=10-22
< at > daily   50     backup.daily            df -k /backup; nice rsnapshot -v -c /etc/rsnapshot.dream.conf daily; df -k /backup
< at > weekly  30     backup.weekly           df -k /backup; nice rsnapshot -v -c /etc/rsnapshot.dream.conf weekly; df -k /backup
< at > monthly 15     backup.monthly          df -k /backup; nice rsnapshot -v -c /etc/rsnapshot.dream.conf monthly; df -k /backup
Are you sure that this does what you think it does? I suspect this happens if rotations are done in the order shown (assuming 4 monthly backups):
daily.6->trash
daily.5->daily.6
daily.4->daily.5
daily.3->daily.4
daily.2->daily.3
daily.1->daily.2
daily.0->daily.1

weekly.3->trash
weekly.2->weekly.3
weekly.1->weekly.2
weekly.0->weekly.1
daily.6->weekly.0

monthly.3->trash
monthly.2->monthly.3
monthly.1->monthly.2
monthly.0->monthly.1
As a result, daily.6 and weekly.3 are missing. If the rotation order is monthly, weekly, daily this doesn't happen.

Or maybe I'm missing something. This is all from the documentation; I haven't studied the code.

jon

2011/8/3 Jonathan Ryshpan <jonrysh < at > pacbell.net>:
On Tue, 2011-08-02 at 19:21 +0200, Martin Schröder wrote:
I have this:
START_HOURS_RANGE=10-22
< at > daily   50     backup.daily            df -k /backup; nice rsnapshot -v -c
/etc/rsnapshot.dream.conf daily; df -k /backup
< at > weekly  30     backup.weekly           df -k /backup; nice rsnapshot -v -c
/etc/rsnapshot.dream.conf weekly; df -k /backup
< at > monthly 15     backup.monthly          df -k /backup; nice rsnapshot -v -c
/etc/rsnapshot.dream.conf monthly; df -k /backup

Are you sure that this does what you think it does?  I suspect this happens
if rotations are done in the order shown (assuming 4 monthly backups):

daily.6->trash
daily.5->daily.6
daily.4->daily.5
daily.3->daily.4
daily.2->daily.3
daily.1->daily.2
daily.0->daily.1

weekly.3->trash
weekly.2->weekly.3
weekly.1->weekly.2
weekly.0->weekly.1
daily.6->weekly.0

monthly.3->trash
monthly.2->monthly.3
monthly.1->monthly.2
monthly.0->monthly.1

As a result, daily.6 and weekly.3 are missing.  If the rotation order is
monthly, weekly, daily this doesn't happen.

Or maybe I'm missing something.  This is all from the documentation; I
haven't studied the code.

I have also

# the maximal random delay added to the base delay of the jobs
RANDOM_DELAY=45

Don't get fooled by the order of the entries; the second column
(minutes) is important. Here monthly is started at start+random+15,
weekly at start+random+30, daily at start+random+45. So far this has
worked very well.

Best
Martin

------------------------------------------------------------------------------
BlackBerry&reg; DevCon Americas, Oct. 18-20, San Francisco, CA
The must-attend event for mobile developers. Connect with experts.
Get tools for creating Super Apps. See the latest technologies.
Sessions, hands-on labs, demos & much more. Register early & save!
http://p.sf.net/sfu/rim-blackberry-1
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

On Tue, Aug 02, 2011 at 09:58:16AM -0700, Jonathan Ryshpan (jonrysh < at > pacbell.net) wrote:

In order for rsnapshot to run properly, it appears that the various
kinds of backup need to be invoked in reverse order of frequency. From
the HOWTO:

It is usually a good idea to schedule the larger intervals to
run a bit before the lower ones. For example,
in the crontab above, notice that daily runs 30 minutes before
hourly. This helps prevent race conditions
where the daily would try to run before the hourly job had
finished.

One possibility is using a wrapper script along these lines:


# save month, day of month and day of week
eval $(date '+MONTH=%M MDAY=%d WDAY=%a')
# yearly on Jan 1 only
case $MONTH-$MDAY in 01-01) rsnapshot yearly ;; esac
# monthly on 1st of each month
case $MDAY in 01) rsnapshot monthly ;; esac
# weekly on Sundays
case $WDAY in Sun) rsnapshot daily ;; esac
# sync & daily every day
rsnapshot sync && rsnapshot daily


That keeps them in right order regardless of how long they take.


--
Tapani Tarvainen

------------------------------------------------------------------------------
BlackBerry&reg; DevCon Americas, Oct. 18-20, San Francisco, CA
The must-attend event for mobile developers. Connect with experts.
Get tools for creating Super Apps. See the latest technologies.
Sessions, hands-on labs, demos & much more. Register early & save!
http://p.sf.net/sfu/rim-blackberry-1
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

On Wed, 2011-08-03 at 08:34 +0200, Martin Schröder wrote:
2011/8/3 Jonathan Ryshpan <jonrysh < at > pacbell.net ([email]jonrysh < at > pacbell.net[/email])>:
On Tue, 2011-08-02 at 19:21 +0200, Martin Schröder wrote:
I have this:
START_HOURS_RANGE=10-22
< at > daily   50     backup.daily            df -k /backup; nice rsnapshot -v -c
/etc/rsnapshot.dream.conf daily; df -k /backup
< at > weekly  30     backup.weekly           df -k /backup; nice rsnapshot -v -c
/etc/rsnapshot.dream.conf weekly; df -k /backup
< at > monthly 15     backup.monthly          df -k /backup; nice rsnapshot -v -c
/etc/rsnapshot.dream.conf monthly; df -k /backup

Are you sure that this does what you think it does?  I suspect this happens
if rotations are done in the order shown (assuming 4 monthly backups):


I have also

# the maximal random delay added to the base delay of the jobs
RANDOM_DELAY=45

Don't get fooled by the order of the entries; the second column
(minutes) is important. Here monthly is started at start+random+15,
weekly at start+random+30, daily at start+random+45. So far this has
worked very well.
It appears that your system invokes anacron without the -s option, whereas mine (Fedora-15) invokes it with this option. As a result (if I understand right) on my system anacron pays no attention to the delays of 15, 30, and 45 minutes in deciding the order in which to execute the commands. It's always: top command, next command, next command, and so on. I like the -s option (as it appears the packages of Fedora do also), since it totally eliminates the possibility of race conditions.

Thanks very much for your help - jon

On Wed, 2011-08-03 at 10:08 +0300, Tapani Tarvainen wrote:
On Tue, Aug 02, 2011 at 09:58:16AM -0700, Jonathan Ryshpan (jonrysh < at > pacbell.net ([email]jonrysh < at > pacbell.net[/email])) wrote:

In order for rsnapshot to run properly, it appears that the various
kinds of backup need to be invoked in reverse order of frequency. From
the HOWTO:

It is usually a good idea to schedule the larger intervals to
run a bit before the lower ones. For example,
in the crontab above, notice that daily runs 30 minutes before
hourly. This helps prevent race conditions
where the daily would try to run before the hourly job had
finished.

One possibility is using a wrapper script along these lines:


# save month, day of month and day of week
eval $(date '+MONTH=%M MDAY=%d WDAY=%a')
# yearly on Jan 1 only
case $MONTH-$MDAY in 01-01) rsnapshot yearly ;; esac
# monthly on 1st of each month
case $MDAY in 01) rsnapshot monthly ;; esac
# weekly on Sundays
case $WDAY in Sun) rsnapshot daily ;; esac
# sync & daily every day
rsnapshot sync && rsnapshot daily


That keeps them in right order regardless of how long they take.
What an elegant script!

I had one like it, though less pretty, in an older version of rsnapshot, modified from Rubel's original shell scripts. Unfortunately, it has a flaw: If the computer isn't run on January 1st, the yearly backup isn't done. Similar problems with monthly and weekly backups.

So ... unless someone can give me a reason why not, I'm going to change anacrontab to run the jobs in the order
cron.monthly
cron.weekly
cron.daily
Thanks for your help - jon

Am Dienstag 2 August 11 schrieb Martin Schröder:
...
I have this:
START_HOURS_RANGE=10-22
< at > daily 50 backup.daily df -k /backup; nice rsnapshot
-v -c /etc/rsnapshot.dream.conf daily; df -k /backup
< at > weekly 30 backup.weekly df -k /backup; nice rsnapshot
-v -c /etc/rsnapshot.dream.conf weekly; df -k /backup
< at > monthly 15 backup.monthly df -k /backup; nice rsnapshot
-v -c /etc/rsnapshot.dream.conf monthly; df -k /backup

Or you could use my do-rsnapshots in my posting:

Am Dienstag 7 Juni 11 schrieb Rolf Muth:
Re: [rsnapshot-discuss] Patch to run all intervals in one go

That runs it all automatically.

--
Herzliche Grüße!
Rolf Muth

Meine Adressen duerfen nicht fuer Werbung verwendet werden! PGP Public Key:
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xF8DC41935544C89A
Analoge Uhr (clock): http://www.heise.de/software/download/analoge_uhr/61872

------------------------------------------------------------------------------
BlackBerry&reg; DevCon Americas, Oct. 18-20, San Francisco, CA
The must-attend event for mobile developers. Connect with experts.
Get tools for creating Super Apps. See the latest technologies.
Sessions, hands-on labs, demos & much more. Register early & save!
http://p.sf.net/sfu/rim-blackberry-1
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss