Backup Central

I have a dilemma and im not sure what the best thing
to do is. I am currently attempting to run rdiff on
several servers. All the servers i am backing up has
a large /home partition of about 100-400 gigs. I
have had success running rdiff backing up partitions
of 30 gigs. But I keep running into some issue with
the larger mount points. Rdiff seems to fail and
since its crond if the mount point backup fails the
night before, the backups continuously fail. Now
since the backups fail on the /home partition, I was
thinking if its more sensible to perhaps
ssh hostname ls /home and pipe that into a file so I
run rdiff on all the directories seperately. I assume
this is better to do in terms of getting backups of
all the directories vs, losing the entire /home
partition.

In regards to the failures of backups. If the backups
fail, is there any way to recover, so that crond jobs
succeed every time, vs me having to monitor if the
backups succeed everyday? If the backups fail, does
it error out in bash to 1, so that it knows that the
backup failed? I wanted to script it so that if it
errors out, to remove the rdiff-data-backup directory
from that directory, and retry the backup. Any one
out there run into these types of issues?

Also I have another question, If it errors out and I
remove the rdiff-backup-directory, I think I would
have to use the --force option to write into the same
directory again correct? If i do this, does rdiff
essentially run incrementals against the destination
directory? Or does it overwrite everything again?





__________________________________
Do you Yahoo!?
Yahoo! Photos: High-quality 4x6 digital prints for 25¢
http://photos.yahoo.com/ph/print_splash

On 19 Apr 2004, Robert Yoon <robertyoon < at > yahoo.com> wrote:

I have a dilemma and im not sure what the best thing
to do is. I am currently attempting to run rdiff on
several servers. All the servers i am backing up has
a large /home partition of about 100-400 gigs. I
have had success running rdiff backing up partitions
of 30 gigs. But I keep running into some issue with
the larger mount points. Rdiff seems to fail and
since its crond if the mount point backup fails the
night before, the backups continuously fail.

What kind of failures?

Now
since the backups fail on the /home partition, I was
thinking if its more sensible to perhaps
ssh hostname ls /home and pipe that into a file so I
run rdiff on all the directories seperately. I assume
this is better to do in terms of getting backups of
all the directories vs, losing the entire /home
partition.

That would make each individual backup directory smaller, which might
give you a better chance of them running to completion.


In regards to the failures of backups. If the backups
fail, is there any way to recover, so that crond jobs
succeed every time, vs me having to monitor if the
backups succeed everyday? If the backups fail, does
it error out in bash to 1, so that it knows that the
backup failed? I wanted to script it so that if it
errors out, to remove the rdiff-data-backup directory
from that directory, and retry the backup. Any one
out there run into these types of issues?

Don't you want --check-destination-dir?


--
Martin

If you're running it as cron, you can make root your email address in
aliases and you'll get the cron output. It's not a fix, but that way
you'll know when it fails.
-jeff

Martin Pool wrote:

On 19 Apr 2004, Robert Yoon <robertyoon < at > yahoo.com> wrote:



I have a dilemma and im not sure what the best thing
to do is. I am currently attempting to run rdiff on
several servers. All the servers i am backing up has
a large /home partition of about 100-400 gigs. I
have had success running rdiff backing up partitions
of 30 gigs. But I keep running into some issue with
the larger mount points. Rdiff seems to fail and
since its crond if the mount point backup fails the
night before, the backups continuously fail.



What kind of failures?



Now
since the backups fail on the /home partition, I was
thinking if its more sensible to perhaps
ssh hostname ls /home and pipe that into a file so I
run rdiff on all the directories seperately. I assume
this is better to do in terms of getting backups of
all the directories vs, losing the entire /home
partition.



That would make each individual backup directory smaller, which might
give you a better chance of them running to completion.




In regards to the failures of backups. If the backups
fail, is there any way to recover, so that crond jobs
succeed every time, vs me having to monitor if the
backups succeed everyday? If the backups fail, does
it error out in bash to 1, so that it knows that the
backup failed? I wanted to script it so that if it
errors out, to remove the rdiff-data-backup directory
from that directory, and retry the backup. Any one
out there run into these types of issues?



Don't you want --check-destination-dir?




------------------------------------------------------------------------

_______________________________________________
rdiff-backup-users mailing list at rdiff-backup-users < at > nongnu.org
http://mail.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

On Mon, Apr 19, 2004 at 05:14:11PM -0700, Robert Yoon wrote:
| I have a dilemma and im not sure what the best thing
| to do is. I am currently attempting to run rdiff on
| several servers. All the servers i am backing up has
| a large /home partition of about 100-400 gigs. I
| have had success running rdiff backing up partitions
| of 30 gigs. But I keep running into some issue with
| the larger mount points.

Have you tried cranking the shell resource limits for the
rdiff-backup process, especially the data size and maximum
number of file descriptors?


I wrote a trivial rdiff-backup front-end that does this for
me. It's at:
http://www.mewburn.net/luke/src/do-rdiff-backup

I've attached the config file I'm using.
It uses /z/backup/ as the backup partition, and I've currently
disabled automatically clean out old incrementals.


Cheers,
Luke.

On Mon, 19 Apr 2004 17:14:11 -0700 (PDT)
Robert Yoon <robertyoon < at > yahoo.com> wrote:

I have a dilemma and im not sure what the best thing
to do is. I am currently attempting to run rdiff on
several servers. All the servers i am backing up has
a large /home partition of about 100-400 gigs. I
have had success running rdiff backing up partitions
of 30 gigs. But I keep running into some issue with
the larger mount points. Rdiff seems to fail

What kind of failure? There is no reason why rdiff-backup should not be
able to handle backups of this size. Rather than investigate ways around
the failure, is it possible to investigate the failure itself?

If the backups fail, does
it error out in bash to 1, so that it knows that the
backup failed?

Don't you have enough failures to establish this for yourself? [] You
could easily force a failure with something like:

rdiff-backup /etc host::/this/path/does/not/exist

However, the answer is that rdiff-backup has a non-zero exit status if
the backup fails.

I wanted to script it so that if it
errors out, to remove the rdiff-data-backup directory
from that directory, and retry the backup. Any one
out there run into these types of issues?

If you remove the rdiff-backup directory then you will lose all the
increments ever stored, which is possibly not what you want. If you
really don't want increments at all, I'd suggest you use rsync rather
than rdiff-backup.

I'd seriously suggest that we start with trying to fix the original
problem.

Keith

Well I guess the main trouble I am running into is the
initial disk syncing. As long as it has a valid
mirror marker to validate against it should be ok.
But that in itself seems to be real big issue I am
having. So I believe i will just run ssh hostname ls
/home and pipe that into a file and run rdiff-backup
against that list. The only real issue with that is
what if /home/dir1 no longer exists, whenever I run
an rdiff-backup --remove-older-than 2w /home/dir1
does that remove all the files within that directory
(assuming all it contents are older than 2 weeks old?)





__________________________________
Do you Yahoo!?
Yahoo! Photos: High-quality 4x6 digital prints for 25¢
http://photos.yahoo.com/ph/print_splash