Deprecated: __autoload() is deprecated, use spl_autoload_register() instead in /home/pbuc/public_html/forum/mods/ext_phorummail/ezc/Base/src/ezc_bootstrap.php on line 36

Deprecated: Methods with the same name as their class will not be constructors in a future version of PHP; KeyCAPTCHA_CLASS has a deprecated constructor in /home/pbuc/public_html/forum/mods/keycaptcha/keycaptcha.php on line 108
Evaluating a backup repository
Welcome! » Log In » Create A New Profile

Evaluating a backup repository

Posted by Anonymous 
Evaluating a backup repository
October 22, 2014 03:23AM
hi,
I did a 'rsnapshot du' on my repository to validate it and have an
overview. Interesting... but I have a few questions:

1-I would have expected all these numbers in REVERSE (monthly.x being
the biggest one and the newest daily.0 just containing the incremental
part, the additions). Wrong thinking?

2-Because of a connection bug leaving daily{0,1,2,3,4,5} empty , I had
to recently cp -al daily.6 to daily{0,1,2,3,4,5} I am not sure I
understand why they take 769M each :/ WOuld have expected just a few Ks
for the hard link. Wrong thinking?

3-And my weekly.1 looks out of sync with the rest at 127G. Any clue how
to investigate this?!

Thanks!

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
My command and result
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
root < at > server:# rsnapshot -c
/root/scripts/backup/rsnapshot.wscpb_cpbackup.conf du
132G /media/backup02/wscpb/daily.0/
769M /media/backup02/wscpb/daily.1/
769M /media/backup02/wscpb/daily.2/
768M /media/backup02/wscpb/daily.3/
768M /media/backup02/wscpb/daily.4/
768M /media/backup02/wscpb/daily.5/
768M /media/backup02/wscpb/daily.6/
8,7G /media/backup02/wscpb/weekly.0/
127G /media/backup02/wscpb/weekly.1/
15G /media/backup02/wscpb/weekly.2/
11G /media/backup02/wscpb/weekly.3/
9,5G /media/backup02/wscpb/monthly.0/
306G total
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

------------------------------------------------------------------------------
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://p.sf.net/sfu/Zoho
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Evaluating a backup repository
October 22, 2014 06:49AM
Hallo, Thierry,

Du meintest am 21.10.14:

[quote]hi,
I did a 'rsnapshot du' on my repository to validate it and have an
overview. Interesting... but I have a few questions:
[/quote]
[quote]1-I would have expected all these numbers in REVERSE (monthly.x being
the biggest one and the newest daily.0 just containing the
incremental part, the additions). Wrong thinking?
[/quote]
Yes: wrong.
The examination starts with the newest backup (in my system: hourly.0,
in your system: daily.0) and then compares what has changed down to the
oldest backup.

[quote]2-Because of a connection bug leaving daily{0,1,2,3,4,5} empty , I
had to recently cp -al daily.6 to daily{0,1,2,3,4,5} I am not sure I
understand why they take 769M each :/ WOuld have expected just a few
Ks for the hard link. Wrong thinking?
[/quote]
Strange. But I won't worry ...

[quote]3-And my weekly.1 looks out of sync with the rest at 127G. Any clue
how to investigate this?!
[/quote]
I wouldn't investigate but instead "hard link" again the whole bundle of
backups, with the "hardlink" program from Dag Wieers in

http://arktur.shuttle.de/CD/beta/slack/ap1/hardlink-1.2-i486-1hln.tgz

My usual way:

cd /path/to/backups
hardlink yearly.? monthly.1? monthly.? weekly.? daily.?

That may need some hours; you should start this command just after a
regular "rsnapshot hourly" run. The above command "hard links" from
daily.0, daily.1 etc. over weekly.0 etc. "down" to yearly.0 etc.

Viele Gruesse!
Helmut

------------------------------------------------------------------------------
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://p.sf.net/sfu/Zoho
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Evaluating a backup repository
October 22, 2014 09:38AM
On Wed, Oct 22, 2014 at 7:30 AM, Helmut Hullen <hhullen < at > aktivanet.de ([email]hhullen < at > aktivanet.de[/email])> wrote:
[quote]> 1-I would have expected all these numbers in REVERSE (monthly.x being
[quote]the biggest one and the newest daily.0 just containing the
incremental part, the additions). Wrong thinking?
[/quote]
Yes: wrong.
The examination starts with the newest backup (in my system: hourly.0,
in your system: daily.0) and then compares what has changed down to the
oldest backup.
[/quote]

Also, it&#39;s incorrect to think of rsnapshot backups as a base set plus incrementals. All the backup sets are complete, they just happen to share disk space and avoid redundant copy operations. This is an important concept which differentiates rsnapshot from most traditional backup systems.

poc
Evaluating a backup repository
October 22, 2014 10:26AM
On Wed, Oct 22, 2014 at 2:30 AM, Helmut Hullen <hhullen < at > aktivanet.de> wrote:
[quote]Hallo, Thierry,

Du meintest am 21.10.14:

[quote]hi,
I did a 'rsnapshot du' on my repository to validate it and have an
overview. Interesting... but I have a few questions:
[/quote]
[quote]1-I would have expected all these numbers in REVERSE (monthly.x being
the biggest one and the newest daily.0 just containing the
incremental part, the additions). Wrong thinking?
[/quote]
Yes: wrong.
The examination starts with the newest backup (in my system: hourly.0,
in your system: daily.0) and then compares what has changed down to the
oldest backup.

[quote]2-Because of a connection bug leaving daily{0,1,2,3,4,5} empty , I
had to recently cp -al daily.6 to daily{0,1,2,3,4,5} I am not sure I
understand why they take 769M each :/ WOuld have expected just a few
Ks for the hard link. Wrong thinking?
[/quote]
Strange. But I won't worry ...

[quote]3-And my weekly.1 looks out of sync with the rest at 127G. Any clue
how to investigate this?!
[/quote]
I wouldn't investigate but instead "hard link" again the whole bundle of
backups, with the "hardlink" program from Dag Wieers in

http://arktur.shuttle.de/CD/beta/slack/ap1/hardlink-1.2-i486-1hln.tgz

My usual way:

cd /path/to/backups
hardlink yearly.? monthly.1? monthly.? weekly.? daily.?

That may need some hours; you should start this command just after a
regular "rsnapshot hourly" run. The above command "hard links" from
daily.0, daily.1 etc. over weekly.0 etc. "down" to yearly.0 etc.

Viele Gruesse!
Helmut

------------------------------------------------------------------------------
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://p.sf.net/sfu/Zoho
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
[/quote]
------------------------------------------------------------------------------
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://p.sf.net/sfu/Zoho
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Evaluating a backup repository
October 22, 2014 10:36AM
On Wed, Oct 22, 2014 at 2:30 AM, Helmut Hullen <hhullen < at > aktivanet.de> wrote:

[quote][quote]3-And my weekly.1 looks out of sync with the rest at 127G. Any clue
how to investigate this?!
[/quote]
I wouldn't investigate but instead "hard link" again the whole bundle of
backups, with the "hardlink" program from Dag Wieers in
[/quote]

According to the manual page, it was written by Jakub Jelinek at Red
Hat. Dag and the Reporforge repository he seems to have founded
repackage most of the software, they don't write much from scratch. I
met Dag about 5 years ago in London and bought him beer. Nice guy, I
don't think he'd want to accidentally take credit for software he
didn't write.

[quote]
http://arktur.shuttle.de/CD/beta/slack/ap1/hardlink-1.2-i486-1hln.tgz

My usual way:

cd /path/to/backups
hardlink yearly.? monthly.1? monthly.? weekly.? daily.?

That may need some hours; you should start this command just after a
regular "rsnapshot hourly" run. The above command "hard links" from
daily.0, daily.1 etc. over weekly.0 etc. "down" to yearly.0 etc.

Viele Gruesse!
Helmut
[/quote]
This approach has risks. Without serous arcanery, the 'hardlink'
software creates hardlinks *inside* each software repository So, for
example, if you've got two working copies of Subversion or git
repositories, suddenly the files from one will be linked to the other
inside one reposotory. Copy a home direcotry with both repositories
elsewhere with 'rsync -avH' and suddenly you've got two repos
hardlinked together in potentially quie dangerous fashion.

------------------------------------------------------------------------------
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://p.sf.net/sfu/Zoho
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Evaluating a backup repository
October 22, 2014 10:46AM
On Wed, Oct 22, 2014 at 10:35:22AM +0100, Patrick O'Callaghan wrote:
[quote]On Wed, Oct 22, 2014 at 7:30 AM, Helmut Hullen <hhullen < at > aktivanet.de> wrote:

[quote]Yes: wrong.
The examination starts with the newest backup (in my system: hourly.0,
in your system: daily.0) and then compares what has changed down to the
oldest backup.

[/quote]
Also, it's incorrect to think of rsnapshot backups as a base set plus
incrementals. All the backup sets are complete, they just happen to share
disk space and avoid redundant copy operations. This is an important
concept which differentiates rsnapshot from most traditional backup systems.

poc
[/quote]
Framed from the perspective of du, the files exist once on disk and are
linked to in multiple backup dirs. Du will only account for the data
once, but will attribute that data to the first dir that it encounters.

Here is an example on my system. Note the ordering of the dirs in du:

[sean < at > rat backup]$ du -sh 20141016 20141017
441G 20141016
723M 20141017
[sean < at > rat backup]$ du -sh 20141017 20141016
441G 20141017
298M 20141016

In this case, the full backup set is 441G. Both directories have this
full set, but it is only counted in the first dir. Hope this clarifies a
bit.

--Sean

------------------------------------------------------------------------------
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://p.sf.net/sfu/Zoho
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Evaluating a backup repository
October 22, 2014 11:24AM
Hallo, Nico,

Du meintest am 22.10.14:

[...]

[quote][quote]I wouldn't investigate but instead "hard link" again the whole
bundle of backups, with the "hardlink" program from Dag Wieers in
[/quote][/quote]
[quote]According to the manual page, it was written by Jakub Jelinek at Red
Hat. Dag and the Reporforge repository he seems to have founded
repackage most of the software, they don't write much from scratch. I
met Dag about 5 years ago in London and bought him beer. Nice guy, I
don't think he'd want to accidentally take credit for software he
didn't write.
[/quote]
Just additional:

"hardlink.spec":

# $Id$
# Authority: dag

Summary: Tool to hardlink duplicate files in a directory tree
Name: hardlink
Version: 1.2
Release: 1.2
License: GPL
Group: Applications/System
URL: ftp://ftp.redhat.com/pub/redhat/mirror-tools/

Source: ftp://ftp.redhat.com/pub/redhat/mirror-tools/hardlink.c
BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root

%description
A utility to hardlink duplicate files in a directory tree.

%prep
%setup -c -T
%{__cp} -afp %{SOURCE0} .

%build
${CC:-%{__cc}} %{optflags} -o hardlink hardlink.c

%install
%{__rm} -rf %{buildroot}
%{__install} -Dp -m0755 hardlink %{buildroot}%{_bindir}/hardlink

%clean
%{__rm} -rf %{buildroot}

%files
%defattr(-, root, root, 0755)
%{_bindir}/hardlink

%changelog
* Sat Apr 08 2006 Dries Verachtert <dries < at > ulyssis.org> - 1.2-1.2
- Rebuild for Fedora Core 5.

* Tue Apr 29 2003 Dag Wieers <dag < at > wieers.com> - 1.2-1
- Build happens in its own buildsubdir.

* Sun Apr 20 2003 Dag Wieers <dag < at > wieers.com> - 1.2-0
- Initial build. (using DAR)

============================================================

and the head lines of "hardlink.c":
/*
* This program was originally written by Jakub Jalunik and called
* cleanftp. I changed the name to hardlink because it could be used on
* other areas than an FTP site.
*
* $Id: hardlink.c,v 1.1 2004/02/24 23:49:14 dag- Exp $
*/

=============================================================

Viele Gruesse!
Helmut

------------------------------------------------------------------------------
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://p.sf.net/sfu/Zoho
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Evaluating a backup repository
October 22, 2014 08:37PM
On Tue, Oct 21, 2014 at 11:22:03PM -0400, Thierry Lavallee wrote:

[quote]2-Because of a connection bug leaving daily{0,1,2,3,4,5} empty , I had
to recently cp -al daily.6 to daily{0,1,2,3,4,5} I am not sure I
understand why they take 769M each :/ WOuld have expected just a few Ks
for the hard link. Wrong thinking?
[/quote]
The cp -al can hard link regular files, but it can't hard link
directory files, nor sym link files, etc.

My first guess would be that most of the 769M is directory files
and some of it sym files, and perhaps a few other non-regular files.

[quote]3-And my weekly.1 looks out of sync with the rest at 127G. Any clue how
to investigate this?!
[/quote]
I think the two most likely explanations are (1) the source files
changed between weekly.1 and weekly.0; or (2) rsnapshot backups were
broken at some point between weekly.1 and weekly.0 and then fixed,
so a daily snapshot (which no longer exists) could not hard link
from the previous daily backup.

You could pick a random relative regular file path foo/bar/baz and
ls -li /media/backup02/wscpb/weekly.{0,1}/foo/bar/baz
Given the du output, you would expect a different i-node number (the
first number in ls -li output) for most files.

The rest of the ls output shows things like size, modification date
and time. If these are different, then it explains why rsnapshot
could not just hard link the files together, hence different i-node
numbers and much more space used (case 1).

Then you could repeat that investigation with a few other files.

Or, an alternate way to do much the same thing which is probably
faster & easier, run
rsync -an --itemize-changes /media/backup02/wscpb/weekly.{1,0}/

For each file (except identical files) you will see a line with some
characters indicating the file type and what rsync thinks changed
between the backups with respect to that file, and the file name.
Look for itemize-changes in rsync(1) for details/explanation.

[quote]Thanks!

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
My command and result
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
root < at > server:# rsnapshot -c
/root/scripts/backup/rsnapshot.wscpb_cpbackup.conf du
132G /media/backup02/wscpb/daily.0/
769M /media/backup02/wscpb/daily.1/
769M /media/backup02/wscpb/daily.2/
768M /media/backup02/wscpb/daily.3/
768M /media/backup02/wscpb/daily.4/
768M /media/backup02/wscpb/daily.5/
768M /media/backup02/wscpb/daily.6/
8,7G /media/backup02/wscpb/weekly.0/
127G /media/backup02/wscpb/weekly.1/
15G /media/backup02/wscpb/weekly.2/
11G /media/backup02/wscpb/weekly.3/
9,5G /media/backup02/wscpb/monthly.0/
306G total
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

------------------------------------------------------------------------------
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://p.sf.net/sfu/Zoho
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

[/quote]
--
___________________________________________________________________________
David Keegel <djk < at > cyber.com.au> Cyber IT Solutions Pty. Ltd.
http://www.cyber.com.au/~djk/ Linux & Unix Systems Administration

------------------------------------------------------------------------------
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Sorry, only registered users may post in this forum.

Click here to login