Backup Central

We are now using rdiff-backup on our debian fileserver without any real
problems.

Our daily backup of ca. 1.2TB starts at 1:00am and lasts for 43 hours.
As 43 hours are more than a day, this is our problem.

rdiff-backup detects half of the files (395041 files with 567GB) as
changed, but rdiff-backups "Increment FileSize" is only 5GB.
The only thing that has changed on most of this files is their ctime. So
rdiff-backup works fine and correct.

We don't know what changes the ctime. The files are shared trough
netatalk and samba. Maybe it's an indexing or searching of some Mac or
Win clients.

Since we have no clue what changes the ctime, we would like rdiff-backup
to ignore the ctime.
Before you add an option like --no-ctime-compare I would like to test if
it would work.

So I would ask you to tell me how to change rdiff-backup manually to
ignore the ctime.

Regards,
Carsten

"Carsten Lorenz" <clorenz < at > hamburg.fcb.com>
wrote the following on Thu, 17 Nov 2005 11:24:03 +0100
We are now using rdiff-backup on our debian fileserver without any real
problems.

Our daily backup of ca. 1.2TB starts at 1:00am and lasts for 43 hours.
As 43 hours are more than a day, this is our problem.

rdiff-backup detects half of the files (395041 files with 567GB) as
changed, but rdiff-backups "Increment FileSize" is only 5GB.
The only thing that has changed on most of this files is their ctime. So
rdiff-backup works fine and correct.

Actually rdiff-backup already ignores ctime, so that isn't your
problem :-)

(The reason it does this is simply because a lot of the existing
automatic tests depend on a successful rdiff-backup mirror comparing
correctly to the source. This is impossible if ctimes are compared,
since rdiff-backup can't set ctimes either.)

So anyway, the problem isn't ctimes. How can you figure out why
rdiff-backup thinks a file has changed? One way is to find a file you
think hasn't changed, and compare its records in two consecutive
mirror_metadata files. (Note that there is no ctime information in
the mirror_metadata file.)


--
Ben Escoto

Ben Escoto wrote: "Carsten Lorenz" <clorenz < at > hamburg.fcb.com> ([email]clorenz < at > hamburg.fcb.com[/email])
wrote the following on Thu, 17 Nov 2005 11:24:03 +0100
We are now using rdiff-backup on our debian fileserver without any real
problems.

Our daily backup of ca. 1.2TB starts at 1:00am and lasts for 43 hours.
As 43 hours are more than a day, this is our problem.

rdiff-backup detects half of the files (395041 files with 567GB) as
changed, but rdiff-backups "Increment FileSize" is only 5GB.
The only thing that has changed on most of this files is their ctime. So
rdiff-backup works fine and correct.

Actually rdiff-backup already ignores ctime, so that isn't your
problem :-)

(The reason it does this is simply because a lot of the existing
automatic tests depend on a successful rdiff-backup mirror comparing
correctly to the source. This is impossible if ctimes are compared,
since rdiff-backup can't set ctimes either.)

So anyway, the problem isn't ctimes. How can you figure out why
rdiff-backup thinks a file has changed? One way is to find a file you
think hasn't changed, and compare its records in two consecutive
mirror_metadata files. (Note that there is no ctime information in
the mirror_metadata file.)
I've got a file which hasn't changed, even the data in the mirror_metadata hasn't changed, but the file_statistic shows that it has changed.

The file doesn't appear in rdiff-backup its extended_attributes or access_control_lists files.
Since this file has an acl this may be the reason why rdiff-backup detects a difference.

I switched of acls and eas to look if this works.

Regards,
Carsten


file_statistic:
.../get the look_02.eps 1 15771551 15771551 219

both mirror_metadata files:
  Permissions 436

stat:
Access: (0664/-rw-rw-r--)  Uid: ( 1032/   mmohr)   Gid: (17000/  beaute)

getfacl:
# owner: mmohr
# group: beaute
user::rw-
group::rwx                      #effective:rw-
group:readonly:r-x              #effective:r--
mask::rw-
other::r--

Ben Escoto wrote:

"Carsten Lorenz" <clorenz < at > hamburg.fcb.com>
wrote the following on Thu, 17 Nov 2005 11:24:03 +0100


We are now using rdiff-backup on our debian fileserver without any real
problems.

Our daily backup of ca. 1.2TB starts at 1:00am and lasts for 43 hours.
As 43 hours are more than a day, this is our problem.

rdiff-backup detects half of the files (395041 files with 567GB) as
changed, but rdiff-backups "Increment FileSize" is only 5GB.
The only thing that has changed on most of this files is their ctime. So
rdiff-backup works fine and correct.


So anyway, the problem isn't ctimes. How can you figure out why
rdiff-backup thinks a file has changed? One way is to find a file you
think hasn't changed, and compare its records in two consecutive
mirror_metadata files. (Note that there is no ctime information in
the mirror_metadata file.)


Some test later ...
There is something wrong with the acls.

With --no-acls it took only 1.5 hours to process 1.17 TB !
Of cause it depends how many files change, but our last full tape-backup
lasts more than 2.5 days.

I have found a test case where rdiff-backup detects every file as
changed although the files are unchanged.
If i add the option --no-acls all works fine!

Looks as if there is a bug in rdiff-backup or one of the library it uses :-(

Here are samples of the output:


With the options -v 8 -no-acls
Executing ssh 192.168.0.1 rdiff-backup --server
Registering connection 1
-----------------------------------------------------------------
Detected abilities for source (read only) file system:
Access control lists On
Extended attributes On
Mac OS X style resource forks Off
Mac OS X Finder information Off
-----------------------------------------------------------------
[...]
-----------------------------------------------------------------
Detected abilities for destination (read/write) file system:
Characters needing quoting ''
Ownership changing On
Hard linking On
fsync() directories On
Directory inc permissions On
Access control lists On
Extended attributes On
Mac OS X style resource forks Off
Mac OS X Finder information Off
-----------------------------------------------------------------
Writing mirror marker
/mnt/raid/backup/test/rdiff-backup-data/current_mirror.2005-11-28T17:32:47+01:00.data
Starting increment operation
/mnt/files/clients/Archiv/NDE_Beaute/NDE-Creative Rumble/Booklet/Rumble
Booklet/alle eps to /mnt/raid/backup/test
--------------[ Session statistics ]--------------
StartTime 1133195567.00 (Mon Nov 28 17:32:47 2005)
EndTime 1133195569.45 (Mon Nov 28 17:32:49 2005)
ElapsedTime 2.45 (2.45 seconds)
SourceFiles 185
SourceFileSize 1406464859 (1.31 GB)
MirrorFiles 185
MirrorFileSize 1406464859 (1.31 GB)
NewFiles 0
NewFileSize 0 (0 bytes)
DeletedFiles 0
DeletedFileSize 0 (0 bytes)
ChangedFiles 0
ChangedSourceSize 0 (0 bytes)
ChangedMirrorSize 0 (0 bytes)
IncrementFiles 0
IncrementFileSize 0 (0 bytes)
TotalDestinationSizeChange 0 (0 bytes)
Errors 0
--------------------------------------------------
Deleting
/mnt/raid/backup/test/rdiff-backup-data/current_mirror.2005-11-28T17:32:35+01:00.data
Cleaning up


Only with the option -v 8
Executing ssh 192.168.0.1 rdiff-backup --server
Registering connection 1
-----------------------------------------------------------------
Detected abilities for source (read only) file system:
Access control lists On
Extended attributes On
Mac OS X style resource forks Off
Mac OS X Finder information Off
-----------------------------------------------------------------
[...]
-----------------------------------------------------------------
Detected abilities for destination (read/write) file system:
Characters needing quoting ''
Ownership changing On
Hard linking On
fsync() directories On
Directory inc permissions On
Access control lists On
Extended attributes On
Mac OS X style resource forks Off
Mac OS X Finder information Off
-----------------------------------------------------------------
Writing mirror marker
/mnt/raid/backup/test/rdiff-backup-data/current_mirror.2005-11-28T17:07:01+01:00.data
Starting increment operation
/mnt/files/clients/Archiv/NDE_Beaute/NDE-Creative Rumble/Booklet/Rumble
Booklet/alle eps to /mnt/raid/backup/test
[...]
Getting signature of the kiss.eps with blocksize 3968
[...]
Getting delta of /mnt/files/clients/Archiv/NDE_Beaute/NDE-Creative
Rumble/Booklet/Rumble Booklet/alle eps/the kiss.eps with signature the
kiss.eps
[...]
Processing changed file the kiss.eps
Writing file object to /mnt/raid/backup/test/rdiff-backup.tmp.183
Copying attributes from ('the kiss.eps',) to
/mnt/raid/backup/test/rdiff-backup.tmp.183
Setting time of /mnt/raid/backup/test/rdiff-backup.tmp.183 to 1113838797
Incrementing mirror file /mnt/raid/backup/test/the kiss.eps
Writing delta /mnt/raid/backup/test/rdiff-backup.tmp.183 from
/mnt/raid/backup/test/the kiss.eps ->
/mnt/raid/backup/test/rdiff-backup-data/increments/the
kiss.eps.2005-11-28T16:54:53+01:00.diff.gz
Getting signature of rdiff-backup.tmp.183 with blocksize 3968
Writing file object to
/mnt/raid/backup/test/rdiff-backup-data/increments/the
kiss.eps.2005-11-28T16:54:53+01:00.diff.gz
Copying inc attrs from ('the kiss.eps',) to
/mnt/raid/backup/test/rdiff-backup-data/increments/the
kiss.eps.2005-11-28T16:54:53+01:00.diff.gz
Setting time of /mnt/raid/backup/test/rdiff-backup-data/increments/the
kiss.eps.2005-11-28T16:54:53+01:00.diff.gz to 1113838797
Renaming /mnt/raid/backup/test/rdiff-backup.tmp.183 to
/mnt/raid/backup/test/the kiss.eps
[...]
--------------[ Session statistics ]--------------
StartTime 1133194021.00 (Mon Nov 28 17:07:01 2005)
EndTime 1133194256.15 (Mon Nov 28 17:10:56 2005)
ElapsedTime 235.15 (3 minutes 55.15 seconds)
SourceFiles 185
SourceFileSize 1406464859 (1.31 GB)
MirrorFiles 185
MirrorFileSize 1406464859 (1.31 GB)
NewFiles 0
NewFileSize 0 (0 bytes)
DeletedFiles 0
DeletedFileSize 0 (0 bytes)
ChangedFiles 185
ChangedSourceSize 1406464859 (1.31 GB)
ChangedMirrorSize 1406464859 (1.31 GB)
IncrementFiles 185
IncrementFileSize 27238 (26.6 KB)
TotalDestinationSizeChange 27238 (26.6 KB)
Errors 0
--------------------------------------------------
Deleting
/mnt/raid/backup/test/rdiff-backup-data/current_mirror.2005-11-28T16:54:53+01:00.data
Cleaning up


In the mirror_metadata:
File the kiss.eps
Type reg
Size 15773010
ModTime 1113838797
Uid 1032
Uname mmohr
Gid 17000
Gname beaute
Permissions 436

In the file_statistics:
the kiss.eps 1 15773010 15773010 126
(Only 11 bytes after unpacking the diff file.)

getfacl of source and destination gives:
# file: the\040kiss.eps
# owner: mmohr
# group: beaute
user::rw-
group::rwx #effective:rw-
group:readonly:r-x #effective:r--
mask::rw-
other::r--

stat of source gives:
File: `the kiss.eps'
Size: 15773010 Blocks: 30816 IO Block: 4096 regular file
Device: 841h/2113d Inode: 1109055492 Links: 1
Access: (0664/-rw-rw-r--) Uid: ( 1032/ mmohr) Gid: (17000/ beaute)
Access: 2005-11-28 17:29:19.987785284 +0100
Modify: 2005-04-18 17:39:57.000000000 +0200
Change: 2005-04-25 10:34:16.455895071 +0200

stat of destination:
File: `the kiss.eps'
Size: 15773010 Blocks: 30816 IO Block: 4096 regular file
Device: fe00h/65024d Inode: -2122278833 Links: 1
Access: (0664/-rw-rw-r--) Uid: ( 1032/ mmohr) Gid: (17000/ beaute)
Access: 2005-11-28 17:31:35.641293223 +0100
Modify: 2005-04-18 17:39:57.000000000 +0200
Change: 2005-11-28 17:31:35.934240451 +0100


The versions i use are:
pylibacl 0.2.1
rdiff-backup 1.0.1
kernel 2.6.12
filesystem is xfs

Carsten