Backup Central

I'm running rdiff-backup on a Western Digital My Book World Edition
II, it's a little NAS server that runs linux and has ssh login
available.

The rdiff-backup is running from one of the NAS server's drives to the
other drive, i.e. it's not being run from another system to (or from)
the NAS and thus is not limited by network bandwidth.

I'm backing up two large[ish] directories, one is around 164Gb, the
other is about 35Gb.

The 35Gb one takes about 7 minutes to run, which is fine, but the
165Gb one takes 4 hours and 5 minutes. Presumably there's some sort
of resource limitation which the bigger backup is hitting and hence
running slower.

Is there anything I can do to improve performance? My first guess
would be that rdiff-backup is running out of 'real' memory and
swapping which is slowing things.

It has 126828 kB of memory, more than I thought, that's 128Mb.

The processor is an ARM926EJ-S, 183 BogoMIPS.

--
Chris Green



_______________________________________________
rdiff-backup-users mailing list at rdiff-backup-users < at > nongnu.org
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

On Thu, Jan 14, 2010 at 06:18:33PM +0000, Chris G wrote:
I'm running rdiff-backup on a Western Digital My Book World Edition
II, it's a little NAS server that runs linux and has ssh login
available.

The rdiff-backup is running from one of the NAS server's drives to the
other drive, i.e. it's not being run from another system to (or from)
the NAS and thus is not limited by network bandwidth.

I'm backing up two large[ish] directories, one is around 164Gb, the
other is about 35Gb.

The 35Gb one takes about 7 minutes to run, which is fine, but the
165Gb one takes 4 hours and 5 minutes. Presumably there's some sort
of resource limitation which the bigger backup is hitting and hence
running slower.

Is there anything I can do to improve performance? My first guess
would be that rdiff-backup is running out of 'real' memory and
swapping which is slowing things.

It has 126828 kB of memory, more than I thought, that's 128Mb.

The processor is an ARM926EJ-S, 183 BogoMIPS.

... and here are the rdiff-backup session_statistics files:-

StartTime 1263436518.00 (Thu Jan 14 02:35:18 2010)
EndTime 1263451841.07 (Thu Jan 14 06:50:41 2010)
ElapsedTime 15323.07 (4 hours 15 minutes 23.07 seconds)
SourceFiles 253324
SourceFileSize 167143773023 (156 GB)
MirrorFiles 253050
MirrorFileSize 167116487371 (156 GB)
NewFiles 274
NewFileSize 23594577 (22.5 MB)
DeletedFiles 0
DeletedFileSize 0 (0 bytes)
ChangedFiles 431
ChangedSourceSize 18543551628 (17.3 GB)
ChangedMirrorSize 18539860553 (17.3 GB)
IncrementFiles 706
IncrementFileSize 90764595 (86.6 MB)
TotalDestinationSizeChange 118050247 (113 MB)
Errors 0


StartTime 1263451844.00 (Thu Jan 14 06:50:44 2010)
EndTime 1263452264.06 (Thu Jan 14 06:57:44 2010)
ElapsedTime 420.06 (7 minutes)
SourceFiles 25405
SourceFileSize 35810108114 (33.4 GB)
MirrorFiles 25405
MirrorFileSize 35810108114 (33.4 GB)
NewFiles 0
NewFileSize 0 (0 bytes)
DeletedFiles 0
DeletedFileSize 0 (0 bytes)
ChangedFiles 0
ChangedSourceSize 0 (0 bytes)
ChangedMirrorSize 0 (0 bytes)
IncrementFiles 0
IncrementFileSize 0 (0 bytes)
TotalDestinationSizeChange 0 (0 bytes)
Errors 0



Hmmm, there's something funny going on here! There should be some
changed files in that second (33Gb) backup, I'll go and investigate.

--
Chris Green


_______________________________________________
rdiff-backup-users mailing list at rdiff-backup-users < at > nongnu.org
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

How about splitting the larger backup into 3 or 4 parts?

Gavin

Chris G wrote:
I'm running rdiff-backup on a Western Digital My Book World Edition
II, it's a little NAS server that runs linux and has ssh login
available.

The rdiff-backup is running from one of the NAS server's drives to the
other drive, i.e. it's not being run from another system to (or from)
the NAS and thus is not limited by network bandwidth.

I'm backing up two large[ish] directories, one is around 164Gb, the
other is about 35Gb.

The 35Gb one takes about 7 minutes to run, which is fine, but the
165Gb one takes 4 hours and 5 minutes. Presumably there's some sort
of resource limitation which the bigger backup is hitting and hence
running slower.

Is there anything I can do to improve performance? My first guess
would be that rdiff-backup is running out of 'real' memory and
swapping which is slowing things.

It has 126828 kB of memory, more than I thought, that's 128Mb.

The processor is an ARM926EJ-S, 183 BogoMIPS.




_______________________________________________
rdiff-backup-users mailing list at rdiff-backup-users < at > nongnu.org
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

On Fri, Jan 15, 2010 at 05:27:59PM +1000, Gavin wrote:
Chris G wrote:
I'm running rdiff-backup on a Western Digital My Book World Edition
II, it's a little NAS server that runs linux and has ssh login
available.

The rdiff-backup is running from one of the NAS server's drives to the
other drive, i.e. it's not being run from another system to (or from)
the NAS and thus is not limited by network bandwidth.

I'm backing up two large[ish] directories, one is around 164Gb, the
other is about 35Gb.

The 35Gb one takes about 7 minutes to run, which is fine, but the
165Gb one takes 4 hours and 5 minutes. Presumably there's some sort
of resource limitation which the bigger backup is hitting and hence
running slower.

Is there anything I can do to improve performance? My first guess
would be that rdiff-backup is running out of 'real' memory and
swapping which is slowing things.

It has 126828 kB of memory, more than I thought, that's 128Mb.

The processor is an ARM926EJ-S, 183 BogoMIPS.

How about splitting the larger backup into 3 or 4 parts?


Yes, that's a possibility, though it is rather one large bit and
several small bits. However it might be that decreasing the size by a
fairly small amount would help so it's worth trying. Thanks.

--
Chris Green



_______________________________________________
rdiff-backup-users mailing list at rdiff-backup-users < at > nongnu.org
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki