That's insightful, Craig; thanks.
None of the PCs in my example is slower than 2.4 GHz, with a 100MBit network
locally, and a 6Mbit/768Kbit Internet connection. These were all full
backups. I had another, local backup take 50% less time on backups 3+, so
some serious reductions on all but one local machine.
-----Original Message-----
From: Craig Barratt [mailto:cbarratt < at > users.sourceforge.net]
Sent: Tuesday, July 27, 2004 1:54 AM
To: Todd Curry
Cc: backuppc-users < at > lists.sourceforge.net
Subject: Re: [BackupPC-users] rsync checksum backup speeds
"Todd Curry" writes:
Just fired up the rsync(d) checksum seeds on all of my backup machines and
I'm both elated and confused.
The elation: some of the PCs I've been backing up over the Internet are
now
*so* much faster to back up that I am stunned!
Example 1:
First Backup: 681 minutes
Second backup: 225 minutes
Third backup: 14 minutes
Example 2:
First backup: 165 minutes
Second backup: 142 minutes
Third backup: 12 minutes
This prompted an audible "WOW" -- way to go CRAIG!!
However, I'm confused by the (comparatively) paltry savings (only 1/3!
)
on the machines I'm backing up over my local ethernet connection
Example 1:
First Backup: 75 minutes
Second Backup: 74 minutes
Third Backup: 53 minutes
Fourth Backup: 54 minutes
I'm trying others now. This ethernet backup is certainly bigger than the
other two backups. Backups 2-4 on the local machine were done about 15
minutes after the previous backup finished (more than enough time to
link/fill).
Any thoughts on why this local backup isn't taking far less time?
Josh's comments are correct. I wish I could take credit for
making the internet backups faster, but I doubt rsync checksum
caching has this sort of effect, since I suspect the backup
speed is dominated by network speed, not server CPU/disk.
Backup speed is affected by many factors, including client speed
(cpu and disk), server speed (cpu and disk) and network throughput
and latency.
With a really slow network, the client and server speed won't matter
as much. With a fast network and fast client, the server speed will
dominate. With a fast network and fast server, the client speed will
dominate. In most cases it will be a combination of all three.
After the first backup, rsync will tend to use a lot less network
traffic. So with rsync the second (and subsequent) full backup
will be a lot faster if the network is slow.
Rsync checksum caching significantly reduces the server cpu and
disk load. That might make a backup run faster if the server
performance is a significant factor in overall backup speed.
The biggest benefit is that the server should be able to run
more concurrent backups.
The benefit you see in the local backups (1/3) are probably due to rsync
checksum caching:
First Backup: 75 minutes
Second Backup: 74 minutes
Third Backup: 53 minutes
Fourth Backup: 54 minutes
The behavior looks correct: you see the benefit on the third backup. On
the first backup the files are all transferred. On the second backup
the checksums are computed, used, and cached. On the third backup the
cached checksums are used. This is not exactly "paltry" since your
server can probably now run perhaps twice the number of concurrent
backups.
Craig
