Backup Central

Hello all,

Firstly I would like to say how great BackupPC is! Amazing product, that
just solves all my backup requirements so well.

Unfortunately though I am having a speed related problem with rsync.

My current setup is 2 x FreeBSD machines (one a p4 with freebsd 5.1, the
other a p3 with freebsd 5.2.1) setup as backup machines to backup a
variety of linux machines. all machines are connected by 100mbps
ethernet.

The two servers were running rsync 2.6.0 and now one is running
2.6.1beta2. The client machines are running a mix of rsync versions from
2.5.7 to 2.6.0, and now one is also running 2.6.1beta2. The version of
rsync doesn't seem to make any difference. Perl is 5.8.0, and
File::RsyncP is 0.50


The problem is that the backup process is taking along time, and then
timing out. For example one client machine that has 3GB of used disk
space completes a full backup in about 4 hours, and times out a
incremental backup after 6 hours. This is with just that one machine
being backed up at the time. Running rsync directly to transfer files
shows good network performance.

BackupPC is reporting that the backup speed is around 0.8MB/s, and this is
confirmed by graphs of the switch the machines are connected to. Running
top on the backup server also shows that there is alot of time when the
backup perl process just sits without using any real cpu.

Further testing by switching over to tar (over ssh) though has shown an
amazing increase in speed. Network utilisation is now up around 20Mb/s

I am happy to do further testing to try to resolve this, any suggestions
on what I can try to get rsync working better?


Regards,

Matt Baker

--
matt baker
matt < at > sevenone.com




-------------------------------------------------------
This SF.Net email is sponsored by: Oracle 10g
Get certified on the hottest thing ever to hit the market... Oracle 10g.
Take an Oracle 10g class now, and we'll give you the exam FREE.
http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/

FWIW, I'm having a very similar problem (same traffic graphs, etc) backing up a
Win2K machine running rsyncd-2.6.0 from the backuppc site.

61GB of data.. > 12hrs to backup (not done with first full yet)

Any other data I can add to help solve this speed issue?

Tony

--
Tony Nelson
Director of IT Operations
Starpoint Solutions LLC
115 Broadway, 2nd Fl
New York, NY 10006


Quoting matt baker <matt < at > sevenone.com>:

Hello all,

Firstly I would like to say how great BackupPC is! Amazing product, that
just solves all my backup requirements so well.

Unfortunately though I am having a speed related problem with rsync.

My current setup is 2 x FreeBSD machines (one a p4 with freebsd 5.1, the
other a p3 with freebsd 5.2.1) setup as backup machines to backup a
variety of linux machines. all machines are connected by 100mbps
ethernet.

The two servers were running rsync 2.6.0 and now one is running
2.6.1beta2. The client machines are running a mix of rsync versions from
2.5.7 to 2.6.0, and now one is also running 2.6.1beta2. The version of
rsync doesn't seem to make any difference. Perl is 5.8.0, and
File::RsyncP is 0.50


The problem is that the backup process is taking along time, and then
timing out. For example one client machine that has 3GB of used disk
space completes a full backup in about 4 hours, and times out a
incremental backup after 6 hours. This is with just that one machine
being backed up at the time. Running rsync directly to transfer files
shows good network performance.

BackupPC is reporting that the backup speed is around 0.8MB/s, and this is
confirmed by graphs of the switch the machines are connected to. Running
top on the backup server also shows that there is alot of time when the
backup perl process just sits without using any real cpu.

Further testing by switching over to tar (over ssh) though has shown an
amazing increase in speed. Network utilisation is now up around 20Mb/s

I am happy to do further testing to try to resolve this, any suggestions
on what I can try to get rsync working better?


Regards,

Matt Baker

--
matt baker
matt < at > sevenone.com




-------------------------------------------------------
This SF.Net email is sponsored by: Oracle 10g
Get certified on the hottest thing ever to hit the market... Oracle 10g.
Take an Oracle 10g class now, and we'll give you the exam FREE.
http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/



-------------------------------------------------------
This SF.Net email is sponsored by: Oracle 10g
Get certified on the hottest thing ever to hit the market... Oracle 10g.
Take an Oracle 10g class now, and we'll give you the exam FREE.
http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/

matt baker writes:

Firstly I would like to say how great BackupPC is! Amazing product, that
just solves all my backup requirements so well.

Thanks for the feedback.

Unfortunately though I am having a speed related problem with rsync.

My current setup is 2 x FreeBSD machines (one a p4 with freebsd 5.1, the
other a p3 with freebsd 5.2.1) setup as backup machines to backup a
variety of linux machines. all machines are connected by 100mbps
ethernet.

The two servers were running rsync 2.6.0 and now one is running
2.6.1beta2. The client machines are running a mix of rsync versions from
2.5.7 to 2.6.0, and now one is also running 2.6.1beta2. The version of
rsync doesn't seem to make any difference. Perl is 5.8.0, and
File::RsyncP is 0.50

The problem is that the backup process is taking along time, and then
timing out. For example one client machine that has 3GB of used disk
space completes a full backup in about 4 hours, and times out a
incremental backup after 6 hours. This is with just that one machine
being backed up at the time. Running rsync directly to transfer files
shows good network performance.

BackupPC is reporting that the backup speed is around 0.8MB/s, and this is
confirmed by graphs of the switch the machines are connected to. Running
top on the backup server also shows that there is alot of time when the
backup perl process just sits without using any real cpu.

Further testing by switching over to tar (over ssh) though has shown an
amazing increase in speed. Network utilisation is now up around 20Mb/s

I am happy to do further testing to try to resolve this, any suggestions
on what I can try to get rsync working better?

3GB in 4 hours and more than 6 hours for an incremental is terrible.
But 0.8MB/sec should be around 3GB/hour. That doesn't reconcile
with your 3GB in 4 hour estimate. Have you excluded /proc?
Rsync should handle /dev correctly, but you might want to
exclude that too while you test.

As one example, my XP laptop with rsyncd/cygwin is about 19GB and
fulls take around 2 hours and incrementals less than an hour.

For the incremental timeout you might want to increase the value
of $Conf{ClientTimeout}. I'd be curious to see the log file
for an incremental with $Conf{RsyncLogLevel} set to, say, 4.

Something is obviously broken, but I'm not sure what. One random
suggestion is to try perl version >= 5.8.2.

Craig