Am Montag, 1. Februar 2010 16:44:10 schrieb Jeff Plotzke:
Hi All,
I recently tried out rsnapshot on my server in order to automate
backups, but I find it running extremely slow. My first backup of about
100GB took about 12 hours to complete. After that, I did another hourly
backup (manually from command line) while watching the rsnapshot log and
it took about 4 additional hours (barely anything (definitely < 1GB)
should have changed between those backups. During the 2nd backup, I
noticed that the cp -la command took about 2 hrs in order to copy the
hourly.1 folder to hourly.0. The remaining 2 hours was rsync completing
the backup.
I then turned on link_dest (as suggested by some others having slow
rsnapshot issues) and ran it again, however about the same situation...
this time, rsync took about 4 hours to run.
Since the cp command took an extreme amount of time, my first guess is
that disk or CPU is slowing things down... do you agree? I admit that
this isn't a super fast server -- running about 1.2 GHz.
Jeff,
there are a couple of potential reasons:
0) USB 1.X connection to source or backup disk. This could be caused by an
ancient USB hub or outdated main board hardware. Please check your manual and
buy an USB 2.0 add in card. Do you use sdg for backup or is it the source?
Please repeat the hdparm test with the other disk. Loose cables also fall into
this category. Please check "dmsg" output.
1) Original and backup are on partitions of the same disk, e. g. in your case
/dev/sdg1 and /dev/sdg2. I'd not call this a reliable backup, but it makes
most use of the given resources. The disk head needs to move a lot between the
partitions, slowing things down without reading/writing much data.
2) No hard linking but full copying in every run. "ls -li
daily.0/path/to/unchanged/file daily.1/path/to/unchanged/file" (the same file)
should have the same number (inode) in the first column. Your varying run
times are a hint that hard linking is done correctly. An obvious reason is
3) Backing up to a file system unable to store hard links, e. g. FAT. Using an
USB disk as bought is the most common cause.
4) Backing up to a file system that is slow in creating and deleting hard
links, e. g. XFS or JFS. Reiser3 becomes slower over time, especially as a
backup partition where free space gets fragmented quite fast. Ext3 and
especially ext4 are fast. Btrfs is lightning fast, but too experimental.
5) Backing up to a file system that is nearly full, e. g. over 80%. It gets
difficult to allocate large chunks of free space and similar to 1) disk seeks
increase.
6) Backing up from a fragmented source. "Format and restore" is the Unix way
of defragmenting and should be done after a partition has been used for some
years.
7) Using file systems slow at traversing the directory structure. "time find
/mount/point" with "cold" caches is a good benchmark for this. Please provide
the tail of the output for both partitions.
This write up could become part of the FAQ.
HTH,
Johannes Niess
------------------------------------------------------------------------------
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
