Backup Central

Hi,

after adding one of our customers' server we see some weird backup
effects with a full backup taking (somewhat expected) 7 hours but
incremental backups taking more than twice that time.

See for yourself:

2012-08-10 01:30:01 full backup started for directory /backup_tree
2012-08-10 08:35:12 full backup 0 complete, 2599979 files, 25649558375
bytes, 0 xferErrs (0 bad files, 0 bad shares, 0 other)
2012-08-11 01:31:22 incr backup started back to 2012-08-10 01:30:01
(backup #0) for directory /backup_tree
2012-08-11 18:05:25 incr backup 1 complete, 11229 files, 464722370
bytes, 0 xferErrs (0 bad files, 0 bad shares, 0 other)
2012-08-12 01:32:23 incr backup started back to 2012-08-10 01:30:01
(backup #0) for directory /backup_tree
2012-08-12 19:33:58 incr backup 2 complete, 16858 files, 351340522
bytes, 0 xferErrs (0 bad files, 0 bad shares, 0 other)

Backups are done using ordinary rsync over ssh, with no "special"
settings except the nice'ing:

$Conf{RsyncClientCmd} = '$sshPath -q -x -l root $host nice -n 9
$rsyncPath $argList+';

One of the reasons I can think of is the file structure on that host. It
serves as a special storage pool for a customer developed application
and as such it has really really many subdirectories with really really
many subdirectories with really really many subdirectories. And by
really really many I mean really really many ... So my best guess would
be that building the file list diff takes much longer than just fetching
the files as they exist.

I've now disabled incremental backups on this server, but maybe someone
has an idea how to enable incremental backups for this host as well.

thanks in advance!

--
Udo Rader, CTO
BestSolution.at EDV Systemhaus GmbH
Eduard-Bodem-Gasse 5-7, A-6020 Innsbruck
http://www.bestsolution.at/
Reg. Nr. FN 222302s am Firmenbuchgericht Innsbruck

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

On 16/08/12 09:47, Udo Rader wrote:
One of the reasons I can think of is the file structure on that host. It
serves as a special storage pool for a customer developed application
and as such it has really really many subdirectories with really really
many subdirectories with really really many subdirectories. And by
really really many I mean really really many ... So my best guess would
be that building the file list diff takes much longer than just fetching
the files as they exist.

I've now disabled incremental backups on this server, but maybe someone
has an idea how to enable incremental backups for this host as well.

thanks in advance!

Try using tar instead of rsync, you are correct that for large file
trees rsync is very slow and also RAM intensive when it's building the
filetrees

--
Tim Fletcher <tim < at > night-shade.org.uk>


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

On 08/16/2012 12:02 PM, Tim Fletcher wrote:
On 16/08/12 09:47, Udo Rader wrote:
One of the reasons I can think of is the file structure on that host. It
serves as a special storage pool for a customer developed application
and as such it has really really many subdirectories with really really
many subdirectories with really really many subdirectories. And by
really really many I mean really really many ... So my best guess would
be that building the file list diff takes much longer than just fetching
the files as they exist.

I've now disabled incremental backups on this server, but maybe someone
has an idea how to enable incremental backups for this host as well.

thanks in advance!

Try using tar instead of rsync, you are correct that for large file
trees rsync is very slow and also RAM intensive when it's building the
filetrees

ok thanks, I will try that.

But IIRC, tar's pitfall is that it does not remove deleted files when
doing incremental backups, right?

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

On Thu, Aug 16, 2012 at 3:47 AM, Udo Rader <listudo < at > bestsolution.at> wrote:
Hi,

after adding one of our customers' server we see some weird backup
effects with a full backup taking (somewhat expected) 7 hours but
incremental backups taking more than twice that time.

[...]

One of the reasons I can think of is the file structure on that host. It
serves as a special storage pool for a customer developed application
and as such it has really really many subdirectories with really really
many subdirectories with really really many subdirectories. And by
really really many I mean really really many ... So my best guess would
be that building the file list diff takes much longer than just fetching
the files as they exist.

Only the first full will 'just fetch', and that's because there is no
local directory tree to compare. Subsequent runs will not only
compare trees, but the fulls do a block-checksum compare of the data.

I've now disabled incremental backups on this server, but maybe someone
has an idea how to enable incremental backups for this host as well.

I think you've jumped to conclusions here - you need to time full runs
other than the first. Other things to keep in mind are:
Incremental runs copy everything that has changed since the previous full.
Using the checksum caching will avoid the need to uncompress/compare
on the server but only after the 2nd full of an unchanged file.
The whole directory tree is held in ram

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

On 2012-08-16 15:51, Les Mikesell wrote:
I've now disabled incremental backups on this server, but maybe someone
has an idea how to enable incremental backups for this host as well.

I think you've jumped to conclusions here - you need to time full runs
other than the first. Other things to keep in mind are:
Incremental runs copy everything that has changed since the previous full.

One way we've improved this is to make incrementals reference the previous
incremental. As long as your filesystem timestamps are accurate (system
dates are synced, file modification times are updated on write), this works
well.

$Conf{IncrLevels} = [ '1', '2', '3', '4', '5', '6', '7' ];

We use seven on the off chance that someone runs a manual one that week.

Regards,
Tyler

Using the checksum caching will avoid the need to uncompress/compare
on the server but only after the 2nd full of an unchanged file.
The whole directory tree is held in ram

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


--
"... my partner and I became exhibit A in a process that I have been
warning Americans about since 2007: first they come for the 'other' –
the 'terrorist', the brown person, the Muslim, the outsider; then they
come for you – while you are standing on a sidewalk in evening dress,
obeying the law."
-- Naomi Wolf, "How I was arrested at Occupy Wall Street", 2011-10-19

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

On Thursday 16 August 2012 16:35:49 Tyler J. Wagner wrote:
On 2012-08-16 15:51, Les Mikesell wrote:
I've now disabled incremental backups on this server, but maybe someone
has an idea how to enable incremental backups for this host as well.

I think you've jumped to conclusions here - you need to time full runs
other than the first. Other things to keep in mind are:
Incremental runs copy everything that has changed since the previous full.

One way we've improved this is to make incrementals reference the previous
incremental. As long as your filesystem timestamps are accurate (system
dates are synced, file modification times are updated on write), this works
well.
$Conf{IncrLevels} = [ '1', '2', '3', '4', '5', '6', '7' ];
We use seven on the off chance that someone runs a manual one that week.

This variable (and also the corresponding FillLevels) works differently than
you expect:
It means keep one backup at IncrInterval, keep 2 backups at 2*IncrInterval
before that, keep 3 backups at 4*IncrInterval before that, keep 5 backups at
8*IncrInterval before that,....

You actually "want" to set this to just plain 7 to make backuppc keep seven
incremental backups at IncrInterval.

For FullLevels you are well advised setting it to [4, 0, 4] or [4, 0, 6].
Given a FullInterval of 7 days this results in keeping the last four full
backups (which happen to be weekly), keep zero with two weeks interval and
keep 4 (or 6) at four-weeks interval which is appr. a month.

Have fun,

Arnold
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

On 2012-08-16 18:20, Arnold Krille wrote:
This variable (and also the corresponding FillLevels) works differently than
you expect:
It means keep one backup at IncrInterval, keep 2 backups at 2*IncrInterval
before that, keep 3 backups at 4*IncrInterval before that, keep 5 backups at
8*IncrInterval before that,....

You are describing a behaviour similar to FullKeepCnt. Please see the
documentation for IncrLevels:

http://backuppc.sourceforge.net/faq/BackupPC.html#_conf_incrlevels_

Regards,
Tyler



--
"If you wish to make an apple pie from scratch, you must first create
the universe."
-- Carl Sagan

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/