Backup Central

Hi,

I'm new to rsnapshot and use BackupPC and bacula for backups right
now.

We are planing to expand our storage infrastructure to some 100+ TB
and consider doing it with some RAID arrays or more expensive products
with snapshot features for taking LAN-less backups.

The cheapest hardware solution would be a couple of big server chassis
with 36 disks on board and an expansion unit with additional 45 disks.
This would give us 160 TB, a dual CPU and 24 GB of RAM in 8 HU.

The data that will be stored there won't be mission critical,
nevertheless having a backup is necessary.

The 160 TB will be used in slices of 10-15 TB with an XFS filesystem.
The Users will access th data with Samba.

The big problem is to backup this amount of data over LAN.

The idea for backup is to add a backup server for each of the data
servers that has app. 1,5 x the size of the data server and take
backups over lan. Maybe over 10GBASE-T, data server and backup server
back to back directly connected.

Here comes my question: could rsnapshot be the right tool for this?
Important for me is, that a file that was backed up once, doesn't get
backed up again. The filesystems will be 10-15 TB, maybe a backup set
could be broken further down, but I doubt that I have controll over
the directories the user will create/remove.

For smaller amounts of data backuppc is working fine, but it has some
significant overhead for pooling and calculating the hashes. And the
pool has to be a single large filesystem holding all files.


Any ideas?

Ralf

------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

Am Freitag 26 Februar 2010 schrieb Ralf Gross:
[...]
Here comes my question: could rsnapshot be the right tool for this?
Important for me is, that a file that was backed up once, doesn't get
backed up again.

You might want to have a look at storeBackup (.org). It calculates md5sums for
each file and makes heavy use of hard links to only backup files once, even
if they are renamed or moved or exist in different places below your backup
root. In addition, it's able to split (big) files, hard link the unchanged
parts and only back up the changed chunks (I use this for VirtualBox disk
images, f.ex.). Of course this also might have some performance issues with
it.

HTH,
Ben

------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

Ben Staude schrieb:
Am Freitag 26 Februar 2010 schrieb Ralf Gross:
[...]
Here comes my question: could rsnapshot be the right tool for this?
Important for me is, that a file that was backed up once, doesn't get
backed up again.

You might want to have a look at storeBackup (.org). It calculates md5sums for
each file and makes heavy use of hard links to only backup files once, even
if they are renamed or moved or exist in different places below your backup
root. In addition, it's able to split (big) files, hard link the unchanged
parts and only back up the changed chunks (I use this for VirtualBox disk
images, f.ex.). Of course this also might have some performance issues with
it.

Thanks for the link. This sounds a lot like backuppc that we use right
now - with all the pros and cons. But I'll have a closer look.

Ralf

------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss