SearchFAQMemberlist Log in
Reply to topic Page 1 of 1
Suggested fs for placing rdiff-backup data stores
Author Message
Post Suggested fs for placing rdiff-backup data stores 
Hi

thought I would break this off the original thread

just to give you some example info on fusecompress datastore

max:/backups/nas/system/boot# du --apparent-size -s --si *
1.3M System.map-2.6.26-1-amd64
1.6M System.map-2.6.30-2-amd64
1.6M System.map-2.6.31-1-amd64
86k config-2.6.26-1-amd64
99k config-2.6.30-2-amd64
102k config-2.6.31-1-amd64
4.0M grub
8.4M initrd.img-2.6.26-1-amd64
7.8M initrd.img-2.6.26-1-amd64.bak
9.8M initrd.img-2.6.30-2-amd64
9.8M initrd.img-2.6.30-2-amd64.bak
11M initrd.img-2.6.31-1-amd64
11M initrd.img-2.6.31-1-amd64.bak
1.8M vmlinuz-2.6.26-1-amd64
2.3M vmlinuz-2.6.30-2-amd64
2.5M vmlinuz-2.6.31-1-amd64
max:/backups/nas/system/boot# du -s --si *
238k System.map-2.6.26-1-amd64
291k System.map-2.6.30-2-amd64
308k System.map-2.6.31-1-amd64
25k config-2.6.26-1-amd64
25k config-2.6.30-2-amd64
25k config-2.6.31-1-amd64
1.8M grub
8.4M initrd.img-2.6.26-1-amd64
7.8M initrd.img-2.6.26-1-amd64.bak
9.8M initrd.img-2.6.30-2-amd64
9.8M initrd.img-2.6.30-2-amd64.bak
11M initrd.img-2.6.31-1-amd64
11M initrd.img-2.6.31-1-amd64.bak
1.8M vmlinuz-2.6.26-1-amd64
2.3M vmlinuz-2.6.30-2-amd64
2.5M vmlinuz-2.6.31-1-amd64

so my primary partition is /backups/.nas and then fuse mounts to /backups/nas/ this is looking at /boot directory for an example. Overall using du I have 1.3 compressed compared to 3.1 uncompressed


du --apparent-size -s --si .nas nas
1.3G .nas
3.1G nas

du -s --si .nas nas
1.6G .nas
1.6G nas


Alex


On Thu, Feb 11, 2010 at 07:45:25AM +1100, Alex Samad wrote:
On Wed, Feb 10, 2010 at 03:05:14PM -0500, Matthew Miller wrote:
On Thu, Feb 11, 2010 at 06:44:46AM +1100, Alex Samad wrote:
I'm experimenting with LessFS. It's another fuse-based filesystem, and in
addition to compressing blocks, it checksums each block and only stores
identical blocks once -- "de-duplication". This seems like a particular win
with rdiff-backup, because of the problem with handling of renamed files.
thats nice.... what compression tec does it use

Read about it yourself here: http://www.lessfs.com/wordpress/?page_id=50

In short, it uses a 192-bit hash function (happens to be Tiger) to uniquely
identify each block, and then compresses each block with LZO or QUICKLZ.

had a quick read of the web site, just wondering how effective it would
be with something like rdiff-backup - my line of thinking is that rd
stores the differences, so I would guess all the original files would
benefit, but the differences wouldn't

Also with fusecompress you can specify by mime type which files pass
through ie don't get affected by fusecompress.

I will have to investigate a bit more, run some tests




_______________________________________________
rdiff-backup-users mailing list at rdiff-backup-users < at > nongnu.org
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

Post Suggested fs for placing rdiff-backup data stores 
Nice work Alex.
Although I'm not sure why you are backing up several apparently standard Debian kernels.
If you just backup the /boot/grub dir and have the apt package list the system can be re-installed
to the exact same state with a much smaller backup.

Cheers
Gavin


Alex Samad wrote: Hi

thought I would break this off the original thread

just to give you some example info on fusecompress datastore

max:/backups/nas/system/boot# du --apparent-size -s --si *
1.3M System.map-2.6.26-1-amd64
1.6M System.map-2.6.30-2-amd64
1.6M System.map-2.6.31-1-amd64
86k config-2.6.26-1-amd64
99k config-2.6.30-2-amd64
102k config-2.6.31-1-amd64
4.0M grub
8.4M initrd.img-2.6.26-1-amd64
7.8M initrd.img-2.6.26-1-amd64.bak
9.8M initrd.img-2.6.30-2-amd64
9.8M initrd.img-2.6.30-2-amd64.bak
11M initrd.img-2.6.31-1-amd64
11M initrd.img-2.6.31-1-amd64.bak
1.8M vmlinuz-2.6.26-1-amd64
2.3M vmlinuz-2.6.30-2-amd64
2.5M vmlinuz-2.6.31-1-amd64
max:/backups/nas/system/boot# du -s --si *
238k System.map-2.6.26-1-amd64
291k System.map-2.6.30-2-amd64
308k System.map-2.6.31-1-amd64
25k config-2.6.26-1-amd64
25k config-2.6.30-2-amd64
25k config-2.6.31-1-amd64
1.8M grub
8.4M initrd.img-2.6.26-1-amd64
7.8M initrd.img-2.6.26-1-amd64.bak
9.8M initrd.img-2.6.30-2-amd64
9.8M initrd.img-2.6.30-2-amd64.bak
11M initrd.img-2.6.31-1-amd64
11M initrd.img-2.6.31-1-amd64.bak
1.8M vmlinuz-2.6.26-1-amd64
2.3M vmlinuz-2.6.30-2-amd64
2.5M vmlinuz-2.6.31-1-amd64

so my primary partition is /backups/.nas and then fuse mounts to /backups/nas/ this is looking at /boot directory for an example. Overall using du I have 1.3 compressed compared to 3.1 uncompressed


du --apparent-size -s --si .nas nas
1.3G .nas
3.1G nas

du -s --si .nas nas
1.6G .nas
1.6G nas


Alex


On Thu, Feb 11, 2010 at 07:45:25AM +1100, Alex Samad wrote:
On Wed, Feb 10, 2010 at 03:05:14PM -0500, Matthew Miller wrote:
On Thu, Feb 11, 2010 at 06:44:46AM +1100, Alex Samad wrote:
I'm experimenting with LessFS. It's another fuse-based filesystem, and in
addition to compressing blocks, it checksums each block and only stores
identical blocks once -- "de-duplication". This seems like a particular win
with rdiff-backup, because of the problem with handling of renamed files.
thats nice.... what compression tec does it use
Read about it yourself here: http://www.lessfs.com/wordpress/?page_id=50

In short, it uses a 192-bit hash function (happens to be Tiger) to uniquely
identify each block, and then compresses each block with LZO or QUICKLZ.
had a quick read of the web site, just wondering how effective it would
be with something like rdiff-backup - my line of thinking is that rd
stores the differences, so I would guess all the original files would
benefit, but the differences wouldn't

Also with fusecompress you can specify by mime type which files pass
through ie don't get affected by fusecompress.

I will have to investigate a bit more, run some tests




_______________________________________________
rdiff-backup-users mailing list at rdiff-backup-users < at > nongnu.org ([email]rdiff-backup-users < at > nongnu.org[/email])
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

Post Suggested fs for placing rdiff-backup data stores 
On Thu, Feb 11, 2010 at 02:09:05PM +1000, Gavin wrote:
Nice work Alex.
Although I'm not sure why you are backing up several apparently standard
Debian kernels.
I showed just a subset of the backup, my system backup setup is / down
excluded things like /exports, /proc, /sys etc

If you just backup the /boot/grub dir and have the apt package list the
system can be re-installed
to the exact same state with a much smaller backup.

Cheers
Gavin


[snip]


_______________________________________________
rdiff-backup-users mailing list at rdiff-backup-users < at > nongnu.org
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

Display posts from previous:
Reply to topic Page 1 of 1
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
  


Magic SEO URL for phpBB