Backup Central

I'm currently working on the early stages of a fuse filesystem for
rdiff-backup repositories. As I look through the rdiff-backup code I
noticed that it is written around the command line interface
(unsurprisingly). As I would like to use the rdiff-backup python
packages directly, rather than forking a subprocess each time the fuse
fs needs data, I wanted to get the developers' thoughts on the API of
rdiff-backup. Are the rdiff-backup packages used by rdiff-backup's
Main.py likely to remain more or less stable, or should I expect a lot
of churn in how they are implemented?

While tinkering with generating listings (for 'ls') I found it would
be a lot more efficient to be able to do a non-recursive
restore.ListAtTime(). Specifically in
MirrorStruct.get_rorp_iter_from_rf(). For example something like:

def get_rorp_iter_from_rt(self, rf, recurse=True):

would do the trick and requires no other changes to rdiff-backup.
Perhaps the recurse option should be available higher up the stack as
well, perhaps in restore.ListAtTime() itself, but the idea is the
same. Would the developers be amenable to such a change? I'm sure I'll
run into several more similar sorts of tweaks that would make the
rdiff-backup packages more usable as a general purpose rdiff-backup
repository access library, and I wanted to get a feel for how likely
it will be for such changes to make it into rdiff-backup proper.

Thanks,

--
Darren Hart

_______________________________________________
rdiff-backup-users mailing list at rdiff-backup-users < at > nongnu.org
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

Hi Darren,

Are you aware that there is already a rdiff-backup FUSE plugin?

http://code.google.com/p/archfs/

Cheers,
Anthony

I'm currently working on the early stages of a fuse filesystem for
rdiff-backup repositories. As I look through the rdiff-backup code I
noticed that it is written around the command line interface
(unsurprisingly). As I would like to use the rdiff-backup python
packages directly, rather than forking a subprocess each time the fuse
fs needs data, I wanted to get the developers' thoughts on the API of
rdiff-backup. Are the rdiff-backup packages used by rdiff-backup's
Main.py likely to remain more or less stable, or should I expect a lot
of churn in how they are implemented?

While tinkering with generating listings (for 'ls') I found it would
be a lot more efficient to be able to do a non-recursive
restore.ListAtTime(). Specifically in
MirrorStruct.get_rorp_iter_from_rf(). For example something like:

def get_rorp_iter_from_rt(self, rf, recurse=True):

would do the trick and requires no other changes to rdiff-backup.
Perhaps the recurse option should be available higher up the stack as
well, perhaps in restore.ListAtTime() itself, but the idea is the
same. Would the developers be amenable to such a change? I'm sure I'll
run into several more similar sorts of tweaks that would make the
rdiff-backup packages more usable as a general purpose rdiff-backup
repository access library, and I wanted to get a feel for how likely
it will be for such changes to make it into rdiff-backup proper.

Thanks,

--
Darren Hart

_______________________________________________
rdiff-backup-users mailing list at rdiff-backup-users < at > nongnu.org
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

On Tue, Jun 22, 2010 at 10:07 AM, Anthony Toole <artoole < at > gmail.com> wrote:
Hi Darren,

Are you aware that there is already a rdiff-backup FUSE plugin?

http://code.google.com/p/archfs/

I had no idea! This is excellent, I'll have a look and see how it
does. Thanks for the link - I'm glad I posted early


Cheers,
Anthony

I'm currently working on the early stages of a fuse filesystem for
rdiff-backup repositories. As I look through the rdiff-backup code I
noticed that it is written around the command line interface
(unsurprisingly). As I would like to use the rdiff-backup python
packages directly, rather than forking a subprocess each time the fuse
fs needs data, I wanted to get the developers' thoughts on the API of
rdiff-backup. Are the rdiff-backup packages used by rdiff-backup's
Main.py likely to remain more or less stable, or should I expect a lot
of churn in how they are implemented?

While tinkering with generating listings (for 'ls') I found it would
be a lot more efficient to be able to do a non-recursive
restore.ListAtTime(). Specifically in
MirrorStruct.get_rorp_iter_from_rf(). For example something like:

def get_rorp_iter_from_rt(self, rf, recurse=True):

would do the trick and requires no other changes to rdiff-backup.
Perhaps the recurse option should be available higher up the stack as
well, perhaps in restore.ListAtTime() itself, but the idea is the
same. Would the developers be amenable to such a change? I'm sure I'll
run into several more similar sorts of tweaks that would make the
rdiff-backup packages more usable as a general purpose rdiff-backup
repository access library, and I wanted to get a feel for how likely
it will be for such changes to make it into rdiff-backup proper.

Thanks,

--
Darren Hart




--
Darren Hart

_______________________________________________
rdiff-backup-users mailing list at rdiff-backup-users < at > nongnu.org
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

On 22/06/2010 19:28, Darren Hart wrote:
On Tue, Jun 22, 2010 at 10:07 AM, Anthony Toole<artoole < at > gmail.com> wrote:

Hi Darren,

Are you aware that there is already a rdiff-backup FUSE plugin?

http://code.google.com/p/archfs/

I had no idea! This is excellent, I'll have a look and see how it
does. Thanks for the link - I'm glad I posted early


When I last used archfs (at v 0.5.3) it worked fine for small
repositories (say 100 files) but for large ones (say 5000 files) it was
very slow and liable to crash the server. Has anyone got any better
experience with it? It's a really neat concept and if someone could make
it faster and reliable, it would be great.

Dominic

_______________________________________________
rdiff-backup-users mailing list at rdiff-backup-users < at > nongnu.org
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

On Mon, Jun 28, 2010 at 2:47 AM, Dominic Raferd <dominic < at > timedicer.co.uk> wrote:
On 22/06/2010 19:28, Darren Hart wrote:

On Tue, Jun 22, 2010 at 10:07 AM, Anthony Toole<artoole < at > gmail.com>  wrote:


Hi Darren,

Are you aware that there is already a rdiff-backup FUSE plugin?

http://code.google.com/p/archfs/


I had no idea! This is excellent, I'll have a look and see how it
does. Thanks for the link - I'm glad I posted early



When I last used archfs (at v 0.5.3) it worked fine for small repositories
(say 100 files) but for large ones (say 5000 files) it was very slow and
liable to crash the server. Has anyone got any better experience with it?
It's a really neat concept and if someone could make it faster and reliable,
it would be great.

That still appears to be the case. I gave it a try on my complete
system backup and it invoked the oom killer it short order. Perhaps
there are some options for the amount of cache to use, etc. But I
think this indicates that it has different goals. I'm not
particularly interested in the sort of performance gains I'd get from
caching the files, etc. What I'm looking to do is occasionally mount
the backup repository, browse for a single file, and restore it from
some time in the past, then unmount the repo.

I think my approach of building the directory listings from the
responses provided by the rdiff-backup API is a fine way to do that.
Each command has some latency involved, but it should be adequate.

So my original questions stands - what do the rdiff-backup developers
think about the API, is it likely to remain relatively stable? What
about the recursive option I asked about, is such a patch likely to be
accepted?

--
Darren


Dominic

_______________________________________________
rdiff-backup-users mailing list at rdiff-backup-users < at > nongnu.org
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL:
http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki




--
Darren Hart

_______________________________________________
rdiff-backup-users mailing list at rdiff-backup-users < at > nongnu.org
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

On 28/06/2010 14:22, Darren Hart wrote:
...What I'm looking to do is occasionally mount
the backup repository, browse for a single file, and restore it from
some time in the past, then unmount the repo.

I think my approach of building the directory listings from the
responses provided by the rdiff-backup API is a fine way to do that.
Each command has some latency involved, but it should be adequate.

So my original questions stands - what do the rdiff-backup developers
think about the API, is it likely to remain relatively stable? What
about the recursive option I asked about, is such a patch likely to be
accepted?

--
Darren
Not answering your direct question (because I can't), but if you want to
recover individual files the best I have found is Josh's rdiffWeb
http://www.rdiffweb.org/. As the name implies it uses a web interface
and although the initial configuration can be a bit tricky, once set up
it works well and makes retrieving individual files, including versions
from the past, a snap.

Dominic

_______________________________________________
rdiff-backup-users mailing list at rdiff-backup-users < at > nongnu.org
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki