Backup Central

One thing I've been pondering recently is rdiff-backup's error handling.
This isn't a complaint - I'm a big fan of rdiff-backup and use it every
day (well, night actually) - but the error handling could be better in
my opinion. Current behaviour is a somewhat unfriendly Python stack dump
in which is it often not immediately obvious what the problem is.
Moreover, if the program was running with a low or zero verbosity
setting, it is usually necessary to run it again "v5 or higher" to
establish the file causing the problem.

Has anyone looked at providing an umbrella exception handling routine
for rdiff-backup? I would envisage something that does the following:

- hides the stack dump (possibly writing it to a file) from the user,
although the final line could be used
- identifies the file being backed up at the time of the error
- perhaps provides the command line used
- gives a summary of the error

Not entirely ideal, but maybe something like:

Oct 23 03:05:01 rdiff-backup failed:
Error: Truncated header string (problem probably originated remotely)
File: /home/fred/widgets.c
Command line: rdiff-backup /home remote::/backup/home
Details: /home/rdiff-backup/errorlog/1.txt

I don't think this is quite as easy to do as I may have implied above -
I've conveniently left some questions unanswered - but I think it is
worthy of consideration for the future.

Keith

--
Keith Edmunds

+---------------------------------------------------------------------+
| Tiger Computing Ltd | Helping businesses make the most of Linux |
| "The Linux Company" | http://www.TheLinuxConsultancy.co.uk |
+---------------------------------------------------------------------+

Keith Edmunds <keith < at > midnighthax.com>
wrote the following on Tue, 25 Oct 2005 08:10:45 +0100
One thing I've been pondering recently is rdiff-backup's error handling.
This isn't a complaint - I'm a big fan of rdiff-backup and use it every
day (well, night actually) - but the error handling could be better in
my opinion. Current behaviour is a somewhat unfriendly Python stack dump
in which is it often not immediately obvious what the problem is.
Moreover, if the program was running with a low or zero verbosity
setting, it is usually necessary to run it again "v5 or higher" to
establish the file causing the problem.

Has anyone looked at providing an umbrella exception handling routine
for rdiff-backup? I would envisage something that does the following:

- hides the stack dump (possibly writing it to a file) from the user,
although the final line could be used
- identifies the file being backed up at the time of the error
- perhaps provides the command line used
- gives a summary of the error

Not entirely ideal, but maybe something like:

Oct 23 03:05:01 rdiff-backup failed:
Error: Truncated header string (problem probably originated remotely)
File: /home/fred/widgets.c
Command line: rdiff-backup /home remote::/backup/home
Details: /home/rdiff-backup/errorlog/1.txt

I don't think this is quite as easy to do as I may have implied above -
I've conveniently left some questions unanswered - but I think it is
worthy of consideration for the future.

Yes, those error reports are a pain, and could definitely be more
friendly. However, the reason things look like that has to do with
rdiff-backup's remote operations. Basically, rdiff-backup set's up a
mini RPC-like system, where each side can execute arbitrary functions
on the other (modulo the security stuff). If one of these functions
raises an exception, the exception will be propagated to the other
side where it may be handled. So when an exception first occurs, the
computer which raises the exception may not know whether the exception
will be handled.

Anyway unhandled exceptions (which causes crashes) often bounce from
computer to computer until they eventually reach the top level.
Unless you run at -v5 or higher, the other sides don't log the
traceback information, so that is lost, so often the final side can
only say "the other guy gave me this exception". It seemed like
overkill for every side to log every exception, since only a small
percentage of these lead to crashes.

Hmm I just thought of a possible solution though.. When one side
raises an exception, maybe it should send the traceback information as
well as the exception itself to the other side.. Then when the
original side reraises the exception, it should also glue the
traceback information its given to its own traceback. Then when
finally the exception gets to the top it should have all the traceback
information from both sides...


--
Ben Escoto

On 10/27/05, Ben Escoto <ben < at > emerose.org> wrote:
Keith Edmunds <keith < at > midnighthax.com>
wrote the following on Tue, 25 Oct 2005 08:10:45 +0100
One thing I've been pondering recently is rdiff-backup's error handling.
This isn't a complaint - I'm a big fan of rdiff-backup and use it every
day (well, night actually) - but the error handling could be better in
my opinion. Current behaviour is a somewhat unfriendly Python stack dump
in which is it often not immediately obvious what the problem is.
Moreover, if the program was running with a low or zero verbosity
setting, it is usually necessary to run it again "v5 or higher" to
establish the file causing the problem.

Has anyone looked at providing an umbrella exception handling routine
for rdiff-backup? I would envisage something that does the following:

- hides the stack dump (possibly writing it to a file) from the user,
although the final line could be used
- identifies the file being backed up at the time of the error
- perhaps provides the command line used
- gives a summary of the error

Not entirely ideal, but maybe something like:

Oct 23 03:05:01 rdiff-backup failed:
Error: Truncated header string (problem probably originated remotely)
File: /home/fred/widgets.c
Command line: rdiff-backup /home remote::/backup/home
Details: /home/rdiff-backup/errorlog/1.txt

I don't think this is quite as easy to do as I may have implied above -
I've conveniently left some questions unanswered - but I think it is
worthy of consideration for the future.

Yes, those error reports are a pain, and could definitely be more
friendly. However, the reason things look like that has to do with
rdiff-backup's remote operations. Basically, rdiff-backup set's up a
mini RPC-like system, where each side can execute arbitrary functions
on the other (modulo the security stuff). If one of these functions
raises an exception, the exception will be propagated to the other
side where it may be handled. So when an exception first occurs, the
computer which raises the exception may not know whether the exception
will be handled.

Anyway unhandled exceptions (which causes crashes) often bounce from
computer to computer until they eventually reach the top level.
Unless you run at -v5 or higher, the other sides don't log the
traceback information, so that is lost, so often the final side can
only say "the other guy gave me this exception". It seemed like
overkill for every side to log every exception, since only a small
percentage of these lead to crashes.

Hmm I just thought of a possible solution though.. When one side
raises an exception, maybe it should send the traceback information as
well as the exception itself to the other side.. Then when the
original side reraises the exception, it should also glue the
traceback information its given to its own traceback. Then when
finally the exception gets to the top it should have all the traceback
information from both sides...


--
Ben Escoto


I don't know the answer, but I will say that the current error
handling is rdiff-backup's biggest shortcoming in my opinion. Anytime
I recommend rdiff-backup, I always warn the potential new user that if
they are not comfortable seeing stack dumps, they will get very
nervious if rdiff-backup ever has a hiccup on their system.

Greg
--
Greg Freemyer
The Norcross Group
Forensics for the 21st Century

Greg Freemyer wrote:
I don't know the answer, but I will say that the current error
handling is rdiff-backup's biggest shortcoming in my opinion. Anytime
I recommend rdiff-backup, I always warn the potential new user that if
they are not comfortable seeing stack dumps, they will get very
nervious if rdiff-backup ever has a hiccup on their system.


I don't disagree with you Greg, but the behaviour of rdiff-backup is
pretty standard for every other open source python program I have used.
I mean if you want to see ugly tracebacks, just try and upgrade Plone or
do anything else with it :)

dave

Greg Freemyer wrote:
I don't know the answer, but I will say that the current error
handling is rdiff-backup's biggest shortcoming in my opinion. Anytime
I recommend rdiff-backup, I always warn the potential new user that if
they are not comfortable seeing stack dumps, they will get very
nervious if rdiff-backup ever has a hiccup on their system.

If I may comment as well, I don't mind the stackdump at all. I see rdiff-backup
mainly as a system-admin's or experienced user's tool, meaning the people most
likely to use it are comfortable with stackdumps. If you're a Linux-newbie, you
wouldn't start using rdiff-backup... In my opinion, it's not necessary to fix
it. If it ain't broke, don't fix it.

And BTW, I think by saying that the error-handling is rdiff-backup's greatest
shortcoming, is actually a compliment on that which it is supposed to do: backup

Greg Freemyer <greg.freemyer < at > gmail.com>
wrote the following on Thu, 27 Oct 2005 17:47:08 -0400

I don't know the answer, but I will say that the current error
handling is rdiff-backup's biggest shortcoming in my opinion. Anytime
I recommend rdiff-backup, I always warn the potential new user that if
they are not comfortable seeing stack dumps, they will get very
nervious if rdiff-backup ever has a hiccup on their system.

I agree that people should rarely if ever see a traceback. But when
would a new user see a traceback? Personally, I haven't had a stable
version of rdiff-backup crash on me in over a year (except user abort
and failend connection). Of course I'm a biased sample - if it
doesn't on my machine I wouldn't call it a stable version :)

But anyway tracebacks are supposed to be rare, and not caused by user
error. So when one happens it may be a bug in rdiff-backup, or at
least a failure of some part of the system. If there is a bug in
rdiff-backup, I like having a lot of technical information spew out,
because it makes my job easier.

So what kind of tracebacks are people seeing regularly? If I can
reproduce it on my system it should be easy to fix.


--
Ben Escoto

Ben Escoto wrote:
So what kind of tracebacks are people seeing regularly? If I can
reproduce it on my system it should be easy to fix.

Ben, the most common traceback I see is that caused by a failure in the
underlying network (some of my customers connect via ADSL and it does
sometimes drop the connection).

Keith

--
Keith Edmunds

+---------------------------------------------------------------------+
| Tiger Computing Ltd | Helping businesses make the most of Linux |
| "The Linux Company" | http://www.TheLinuxConsultancy.co.uk |
+---------------------------------------------------------------------+