Backup Central

Hi,

OS: Solaris8,
Hardware: Sunfire V120

amanda backup is failing regularly.

Following is the report generated.

These dumps were to tape ARCH_DE2_003.
*** A TAPE ERROR OCCURRED: [[writing file: Bad file
number]].
Some dumps may have been left in the holding disk.
Run amflush to flush them to tape.
The next tape Amanda expects to use is: a new tape.

FAILURE AND STRANGE DUMP SUMMARY:
jafar2 /home/host lev 0 FAILED [out of tape]
jafar2 /home/host lev 0 FAILED ["data write:
Broken pipe"]
jafar2 /home/host lev 0 FAILED [dump to tape
failed]
serv5 /raid/sql1_mysql_var lev 0 FAILED [out of
tape]
serv5 /raid/sql1_mysql_var lev 0 FAILED ["data
write: Broken pipe"]
serv5 /raid/sql1_mysql_var lev 0 FAILED [dump to
tape failed]
serv5 /raid/mysql_backup lev 0 FAILED [out of
tape]
serv5 /raid/mysql_backup lev 0 FAILED ["data
write: Broken pipe"]
serv5 /raid/mysql_backup lev 0 FAILED [dump to
tape failed]
serv3 /home/host lev 0 FAILED [out of tape]
serv3 /home/host lev 0 FAILED ["data write:
Broken pipe"]
serv3 /home/host lev 0 FAILED [dump to tape
failed]
serv5 /raid/clring_rpts/Fortisdown lev 0 FAILED
[out of tape]
serv5 /raid/clring_rpts/Fortisdown lev 0 FAILED
["data write: Broken pipe"]
serv5 /raid/clring_rpts/Fortisdown lev 0 FAILED
[dump to tape failed]

STATISTICS:
Total
-------
Estimate Time (hrs:min).. 0:07
Run Time (hrs:min)....... 0:57
Dump Time (hrs:min)...... 0:39
Output Size (GB)......... 4.71
Original Size (GB)....... 4.71
Filesystems Dumped....... 6
Avg Dump Rate (MB/s)..... 2.05
Tape Time (hrs:min)...... 0:32
Tape Size (GB)........... 4.71
Tape Used (%)............ 9.65
Filesystems Taped........ 6
Avg Tp Write Rate (MB/s). 2.52

FAILED AND STRANGE DUMP DETAILS:

/-- jafar2 /home/host lev 0 FAILED ["data write:
Broken pipe"]
sendbackup: start [jafar2:/home/host level 0]
sendbackup: info BACKUP=/usr/sbin/ufsdump
sendbackup: info RECOVER_CMD=/usr/sbin/ufsrestore
-f... -
sendbackup: info end
| DUMP: Writing 32 Kilobyte records
| DUMP: Date of this level 0 dump: Fri Jun 18
00:56:44 2004
| DUMP: Date of last level 0 dump: the epoch
| DUMP: Dumping /dev/rdsk/c0t0d0s7
(jafar2:/home/host) to standard output.
| DUMP: Mapping (Pass I) [regular files]
| DUMP: Mapping (Pass II) [directories]
| DUMP: Estimated 11526716 blocks (5628.28MB) on
0.08 tapes.
| DUMP: Dumping (Pass III) [directories]
| DUMP: Dumping (Pass IV) [regular files]
\--------

/-- serv5 /raid/sql1_mysql_var lev 0 FAILED
["data write: Broken pipe"]
sendbackup: start [serv5:/raid/sql1_mysql_var level 0]
sendbackup: info BACKUP=/usr/sbin/ufsdump
sendbackup: info RECOVER_CMD=/usr/sbin/ufsrestore
-f... -
sendbackup: info end
| DUMP: Writing 32 Kilobyte records
| DUMP: Date of this level 0 dump: Fri Jun 18
01:06:06 2004
| DUMP: Date of last level 0 dump: the epoch
| DUMP: Dumping /dev/rdsk/c2t6d2s0
(serv5:/raid/sql1_mysql_var) to standard output.
| DUMP: Mapping (Pass I) [regular files]
| DUMP: Mapping (Pass II) [directories]
| DUMP: Estimated 9108784 blocks (4447.65MB) on 0.07
tapes.
\--------

/-- serv5 /raid/mysql_backup lev 0 FAILED ["data
write: Broken pipe"]
sendbackup: start [serv5:/raid/mysql_backup level 0]
sendbackup: info BACKUP=/usr/sbin/ufsdump
sendbackup: info RECOVER_CMD=/usr/sbin/ufsrestore
-f... -
sendbackup: info end
| DUMP: Writing 32 Kilobyte records
| DUMP: Date of this level 0 dump: Fri Jun 18
01:06:43 2004
| DUMP: Date of last level 0 dump: the epoch
| DUMP: Dumping /dev/rdsk/c2t6d1s5
(serv5:/raid/mysql_backup) to standard output.
| DUMP: Mapping (Pass I) [regular files]
| DUMP: Mapping (Pass II) [directories]
| DUMP: Estimated 17408044 blocks (8500.02MB) on
0.13 tapes.
| DUMP: Dumping (Pass III) [directories]
\--------

/-- serv3 /home/host lev 0 FAILED ["data write:
Broken pipe"]
sendbackup: start [serv3:/home/host level 0]
sendbackup: info BACKUP=/usr/sbin/ufsdump
sendbackup: info RECOVER_CMD=/usr/sbin/ufsrestore
-f... -
sendbackup: info end
| DUMP: Writing 32 Kilobyte records
| DUMP: Date of this level 0 dump: Fri Jun 18
01:07:04 2004
| DUMP: Date of last level 0 dump: the epoch
| DUMP: Dumping /dev/rdsk/c0t0d0s7
(serv3a:/home/host) to standard output.
| DUMP: Mapping (Pass I) [regular files]
| DUMP: Mapping (Pass II) [directories]
| DUMP: Estimated 9341590 blocks (4561.32MB) on 0.07
tapes.
| DUMP: Dumping (Pass III) [directories]
\--------

/-- serv5 /raid/clring_rpts/Fortisdown lev 0
FAILED ["data write: Broken pipe"]
sendbackup: start [serv5:/raid/clring_rpts/Fortisdown
level 0]
sendbackup: info BACKUP=/usr/sbin/ufsdump
sendbackup: info RECOVER_CMD=/usr/sbin/ufsrestore
-f... -
sendbackup: info end
| DUMP: Writing 32 Kilobyte records
| DUMP: Date of this level 0 dump: Fri Jun 18
01:07:15 2004
| DUMP: Date of last level 0 dump: the epoch
| DUMP: Dumping /dev/rdsk/c2t6d2s1
(serv5:/raid/clring_rpts) to standard output.
| DUMP: Mapping (Pass I) [regular files]
| DUMP: Mapping (Pass II) [directories]
| DUMP: Estimated 5134274 blocks (2506.97MB) on 0.04
tapes.
\--------

NOTES:
planner: Last full dump of serv3:/ on tape
ARCH_DE2_000 overwritten in 1 run.
planner: Last full dump of serv3:/home/host on tape
overwritten in 1 run.
planner: Last full dump of serv3:/usr on tape
ARCH_DE2_000 overwritten in 1 run.
planner: Last full dump of serv3:/var on tape
ARCH_DE2_000 overwritten in 1 run.
planner: Last full dump of serv5:/raid/mysql_backup
on tape overwritten in 1 run.
planner: Last full dump of serv5:/raid/sql1_mysql_var
on tape overwritten in 1 run.
planner: Last full dump of
serv5:/raid/clring_rpts/Fortisdown on tape
overwritten in 1 run.
planner: Last full dump of
serv5:/raid/clring_rpts/EurexReports on tape
ARCH_DE2_000 overwritten in 1 run.
planner: Last full dump of
serv5:/raid/clring_rpts/XetraReports on tape
ARCH_DE2_000 overwritten in 1 run.
planner: Last full dump of
serv5:/raid/clring_rpts/XetraRpts on tape overwritten
in 1 run.
planner: Last full dump of jafar2:/home/host on tape
overwritten in 1 run.
taper: tape ARCH_DE2_003 kb 6878912 fm 7 writing
file: I/O error

DUMP SUMMARY:
DUMPER STATS
TAPER STATS
HOSTNAME DISK L ORIG-MB OUT-MB mm:ss
MB/s mm:ss MB/s
-------------------------- ---------------------------
------------
jafar2 /home/host 0 FAILED
---------------------------------
serv3 / 0 44 44 0:19
2.3 0:04 11.2
serv3 /home/host 0 FAILED
---------------------------------
serv3 /usr 0 380 380 2:54
2.2 0:30 12.5
serv3 /var 0 20 20 0:12
1.7 0:07 2.7
serv5 -rexReports 0 1010 1010 6:20
2.7 1:28 11.4
serv5 -Fortisdown 0 FAILED
---------------------------------
serv5 -traReports 0 2996 2996 27:26
1.8 27:27 1.8
serv5 -/XetraRpts 0 375 375 2:04
3.0 2:16 2.7
serv5 -sql_backup 0 FAILED
---------------------------------
serv5 -_mysql_var 0 FAILED
---------------------------------







__________________________________
Do you Yahoo!?
Yahoo! Mail - Helps protect you from nasty viruses.
http://promotions.yahoo.com/new_mail

Sal Sharief wrote:

FAILURE AND STRANGE DUMP SUMMARY:
jafar2 /home/host lev 0 FAILED [out of tape]

It says "out of tape", but a tape error is more likely.

...
Tape Size (GB)........... 4.71
Tape Used (%)............ 9.65
Filesystems Taped........ 6

Am I correct that your tapesize is configured to be about 45 Gbyte.
(What kind of tape is that, I've never seen a tape with
such a capacity.)

...
taper: tape ARCH_DE2_003 kb 6878912 fm 7 writing
file: I/O error

While writing the 7th backup image to tape, there was an IO error
around 6878912 Kbyte or about 6.6 Gbyte. That seems to be far
from the expected capacity of 45 Gbyte, so it's probably a tape
error.
Try a cleaning tape.
Look for scsi errors in /var/adm/messages.

(PS. what version of amanda has only one column in the
statistics section? Must be quite old.)


--
Paul Bijnens, Xplanation Tel +32 16 397.511
Technologielaan 21 bus 2, B-3001 Leuven, BELGIUM Fax +32 16 397.512
http://www.xplanation.com/ email: Paul.Bijnens < at > xplanation.com
***********************************************************************
* I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, F6, *
* quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, *
* stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, *
* PF4, F20, ^X^X, :, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, *
* kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... *
* ... "Are you sure?" ... YES ... Phew ... I'm out *
***********************************************************************

--On Friday, June 18, 2004 00:36:43 -0700 Sal Sharief <mastersalx < at > yahoo.com> wrote:

Hi,

OS: Solaris8,
Hardware: Sunfire V120

amanda backup is failing regularly.

Following is the report generated.

These dumps were to tape ARCH_DE2_003.
*** A TAPE ERROR OCCURRED: [[writing file: Bad file
number]].
Some dumps may have been left in the holding disk.
Run amflush to flush them to tape.
The next tape Amanda expects to use is: a new tape.

FAILURE AND STRANGE DUMP SUMMARY:
jafar2 /home/host lev 0 FAILED [out of tape]
jafar2 /home/host lev 0 FAILED ["data write:
Broken pipe"]
jafar2 /home/host lev 0 FAILED [dump to tape
failed]
serv5 /raid/sql1_mysql_var lev 0 FAILED [out of
tape]
serv5 /raid/sql1_mysql_var lev 0 FAILED ["data
write: Broken pipe"]
serv5 /raid/sql1_mysql_var lev 0 FAILED [dump to
tape failed]
serv5 /raid/mysql_backup lev 0 FAILED [out of
tape]
serv5 /raid/mysql_backup lev 0 FAILED ["data
write: Broken pipe"]
serv5 /raid/mysql_backup lev 0 FAILED [dump to
tape failed]
serv3 /home/host lev 0 FAILED [out of tape]
serv3 /home/host lev 0 FAILED ["data write:
Broken pipe"]
serv3 /home/host lev 0 FAILED [dump to tape
failed]
serv5 /raid/clring_rpts/Fortisdown lev 0 FAILED
[out of tape]
serv5 /raid/clring_rpts/Fortisdown lev 0 FAILED
["data write: Broken pipe"]
serv5 /raid/clring_rpts/Fortisdown lev 0 FAILED
[dump to tape failed]

STATISTICS:
Total
-------
Estimate Time (hrs:min).. 0:07
Run Time (hrs:min)....... 0:57
Dump Time (hrs:min)...... 0:39
Output Size (GB)......... 4.71
Original Size (GB)....... 4.71
Filesystems Dumped....... 6
Avg Dump Rate (MB/s)..... 2.05
Tape Time (hrs:min)...... 0:32
Tape Size (GB)........... 4.71
Tape Used (%)............ 9.65
Filesystems Taped........ 6
Avg Tp Write Rate (MB/s). 2.52

FAILED AND STRANGE DUMP DETAILS:

/-- jafar2 /home/host lev 0 FAILED ["data write:
Broken pipe"]
sendbackup: start [jafar2:/home/host level 0]
sendbackup: info BACKUP=/usr/sbin/ufsdump
sendbackup: info RECOVER_CMD=/usr/sbin/ufsrestore
-f... -
sendbackup: info end
| DUMP: Writing 32 Kilobyte records
| DUMP: Date of this level 0 dump: Fri Jun 18
00:56:44 2004
| DUMP: Date of last level 0 dump: the epoch
| DUMP: Dumping /dev/rdsk/c0t0d0s7
(jafar2:/home/host) to standard output.
| DUMP: Mapping (Pass I) [regular files]
| DUMP: Mapping (Pass II) [directories]
| DUMP: Estimated 11526716 blocks (5628.28MB) on
0.08 tapes.
| DUMP: Dumping (Pass III) [directories]
| DUMP: Dumping (Pass IV) [regular files]
\--------

/-- serv5 /raid/sql1_mysql_var lev 0 FAILED
["data write: Broken pipe"]
sendbackup: start [serv5:/raid/sql1_mysql_var level 0]
sendbackup: info BACKUP=/usr/sbin/ufsdump
sendbackup: info RECOVER_CMD=/usr/sbin/ufsrestore
-f... -
sendbackup: info end
| DUMP: Writing 32 Kilobyte records
| DUMP: Date of this level 0 dump: Fri Jun 18
01:06:06 2004
| DUMP: Date of last level 0 dump: the epoch
| DUMP: Dumping /dev/rdsk/c2t6d2s0
(serv5:/raid/sql1_mysql_var) to standard output.
| DUMP: Mapping (Pass I) [regular files]
| DUMP: Mapping (Pass II) [directories]
| DUMP: Estimated 9108784 blocks (4447.65MB) on 0.07
tapes.
\--------

/-- serv5 /raid/mysql_backup lev 0 FAILED ["data
write: Broken pipe"]
sendbackup: start [serv5:/raid/mysql_backup level 0]
sendbackup: info BACKUP=/usr/sbin/ufsdump
sendbackup: info RECOVER_CMD=/usr/sbin/ufsrestore
-f... -
sendbackup: info end
| DUMP: Writing 32 Kilobyte records
| DUMP: Date of this level 0 dump: Fri Jun 18
01:06:43 2004
| DUMP: Date of last level 0 dump: the epoch
| DUMP: Dumping /dev/rdsk/c2t6d1s5
(serv5:/raid/mysql_backup) to standard output.
| DUMP: Mapping (Pass I) [regular files]
| DUMP: Mapping (Pass II) [directories]
| DUMP: Estimated 17408044 blocks (8500.02MB) on
0.13 tapes.
| DUMP: Dumping (Pass III) [directories]
\--------

/-- serv3 /home/host lev 0 FAILED ["data write:
Broken pipe"]
sendbackup: start [serv3:/home/host level 0]
sendbackup: info BACKUP=/usr/sbin/ufsdump
sendbackup: info RECOVER_CMD=/usr/sbin/ufsrestore
-f... -
sendbackup: info end
| DUMP: Writing 32 Kilobyte records
| DUMP: Date of this level 0 dump: Fri Jun 18
01:07:04 2004
| DUMP: Date of last level 0 dump: the epoch
| DUMP: Dumping /dev/rdsk/c0t0d0s7
(serv3a:/home/host) to standard output.
| DUMP: Mapping (Pass I) [regular files]
| DUMP: Mapping (Pass II) [directories]
| DUMP: Estimated 9341590 blocks (4561.32MB) on 0.07
tapes.
| DUMP: Dumping (Pass III) [directories]
\--------

/-- serv5 /raid/clring_rpts/Fortisdown lev 0
FAILED ["data write: Broken pipe"]
sendbackup: start [serv5:/raid/clring_rpts/Fortisdown
level 0]
sendbackup: info BACKUP=/usr/sbin/ufsdump
sendbackup: info RECOVER_CMD=/usr/sbin/ufsrestore
-f... -
sendbackup: info end
| DUMP: Writing 32 Kilobyte records
| DUMP: Date of this level 0 dump: Fri Jun 18
01:07:15 2004
| DUMP: Date of last level 0 dump: the epoch
| DUMP: Dumping /dev/rdsk/c2t6d2s1
(serv5:/raid/clring_rpts) to standard output.
| DUMP: Mapping (Pass I) [regular files]
| DUMP: Mapping (Pass II) [directories]
| DUMP: Estimated 5134274 blocks (2506.97MB) on 0.04
tapes.
\--------

NOTES:
planner: Last full dump of serv3:/ on tape
ARCH_DE2_000 overwritten in 1 run.
planner: Last full dump of serv3:/home/host on tape
overwritten in 1 run.
planner: Last full dump of serv3:/usr on tape
ARCH_DE2_000 overwritten in 1 run.
planner: Last full dump of serv3:/var on tape
ARCH_DE2_000 overwritten in 1 run.
planner: Last full dump of serv5:/raid/mysql_backup
on tape overwritten in 1 run.
planner: Last full dump of serv5:/raid/sql1_mysql_var
on tape overwritten in 1 run.
planner: Last full dump of
serv5:/raid/clring_rpts/Fortisdown on tape
overwritten in 1 run.
planner: Last full dump of
serv5:/raid/clring_rpts/EurexReports on tape
ARCH_DE2_000 overwritten in 1 run.
planner: Last full dump of
serv5:/raid/clring_rpts/XetraReports on tape
ARCH_DE2_000 overwritten in 1 run.
planner: Last full dump of
serv5:/raid/clring_rpts/XetraRpts on tape overwritten
in 1 run.
planner: Last full dump of jafar2:/home/host on tape
overwritten in 1 run.

You did notice the above messages from planner that unless you
add a new tape you are going to lose your only level 0's of
all these filesystems on your next run?


taper: tape ARCH_DE2_003 kb 6878912 fm 7 writing
file: I/O error

Someone else already mentioned possible causes for your tape
error.

Frank


DUMP SUMMARY:
DUMPER STATS
TAPER STATS
HOSTNAME DISK L ORIG-MB OUT-MB mm:ss
MB/s mm:ss MB/s
-------------------------- ---------------------------
------------
jafar2 /home/host 0 FAILED
---------------------------------
serv3 / 0 44 44 0:19
2.3 0:04 11.2
serv3 /home/host 0 FAILED
---------------------------------
serv3 /usr 0 380 380 2:54
2.2 0:30 12.5
serv3 /var 0 20 20 0:12
1.7 0:07 2.7
serv5 -rexReports 0 1010 1010 6:20
2.7 1:28 11.4
serv5 -Fortisdown 0 FAILED
---------------------------------
serv5 -traReports 0 2996 2996 27:26
1.8 27:27 1.8
serv5 -/XetraRpts 0 375 375 2:04
3.0 2:16 2.7
serv5 -sql_backup 0 FAILED
---------------------------------
serv5 -_mysql_var 0 FAILED
---------------------------------







__________________________________
Do you Yahoo!?
Yahoo! Mail - Helps protect you from nasty viruses.
http://promotions.yahoo.com/new_mail



--
Frank Smith fsmith < at > hoovers.com
Sr. Systems Administrator Voice: 512-374-4673
Hoover's Online Fax: 512-374-4501

--- Paul Bijnens <paul.bijnens < at > xplanation.com> wrote:
Sal Sharief wrote:

FAILURE AND STRANGE DUMP SUMMARY:
jafar2 /home/host lev 0 FAILED [out of tape]

It says "out of tape", but a tape error is more
likely.

....
Tape Size (GB)........... 4.71
Tape Used (%)............ 9.65
Filesystems Taped........ 6

Am I correct that your tapesize is configured to be
about 45 Gbyte.
(What kind of tape is that, I've never seen a tape
with
such a capacity.)

....
taper: tape ARCH_DE2_003 kb 6878912 fm 7 writing
file: I/O error

While writing the 7th backup image to tape, there
was an IO error
around 6878912 Kbyte or about 6.6 Gbyte. That seems
to be far
from the expected capacity of 45 Gbyte, so it's
probably a tape
error.
Try a cleaning tape.
Look for scsi errors in /var/adm/messages.

--snip from /var/adm/messages ----
Jun 17 20:38:57 serv5c scsi: [ID 107833 kern.notice]
/pci < at > 1f,0/pci < at > 1/scsi < at > 8,1/st < at > 4,0 (st11):
Jun 17 20:38:57 serv5c Fixed record length (1024 byte
blocks) I/O
Jun 17 20:39:12 serv5c scsi: [ID 193665 kern.info]
sgen26 at glm1: target a lun 0
Jun 17 20:39:12 serv5c genunix: [ID 936769 kern.info]
sgen26 is /pci < at > 1f,0/pci < at > 1/scsi < at > 8,1/sgen < at > a,0
Jun 18 01:04:52 serv5c scsi: [ID 107833 kern.warning]
WARNING: /pci < at > 1f,0/pci < at > 1/scsi < at > 8,1/st < at > 4,0 (st11):
Jun 18 01:04:52 serv5c Error for Command: write
Error Level: Fatal
Jun 18 01:04:52 serv5c scsi: [ID 107833 kern.notice]
Requested Block: 1938880 Error
Block: 1938880
Jun 18 01:04:52 serv5c scsi: [ID 107833 kern.notice]
Vendor: EXABYTE Serial
Number: 8E002106
Jun 18 01:04:52 serv5c scsi: [ID 107833 kern.notice]
Sense Key: Media Error
Jun 18 01:04:52 serv5c scsi: [ID 107833 kern.notice]
ASC: 0xc (write error), ASCQ: 0x0, FRU: 0x0

-- end snip --

I have a 15 slot Exabyte 215M tape library with
225m AME with SmartClean cartridges. I have tried to
run backups on more than 3 tapes, and got the same
error. These tapes are approx. four months old.

I went ahead and ordered a cleaning cartridge. Exabyte
claimed that I wouldn't need a cleaning tape since the
media comes with smart clean automatically. Now they
backed out of their claim after creating an incident
report. Go figure.


(PS. what version of amanda has only one column in
the
statistics section? Must be quite old.)

Its an amanda version 2.4.2p2 - modified to show data
in MB and minus other stats.



--
Paul Bijnens, Xplanation
Tel +32 16 397.511
Technologielaan 21 bus 2, B-3001 Leuven, BELGIUM
Fax +32 16 397.512
http://www.xplanation.com/ email:
Paul.Bijnens < at > xplanation.com




__________________________________
Do you Yahoo!?
Yahoo! Mail - You care about security. So do we.
http://promotions.yahoo.com/new_mail