Backup Central

Hi!

I'm trying to use amanda with HP Storageworks MSI5030 library.

It uses C7971A tpaes (100GB/200GB), but I can't find any amanda
tapetypes in th Net. So, I ran amtapetype. It was working for
about 12 hours with the folloeing results:

$ /usr/sbin/amtapetype -f /dev/nst0 -t "HP-C7971A"
Writing 1024 Mbyte compresseable data: 26 sec
Writing 1024 Mbyte uncompresseable data: 71 sec
WARNING: Tape drive has hardware compression enabled
Estimated time to write 2 * 1024 Mbyte: 142 sec = 0 h 2 min
wrote 3174843 32Kb blocks in 9709 files in 46846 seconds (short write)
wrote 847763 32Kb blocks in 5201 files in 20794 seconds (short write)
define tapetype HP-C7971A {
comment "just produced by tapetype prog (hardware compression on)"
length 1107 gbytes
filemark 74466560 kbytes
speed 1736 kps
}
amtapetype: could not rewind /dev/nst0: Input/output error

I think length and filemark are too large for this tapes. Does
anybody uses same tapes with amanda?

P.S. Right now I figured how to turn HW compression off (with mt
utility) and started amtapetype again...

--
Regards, Alexey I. Froloff
AIF5-RIPN, AIF5-RIPE
-------------------------------------------
Inform-Mobil, Ltd. System Administrator
http://www.inform-mobil.ru/
Tel: +7(095)504-4709, Fax: +7(095)513-1006

Alexey I. Froloff wrote:

I'm trying to use amanda with HP Storageworks MSI5030 library.

It uses C7971A tpaes (100GB/200GB), but I can't find any amanda
tapetypes in th Net. So, I ran amtapetype. It was working for
about 12 hours with the folloeing results:

It runs much faster if you give it a realistic capacity estimate,
as the manual page warns you. Something like:
amtapetape -e 100g -f /dev/nst0 -t "HP-C7971A"



$ /usr/sbin/amtapetype -f /dev/nst0 -t "HP-C7971A"
Writing 1024 Mbyte compresseable data: 26 sec
Writing 1024 Mbyte uncompresseable data: 71 sec
WARNING: Tape drive has hardware compression enabled
Estimated time to write 2 * 1024 Mbyte: 142 sec = 0 h 2 min
wrote 3174843 32Kb blocks in 9709 files in 46846 seconds (short write)
wrote 847763 32Kb blocks in 5201 files in 20794 seconds (short write)
define tapetype HP-C7971A {
comment "just produced by tapetype prog (hardware compression on)"
length 1107 gbytes
filemark 74466560 kbytes
speed 1736 kps
}
amtapetype: could not rewind /dev/nst0: Input/output error


I think you ran into a tape write error the second time.
The second pass should write about twice as much files as the first
pass. (When giving a realistic estimate the first pass writes about
100 files instead of 9709 files. Much much faster.)

The size of a filemark is computed as the difference in data written
between pass 1 and pass 2 divided by the difference in number of
file marks written between pass 1 and pass 2.
Because you hit some error, that computation was wrong.
The tape length is the average of the two passes included the computed
bogus filemarks.


I think length and filemark are too large for this tapes. Does
anybody uses same tapes with amanda?

P.S. Right now I figured how to turn HW compression off (with mt
utility) and started amtapetype again...


The hardware compression algorithm in the drive is capable of detecting
nearly uncompressable data and does not behave as most other drives do
(expanding that data by blindly applying the compression algorithm).
That means it is not that important to turn off hardware compression
for these drives.

But it is important to find out where that error came from.


--
Paul Bijnens, Xplanation Tel +32 16 397.511
Technologielaan 21 bus 2, B-3001 Leuven, BELGIUM Fax +32 16 397.512
http://www.xplanation.com/ email: Paul.Bijnens < at > xplanation.com
***********************************************************************
* I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, F6, *
* quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, *
* stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, *
* PF4, F20, ^X^X, :, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, *
* kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... *
* ... "Are you sure?" ... YES ... Phew ... I'm out *
***********************************************************************

* Alexey I. Froloff <raorn < at > > [040707 14:10]:
P.S. Right now I figured how to turn HW compression off (with mt
utility) and started amtapetype again...
Thanks to all for your help. amtapetype finished successfully.

$ /usr/sbin/amtapetype -e 100g -f /dev/nst0 -t "HP-C7971A"
Writing 1024 Mbyte compresseable data: 25 sec
Writing 1024 Mbyte uncompresseable data: 72 sec
WARNING: Tape drive has hardware compression enabled
Estimated time to write 2 * 102400 Mbyte: 14400 sec = 4 h 0 min
wrote 3244032 32Kb blocks in 99 files in 7310 seconds (short write)
wrote 3244032 32Kb blocks in 198 files in 7666 seconds (short write)
define tapetype HP-C7971A {
comment "just produced by tapetype prog (hardware compression on)"
length 101376 mbytes
filemark 0 kbytes
speed 13871 kps
}

--
Regards, Alexey I. Froloff
AIF5-RIPN, AIF5-RIPE
-------------------------------------------
Inform-Mobil, Ltd. System Administrator
http://www.inform-mobil.ru/
Tel: +7(095)504-4709, Fax: +7(095)513-1006

Alexey I. Froloff wrote:

* Alexey I. Froloff <raorn < at > > [040707 14:10]:

P.S. Right now I figured how to turn HW compression off (with mt
utility) and started amtapetype again...

Thanks to all for your help. amtapetype finished successfully.

$ /usr/sbin/amtapetype -e 100g -f /dev/nst0 -t "HP-C7971A"
Writing 1024 Mbyte compresseable data: 25 sec
Writing 1024 Mbyte uncompresseable data: 72 sec
WARNING: Tape drive has hardware compression enabled
Estimated time to write 2 * 102400 Mbyte: 14400 sec = 4 h 0 min
wrote 3244032 32Kb blocks in 99 files in 7310 seconds (short write)
wrote 3244032 32Kb blocks in 198 files in 7666 seconds (short write)
define tapetype HP-C7971A {
comment "just produced by tapetype prog (hardware compression on)"
length 101376 mbytes
filemark 0 kbytes
speed 13871 kps
}



Noticed the speed increase when specifying '-e 100g' !

And notice that even with hardware compression on, these drives can put
100 Gbyte on tape, thanks to the intelligent hwcompr algorithm that
somehow avoids compressing random data.

I wish I owned one of those drives.

--
Paul Bijnens, Xplanation Tel +32 16 397.511
Technologielaan 21 bus 2, B-3001 Leuven, BELGIUM Fax +32 16 397.512
http://www.xplanation.com/ email: Paul.Bijnens < at > xplanation.com
***********************************************************************
* I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, F6, *
* quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, *
* stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, *
* PF4, F20, ^X^X, :, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, *
* kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... *
* ... "Are you sure?" ... YES ... Phew ... I'm out *
***********************************************************************

* Paul Bijnens <paul.bijnens < at > > [040708 12:18]:
Noticed the speed increase when specifying '-e 100g' !
Yes!

And notice that even with hardware compression on, these drives can put
100 Gbyte on tape, thanks to the intelligent hwcompr algorithm that
somehow avoids compressing random data.
I think it worth a try with -e 200g option.

I wish I owned one of those drives.
:-)

--
Regards, Alexey I. Froloff
AIF5-RIPN, AIF5-RIPE
-------------------------------------------
Inform-Mobil, Ltd. System Administrator
http://www.inform-mobil.ru/
Tel: +7(095)504-4709, Fax: +7(095)513-1006

On Thursday 08 July 2004 01:32, Alexey I. Froloff wrote:
* Alexey I. Froloff <raorn < at > > [040707 14:10]:
P.S. Right now I figured how to turn HW compression off (with mt
utility) and started amtapetype again...

Thanks to all for your help. amtapetype finished successfully.

$ /usr/sbin/amtapetype -e 100g -f /dev/nst0 -t "HP-C7971A"
Writing 1024 Mbyte compresseable data: 25 sec
Writing 1024 Mbyte uncompresseable data: 72 sec
WARNING: Tape drive has hardware compression enabled

Please disable this as it hides the true tapesize from amtapetype AND
amanda. amanda can do a much better job of filling up a tape if
amanda knows how much a tape can hold. You can use the optional gzip
compressor on portions of the system to much better advantage in
terms of achieved compression ratios, and generally can stuff more
data onto a tape than using the hardware compressor can do. Here,
under what would be termed ideal conditions, I've had amanda email me
and rep[ort that it just put a bit over 12Gb on a 4Gb tape that has
the last 370megs reserved for something else I'm doing here.

amtapetype uses /dev/urandom as a data source, which will overpower
the hardware compressor and make the data on the tape larger, by
maybe 10-15%, than what was fed in, so you do not get a good estimate
of the tapesize from that effect alone.

Estimated time to write 2 * 102400 Mbyte: 14400 sec = 4 h 0 min
wrote 3244032 32Kb blocks in 99 files in 7310 seconds (short write)
wrote 3244032 32Kb blocks in 198 files in 7666 seconds (short write)
define tapetype HP-C7971A {
comment "just produced by tapetype prog (hardware compression
on)" length 101376 mbytes
filemark 0 kbytes
speed 13871 kps
}

--
Cheers, Gene
There are 4 boxes to be used in defense of liberty.
Soap, ballot, jury, and ammo.
Please use in that order, starting now. -Ed Howdershelt, Author
Additions to this message made by Gene Heskett are Copyright 2004,
Maurice E. Heskett, all rights reserved.

On Thu, 8 Jul 2004 at 8:35am, Gene Heskett wrote

On Thursday 08 July 2004 01:32, Alexey I. Froloff wrote:
* Alexey I. Froloff <raorn < at > > [040707 14:10]:
P.S. Right now I figured how to turn HW compression off (with mt
utility) and started amtapetype again...

Thanks to all for your help. amtapetype finished successfully.

$ /usr/sbin/amtapetype -e 100g -f /dev/nst0 -t "HP-C7971A"
Writing 1024 Mbyte compresseable data: 25 sec
Writing 1024 Mbyte uncompresseable data: 72 sec
WARNING: Tape drive has hardware compression enabled

Please disable this as it hides the true tapesize from amtapetype AND
amanda. amanda can do a much better job of filling up a tape if
amanda knows how much a tape can hold. You can use the optional gzip
compressor on portions of the system to much better advantage in
terms of achieved compression ratios, and generally can stuff more
data onto a tape than using the hardware compressor can do. Here,
under what would be termed ideal conditions, I've had amanda email me
and rep[ort that it just put a bit over 12Gb on a 4Gb tape that has
the last 370megs reserved for something else I'm doing here.

Actually Gene, for this drive disabling hardware compression isn't
necessary. It's an LTO/Ultrium drive, and those are actually smart enough
to not compress already compressed data. Notice (below) how he still gets
the full 100GB native capacity? With those drives, you get the best of
both worlds -- you can mix software and hardware compression. Very, very
slick.

Estimated time to write 2 * 102400 Mbyte: 14400 sec = 4 h 0 min
wrote 3244032 32Kb blocks in 99 files in 7310 seconds (short write)
wrote 3244032 32Kb blocks in 198 files in 7666 seconds (short write)
define tapetype HP-C7971A {
comment "just produced by tapetype prog (hardware compression
on)" length 101376 mbytes
filemark 0 kbytes
speed 13871 kps
}

--
Joshua Baker-LePain
Department of Biomedical Engineering
Duke University

Actually Gene, for this drive disabling hardware compression isn't
necessary. It's an LTO/Ultrium drive, and those are actually smart enough
to not compress already compressed data. Notice (below) how he still gets
the full 100GB native capacity? With those drives, you get the best of
both worlds -- you can mix software and hardware compression. Very, very
slick.

I can agree - We have a couple of autoloaders with these in and they have
reported very similar results while using amtapetype.

They are very neat indeed - And reliable so far

thanks

On Thursday 08 July 2004 08:50, Joshua Baker-LePain wrote:
On Thu, 8 Jul 2004 at 8:35am, Gene Heskett wrote

On Thursday 08 July 2004 01:32, Alexey I. Froloff wrote:
* Alexey I. Froloff <raorn < at > > [040707 14:10]:
P.S. Right now I figured how to turn HW compression off (with
mt utility) and started amtapetype again...

Thanks to all for your help. amtapetype finished successfully.

$ /usr/sbin/amtapetype -e 100g -f /dev/nst0 -t "HP-C7971A"
Writing 1024 Mbyte compresseable data: 25 sec
Writing 1024 Mbyte uncompresseable data: 72 sec
WARNING: Tape drive has hardware compression enabled

Please disable this as it hides the true tapesize from amtapetype
AND amanda. amanda can do a much better job of filling up a tape
if amanda knows how much a tape can hold. You can use the
optional gzip compressor on portions of the system to much better
advantage in terms of achieved compression ratios, and generally
can stuff more data onto a tape than using the hardware compressor
can do. Here, under what would be termed ideal conditions, I've
had amanda email me and rep[ort that it just put a bit over 12Gb
on a 4Gb tape that has the last 370megs reserved for something
else I'm doing here.

Actually Gene, for this drive disabling hardware compression isn't
necessary. It's an LTO/Ultrium drive, and those are actually smart
enough to not compress already compressed data. Notice (below) how
he still gets the full 100GB native capacity? With those drives,
you get the best of both worlds -- you can mix software and
hardware compression. Very, very slick.

And a bit expen$ive for my retired blood. I've spent $200+ on a fresh
pair of drives here since the first of the year, and they are both
dead now, acting like worn out heads. So if I ever get this machine
stable enough to worry about amanda, I'll see if I can make amanda
use dvd+-rw's instead. But right now the video card died yesterday,
and an atempt to recomipile kde-3.2.3 is meeting with a bus error in
the middle of the qt-x11-free-3.2.3 build. Hello... Bus error?
Nothing in the logs, nada, zip zero.

Estimated time to write 2 * 102400 Mbyte: 14400 sec = 4 h 0 min
wrote 3244032 32Kb blocks in 99 files in 7310 seconds (short
write) wrote 3244032 32Kb blocks in 198 files in 7666 seconds
(short write) define tapetype HP-C7971A {
comment "just produced by tapetype prog (hardware compression
on)" length 101376 mbytes
filemark 0 kbytes
speed 13871 kps
}

--
Cheers, Gene
There are 4 boxes to be used in defense of liberty.
Soap, ballot, jury, and ammo.
Please use in that order, starting now. -Ed Howdershelt, Author
Additions to this message made by Gene Heskett are Copyright 2004,
Maurice E. Heskett, all rights reserved.

On Thu, Jul 08, 2004 at 08:50:17AM -0400, Joshua Baker-LePain wrote:
On Thu, 8 Jul 2004 at 8:35am, Gene Heskett wrote

On Thursday 08 July 2004 01:32, Alexey I. Froloff wrote:
* Alexey I. Froloff <raorn < at > > [040707 14:10]:
P.S. Right now I figured how to turn HW compression off (with mt
utility) and started amtapetype again...

Thanks to all for your help. amtapetype finished successfully.

$ /usr/sbin/amtapetype -e 100g -f /dev/nst0 -t "HP-C7971A"
Writing 1024 Mbyte compresseable data: 25 sec
Writing 1024 Mbyte uncompresseable data: 72 sec
WARNING: Tape drive has hardware compression enabled

Please disable this as it hides the true tapesize from amtapetype AND
amanda. amanda can do a much better job of filling up a tape if
amanda knows how much a tape can hold.
...

Actually Gene, for this drive disabling hardware compression isn't
necessary. It's an LTO/Ultrium drive, and those are actually smart enough
to not compress already compressed data. Notice (below) how he still gets
the full 100GB native capacity? With those drives, you get the best of
both worlds -- you can mix software and hardware compression. Very, very
slick.


Gene's second point is still valid though.
What is the actual "real world data" capacity of the tape?
Amanda will think 100GB and yet the HW compressor will be
shrinking that 0-80% depending on the data compressability.
So running tapetype if you plan to use HW compression is
basically a time-waster as you will be guess-timating the
actual data capacity anyway.

--
Jon H. LaBadie jon < at > jgcomp.com
JG Computing
4455 Province Line Road (609) 252-0159
Princeton, NJ 08540-4322 (609) 683-7220 (fax)