Welcome! » Log In » Create A New Profile

Cloning from VTL to Tape

Posted by JesseGill 
Cloning from VTL to Tape
October 09, 2008 09:50AM
Hey all, I looked through the forum and couldn't really find anything on this, but I hope I'm not revisiting an old topic.

I'm a new backup admin for Directv and need a little help. In our datacenter, we have two Networker servers, bkupcr1 and bkupcr2, both running Networker 7.3.2 on RedHat Linux 4.3 32-bit. Bkupcr1 is the nsrhost with an attached ADIC Scalar 24 tape drive, while bkupcr2 is a storage node with an attached Clariion Disk Library.

Each night, data is backed up to the CDL. There are three pools (INDEX, SERVERS, and FS) on the CDL. Indexes go to the INDEX pool, bkupcr1-2 filesystems and MMC database go to SERVERS, and all other client filesystems go to FS. What I want to do is have that data cloned to a clone pool on tape. Maybe not every day, but at least once a week.

I've tested the following command:

# nsrclone -s bkupcr1 -b 'CDL Clone' VT0025

This clones a volume on the CDL to a Clone pool on tape. This test was successful. Is there a way to clone an entire pool at once? What would be the best way to do this?

If there's any other information I need to include, please let me know. Thanks in adnvance!

Jesse
Cloning from VTL to Tape
October 09, 2008 03:15PM
[quote]
<snip>

Is there a way to clone an entire pool at once? What would be the best way to do this?

If there's any other information I need to include, please let me know. Thanks in adnvance!

Jesse[/quote]

You can clone by volume name, volume id, or (like we do) by saveset. If you want to clone more than one thing at a time, you need to get what you want into a file, and pass the file name to the nsrclone executable. In our case, we use an mminfo command to populate a text file with all of the saveset numbers we intend to clone and then reference that file with the -f option using nsrclone. By using mminfo, you can craft a query to return just the items, be it volume names or savesets, that need to be cloned on to tape. Our script uses an mminfo query that looks for savesets that only have one copy (i.e. no clone copy already exists), and I'm guessing that you could craft something that achieves the same for volumes.

HTH
-brerrabbit
Cloning from VTL to Tape
October 09, 2008 06:16PM
Well, generally there's two ways to do cloning. The first is as a save
group, you flip a flag to clone at group completion. The only other
option
IMO is by listing the savesets into a file and using nsrclone commands.

The issue I've had with cloning is it inevitably fails at some point,
leaving you to cleanup the mess and try again!

HTH
Jerry

On Oct 9, 2008, at 12:53 PM, "JesseGill" <networker-forum < at > BACKUPCENTRAL.COM
[quote]wrote:
[/quote]
[quote]
Hey all, I looked through the forum and couldn't really find
anything on this, but I hope I'm not revisiting an old topic.

I'm a new backup admin for Directv and need a little help. In our
datacenter, we have two Networker servers, bkupcr1 and bkupcr2, both
running Networker 7.3.2 on RedHat Linux 4.3 32-bit. Bkupcr1 is the
nsrhost with an attached ADIC Scalar 24 tape drive, while bkupcr2 is
a storage node with an attached Clariion Disk Library.

Each night, data is backed up to the CDL. There are three pools
(INDEX, SERVERS, and FS) on the CDL. Indexes go to the INDEX pool,
bkupcr1-2 filesystems and MMC database go to SERVERS, and all other
client filesystems go to FS. What I want to do is have that data
cloned to a clone pool on tape. Maybe not every day, but at least
once a week.

I've tested the following command:

# nsrclone -s bkupcr1 -b 'CDL Clone' VT0025

This clones a volume on the CDL to a Clone pool on tape. This test
was successful. Is there a way to clone an entire pool at once? What
would be the best way to do this?

If there's any other information I need to include, please let me
know. Thanks in adnvance!

Jesse

+--
--------------------------------------------------------------------
|This was sent by jjgill < at > directv.com via Backup Central.
|Forward SPAM to abuse < at > backupcentral.com.
+--
--------------------------------------------------------------------

and type "signoff networker" in the body of the email. Please write
to networker-request < at > listserv.temple.edu if you have any problems
with this list. You can access the archives at http://listserv.temple.edu/archives/networker.html
or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
[/quote]

via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
Cloning from VTL to Tape
October 09, 2008 07:09PM
[quote]Well, generally there's two ways to do cloning. The first is as a save
group, you flip a flag to clone at group completion. The only other
option
IMO is by listing the savesets into a file and using nsrclone commands.

The issue I've had with cloning is it inevitably fails at some point,
leaving you to cleanup the mess and try again!

HTH
Jerry

On Oct 9, 2008, at 12:53 PM, "JesseGill" <networker-forum < at > BACKUPCENTRAL.COM

[/quote]

Jerry, if your clones inevitably fail then there's something else broken in your system. We're running 7.3.3 in a Windows-only environment, and I've got eight clone jobs running from four VTL's to a single tape library every day. We might have something weird happen to a single clone job once every six months. We've had SAN connectivity issues from time to time that would preclude a clone job from running, but once the connectivity is working correctly, the clones just flat work. Can you elaborate on the issues you're encountering and your environment?

-brerrabbit
Cloning from VTL to Tape
October 10, 2008 09:43AM
I agree with brerrabbit. We autoclone and have very few problems,
primarily with hardware issues. Maybe 2 or 3 times a year.

Dave Werth
Garmin AT, Inc.
Salem, Oregon
-----Original Message-----
From: EMC NetWorker discussion [mailto] On
Behalf Of brerrabbit
Sent: Thursday, October 09, 2008 7:09 PM
To: NETWORKER < at > LISTSERV.TEMPLE.EDU
Subject: [Networker] Cloning from VTL to Tape

Landwehr, Jerome wrote:
[quote]Well, generally there's two ways to do cloning. The first is as a save
[/quote]
[quote]group, you flip a flag to clone at group completion. The only other
option
IMO is by listing the savesets into a file and using nsrclone
[/quote]commands.
[quote]
The issue I've had with cloning is it inevitably fails at some point,
[/quote]
[quote]leaving you to cleanup the mess and try again!

HTH
Jerry

On Oct 9, 2008, at 12:53 PM, "JesseGill" <networker-forum < at >
[/quote]BACKUPCENTRAL.COM

Jerry, if your clones inevitably fail then there's something else broken
in your system. We're running 7.3.3 in a Windows-only environment, and
I've got eight clone jobs running from four VTL's to a single tape
library every day. We might have something weird happen to a single
clone job once every six months. We've had SAN connectivity issues from
time to time that would preclude a clone job from running, but once the
connectivity is working correctly, the clones just flat work. Can you
elaborate on the issues you're encountering and your environment?

-brerrabbit

+----------------------------------------------------------------------
|This was sent by drhulme < at > tarrantcounty.com via Backup Central.
|Forward SPAM to abuse < at > backupcentral.com.
+----------------------------------------------------------------------

type "signoff networker" in the body of the email. Please write to
networker-request < at > listserv.temple.edu if you have any problems with this
list. You can access the archives at
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER

via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
Cloning from VTL to Tape
October 10, 2008 04:17PM
Well, I've been testing GUI-based cloning for the past couple days and I'm confused. I have a save group TEST that I checked the 'Clone' box and set a clone pool on our tape library. I ran that save group and it ran fine.

Today, I set two of our smaller production save groups to clone and while the backups completed, the cloning operation failed. In the log files, it seemed to state the cloning completed, but then Networker was trying to load a different volume and would automatically terminate the operation before it could complete. This would loop unless aborted.

My command line test or nsrclone was successful. Unless I can figure out what exactly is going on with the Networker GUI, I might have to write up an nsrclone script. :S
Cloning from VTL to Tape
October 10, 2008 04:57PM
It seems too obvious to mention but you didn't say if you set the clone
pool on your "smaller production save groups".

Dave Werth
Garmin AT, Inc.
Salem, Oregon

-----Original Message-----
From: EMC NetWorker discussion [mailto] On
Behalf Of JesseGill
Sent: Friday, October 10, 2008 4:18 PM
To: NETWORKER < at > LISTSERV.TEMPLE.EDU
Subject: [Networker] Cloning from VTL to Tape

Well, I've been testing GUI-based cloning for the past couple days and
I'm confused. I have a save group TEST that I checked the 'Clone' box
and set a clone pool on our tape library. I ran that save group and it
ran fine.

Today, I set two of our smaller production save groups to clone and
while the backups completed, the cloning operation failed. In the log
files, it seemed to state the cloning completed, but then Networker was
trying to load a different volume and would automatically terminate the
operation before it could complete. This would loop unless aborted.

My command line test or nsrclone was successful. Unless I can figure out
what exactly is going on with the Networker GUI, I might have to write
up an nsrclone script. :S

+----------------------------------------------------------------------
|This was sent by jjgill < at > directv.com via Backup Central.
|Forward SPAM to abuse < at > backupcentral.com.
+----------------------------------------------------------------------

type "signoff networker" in the body of the email. Please write to
networker-request < at > listserv.temple.edu if you have any problems with this
list. You can access the archives at
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER

via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
Cloning from VTL to Tape
October 11, 2008 11:17PM
Hi there,
nsrclone with the -t and -e options enables you to specify a date range so that only save sets that were backed up for a group within a specific period of time, such as the last 24 hours, will be cloned.

Warm regards,
Mesut
Cloning from VTL to Tape
October 13, 2008 07:38AM
Not that I'm at a point that I want to fix cloning, or even at the same
version anymore (I think it was 7.3.1), but suffice it to say we had
issues cloning about 60TB. It would take a week to do so and the
failures are usually on only one or two clonesets. I had it scripted to
split into ~15 2-3TB data sets and someplace one would die. So I had to
look at the copies, check that they were all complete and not errored
out, delete the errored ones so the count was accurate (even errored
clones increment the copies field), recreate the lists of SSIDs, rinse
and repeat...

That's the cloning mess I was referring to

cheers
Jerry

-----Original Message-----
From: EMC NetWorker discussion [mailto] On
Behalf Of brerrabbit
Sent: Thursday, October 09, 2008 10:09 PM
To: NETWORKER < at > LISTSERV.TEMPLE.EDU
Subject: [Networker] Cloning from VTL to Tape

Landwehr, Jerome wrote:
[quote]Well, generally there's two ways to do cloning. The first is as a save
[/quote]
[quote]group, you flip a flag to clone at group completion. The only other
option
IMO is by listing the savesets into a file and using nsrclone
[/quote]commands.
[quote]
The issue I've had with cloning is it inevitably fails at some point,
[/quote]
[quote]leaving you to cleanup the mess and try again!

HTH
Jerry

On Oct 9, 2008, at 12:53 PM, "JesseGill" <networker-forum < at >
[/quote]BACKUPCENTRAL.COM

Jerry, if your clones inevitably fail then there's something else broken
in your system. We're running 7.3.3 in a Windows-only environment, and
I've got eight clone jobs running from four VTL's to a single tape
library every day. We might have something weird happen to a single
clone job once every six months. We've had SAN connectivity issues from
time to time that would preclude a clone job from running, but once the
connectivity is working correctly, the clones just flat work. Can you
elaborate on the issues you're encountering and your environment?

-brerrabbit

+----------------------------------------------------------------------
|This was sent by drhulme < at > tarrantcounty.com via Backup Central.
|Forward SPAM to abuse < at > backupcentral.com.
+----------------------------------------------------------------------

type "signoff networker" in the body of the email. Please write to
networker-request < at > listserv.temple.edu if you have any problems with this
list. You can access the archives at
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER

via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
Cloning from VTL to Tape
October 13, 2008 12:01PM
[quote]It seems too obvious to mention but you didn't say if you set the clone
pool on your "smaller production save groups".

Dave Werth
Garmin AT, Inc.
Salem, Oregon[/quote]

I set the production groups up exactly as I had the test group, but the result was failure:

[quote]Today, I set two of our smaller production save groups [b]to clone[/b] and while the backups completed, the cloning operation failed[/quote]

I'm trying to understand why the test was successfull when the production groups failed under the same configuration.
Cloning from VTL to Tape
October 17, 2008 07:29AM
We use a perl script to clone from CDL to a sl8500 clone pool. The script is too big to post. I can email it to you if you would like it. This script clones the entire pool.
Cloning from VTL to Tape
October 17, 2008 03:14PM
[quote]We use a perl script to clone from CDL to a sl8500 clone pool. The script is too big to post. I can email it to you if you would like it. This script clones the entire pool.[/quote]

That would be great! jjgill(AT)directv(DOT)com
Much appreciated!!!!
Cloning from VTL to Tape
January 20, 2009 07:52AM
Hello, I'm having a terrible time trying to clone volumes from my VTL to physical tape. I created a script to do the following:

mminfo -m -q 'savetime>=yesterday 10:00:00,savetime<=01:00:00 today' > 13hr_report

then:
awk '{print $1}' 14hr_report | grep VT* > vtlist8day
- which pulls out any barcodes with VT* written in the last 13hrs so as to only clone VT### tapes and not all tapes

then:
nsrclone -v -s cinatl01 -b vtlclone8day -f vtlist8day
- this clones all the VT## volumes from vtlist8day

The problem is that nsrclone will only clone 1 volume at a time, therefore, all I/O is put on one drive, reading from 1 tape and writing to 1 tape

I've been reading up on cloning from SSIDs to get around this issue but cannot find anything that really helps.

I would like to clone at least 3 or 4 VT## volumes at once to physical tape. Does anyone have recommendations or how to pull SSIDs writtent to VT## volumes written in the last 13hrs and then clone those volumes in parallel?

Thank you.

That pearl script sounds like something I could use
Cloning from VTL to Tape
January 20, 2009 08:09AM
[quote]

<snip>

The problem is that nsrclone will only clone 1 volume at a time, therefore, all I/O is put on one drive, reading from 1 tape and writing to 1 tape

<snip>

[/quote]

halomerchant -

Unfortunately, the phenomenon you are encountering is inherent to the nsrclone executable. It is possible to work around that by getting creative with your scripting, e.g. create multiple lists of tapes or SSIDs and spawn a clone process for each, but you need to be careful about how you set your lists up so that savesets that span multiple tapes are accounted for.

It would be REALLY NICE if someone from EMC would address this in a release of Networker. I haven't had a chance to crawl through the 7.5 release notes, but I seem to recall reading about "improvements" to cloning, maybe it's there, you might take a look. Otherwise, you're likely going to be playing with your fav scripting engine for a bit...

--brerrabbit
Cloning from VTL to Tape
January 20, 2009 09:56AM
On 1/20/09 10:52 AM, halomerchant wrote:
[quote]That pearl script sounds like something I could use

[/quote]
Is this statement addressed to me or to someone else that has offered up
a perl script for cloning over the years?

--
Frank Swasey | http://www.uvm.edu/~fcs
Sr Systems Administrator | Always remember: You are UNIQUE,
University of Vermont | just like everyone else.
"I am not young enough to know everything." - Oscar Wilde (1854-1900)

via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
Sorry, only registered users may post in this forum.

Click here to login