Backup Central

Today I had 4 NDMP jobs, streaming to 4 drives. I noticed 1 job had halted with the error that a tape was stuck in one of the drives. I reset the drive from within the Commcell GUI. The Commcell Event viewer then reported that the tape had been successfully recovered. I physically checked the Tape Library itself, and the drive was indeed empty. However, the status of the drive in the Commcell GUI was that the tape is still stuck, and hence the job won't run. This same situation, does happen every so often, and rebooting the Commcell Server usually syncs the correct drive status back up again, freeing things up, but obviously i shouldn't have to do this. Rebooting the server isn't ideal, especially when i'm doing NDMP backups of large volumes as i am now.

The drives in question are IBM LTO4 Fibre Connected drives which are connected to the Commserve from our DR site, via 2 Fibre switches. I'm not sure its a fibre connectivity problem as such, as the drive responded quite happily when i reset it through the Commcell interface, and recovered the tape. It just didn't seem to update the status correctly after doing this.

My questions are:
* What causes a tape to get 'stuck' in the first place?
* Does the fact that this is an NDMP job, make it more difficult to recover from this situation cleanly?
* Is 'resetting the drive' the correct action to take when you see a 'stuck' tape in a drive?
* Is there a way of force/kick the server and drive to 'resync' their status without rebooting the server? I could restart the Galaxy Services, but this will also stop any running jobs, which isn't ideal?
* Am i missing anything else that helps deal with this type of situation?

TIA for any thoughts/advice.

Mark








__._,_.___
Reply to sender ([email]markey164 < at > yahoo.com?subject=Issues with 'stuck' tape in drive[/email]) | Reply to group ([email]commvault < at > yahoogroups.com?subject=Issues with 'stuck' tape in drive[/email]) | Reply via web post | Start a New Topic
Messages in this topic (1)
Recent Activity:

• New Members 7

Visit Your Group

Commvault Documentation here:
http://documentation.commvault.com/
Text-Only ([email]commvault-traditional < at > yahoogroups.com?subject=Change Delivery Format: Traditional[/email]), Daily Digest ([email]commvault-digest < at > yahoogroups.com?subject=Email Delivery: Digest[/email]) &bull; Unsubscribe ([email]commvault-unsubscribe < at > yahoogroups.com?subject=Unsubscribe[/email]) &bull; Terms of Use




.

[img]http://geo.yahoo.com/serv?s=97359714/grpId=7450851/grpspId=1707277520/msgId=3518/stime=1268440326/nc1=4507179/nc2=4025321/nc3=5741391[/img]
__,_._,___

Due to the mechanical nature of the whole robot arm thing, coupled with alignment errors, tapes do - sometimes - get 'stuck' in drives. It's generally quite rare, but it can happen. If a drive is particularly misaliged, or has suffered some sort of physical shock, this can throw the alignment out. Libraries usually calibrate their alignment position when they are first powered on. Environmental condition issues/dust can also have an effect on tape and drive performance - best to keep libraries in properly air-conditioned and temperature controlled environments.

Having said that, if a tape does get stuck, I generally find that a combination of 'unload drive', 'full scan' and 'reset library' normally will do the trick. Sometimes a reset drive helps. If it is actually the case that the tape is physically removable. Worst case scenario I have seen is that the drive had to be physically removed from the library and the tape manually extracted. IT's also helpful to have access to the library GUI via web console to do any tape moving and diagnostics outside of the Commvault environment.


A "reset library" and a "full scan" (from the library node in the browser) should get the library status re-synced. I don't thing you will need to bounce the commvault services.  


The one other thing I have found is that some of these operations (reset library / full scan) take a long while to complete - particularly when there are empty slots in the library as the laser has to scan the empty slot multiple times to be sure that there is no tape in there.


You can also mark the media agent 'offline' and then mark it 'online' again, although I agree this will affect running jobs.


cheers,
m
On Fri, Mar 12, 2010 at 00:51, Mark Edwards <markey164 < at > yahoo.com ([email]markey164 < at > yahoo.com[/email])> wrote:
 

Today I had 4 NDMP jobs, streaming to 4 drives. I noticed 1 job had halted with the error that a tape was stuck in one of the drives. I reset the drive from within the Commcell GUI. The Commcell Event viewer then reported that the tape had been successfully recovered. I physically checked the Tape Library itself, and the drive was indeed empty. However, the status of the drive in the Commcell GUI was that the tape is still stuck, and hence the job won't run. This same situation, does happen every so often, and rebooting the Commcell Server usually syncs the correct drive status back up again, freeing things up, but obviously i shouldn't have to do this. Rebooting the server isn't ideal, especially when i'm doing NDMP backups of large volumes as i am now.

The drives in question are IBM LTO4 Fibre Connected drives which are connected to the Commserve from our DR site, via 2 Fibre switches. I'm not sure its a fibre connectivity problem as such, as the drive responded quite happily when i reset it through the Commcell interface, and recovered the tape. It just didn't seem to update the status correctly after doing this.

My questions are:
* What causes a tape to get 'stuck' in the first place?
* Does the fact that this is an NDMP job, make it more difficult to recover from this situation cleanly?
* Is 'resetting the drive' the correct action to take when you see a 'stuck' tape in a drive?
* Is there a way of force/kick the server and drive to 'resync' their status without rebooting the server? I could restart the Galaxy Services, but this will also stop any running jobs, which isn't ideal?
* Am i missing anything else that helps deal with this type of situation?

TIA for any thoughts/advice.

Mark
















--
regards,

Mark P. Edwards


__._,_.___
Reply to sender ([email]mark.p.edwards < at > gmail.com?subject=Re: [commvault] Issues with 'stuck' tape in drive[/email]) | Reply to group ([email]commvault < at > yahoogroups.com?subject=Re: [commvault] Issues with 'stuck' tape in drive[/email]) | Reply via web post | Start a New Topic
Messages in this topic (2)
Recent Activity:

• New Members 7

Visit Your Group

Commvault Documentation here:
http://documentation.commvault.com/
Text-Only ([email]commvault-traditional < at > yahoogroups.com?subject=Change Delivery Format: Traditional[/email]), Daily Digest ([email]commvault-digest < at > yahoogroups.com?subject=Email Delivery: Digest[/email]) &bull; Unsubscribe ([email]commvault-unsubscribe < at > yahoogroups.com?subject=Unsubscribe[/email]) &bull; Terms of Use




.

[img]http://geo.yahoo.com/serv?s=97359714/grpId=7450851/grpspId=1707277520/msgId=3520/stime=1268441369/nc1=4507179/nc2=4025321/nc3=5191952[/img]
__,_._,___

I was on the phone with a CV tech just this morning on a different issue. While we were on the phone, a tape drive got an error. It thought a tape was stuck in the drive, but the tape in question was in a slot according to the library GUI. the tech reset the library and did a full scan and those did not clear the issue (I agree with Mark, that usually clears it). The tech recommended cycling the services on the media agent in order to clear the issue. I just right clicked the drive and reset it and everything instantly started working again with no issues.

--- On Fri, 3/12/10, Mark Edwards <mark.p.edwards < at > gmail.com> wrote:

From: Mark Edwards <mark.p.edwards < at > gmail.com>
Subject: Re: [commvault] Issues with 'stuck' tape in drive
To: commvault < at > yahoogroups.com
Date: Friday, March 12, 2010, 6:49 PM


Due to the mechanical nature of the whole robot arm thing, coupled with alignment errors, tapes do - sometimes - get 'stuck' in drives. It's generally quite rare, but it can happen. If a drive is particularly misaliged, or has suffered some sort of physical shock, this can throw the alignment out. Libraries usually calibrate their alignment position when they are first powered on. Environmental condition issues/dust can also have an effect on tape and drive performance - best to keep libraries in properly air-conditioned and temperature controlled environments.

Having said that, if a tape does get stuck, I generally find that a combination of 'unload drive', 'full scan' and 'reset library' normally will do the trick. Sometimes a reset drive helps. If it is actually the case that the tape is physically removable. Worst case scenario I have seen is that the drive had to be physically removed from the library and the tape manually extracted. IT's also helpful to have access to the library GUI via web console to do any tape moving and diagnostics outside of the Commvault environment.


A "reset library" and a "full scan" (from the library node in the browser) should get the library status re-synced. I don't thing you will need to bounce the commvault services.


The one other thing I have found is that some of these operations (reset library / full scan) take a long while to complete - particularly when there are empty slots in the library as the laser has to scan the empty slot multiple times to be sure that there is no tape in there.


You can also mark the media agent 'offline' and then mark it 'online' again, although I agree this will affect running jobs.


cheers,
m
On Fri, Mar 12, 2010 at 00:51, Mark Edwards <[url=/mc/compose?to=markey164 < at > yahoo.com]markey164 < at > yahoo. com[/url]> wrote:


Today I had 4 NDMP jobs, streaming to 4 drives. I noticed 1 job had halted with the error that a tape was stuck in one of the drives. I reset the drive from within the Commcell GUI. The Commcell Event viewer then reported that the tape had been successfully recovered. I physically checked the Tape Library itself, and the drive was indeed empty. However, the status of the drive in the Commcell GUI was that the tape is still stuck, and hence the job won't run. This same situation, does happen every so often, and rebooting the Commcell Server usually syncs the correct drive status back up again, freeing things up, but obviously i shouldn't have to do this. Rebooting the server isn't ideal, especially when i'm doing NDMP backups of large volumes as i am now.

The drives in question are IBM LTO4 Fibre Connected drives which are connected to the Commserve from our DR site, via 2 Fibre switches. I'm not sure its a fibre connectivity problem as such, as the drive responded quite happily when i reset it through the Commcell interface, and recovered the tape. It just didn't seem to update the status correctly after doing this.

My questions are:
* What causes a tape to get 'stuck' in the first place?
* Does the fact that this is an NDMP job, make it more difficult to recover from this situation cleanly?
* Is 'resetting the drive' the correct action to take when you see a 'stuck' tape in a drive?
* Is there a way of force/kick the server and drive to 'resync' their status without rebooting the server? I could restart the Galaxy Services, but this will also stop any running jobs, which isn't ideal?
* Am i missing anything else that helps deal with this type of situation?

TIA for any thoughts/advice.

Mark
















--
regards,

Mark P. Edwards




__._,_.___
Reply to sender ([email]hitnrunxx < at > yahoo.com?subject=Re: [commvault] Issues with 'stuck' tape in drive[/email]) | Reply to group ([email]commvault < at > yahoogroups.com?subject=Re: [commvault] Issues with 'stuck' tape in drive[/email]) | Reply via web post | Start a New Topic
Messages in this topic (3)
Recent Activity:

• New Members 7

Visit Your Group

Commvault Documentation here:
http://documentation.commvault.com/
Text-Only ([email]commvault-traditional < at > yahoogroups.com?subject=Change Delivery Format: Traditional[/email]), Daily Digest ([email]commvault-digest < at > yahoogroups.com?subject=Email Delivery: Digest[/email]) &bull; Unsubscribe ([email]commvault-unsubscribe < at > yahoogroups.com?subject=Unsubscribe[/email]) &bull; Terms of Use




.

[img]http://geo.yahoo.com/serv?s=97359714/grpId=7450851/grpspId=1707277520/msgId=3521/stime=1268442145/nc1=4507179/nc2=4025321/nc3=5741398[/img]
__,_._,___