Welcome! » Log In » Create A New Profile

VE backup failures after disk expansion

Posted by Schaub, Steve 
Schaub, Steve
VE backup failures after disk expansion
June 29, 2018 07:59AM
Any help appreciated.

Started failing backups after drive was added or expanded to 2.5 TB.
On 6/25 the max drive size was increased from 2 to 7 via VMMAXVIRTUALDISKS 7
As of this the backups will only complete with non quiesced backup snapshots.
6/28 it was rebooted and still fails to quiesce.

Solution searches are not giving me much to go on.

Windows events :
Volume Shadow Copy Service error: Unexpected error calling routine IOCTL_DISK_SET_SNAPSHOT_INFO(\\.\PHYSICALDRIVE2) fails with winerror 1168. hr = 0x80070490, Element not found.


Dsmerror.log

6/25/2018 01:48:04 ANS2687W The virtual machine 'XXXXXXXX' contains one or more disks that exceed the maximum size set by the VMMAXVIRTUALDISKS option.. You can use the VMMAXVIRTUALDISKS option to change the maximum size or specify the '-VMSKIPMAXVIRTUALDISKS=yes' option to exclude these disks from the backup.
06/25/2018 01:48:04 ANS4174E Full VM backup of VMware Virtual Machine 'XXXXXXXX' failed with RC=4379 mode=Incremental Forever - Incremental, target node name='XXXXXXXX', data mover node name='XXXXXXXX'
06/25/2018 01:48:04 ANS1228E Sending of object 'XXXXXXXX' failed.
06/25/2018 01:48:04 ANS5226E The virtual machine backup operation failed.

06/25/2018 11:13:08 ANS9924W A full backup for the XXXXXXXX virtual machine will be performed because of added disk capacity.
06/25/2018 11:13:29 ANS9365E VMware vStorage API error for virtual machine 'XXXXXXXX'.
IBM Spectrum Protect function name : CreateSnapshot_USCORETask
IBM Spectrum Protect file : ..\..\common\vm\vmvisdk.cpp (6704)
API return code : 2094
API error message : An error occurred while quiescing the virtual machine. See the virtual machine's event log for details.
06/25/2018 11:13:29 ANS2712W The virtual machine 'XXXXXXXX' has a VMware Tools running, but VMware Tools is out of date.

06/25/2018 11:13:29 ANS4066I Snapshot operation attempt 1 of 3 for the guest virtual machine 'XXXXXXXX' failed using "VMware Tools" snapshot.
Reattempting snapshot with "VMware Tools" snapshot.
06/25/2018 11:15:32 ANS9924W A full backup for the XXXXXXXX virtual machine will be performed because of added disk capacity.
06/25/2018 11:15:42 ANS9365E VMware vStorage API error for virtual machine 'XXXXXXXX'.
IBM Spectrum Protect function name : CreateSnapshot_USCORETask
IBM Spectrum Protect file : ..\..\common\vm\vmvisdk.cpp (6704)
API return code : 2094
API error message : An error occurred while quiescing the virtual machine. See the virtual machine's event log for details.
06/25/2018 11:15:42 ANS2712W The virtual machine 'XXXXXXXX' has a VMware Tools running, but VMware Tools is out of date.

06/25/2018 11:15:42 ANS4066I Snapshot operation attempt 2 of 3 for the guest virtual machine 'XXXXXXXX' failed using "VMware Tools" snapshot.
Reattempting snapshot with "VMware Tools with file system quiescing and application quiescing (VSS) disabled" snapshot.
06/25/2018 11:17:18 ANS9924W A full backup for the XXXXXXXX virtual machine will be performed because of added disk capacity.
06/25/2018 12:41:25 ANS9924W A full backup for the YYYYYYYY virtual machine will be performed because of added disk capacity.
06/25/2018 12:41:41 ANS9365E VMware vStorage API error for virtual machine 'YYYYYYYY'.
IBM Spectrum Protect function name : CreateSnapshot_USCORETask
IBM Spectrum Protect file : ..\..\common\vm\vmvisdk.cpp (6704)
API return code : 2094
API error message : An error occurred while quiescing the virtual machine. See the virtual machine's event log for details.
06/25/2018 12:41:41 ANS2712W The virtual machine 'YYYYYYYY' has a VMware Tools running, but VMware Tools is out of date.

06/25/2018 12:41:41 ANS4066I Snapshot operation attempt 1 of 3 for the guest virtual machine 'YYYYYYYY' failed using "VMware Tools" snapshot.
Reattempting snapshot with "VMware Tools" snapshot.
06/25/2018 12:42:13 ANS9924W A full backup for the YYYYYYYY virtual machine will be performed because of added disk capacity.
06/25/2018 12:42:23 ANS9365E VMware vStorage API error for virtual machine 'YYYYYYYY'.
IBM Spectrum Protect function name : CreateSnapshot_USCORETask
IBM Spectrum Protect file : ..\..\common\vm\vmvisdk.cpp (6704)
API return code : 2094
API error message : An error occurred while quiescing the virtual machine. See the virtual machine's event log for details.
06/25/2018 12:42:23 ANS2712W The virtual machine 'YYYYYYYY' has a VMware Tools running, but VMware Tools is out of date.

06/25/2018 12:42:23 ANS4066I Snapshot operation attempt 2 of 3 for the guest virtual machine 'YYYYYYYY' failed using "VMware Tools" snapshot.
Reattempting snapshot with "VMware Tools with file system quiescing and application quiescing (VSS) disabled" snapshot.
06/25/2018 12:43:07 ANS9924W A full backup for the YYYYYYYY virtual machine will be performed because of added disk capacity.
06/25/2018 18:09:13 ANS9365E VMware vStorage API error for virtual machine 'YYYYYYYY'.
IBM Spectrum Protect function name : CreateSnapshot_USCORETask
IBM Spectrum Protect file : ..\..\common\vm\vmvisdk.cpp (6704)
API return code : 2094
API error message : An error occurred while quiescing the virtual machine. See the virtual machine's event log for details.
06/25/2018 18:09:13 ANS2712W The virtual machine 'YYYYYYYY' has a VMware Tools running, but VMware Tools is out of date.

06/25/2018 18:09:13 ANS4066I Snapshot operation attempt 1 of 3 for the guest virtual machine 'YYYYYYYY' failed using "VMware Tools" snapshot.
Reattempting snapshot with "VMware Tools" snapshot.
06/25/2018 18:10:20 ANS9365E VMware vStorage API error for virtual machine 'YYYYYYYY'.
IBM Spectrum Protect function name : CreateSnapshot_USCORETask
IBM Spectrum Protect file : ..\..\common\vm\vmvisdk.cpp (6704)
API return code : 2094
API error message : An error occurred while quiescing the virtual machine. See the virtual machine's event log for details.
06/25/2018 18:10:20 ANS2712W The virtual machine 'YYYYYYYY' has a VMware Tools running, but VMware Tools is out of date.

06/25/2018 18:10:20 ANS4066I Snapshot operation attempt 2 of 3 for the guest virtual machine 'YYYYYYYY' failed using "VMware Tools" snapshot.
Reattempting snapshot with "VMware Tools with file system quiescing and application quiescing (VSS) disabled" snapshot.
06/25/2018 18:10:35 ANS0361I DIAG: psNpOpen(): Timed out waiting for pipe '\\.\pipe\jnlSessionMgr' to become free.
06/25/2018 18:10:39 ANS0361I DIAG: psNpOpen(): Timed out waiting for pipe '\\.\pipe\jnlSessionMgr' to become free.
06/25/2018 18:11:26 ANS9365E VMware vStorage API error for virtual machine 'XXXXXXXX'.
IBM Spectrum Protect function name : CreateSnapshot_USCORETask
IBM Spectrum Protect file : ..\..\common\vm\vmvisdk.cpp (6704)
API return code : 2094
API error message : An error occurred while quiescing the virtual machine. See the virtual machine's event log for details.
06/25/2018 18:11:26 ANS2712W The virtual machine 'XXXXXXXX' has a VMware Tools running, but VMware Tools is out of date.

06/25/2018 18:11:26 ANS4066I Snapshot operation attempt 1 of 3 for the guest virtual machine 'XXXXXXXX' failed using "VMware Tools" snapshot.
Reattempting snapshot with "VMware Tools" snapshot.
06/25/2018 18:12:03 ANS9365E VMware vStorage API error for virtual machine 'XXXXXXXX'.
IBM Spectrum Protect function name : CreateSnapshot_USCORETask
IBM Spectrum Protect file : ..\..\common\vm\vmvisdk.cpp (6704)
API return code : 2094
API error message : An error occurred while quiescing the virtual machine. See the virtual machine's event log for details.
06/25/2018 18:12:03 ANS2712W The virtual machine 'XXXXXXXX' has a VMware Tools running, but VMware Tools is out of date.

06/25/2018 18:12:03 ANS4066I Snapshot operation attempt 2 of 3 for the guest virtual machine 'XXXXXXXX' failed using "VMware Tools" snapshot.
Reattempting snapshot with "VMware Tools with file system quiescing and application quiescing (VSS) disabled" snapshot.
06/25/2018 20:04:05 ANS0361I DIAG: psNpOpen(): Timed out waiting for pipe '\\.\pipe\jnlSessionMgr' to become free.
06/25/2018 20:04:07 ANS0361I DIAG: psNpOpen(): Timed out waiting for pipe '\\.\pipe\jnlSessionMgr' to become free.
06025/25/20 20:04:09 ANANS06161IDIAG:G: sNpNpOpen): Timimed ut w waiing g or r pipe '\\\.\pipe\\jnlSessiionMgr' to becomme free..
06/25/2018 20:04:10 ANS0361I DIAG: psNpOpen(): Timed out waiting for pipe '\\.\pipe\jnlSessionMgr' to become free.
06/25/2018 20:04:12 ANS0361I DIAG: psNpOpen(): Timed out waiting for pipe '\\.\pipe\jnlSessionMgr' to become free.
06/25/2018 20:04:14 ANS0361I DIAG: psNpOpen(): Timed out waiting for pipe '\\.\pipe\jnlSessionMgr' to become free.
06/25/2018 20:04:15 ANS0361I DIAG: psNpOpen(): Timed out waiting for pipe '\\.\pipe\jnlSessionMgr' to become free.
06/25/2018 20:04:16 ANS0361I DIAG: psNpOpen(): Timed out waiting for pipe '\\.\pipe\jnlSessionMgr' to become free.
06/25/2018 20:04:20 ANS0361I0DIA/G: psNpOpen(): TiANS03m1I DIAt wasiting for pipe '\\.\paiting for pipe '\\.\pipe\jnlSessio
gr' to become free.
06/25/2018 20:04:23 ANS0361I DIAG: psNpOpen(): Timed out waiting for pipe '\\.\pipe\jnlSessionMgr' to become free.
06/25/2018 20:04:30 ANS0361I DIAG: psNpOpen(): Timed out waiting for pipe '\\.\pipe\jnlSessionMgr' to become free.
06/25/2018 20:04:36 ANS0361I DIAG: psNpOpen(): Timed out waiting for pipe '\\.\pipe\jnlSessionMgr' to become free.
06/25/2018 20:10:11 ANS0361I DIAG: psNpOpen(): Timed out waiting for pipe '\\.\pipe\jnlSessionMgr' to become free.

Steve Schaub, BCBST


------------------------------------------------------------------------------
Please see the following link for the BlueCross BlueShield of Tennessee E-mail disclaimer: http://www.bcbst.com/email_disclaimer.shtm
This message was imported via the External PhorumMail Module
Andrew Raibeck
Re: VE backup failures after disk expansion
June 29, 2018 08:59AM
Hi Steve,

Not sure what is going on here, but have you tried taking quiesced
snapshots manually, to see if this occurs apart from IBM Spectrum Protect
backups?

http://www.ibm.com/support/docview.wss?uid=swg21585387

Andy

____________________________________________________________________________

Andrew Raibeck | IBM Spectrum Protect Level 3 | storman@us.ibm.com

IBM Tivoli Storage Manager links:
Product support:
https://www.ibm.com/support/entry/portal/product/tivoli/tivoli_storage_manager

Online documentation:
http://www.ibm.com/support/knowledgecenter/SSGSG7/landing/welcome_ssgsg7.html

Product Wiki:
https://www.ibm.com/developerworks/community/wikis/home/wiki/Tivoli%20Storage%20Manager

"ADSM: Dist Stor Manager" <ADSM-L@VM.MARIST.EDU> wrote on 2018-06-29
10:02:42:

> From: "Schaub, Steve" <Steve_Schaub@BCBST.COM>
> To: ADSM-L@VM.MARIST.EDU
> Date: 2018-06-29 10:06
> Subject: VE backup failures after disk expansion
> Sent by: "ADSM: Dist Stor Manager" <ADSM-L@VM.MARIST.EDU>
>
> Any help appreciated.
>
> Started failing backups after drive was added or expanded to 2.5 TB.
> On 6/25 the max drive size was increased from 2 to 7 via
VMMAXVIRTUALDISKS 7
> As of this the backups will only complete with non quiesced backup
snapshots.
> 6/28 it was rebooted and still fails to quiesce.
>
> Solution searches are not giving me much to go on.
>
> Windows events :
> Volume Shadow Copy Service error: Unexpected error calling routine
> IOCTL_DISK_SET_SNAPSHOT_INFO(\\.\PHYSICALDRIVE2) fails with winerror
> 1168. hr = 0x80070490, Element not found.
>
>
> Dsmerror.log
>
> 6/25/2018 01:48:04 ANS2687W The virtual machine 'XXXXXXXX' contains
> one or more disks that exceed the maximum size set by the
> VMMAXVIRTUALDISKS option. You can use the VMMAXVIRTUALDISKS option
> to change the maximum size or specify the '-
> VMSKIPMAXVIRTUALDISKS=yes' option to exclude these disks from the backup.
> 06/25/2018 01:48:04 ANS4174E Full VM backup of VMware Virtual
> Machine 'XXXXXXXX' failed with RC=4379 mode=Incremental Forever -
> Incremental, target node name='XXXXXXXX', data mover node name='XXXXXXXX'
> 06/25/2018 01:48:04 ANS1228E Sending of object 'XXXXXXXX' failed.
> 06/25/2018 01:48:04 ANS5226E The virtual machine backup operation failed.
>
> 06/25/2018 11:13:08 ANS9924W A full backup for the XXXXXXXX virtual
> machine will be performed because of added disk capacity.
> 06/25/2018 11:13:29 ANS9365E VMware vStorage API error for virtual
> machine 'XXXXXXXX'.
> IBM Spectrum Protect function name : CreateSnapshot_USCORETask
> IBM Spectrum Protect file : ..\..\common\vm\vmvisdk.cpp (6704)
> API return code : 2094
> API error message : An error occurred while quiescing the virtual
> machine. See the virtual machine's event log for details.
> 06/25/2018 11:13:29 ANS2712W The virtual machine 'XXXXXXXX' has a
> VMware Tools running, but VMware Tools is out of date.
>
> 06/25/2018 11:13:29 ANS4066I Snapshot operation attempt 1 of 3 for
> the guest virtual machine 'XXXXXXXX' failed using "VMware Tools"
snapshot.
> Reattempting snapshot with "VMware Tools" snapshot.
> 06/25/2018 11:15:32 ANS9924W A full backup for the XXXXXXXX virtual
> machine will be performed because of added disk capacity.
> 06/25/2018 11:15:42 ANS9365E VMware vStorage API error for virtual
> machine 'XXXXXXXX'.
> IBM Spectrum Protect function name : CreateSnapshot_USCORETask
> IBM Spectrum Protect file : ..\..\common\vm\vmvisdk.cpp (6704)
> API return code : 2094
> API error message : An error occurred while quiescing the virtual
> machine. See the virtual machine's event log for details.
> 06/25/2018 11:15:42 ANS2712W The virtual machine 'XXXXXXXX' has a
> VMware Tools running, but VMware Tools is out of date.
>
> 06/25/2018 11:15:42 ANS4066I Snapshot operation attempt 2 of 3 for
> the guest virtual machine 'XXXXXXXX' failed using "VMware Tools"
snapshot.
> Reattempting snapshot with "VMware Tools with file system quiescing
> and application quiescing (VSS) disabled" snapshot.
> 06/25/2018 11:17:18 ANS9924W A full backup for the XXXXXXXX virtual
> machine will be performed because of added disk capacity.
> 06/25/2018 12:41:25 ANS9924W A full backup for the YYYYYYYY virtual
> machine will be performed because of added disk capacity.
> 06/25/2018 12:41:41 ANS9365E VMware vStorage API error for virtual
> machine 'YYYYYYYY'.
> IBM Spectrum Protect function name : CreateSnapshot_USCORETask
> IBM Spectrum Protect file : ..\..\common\vm\vmvisdk.cpp (6704)
> API return code : 2094
> API error message : An error occurred while quiescing the virtual
> machine. See the virtual machine's event log for details.
> 06/25/2018 12:41:41 ANS2712W The virtual machine 'YYYYYYYY' has a
> VMware Tools running, but VMware Tools is out of date.
>
> 06/25/2018 12:41:41 ANS4066I Snapshot operation attempt 1 of 3 for
> the guest virtual machine 'YYYYYYYY' failed using "VMware Tools"
snapshot.
> Reattempting snapshot with "VMware Tools" snapshot.
> 06/25/2018 12:42:13 ANS9924W A full backup for the YYYYYYYY virtual
> machine will be performed because of added disk capacity.
> 06/25/2018 12:42:23 ANS9365E VMware vStorage API error for virtual
> machine 'YYYYYYYY'.
> IBM Spectrum Protect function name : CreateSnapshot_USCORETask
> IBM Spectrum Protect file : ..\..\common\vm\vmvisdk.cpp (6704)
> API return code : 2094
> API error message : An error occurred while quiescing the virtual
> machine. See the virtual machine's event log for details.
> 06/25/2018 12:42:23 ANS2712W The virtual machine 'YYYYYYYY' has a
> VMware Tools running, but VMware Tools is out of date.
>
> 06/25/2018 12:42:23 ANS4066I Snapshot operation attempt 2 of 3 for
> the guest virtual machine 'YYYYYYYY' failed using "VMware Tools"
snapshot.
> Reattempting snapshot with "VMware Tools with file system quiescing
> and application quiescing (VSS) disabled" snapshot.
> 06/25/2018 12:43:07 ANS9924W A full backup for the YYYYYYYY virtual
> machine will be performed because of added disk capacity.
> 06/25/2018 18:09:13 ANS9365E VMware vStorage API error for virtual
> machine 'YYYYYYYY'.
> IBM Spectrum Protect function name : CreateSnapshot_USCORETask
> IBM Spectrum Protect file : ..\..\common\vm\vmvisdk.cpp (6704)
> API return code : 2094
> API error message : An error occurred while quiescing the virtual
> machine. See the virtual machine's event log for details.
> 06/25/2018 18:09:13 ANS2712W The virtual machine 'YYYYYYYY' has a
> VMware Tools running, but VMware Tools is out of date.
>
> 06/25/2018 18:09:13 ANS4066I Snapshot operation attempt 1 of 3 for
> the guest virtual machine 'YYYYYYYY' failed using "VMware Tools"
snapshot.
> Reattempting snapshot with "VMware Tools" snapshot.
> 06/25/2018 18:10:20 ANS9365E VMware vStorage API error for virtual
> machine 'YYYYYYYY'.
> IBM Spectrum Protect function name : CreateSnapshot_USCORETask
> IBM Spectrum Protect file : ..\..\common\vm\vmvisdk.cpp (6704)
> API return code : 2094
> API error message : An error occurred while quiescing the virtual
> machine. See the virtual machine's event log for details.
> 06/25/2018 18:10:20 ANS2712W The virtual machine 'YYYYYYYY' has a
> VMware Tools running, but VMware Tools is out of date.
>
> 06/25/2018 18:10:20 ANS4066I Snapshot operation attempt 2 of 3 for
> the guest virtual machine 'YYYYYYYY' failed using "VMware Tools"
snapshot.
> Reattempting snapshot with "VMware Tools with file system quiescing
> and application quiescing (VSS) disabled" snapshot.
> 06/25/2018 18:10:35 ANS0361I DIAG: psNpOpen(): Timed out waiting for
> pipe '\\.\pipe\jnlSessionMgr' to become free.
> 06/25/2018 18:10:39 ANS0361I DIAG: psNpOpen(): Timed out waiting for
> pipe '\\.\pipe\jnlSessionMgr' to become free.
> 06/25/2018 18:11:26 ANS9365E VMware vStorage API error for virtual
> machine 'XXXXXXXX'.
> IBM Spectrum Protect function name : CreateSnapshot_USCORETask
> IBM Spectrum Protect file : ..\..\common\vm\vmvisdk.cpp (6704)
> API return code : 2094
> API error message : An error occurred while quiescing the virtual
> machine. See the virtual machine's event log for details.
> 06/25/2018 18:11:26 ANS2712W The virtual machine 'XXXXXXXX' has a
> VMware Tools running, but VMware Tools is out of date.
>
> 06/25/2018 18:11:26 ANS4066I Snapshot operation attempt 1 of 3 for
> the guest virtual machine 'XXXXXXXX' failed using "VMware Tools"
snapshot.
> Reattempting snapshot with "VMware Tools" snapshot.
> 06/25/2018 18:12:03 ANS9365E VMware vStorage API error for virtual
> machine 'XXXXXXXX'.
> IBM Spectrum Protect function name : CreateSnapshot_USCORETask
> IBM Spectrum Protect file : ..\..\common\vm\vmvisdk.cpp (6704)
> API return code : 2094
> API error message : An error occurred while quiescing the virtual
> machine. See the virtual machine's event log for details.
> 06/25/2018 18:12:03 ANS2712W The virtual machine 'XXXXXXXX' has a
> VMware Tools running, but VMware Tools is out of date.
>
> 06/25/2018 18:12:03 ANS4066I Snapshot operation attempt 2 of 3 for
> the guest virtual machine 'XXXXXXXX' failed using "VMware Tools"
snapshot.
> Reattempting snapshot with "VMware Tools with file system quiescing
> and application quiescing (VSS) disabled" snapshot.
> 06/25/2018 20:04:05 ANS0361I DIAG: psNpOpen(): Timed out waiting for
> pipe '\\.\pipe\jnlSessionMgr' to become free.
> 06/25/2018 20:04:07 ANS0361I DIAG: psNpOpen(): Timed out waiting for
> pipe '\\.\pipe\jnlSessionMgr' to become free.
> 06025/25/20 20:04:09 ANANS06161IDIAG:G: sNpNpOpen): Timimed ut w
> waiing g or r pipe '\\\.\pipe\\jnlSessiionMgr' to becomme free..
> 06/25/2018 20:04:10 ANS0361I DIAG: psNpOpen(): Timed out waiting for
> pipe '\\.\pipe\jnlSessionMgr' to become free.
> 06/25/2018 20:04:12 ANS0361I DIAG: psNpOpen(): Timed out waiting for
> pipe '\\.\pipe\jnlSessionMgr' to become free.
> 06/25/2018 20:04:14 ANS0361I DIAG: psNpOpen(): Timed out waiting for
> pipe '\\.\pipe\jnlSessionMgr' to become free.
> 06/25/2018 20:04:15 ANS0361I DIAG: psNpOpen(): Timed out waiting for
> pipe '\\.\pipe\jnlSessionMgr' to become free.
> 06/25/2018 20:04:16 ANS0361I DIAG: psNpOpen(): Timed out waiting for
> pipe '\\.\pipe\jnlSessionMgr' to become free.
> 06/25/2018 20:04:20 ANS0361I0DIA/G: psNpOpen(): TiANS03m1I DIAt
> wasiting for pipe '\\.\paiting for pipe '\\.\pipe\jnlSessio
> gr' to become free.
> 06/25/2018 20:04:23 ANS0361I DIAG: psNpOpen(): Timed out waiting for
> pipe '\\.\pipe\jnlSessionMgr' to become free.
> 06/25/2018 20:04:30 ANS0361I DIAG: psNpOpen(): Timed out waiting for
> pipe '\\.\pipe\jnlSessionMgr' to become free.
> 06/25/2018 20:04:36 ANS0361I DIAG: psNpOpen(): Timed out waiting for
> pipe '\\.\pipe\jnlSessionMgr' to become free.
> 06/25/2018 20:10:11 ANS0361I DIAG: psNpOpen(): Timed out waiting for
> pipe '\\.\pipe\jnlSessionMgr' to become free.
>
> Steve Schaub, BCBST
>
>
>
------------------------------------------------------------------------------

> Please see the following link for the BlueCross BlueShield of
> Tennessee E-mail disclaimer: https://urldefense.proofpoint.com/v2/
> url?
>
u=http-3A__www.bcbst.com_email-5Fdisclaimer.shtm&d=DwIFAg&c=jf_iaSHvJObTbx-
>
siA1ZOg&r=Ij6DLy1l7wDpCbTfcDkLC_KknvhyGdCy_RnAGnhV37I&m=VGWq3pcClzYcsflyR1TSMB0D4AwJ5gk3zlyzEROL8Yg&s=iOcgwWdouf9eCWzMvt5GAW86ty8IkABmUkXp9_lxykU&e=

>
This message was imported via the External PhorumMail Module
Rick Adamson
Re: VE backup failures after disk expansion
June 29, 2018 08:59AM
Steve,
Did you try using 999 ?

-VMMAXVIRTUALDISKS=999

Are you also using the vmskipmaxvirtualdisks option?

These references may help:
https://www.ibm.com/support/knowledgecenter/en/SSGSG7_7.1.8/client/r_opt_vmmaxvirtualdisks.html
http://www-01.ibm.com/support/docview.wss?uid=swg22011779

-Rick Adamson


-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Schaub, Steve
Sent: Friday, June 29, 2018 10:03 AM
To: ADSM-L@VM.MARIST.EDU
Subject: [ADSM-L] VE backup failures after disk expansion

* This email originated outside of the organization. Use caution when opening attachments or clicking links. *

----------------------------------------------------------------------
Any help appreciated.

Started failing backups after drive was added or expanded to 2.5 TB.
On 6/25 the max drive size was increased from 2 to 7 via VMMAXVIRTUALDISKS 7 As of this the backups will only complete with non quiesced backup snapshots.
6/28 it was rebooted and still fails to quiesce.

Solution searches are not giving me much to go on.

Windows events :
Volume Shadow Copy Service error: Unexpected error calling routine IOCTL_DISK_SET_SNAPSHOT_INFO(\\.\PHYSICALDRIVE2) fails with winerror 1168. hr = 0x80070490, Element not found.


Dsmerror.log

6/25/2018 01:48:04 ANS2687W The virtual machine 'XXXXXXXX' contains one or more disks that exceed the maximum size set by the VMMAXVIRTUALDISKS option.. You can use the VMMAXVIRTUALDISKS option to change the maximum size or specify the '-VMSKIPMAXVIRTUALDISKS=yes' option to exclude these disks from the backup.
06/25/2018 01:48:04 ANS4174E Full VM backup of VMware Virtual Machine 'XXXXXXXX' failed with RC=4379 mode=Incremental Forever - Incremental, target node name='XXXXXXXX', data mover node name='XXXXXXXX'
06/25/2018 01:48:04 ANS1228E Sending of object 'XXXXXXXX' failed.
06/25/2018 01:48:04 ANS5226E The virtual machine backup operation failed.

06/25/2018 11:13:08 ANS9924W A full backup for the XXXXXXXX virtual machine will be performed because of added disk capacity.
06/25/2018 11:13:29 ANS9365E VMware vStorage API error for virtual machine 'XXXXXXXX'.
IBM Spectrum Protect function name : CreateSnapshot_USCORETask
IBM Spectrum Protect file : ..\..\common\vm\vmvisdk.cpp (6704)
API return code : 2094
API error message : An error occurred while quiescing the virtual machine. See the virtual machine's event log for details.
06/25/2018 11:13:29 ANS2712W The virtual machine 'XXXXXXXX' has a VMware Tools running, but VMware Tools is out of date.

06/25/2018 11:13:29 ANS4066I Snapshot operation attempt 1 of 3 for the guest virtual machine 'XXXXXXXX' failed using "VMware Tools" snapshot.
Reattempting snapshot with "VMware Tools" snapshot.
06/25/2018 11:15:32 ANS9924W A full backup for the XXXXXXXX virtual machine will be performed because of added disk capacity.
06/25/2018 11:15:42 ANS9365E VMware vStorage API error for virtual machine 'XXXXXXXX'.
IBM Spectrum Protect function name : CreateSnapshot_USCORETask
IBM Spectrum Protect file : ..\..\common\vm\vmvisdk.cpp (6704)
API return code : 2094
API error message : An error occurred while quiescing the virtual machine. See the virtual machine's event log for details.
06/25/2018 11:15:42 ANS2712W The virtual machine 'XXXXXXXX' has a VMware Tools running, but VMware Tools is out of date.

06/25/2018 11:15:42 ANS4066I Snapshot operation attempt 2 of 3 for the guest virtual machine 'XXXXXXXX' failed using "VMware Tools" snapshot.
Reattempting snapshot with "VMware Tools with file system quiescing and application quiescing (VSS) disabled" snapshot.
06/25/2018 11:17:18 ANS9924W A full backup for the XXXXXXXX virtual machine will be performed because of added disk capacity.
06/25/2018 12:41:25 ANS9924W A full backup for the YYYYYYYY virtual machine will be performed because of added disk capacity.
06/25/2018 12:41:41 ANS9365E VMware vStorage API error for virtual machine 'YYYYYYYY'.
IBM Spectrum Protect function name : CreateSnapshot_USCORETask
IBM Spectrum Protect file : ..\..\common\vm\vmvisdk.cpp (6704)
API return code : 2094
API error message : An error occurred while quiescing the virtual machine. See the virtual machine's event log for details.
06/25/2018 12:41:41 ANS2712W The virtual machine 'YYYYYYYY' has a VMware Tools running, but VMware Tools is out of date.

06/25/2018 12:41:41 ANS4066I Snapshot operation attempt 1 of 3 for the guest virtual machine 'YYYYYYYY' failed using "VMware Tools" snapshot.
Reattempting snapshot with "VMware Tools" snapshot.
06/25/2018 12:42:13 ANS9924W A full backup for the YYYYYYYY virtual machine will be performed because of added disk capacity.
06/25/2018 12:42:23 ANS9365E VMware vStorage API error for virtual machine 'YYYYYYYY'.
IBM Spectrum Protect function name : CreateSnapshot_USCORETask
IBM Spectrum Protect file : ..\..\common\vm\vmvisdk.cpp (6704)
API return code : 2094
API error message : An error occurred while quiescing the virtual machine. See the virtual machine's event log for details.
06/25/2018 12:42:23 ANS2712W The virtual machine 'YYYYYYYY' has a VMware Tools running, but VMware Tools is out of date.

06/25/2018 12:42:23 ANS4066I Snapshot operation attempt 2 of 3 for the guest virtual machine 'YYYYYYYY' failed using "VMware Tools" snapshot.
Reattempting snapshot with "VMware Tools with file system quiescing and application quiescing (VSS) disabled" snapshot.
06/25/2018 12:43:07 ANS9924W A full backup for the YYYYYYYY virtual machine will be performed because of added disk capacity.
06/25/2018 18:09:13 ANS9365E VMware vStorage API error for virtual machine 'YYYYYYYY'.
IBM Spectrum Protect function name : CreateSnapshot_USCORETask
IBM Spectrum Protect file : ..\..\common\vm\vmvisdk.cpp (6704)
API return code : 2094
API error message : An error occurred while quiescing the virtual machine. See the virtual machine's event log for details.
06/25/2018 18:09:13 ANS2712W The virtual machine 'YYYYYYYY' has a VMware Tools running, but VMware Tools is out of date.

06/25/2018 18:09:13 ANS4066I Snapshot operation attempt 1 of 3 for the guest virtual machine 'YYYYYYYY' failed using "VMware Tools" snapshot.
Reattempting snapshot with "VMware Tools" snapshot.
06/25/2018 18:10:20 ANS9365E VMware vStorage API error for virtual machine 'YYYYYYYY'.
IBM Spectrum Protect function name : CreateSnapshot_USCORETask
IBM Spectrum Protect file : ..\..\common\vm\vmvisdk.cpp (6704)
API return code : 2094
API error message : An error occurred while quiescing the virtual machine. See the virtual machine's event log for details.
06/25/2018 18:10:20 ANS2712W The virtual machine 'YYYYYYYY' has a VMware Tools running, but VMware Tools is out of date.

06/25/2018 18:10:20 ANS4066I Snapshot operation attempt 2 of 3 for the guest virtual machine 'YYYYYYYY' failed using "VMware Tools" snapshot.
Reattempting snapshot with "VMware Tools with file system quiescing and application quiescing (VSS) disabled" snapshot.
06/25/2018 18:10:35 ANS0361I DIAG: psNpOpen(): Timed out waiting for pipe '\\.\pipe\jnlSessionMgr' to become free.
06/25/2018 18:10:39 ANS0361I DIAG: psNpOpen(): Timed out waiting for pipe '\\.\pipe\jnlSessionMgr' to become free.
06/25/2018 18:11:26 ANS9365E VMware vStorage API error for virtual machine 'XXXXXXXX'.
IBM Spectrum Protect function name : CreateSnapshot_USCORETask
IBM Spectrum Protect file : ..\..\common\vm\vmvisdk.cpp (6704)
API return code : 2094
API error message : An error occurred while quiescing the virtual machine. See the virtual machine's event log for details.
06/25/2018 18:11:26 ANS2712W The virtual machine 'XXXXXXXX' has a VMware Tools running, but VMware Tools is out of date.

06/25/2018 18:11:26 ANS4066I Snapshot operation attempt 1 of 3 for the guest virtual machine 'XXXXXXXX' failed using "VMware Tools" snapshot.
Reattempting snapshot with "VMware Tools" snapshot.
06/25/2018 18:12:03 ANS9365E VMware vStorage API error for virtual machine 'XXXXXXXX'.
IBM Spectrum Protect function name : CreateSnapshot_USCORETask
IBM Spectrum Protect file : ..\..\common\vm\vmvisdk.cpp (6704)
API return code : 2094
API error message : An error occurred while quiescing the virtual machine. See the virtual machine's event log for details.
06/25/2018 18:12:03 ANS2712W The virtual machine 'XXXXXXXX' has a VMware Tools running, but VMware Tools is out of date.

06/25/2018 18:12:03 ANS4066I Snapshot operation attempt 2 of 3 for the guest virtual machine 'XXXXXXXX' failed using "VMware Tools" snapshot.
Reattempting snapshot with "VMware Tools with file system quiescing and application quiescing (VSS) disabled" snapshot.
06/25/2018 20:04:05 ANS0361I DIAG: psNpOpen(): Timed out waiting for pipe '\\.\pipe\jnlSessionMgr' to become free.
06/25/2018 20:04:07 ANS0361I DIAG: psNpOpen(): Timed out waiting for pipe '\\.\pipe\jnlSessionMgr' to become free.
06025/25/20 20:04:09 ANANS06161IDIAG:G: sNpNpOpen): Timimed ut w waiing g or r pipe '\\\.\pipe\\jnlSessiionMgr' to becomme free..
06/25/2018 20:04:10 ANS0361I DIAG: psNpOpen(): Timed out waiting for pipe '\\.\pipe\jnlSessionMgr' to become free.
06/25/2018 20:04:12 ANS0361I DIAG: psNpOpen(): Timed out waiting for pipe '\\.\pipe\jnlSessionMgr' to become free.
06/25/2018 20:04:14 ANS0361I DIAG: psNpOpen(): Timed out waiting for pipe '\\.\pipe\jnlSessionMgr' to become free.
06/25/2018 20:04:15 ANS0361I DIAG: psNpOpen(): Timed out waiting for pipe '\\.\pipe\jnlSessionMgr' to become free.
06/25/2018 20:04:16 ANS0361I DIAG: psNpOpen(): Timed out waiting for pipe '\\.\pipe\jnlSessionMgr' to become free.
06/25/2018 20:04:20 ANS0361I0DIA/G: psNpOpen(): TiANS03m1I DIAt wasiting for pipe '\\.\paiting for pipe '\\.\pipe\jnlSessio gr' to become free.
06/25/2018 20:04:23 ANS0361I DIAG: psNpOpen(): Timed out waiting for pipe '\\.\pipe\jnlSessionMgr' to become free.
06/25/2018 20:04:30 ANS0361I DIAG: psNpOpen(): Timed out waiting for pipe '\\.\pipe\jnlSessionMgr' to become free.
06/25/2018 20:04:36 ANS0361I DIAG: psNpOpen(): Timed out waiting for pipe '\\.\pipe\jnlSessionMgr' to become free.
06/25/2018 20:10:11 ANS0361I DIAG: psNpOpen(): Timed out waiting for pipe '\\.\pipe\jnlSessionMgr' to become free.

Steve Schaub, BCBST


------------------------------------------------------------------------------
Please see the following link for the BlueCross BlueShield of Tennessee E-mail disclaimer: https://urldefense.proofpoint.com/v2/url?u=http-3A__www.bcbst.com_email-5Fdisclaimer.shtm&d=DwIFAg&c=AzgFQeXLLKhxSQaoFCm29A&r=eqh5PzQPIsPArLoI_uV1mKvhIpcNP1MsClDPSJjFfxw&m=3ZYPCbhajlmKWKX8adt9QqUBWf2Y35uXHelD62NouAk&s=l8eNAE4fSprw99LGo5jkR2lNQ5UveAULVboFErt2UwA&e=

**CONFIDENTIALITY NOTICE** This electronic message contains information from Southeastern Grocers, LLC and is intended only for the use of the addressee. This message may contain information that is privileged, confidential and/or exempt from disclosure under applicable Law. This message may not be read, used, distributed, forwarded, reproduced or stored by any other than the intended recipient. If you are not the intended recipient, please delete and notify the sender
This message was imported via the External PhorumMail Module
Sorry, only registered users may post in this forum.

Click here to login