Backup Central

I upgraded from 7.6.1 to 7.6.3.4.Build.879 on May 30. Starting this weekend, an unusually high number of backups failed, many with an inactivity error message in the savegroup report.

This is a 64-bit Red Hat Linux 5.1 box with 64-bit NetWorker. All backups write to any of 11 DD Boost devices. No storage nodes are in use. Some of the savegroups are set up to clone to LTO-5 tape after their backups finish.

In the daemon.raw file and in nsrwatch, I am seeing errors such as the following

Mon 02:36:34 PM media emergency: Cannot write to /nw11/73/97/d49401a3-00000006-7fd606b0-4fd606b0-00821000-60d99518 - No such file or directory
Mon 02:36:34 PM media info: save set F:\ for client remotefile.dental.temple.edu was aborted and removed from volume DDBoost11.001
Mon 02:36:34 PM media emergency: Cannot write to /nw11/43/26/f63a037c-00000006-77d606d1-4fd606d1-008a1000-60d99518 - No such file or directory
Mon 02:36:34 PM media info: save set D:\ for client remotefile.dental.temple.edu was aborted and removed from volume DDBoost11.001

I opened a case with EMC open on this, this morning. I restarted NetWorker and restarted some of the failed groups. Some of the failed groups are still in progress; others have completed successfully.

The EMC person who first began working with me on this case switched it over to someone else, who has not yet picked up the case, but before handing over the case, he recommend I upgrade to the latest DD system OS due to a timing issue. I am at 5.0 now. I am a little skeptical of that recommendation.

Has anyone else seen this problem? If so, what did you do about it to fix it?

If your ticket is open with the networker side of the house I'd open one with data domain support, or have networker support do it for you. They may want to check to make sure your DD file system isn't crashing, and they can also look through the ddfs info file (I forgot the actual name of the file). The filesystem normally restarts itself after crashing, but of course that will wreak havoc on your currently running backups if it does crash.

-----Original Message-----
From: EMC NetWorker discussion [mailto:NETWORKER < at > LISTSERV.TEMPLE.EDU] On Behalf Of Stan Horwitz
Sent: Monday, June 11, 2012 1:53 PM
To: NETWORKER < at > LISTSERV.TEMPLE.EDU
Subject: [Networker] NetWorker 7.6.3 with DD Boost

I upgraded from 7.6.1 to 7.6.3.4.Build.879 on May 30. Starting this weekend, an unusually high number of backups failed, many with an inactivity error message in the savegroup report.

This is a 64-bit Red Hat Linux 5.1 box with 64-bit NetWorker. All backups write to any of 11 DD Boost devices. No storage nodes are in use. Some of the savegroups are set up to clone to LTO-5 tape after their backups finish.

In the daemon.raw file and in nsrwatch, I am seeing errors such as the following

Mon 02:36:34 PM media emergency: Cannot write to /nw11/73/97/d49401a3-00000006-7fd606b0-4fd606b0-00821000-60d99518 - No such file or directory Mon 02:36:34 PM media info: save set F:\ for client remotefile.dental.temple.edu was aborted and removed from volume DDBoost11.001 Mon 02:36:34 PM media emergency: Cannot write to /nw11/43/26/f63a037c-00000006-77d606d1-4fd606d1-008a1000-60d99518 - No such file or directory Mon 02:36:34 PM media info: save set D:\ for client remotefile.dental.temple.edu was aborted and removed from volume DDBoost11.001

I opened a case with EMC open on this, this morning. I restarted NetWorker and restarted some of the failed groups. Some of the failed groups are still in progress; others have completed successfully.

The EMC person who first began working with me on this case switched it over to someone else, who has not yet picked up the case, but before handing over the case, he recommend I upgrade to the latest DD system OS due to a timing issue. I am at 5.0 now. I am a little skeptical of that recommendation.

Has anyone else seen this problem? If so, what did you do about it to fix it?

Before Upgrading the Networker software you need to be check which version is stable, I think you have installed latest networker version.

I have found that the devices go to an unmounted state during the upgrade and require you to manually remount them.

DDboost performance issue with the SQL/Exchange backups.
 
They are very slow, couldn't find the reason why, NW 7.6.3 and DDOS 5.1
 
 


________________________________
From: Chester Martin <cmartin < at > SPP.ORG>
To: NETWORKER < at > LISTSERV.TEMPLE.EDU
Sent: Monday, June 11, 2012 10:25 PM
Subject: Re: [Networker] NetWorker 7.6.3 with DD Boost

If your ticket is open with the networker side of the house I'd open one with data domain support, or have networker support do it for you.  They may want to check to make sure your DD file system isn't crashing, and they can also look through the ddfs info file (I forgot the actual name of the file).  The filesystem normally restarts itself after crashing, but of course that will wreak havoc on your currently running backups if it does crash.

-----Original Message-----
From: EMC NetWorker discussion [mailto:NETWORKER < at > LISTSERV.TEMPLE.EDU] On Behalf Of Stan Horwitz
Sent: Monday, June 11, 2012 1:53 PM
To: NETWORKER < at > LISTSERV.TEMPLE.EDU
Subject: [Networker] NetWorker 7.6.3 with DD Boost

I upgraded from 7.6.1 to 7.6.3.4.Build.879 on May 30. Starting this weekend, an unusually high number of backups failed, many with an inactivity error message in the savegroup report.

This is a 64-bit Red Hat Linux 5.1 box with 64-bit NetWorker. All backups write to any of 11 DD Boost devices. No storage nodes are in use. Some of the savegroups are set up to clone to LTO-5 tape after their backups finish.

In the daemon.raw file and in nsrwatch, I am seeing errors such as the following

Mon 02:36:34 PM media emergency: Cannot write to /nw11/73/97/d49401a3-00000006-7fd606b0-4fd606b0-00821000-60d99518 - No such file or directory  Mon 02:36:34 PM media info: save set F:\ for client remotefile.dental.temple.edu was aborted and removed from volume DDBoost11.001  Mon 02:36:34 PM media emergency: Cannot write to /nw11/43/26/f63a037c-00000006-77d606d1-4fd606d1-008a1000-60d99518 - No such file or directory  Mon 02:36:34 PM media info: save set D:\ for client remotefile.dental.temple.edu was aborted and removed from volume DDBoost11.001

I opened a case with EMC open on this, this morning. I restarted NetWorker and restarted some of the failed groups. Some of the failed groups are still in progress; others have completed successfully.

The EMC person who first began working with me on this case switched it over to someone else, who has not yet picked up the case, but before handing over the case, he recommend I upgrade to the  latest DD system OS due to a timing issue. I am at 5.0 now. I am a little skeptical of that recommendation.

Has anyone else seen this problem? If so, what did  you do about it to fix it?

Barbara,

The problem I had was fixed by upgrading the OS on our DD system from 5.0 to the current 5.1 version., but it looks like you might already be at the current version. If you haven't checked your DD system for any problems, you certainly should.

On Jun 23, 2012, at 7:31 AM, Barbara Zahn wrote:

DDboost performance issue with the SQL/Exchange backups.

They are very slow, couldn't find the reason why, NW 7.6.3 and DDOS 5.1




________________________________
From: Chester Martin <cmartin < at > SPP.ORG>
To: NETWORKER < at > LISTSERV.TEMPLE.EDU
Sent: Monday, June 11, 2012 10:25 PM
Subject: Re: [Networker] NetWorker 7.6.3 with DD Boost

If your ticket is open with the networker side of the house I'd open one with data domain support, or have networker support do it for you. They may want to check to make sure your DD file system isn't crashing, and they can also look through the ddfs info file (I forgot the actual name of the file). The filesystem normally restarts itself after crashing, but of course that will wreak havoc on your currently running backups if it does crash.

-----Original Message-----
From: EMC NetWorker discussion [mailto:NETWORKER < at > LISTSERV.TEMPLE.EDU] On Behalf Of Stan Horwitz
Sent: Monday, June 11, 2012 1:53 PM
To: NETWORKER < at > LISTSERV.TEMPLE.EDU
Subject: [Networker] NetWorker 7.6.3 with DD Boost

I upgraded from 7.6.1 to 7.6.3.4.Build.879 on May 30. Starting this weekend, an unusually high number of backups failed, many with an inactivity error message in the savegroup report.

This is a 64-bit Red Hat Linux 5.1 box with 64-bit NetWorker. All backups write to any of 11 DD Boost devices. No storage nodes are in use. Some of the savegroups are set up to clone to LTO-5 tape after their backups finish.

In the daemon.raw file and in nsrwatch, I am seeing errors such as the following

Mon 02:36:34 PM media emergency: Cannot write to /nw11/73/97/d49401a3-00000006-7fd606b0-4fd606b0-00821000-60d99518 - No such file or directory Mon 02:36:34 PM media info: save set F:\ for client remotefile.dental.temple.edu was aborted and removed from volume DDBoost11.001 Mon 02:36:34 PM media emergency: Cannot write to /nw11/43/26/f63a037c-00000006-77d606d1-4fd606d1-008a1000-60d99518 - No such file or directory Mon 02:36:34 PM media info: save set D:\ for client remotefile.dental.temple.edu was aborted and removed from volume DDBoost11.001

I opened a case with EMC open on this, this morning. I restarted NetWorker and restarted some of the failed groups. Some of the failed groups are still in progress; others have completed successfully.

The EMC person who first began working with me on this case switched it over to someone else, who has not yet picked up the case, but before handing over the case, he recommend I upgrade to the latest DD system OS due to a timing issue. I am at 5.0 now. I am a little skeptical of that recommendation.

Has anyone else seen this problem? If so, what did you do about it to fix it?