Backup Central

I have a machine that has 36 NTFS Mount Points, each with millions of files.
Since they are all hosted on the same machines, TSM Support highly
recommended setting up a node for each, and to have TSM journal each. Now I
am running into a situation where every time that machine is rebooted for
patches/etc., the incrementals have to rebuild the journal database. Is there
a way to avoid that, as I am only 1/3 of the way through the Mount Point
Usage (They have only used 12 of the 36 thus far). Please help. J



God bless you!!!

Chip Bell
Network Engineer I
IBM Tivoli Certified Deployment Professional (ITSM)
Baptist Health System
Birmingham, AL










-----------------------------------------
Confidentiality Notice:
The information contained in this email message is privileged and
confidential information and intended only for the use of the
individual or entity named in the address. If you are not the
intended recipient, you are hereby notified that any dissemination,
distribution, or copying of this information is strictly
prohibited. If you received this information in error, please
notify the sender and delete this information from your computer
and retain no copies of any of this information.

Hi Charles,

Look at the PreserveDbOnExit parameter of the tsmjbbd.ini configuration
file.

As a general reference about JBB, read the FAQ :
http://www-1.ibm.com/support/docview.wss?uid=swg21155524

--
Best regards / Cordialement / مع تحياتي
Erwann SIMON

Bell, Charles (Chip) a écrit :
I have a machine that has 36 NTFS Mount Points, each with millions of files.
Since they are all hosted on the same machines, TSM Support highly
recommended setting up a node for each, and to have TSM journal each. Now I
am running into a situation where every time that machine is rebooted for
patches/etc., the incrementals have to rebuild the journal database. Is there
a way to avoid that, as I am only 1/3 of the way through the Mount Point
Usage (They have only used 12 of the 36 thus far). Please help. J



God bless you!!!

Chip Bell
Network Engineer I
IBM Tivoli Certified Deployment Professional (ITSM)
Baptist Health System
Birmingham, AL










-----------------------------------------
Confidentiality Notice:
The information contained in this email message is privileged and
confidential information and intended only for the use of the
individual or entity named in the address. If you are not the
intended recipient, you are hereby notified that any dissemination,
distribution, or copying of this information is strictly
prohibited. If you received this information in error, please
notify the sender and delete this information from your computer
and retain no copies of any of this information.

--------------------------------------------------------------
Ce message ne contient aucun virus connu. http://www.astaro.fr


--------------------------------------------------------------
Ce message ne contient aucun virus connu. http://www.astaro.fr

Yes, that's it. You set it to 1 if you want it to retain/preserve after going
offline.

I wonder why IBM recommended I keep it to zero in my situation?

-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L < at > VM.MARIST.EDU] On Behalf Of
Erwann Simon
Sent: Wednesday, July 09, 2008 9:38 AM
To: ADSM-L < at > VM.MARIST.EDU
Subject: Re: [ADSM-L] Journaling database retaining upon reboot?

Hi Charles,

Look at the PreserveDbOnExit parameter of the tsmjbbd.ini configuration
file.

As a general reference about JBB, read the FAQ :
http://www-1.ibm.com/support/docview.wss?uid=swg21155524

--
Best regards / Cordialement / مع تحياتي
Erwann SIMON

Bell, Charles (Chip) a écrit :
I have a machine that has 36 NTFS Mount Points, each with millions of
files.
Since they are all hosted on the same machines, TSM Support highly
recommended setting up a node for each, and to have TSM journal each. Now I
am running into a situation where every time that machine is rebooted for
patches/etc., the incrementals have to rebuild the journal database. Is
there
a way to avoid that, as I am only 1/3 of the way through the Mount Point
Usage (They have only used 12 of the 36 thus far). Please help. J



God bless you!!!

Chip Bell
Network Engineer I
IBM Tivoli Certified Deployment Professional (ITSM)
Baptist Health System
Birmingham, AL










-----------------------------------------
Confidentiality Notice:
The information contained in this email message is privileged and
confidential information and intended only for the use of the
individual or entity named in the address. If you are not the
intended recipient, you are hereby notified that any dissemination,
distribution, or copying of this information is strictly
prohibited. If you received this information in error, please
notify the sender and delete this information from your computer
and retain no copies of any of this information.

--------------------------------------------------------------
Ce message ne contient aucun virus connu. http://www.astaro.fr


--------------------------------------------------------------
Ce message ne contient aucun virus connu. http://www.astaro.fr

There is some risk in using this feature. Read the cautions documented for
this option.

At 11:09 AM 7/9/2008, Bell, Charles (Chip) wrote:
I wonder why IBM recommended I keep it to zero in my situation?


--
Paul Zarnowski Ph: 607-255-4757
Manager, Storage Services Fx: 607-255-8521
719 Rhodes Hall, Ithaca, NY 14853-3801 Em: psz1 < at > cornell.edu

While the journal service is down, any file system changes are not
tracked. If a file changes while the journal service is down, it will not
be backed up until either another full (non-journaled) backup runs, OR the
file changes again and the journal service is able to track it at that
time.

If the journal service is not shut down cleanly (e.g., a cluster failover
in a cluster setup, or something else that causes the journal to not shut
down cleanly), the journal database might be in an inconsistent
("corrupt") state, after which behavior is unpredictable. "Unpredictable"
could run the gamut of files not getting backed up when they should or a
journal service crash when the corruption is encountered.

So PreserveDbOnExit=1 should only be used when you know you can stop the
journal service cleanly and you expect it to start shortly thereafter
(like in a controlled reboot scenario).

Best regards,

Andy

Andy Raibeck
IBM Software Group
Tivoli Storage Manager Client Product Development
Level 3 Team Lead
Internal Notes e-mail: Andrew Raibeck/Tucson/IBM < at > IBMUS
Internet e-mail: storman < at > us.ibm.com

IBM Tivoli Storage Manager support web page:
http://www.ibm.com/software/sysmgmt/products/support/IBMTivoliStorageManager.html

The only dumb question is the one that goes unasked.
The command line is your friend.
"Good enough" is the enemy of excellence.

"ADSM: Dist Stor Manager" <ADSM-L < at > VM.MARIST.EDU> wrote on 07/09/2008
08:09:54 AM:

Yes, that's it. You set it to 1 if you want it to retain/preserve after
going
offline.

I wonder why IBM recommended I keep it to zero in my situation?

-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L < at > VM.MARIST.EDU] On Behalf Of
Erwann Simon
Sent: Wednesday, July 09, 2008 9:38 AM
To: ADSM-L < at > VM.MARIST.EDU
Subject: Re: [ADSM-L] Journaling database retaining upon reboot?

Hi Charles,

Look at the PreserveDbOnExit parameter of the tsmjbbd.ini configuration
file.

As a general reference about JBB, read the FAQ :
http://www-1.ibm.com/support/docview.wss?uid=swg21155524

--
Best regards / Cordialement / ãÚ ÊÍíÇÊí
Erwann SIMON

Bell, Charles (Chip) a écrit :
I have a machine that has 36 NTFS Mount Points, each with millions of
files.
Since they are all hosted on the same machines, TSM Support highly
recommended setting up a node for each, and to have TSM journal each.
Now I
am running into a situation where every time that machine is rebooted
for
patches/etc., the incrementals have to rebuild the journal database.
Is
there
a way to avoid that, as I am only 1/3 of the way through the Mount
Point
Usage (They have only used 12 of the 36 thus far). Please help. J



God bless you!!!

Chip Bell
Network Engineer I
IBM Tivoli Certified Deployment Professional (ITSM)
Baptist Health System
Birmingham, AL










-----------------------------------------
Confidentiality Notice:
The information contained in this email message is privileged and
confidential information and intended only for the use of the
individual or entity named in the address. If you are not the
intended recipient, you are hereby notified that any dissemination,
distribution, or copying of this information is strictly
prohibited. If you received this information in error, please
notify the sender and delete this information from your computer
and retain no copies of any of this information.

--------------------------------------------------------------
Ce message ne contient aucun virus connu. http://www.astaro.fr


--------------------------------------------------------------
Ce message ne contient aucun virus connu. http://www.astaro.fr

The way the downtime for this particular machine goes, there should be no
file system changes occurring.

Sooo...with all of these issues with jbb, are they being worked
on/fine-tuned?

-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L < at > VM.MARIST.EDU] On Behalf Of
Andrew Raibeck
Sent: Wednesday, July 09, 2008 10:40 AM
To: ADSM-L < at > VM.MARIST.EDU
Subject: Re: [ADSM-L] Journaling database retaining upon reboot?

While the journal service is down, any file system changes are not
tracked. If a file changes while the journal service is down, it will not
be backed up until either another full (non-journaled) backup runs, OR the
file changes again and the journal service is able to track it at that
time.

If the journal service is not shut down cleanly (e.g., a cluster failover
in a cluster setup, or something else that causes the journal to not shut
down cleanly), the journal database might be in an inconsistent
("corrupt") state, after which behavior is unpredictable. "Unpredictable"
could run the gamut of files not getting backed up when they should or a
journal service crash when the corruption is encountered.

So PreserveDbOnExit=1 should only be used when you know you can stop the
journal service cleanly and you expect it to start shortly thereafter
(like in a controlled reboot scenario).

Best regards,

Andy

Andy Raibeck
IBM Software Group
Tivoli Storage Manager Client Product Development
Level 3 Team Lead
Internal Notes e-mail: Andrew Raibeck/Tucson/IBM < at > IBMUS
Internet e-mail: storman < at > us.ibm.com

IBM Tivoli Storage Manager support web page:
http://www.ibm.com/software/sysmgmt/products/support/IBMTivoliStorageManager.
html

The only dumb question is the one that goes unasked.
The command line is your friend.
"Good enough" is the enemy of excellence.

"ADSM: Dist Stor Manager" <ADSM-L < at > VM.MARIST.EDU> wrote on 07/09/2008
08:09:54 AM:

Yes, that's it. You set it to 1 if you want it to retain/preserve after
going
offline.

I wonder why IBM recommended I keep it to zero in my situation?

-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L < at > VM.MARIST.EDU] On Behalf Of
Erwann Simon
Sent: Wednesday, July 09, 2008 9:38 AM
To: ADSM-L < at > VM.MARIST.EDU
Subject: Re: [ADSM-L] Journaling database retaining upon reboot?

Hi Charles,

Look at the PreserveDbOnExit parameter of the tsmjbbd.ini configuration
file.

As a general reference about JBB, read the FAQ :
http://www-1.ibm.com/support/docview.wss?uid=swg21155524

--
Best regards / Cordialement / ãÚ ÊÍíÇÊí
Erwann SIMON

Bell, Charles (Chip) a écrit :
I have a machine that has 36 NTFS Mount Points, each with millions of
files.
Since they are all hosted on the same machines, TSM Support highly
recommended setting up a node for each, and to have TSM journal each.
Now I
am running into a situation where every time that machine is rebooted
for
patches/etc., the incrementals have to rebuild the journal database.
Is
there
a way to avoid that, as I am only 1/3 of the way through the Mount
Point
Usage (They have only used 12 of the 36 thus far). Please help. J



God bless you!!!

Chip Bell
Network Engineer I
IBM Tivoli Certified Deployment Professional (ITSM)
Baptist Health System
Birmingham, AL










-----------------------------------------
Confidentiality Notice:
The information contained in this email message is privileged and
confidential information and intended only for the use of the
individual or entity named in the address. If you are not the
intended recipient, you are hereby notified that any dissemination,
distribution, or copying of this information is strictly
prohibited. If you received this information in error, please
notify the sender and delete this information from your computer
and retain no copies of any of this information.

--------------------------------------------------------------
Ce message ne contient aucun virus connu. http://www.astaro.fr


--------------------------------------------------------------
Ce message ne contient aucun virus connu. http://www.astaro.fr

Sooo...with all of these issues with jbb, are they being worked
on/fine-tuned?

Yes and no.

For the case where you use PreserveDbOnExit=1 and the journal service is
shut down for some period of time, the behavior will remain unchanged. Use
at your own risk. If the journal service is down, it cannot effectively
monitor file system changes so there is no known solution for this.

For the journal corruption issue, we are currently targeting an
enhancement that allows the journal service to know whether the database
was closed cleanly (by a clean journal service shutdown). Upon restart, if
the journal service does not detect the "I was closed cleanly" flag in the
journal database, it will effectively ignore PreserveDbOnExit=1 and reset
the journal database, as if PreserveDbOnExit=0 was in effect. This would
be in a future release of the product. Note that this does not represent a
formal announcement or commitment, so this is subject to change.

A THEORETICALLY cleaner solution could include implementation of of a
journal database with transactional capabilities, including rollback, more
thorough structural integrity checking, etc. Even in a rollback scenario,
in-flight file system changes would be lost (like the case where you use
PreserveDbOnExit=1 and journal service is down for some period of time),
leaving a less than perfect solution. But this is all hypothetical, and
there are no plans in place.

Best regards,

Andy

Andy Raibeck
IBM Software Group
Tivoli Storage Manager Client Product Development
Level 3 Team Lead
Internal Notes e-mail: Andrew Raibeck/Tucson/IBM < at > IBMUS
Internet e-mail: storman < at > us.ibm.com

IBM Tivoli Storage Manager support web page:
http://www.ibm.com/software/sysmgmt/products/support/IBMTivoliStorageManager.html

The only dumb question is the one that goes unasked.
The command line is your friend.
"Good enough" is the enemy of excellence.

"ADSM: Dist Stor Manager" <ADSM-L < at > VM.MARIST.EDU> wrote on 07/09/2008
09:02:05 AM:

The way the downtime for this particular machine goes, there should be
no
file system changes occurring.

Sooo...with all of these issues with jbb, are they being worked
on/fine-tuned?

-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L < at > VM.MARIST.EDU] On Behalf Of
Andrew Raibeck
Sent: Wednesday, July 09, 2008 10:40 AM
To: ADSM-L < at > VM.MARIST.EDU
Subject: Re: [ADSM-L] Journaling database retaining upon reboot?

While the journal service is down, any file system changes are not
tracked. If a file changes while the journal service is down, it will
not
be backed up until either another full (non-journaled) backup runs, OR
the
file changes again and the journal service is able to track it at that
time.

If the journal service is not shut down cleanly (e.g., a cluster
failover
in a cluster setup, or something else that causes the journal to not
shut
down cleanly), the journal database might be in an inconsistent
("corrupt") state, after which behavior is unpredictable.
"Unpredictable"
could run the gamut of files not getting backed up when they should or a

journal service crash when the corruption is encountered.

So PreserveDbOnExit=1 should only be used when you know you can stop the

journal service cleanly and you expect it to start shortly thereafter
(like in a controlled reboot scenario).

Best regards,

Andy

Andy Raibeck
IBM Software Group
Tivoli Storage Manager Client Product Development
Level 3 Team Lead
Internal Notes e-mail: Andrew Raibeck/Tucson/IBM < at > IBMUS
Internet e-mail: storman < at > us.ibm.com

IBM Tivoli Storage Manager support web page:

http://www.ibm.com/software/sysmgmt/products/support/IBMTivoliStorageManager.
html

The only dumb question is the one that goes unasked.
The command line is your friend.
"Good enough" is the enemy of excellence.

"ADSM: Dist Stor Manager" <ADSM-L < at > VM.MARIST.EDU> wrote on 07/09/2008
08:09:54 AM:

Yes, that's it. You set it to 1 if you want it to retain/preserve
after
going
offline.

I wonder why IBM recommended I keep it to zero in my situation?

-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L < at > VM.MARIST.EDU] On Behalf
Of
Erwann Simon
Sent: Wednesday, July 09, 2008 9:38 AM
To: ADSM-L < at > VM.MARIST.EDU
Subject: Re: [ADSM-L] Journaling database retaining upon reboot?

Hi Charles,

Look at the PreserveDbOnExit parameter of the tsmjbbd.ini
configuration
file.

As a general reference about JBB, read the FAQ :
http://www-1.ibm.com/support/docview.wss?uid=swg21155524

--
Best regards / Cordialement / ãÚ ÊÍíÇÊí
Erwann SIMON

Bell, Charles (Chip) a écrit :
I have a machine that has 36 NTFS Mount Points, each with millions
of
files.
Since they are all hosted on the same machines, TSM Support highly
recommended setting up a node for each, and to have TSM journal
each.
Now I
am running into a situation where every time that machine is
rebooted
for
patches/etc., the incrementals have to rebuild the journal database.

Is
there
a way to avoid that, as I am only 1/3 of the way through the Mount
Point
Usage (They have only used 12 of the 36 thus far). Please help. J



God bless you!!!

Chip Bell
Network Engineer I
IBM Tivoli Certified Deployment Professional (ITSM)
Baptist Health System
Birmingham, AL










-----------------------------------------
Confidentiality Notice:
The information contained in this email message is privileged and
confidential information and intended only for the use of the
individual or entity named in the address. If you are not the
intended recipient, you are hereby notified that any dissemination,
distribution, or copying of this information is strictly
prohibited. If you received this information in error, please
notify the sender and delete this information from your computer
and retain no copies of any of this information.

--------------------------------------------------------------
Ce message ne contient aucun virus connu. http://www.astaro.fr


--------------------------------------------------------------
Ce message ne contient aucun virus connu. http://www.astaro.fr

Forgive me here, but I think whoever is setting this up is going down the wrong path. It's getting very irritating to say the least and I share the frustration others are having with IBM's apparent inability to get this right.

The "Journaling system" should not be "actively" monitoring the file system anyway. It should only be a means of tracking what was backed up so that TSM doesn't have to download the entire inventory from the server when it does a backup.

If we take this approach, then it doesn't matter whether the service is left running or not. If the journal, or record, was saved when the backup finished including only what was backed up and all of the pertinent data needed to make sure files have not changed when the next backup starts.

You see the problem here is that if the Journal db gets deleted every time you reboot the server then you have to do a complete incremental, and from what I have seen a selective doesn't cut it. That means I have to delete the file space from TSM to get an incremental to complete when I have millions of small files. This is just crappy, and I can't think of a better way to put it.

I think the Devs should either fix the journaling system, or fix TSM so that it doesn't have to download 4 GB of inventory to do a backup if you have millions of small files.

See Ya'
Howard


Yes and no.

For the case where you use PreserveDbOnExit=1 and the journal service
is
shut down for some period of time, the behavior will remain unchanged.
Use
at your own risk. If the journal service is down, it cannot effectively
monitor file system changes so there is no known solution for this.

For the journal corruption issue, we are currently targeting an
enhancement that allows the journal service to know whether the
database
was closed cleanly (by a clean journal service shutdown). Upon restart,
if
the journal service does not detect the "I was closed cleanly" flag in
the
journal database, it will effectively ignore PreserveDbOnExit=1 and
reset
the journal database, as if PreserveDbOnExit=0 was in effect. This
would
be in a future release of the product. Note that this does not
represent a
formal announcement or commitment, so this is subject to change.

A THEORETICALLY cleaner solution could include implementation of of a
journal database with transactional capabilities, including rollback,
more
thorough structural integrity checking, etc. Even in a rollback
scenario,
in-flight file system changes would be lost (like the case where you
use
PreserveDbOnExit=1 and journal service is down for some period of
time),
leaving a less than perfect solution. But this is all hypothetical, and
there are no plans in place.

Best regards,

Andy

Andy Raibeck
IBM Software Group
Tivoli Storage Manager Client Product Development
Level 3 Team Lead
Internal Notes e-mail: Andrew Raibeck/Tucson/IBM < at > IBMUS
Internet e-mail: storman < at > us.ibm.com

IBM Tivoli Storage Manager support web page:
http://www.ibm.com/software/sysmgmt/products/support/IBMTivoliStorageMa
nager.html

The only dumb question is the one that goes unasked.
The command line is your friend.
"Good enough" is the enemy of excellence.

"ADSM: Dist Stor Manager" <ADSM-L < at > VM.MARIST.EDU> wrote on 07/09/2008
09:02:05 AM:

The way the downtime for this particular machine goes, there should
be
no
file system changes occurring.

Sooo...with all of these issues with jbb, are they being worked
on/fine-tuned?

-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L < at > VM.MARIST.EDU] On Behalf
Of
Andrew Raibeck
Sent: Wednesday, July 09, 2008 10:40 AM
To: ADSM-L < at > VM.MARIST.EDU
Subject: Re: [ADSM-L] Journaling database retaining upon reboot?

While the journal service is down, any file system changes are not
tracked. If a file changes while the journal service is down, it will
not
be backed up until either another full (non-journaled) backup runs,
OR
the
file changes again and the journal service is able to track it at
that
time.

If the journal service is not shut down cleanly (e.g., a cluster
failover
in a cluster setup, or something else that causes the journal to not
shut
down cleanly), the journal database might be in an inconsistent
("corrupt") state, after which behavior is unpredictable.
"Unpredictable"
could run the gamut of files not getting backed up when they should
or a

journal service crash when the corruption is encountered.

So PreserveDbOnExit=1 should only be used when you know you can stop
the

journal service cleanly and you expect it to start shortly thereafter
(like in a controlled reboot scenario).

Best regards,

Andy

Andy Raibeck
IBM Software Group
Tivoli Storage Manager Client Product Development
Level 3 Team Lead
Internal Notes e-mail: Andrew Raibeck/Tucson/IBM < at > IBMUS
Internet e-mail: storman < at > us.ibm.com

IBM Tivoli Storage Manager support web page:

http://www.ibm.com/software/sysmgmt/products/support/IBMTivoliStorageMa
nager.
html

The only dumb question is the one that goes unasked.
The command line is your friend.
"Good enough" is the enemy of excellence.

"ADSM: Dist Stor Manager" <ADSM-L < at > VM.MARIST.EDU> wrote on 07/09/2008
08:09:54 AM:

Yes, that's it. You set it to 1 if you want it to retain/preserve
after
going
offline.

I wonder why IBM recommended I keep it to zero in my situation?

-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L < at > VM.MARIST.EDU] On
Behalf
Of
Erwann Simon
Sent: Wednesday, July 09, 2008 9:38 AM
To: ADSM-L < at > VM.MARIST.EDU
Subject: Re: [ADSM-L] Journaling database retaining upon reboot?

Hi Charles,

Look at the PreserveDbOnExit parameter of the tsmjbbd.ini
configuration
file.

As a general reference about JBB, read the FAQ :
http://www-1.ibm.com/support/docview.wss?uid=swg21155524

--
Best regards / Cordialement / ãÚ ÊÍíÇÊí
Erwann SIMON

Bell, Charles (Chip) a écrit :
I have a machine that has 36 NTFS Mount Points, each with
millions
of
files.
Since they are all hosted on the same machines, TSM Support
highly
recommended setting up a node for each, and to have TSM journal
each.
Now I
am running into a situation where every time that machine is
rebooted
for
patches/etc., the incrementals have to rebuild the journal
database.

Is
there
a way to avoid that, as I am only 1/3 of the way through the
Mount
Point
Usage (They have only used 12 of the 36 thus far). Please help.
J



God bless you!!!

Chip Bell
Network Engineer I
IBM Tivoli Certified Deployment Professional (ITSM)
Baptist Health System
Birmingham, AL










-----------------------------------------
Confidentiality Notice:
The information contained in this email message is privileged and
confidential information and intended only for the use of the
individual or entity named in the address. If you are not the
intended recipient, you are hereby notified that any
dissemination,
distribution, or copying of this information is strictly
prohibited. If you received this information in error, please
notify the sender and delete this information from your computer
and retain no copies of any of this information.

--------------------------------------------------------------
Ce message ne contient aucun virus connu. http://www.astaro.fr


--------------------------------------------------------------
Ce message ne contient aucun virus connu. http://www.astaro.fr

Hi, Andy.

Maybe add an option to the journal service setup that could have the
filesystem filter driver fail writes to the filesystem if the journal
service is down? Of course, this would have the obvious serious
drawback, but maybe in some horrendously large filecount environments
users might decide it's worth it to enforce journal db/filesystem
synchronization.


-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L < at > VM.MARIST.EDU] On Behalf Of
Andrew Raibeck
Sent: Wednesday, July 09, 2008 9:42 AM
To: ADSM-L < at > VM.MARIST.EDU
Subject: Re: [ADSM-L] Journaling database retaining upon reboot?

Sooo...with all of these issues with jbb, are they being worked
on/fine-tuned?

Yes and no.

For the case where you use PreserveDbOnExit=1 and the journal service is

shut down for some period of time, the behavior will remain unchanged.
Use
at your own risk. If the journal service is down, it cannot effectively
monitor file system changes so there is no known solution for this.

For the journal corruption issue, we are currently targeting an
enhancement that allows the journal service to know whether the database

was closed cleanly (by a clean journal service shutdown). Upon restart,
if
the journal service does not detect the "I was closed cleanly" flag in
the
journal database, it will effectively ignore PreserveDbOnExit=1 and
reset
the journal database, as if PreserveDbOnExit=0 was in effect. This would

be in a future release of the product. Note that this does not represent
a
formal announcement or commitment, so this is subject to change.

A THEORETICALLY cleaner solution could include implementation of of a
journal database with transactional capabilities, including rollback,
more
thorough structural integrity checking, etc. Even in a rollback
scenario,
in-flight file system changes would be lost (like the case where you use

PreserveDbOnExit=1 and journal service is down for some period of time),

leaving a less than perfect solution. But this is all hypothetical, and
there are no plans in place.

Best regards,

Andy

Andy Raibeck
IBM Software Group
Tivoli Storage Manager Client Product Development
Level 3 Team Lead
Internal Notes e-mail: Andrew Raibeck/Tucson/IBM < at > IBMUS
Internet e-mail: storman < at > us.ibm.com

IBM Tivoli Storage Manager support web page:
http://www.ibm.com/software/sysmgmt/products/support/IBMTivoliStorageMan
ager.html

The only dumb question is the one that goes unasked.
The command line is your friend.
"Good enough" is the enemy of excellence.


This message (including any attachments) is intended only for
the use of the individual or entity to which it is addressed and
may contain information that is non-public, proprietary,
privileged, confidential, and exempt from disclosure under
applicable law or may constitute as attorney work product.
If you are not the intended recipient, you are hereby notified
that any use, dissemination, distribution, or copying of this
communication is strictly prohibited. If you have received this
communication in error, notify us immediately by telephone and
(i) destroy this message if a facsimile or (ii) delete this message
immediately if this is an electronic communication.

Thank you.

Your situation is screaming for hardware snapshots instead of TSM. Try to get management to look at something like NetApp vfilers and remote mirrors.

Orville L. Lantto


________________________________

From: ADSM: Dist Stor Manager on behalf of Bell, Charles (Chip)
Sent: Wed 7/9/2008 09:26
To: ADSM-L < at > VM.MARIST.EDU
Subject: [ADSM-L] Journaling database retaining upon reboot?



I have a machine that has 36 NTFS Mount Points, each with millions of files.
Since they are all hosted on the same machines, TSM Support highly
recommended setting up a node for each, and to have TSM journal each. Now I
am running into a situation where every time that machine is rebooted for
patches/etc., the incrementals have to rebuild the journal database. Is there
a way to avoid that, as I am only 1/3 of the way through the Mount Point
Usage (They have only used 12 of the 36 thus far). Please help. J



God bless you!!!

Chip Bell
Network Engineer I
IBM Tivoli Certified Deployment Professional (ITSM)
Baptist Health System
Birmingham, AL










-----------------------------------------
Confidentiality Notice:
The information contained in this email message is privileged and
confidential information and intended only for the use of the
individual or entity named in the address. If you are not the
intended recipient, you are hereby notified that any dissemination,
distribution, or copying of this information is strictly
prohibited. If you received this information in error, please
notify the sender and delete this information from your computer
and retain no copies of any of this information.







This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail.

Well, I have to have some relief here, considering there's nothing that I can
do about the setup. That means after they are using all 36 mount point
(millions of files each), and there's a reboot, 36 different backups with
journaling are going to have to start from scratch. Not only is that
terrible, but there's nothing I can do about it! So for those reading this,
what would you do in my situation, set PreserveDbOnExit to 1 or 0?

The TSM client host in question has 36 NTFS mount points, each will 5
million+ files. Per IBM TSM Support recommendations, I set the host machine
up with a TSM node for each mount point (therefore, 36), and configured
journaling for each mount point as well. All per their guidelines and
instruction. The application (an imaging app) on the host machine writes to a
mount point until reaching a 80% utilized threshold, and then begins writing
to the next numerical mount point. Last February, they started with mount
point 1, and now they are on 12. That means when they patched the machine
last night and rebooted, several backups have failed with memory errors, and
there are 4 running right now, each just spinning their wheels on directories
that mostly unchanged. I am really trying to remain calm, though it's
sickening to think that there is simply nothing that I can do about it.

-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L < at > VM.MARIST.EDU] On Behalf Of
Andrew Raibeck
Sent: Wednesday, July 09, 2008 11:42 AM
To: ADSM-L < at > VM.MARIST.EDU
Subject: Re: [ADSM-L] Journaling database retaining upon reboot?

Sooo...with all of these issues with jbb, are they being worked
on/fine-tuned?

Yes and no.

For the case where you use PreserveDbOnExit=1 and the journal service is
shut down for some period of time, the behavior will remain unchanged. Use
at your own risk. If the journal service is down, it cannot effectively
monitor file system changes so there is no known solution for this.

For the journal corruption issue, we are currently targeting an
enhancement that allows the journal service to know whether the database
was closed cleanly (by a clean journal service shutdown). Upon restart, if
the journal service does not detect the "I was closed cleanly" flag in the
journal database, it will effectively ignore PreserveDbOnExit=1 and reset
the journal database, as if PreserveDbOnExit=0 was in effect. This would
be in a future release of the product. Note that this does not represent a
formal announcement or commitment, so this is subject to change.

A THEORETICALLY cleaner solution could include implementation of of a
journal database with transactional capabilities, including rollback, more
thorough structural integrity checking, etc. Even in a rollback scenario,
in-flight file system changes would be lost (like the case where you use
PreserveDbOnExit=1 and journal service is down for some period of time),
leaving a less than perfect solution. But this is all hypothetical, and
there are no plans in place.

Best regards,

Andy

Andy Raibeck
IBM Software Group
Tivoli Storage Manager Client Product Development
Level 3 Team Lead
Internal Notes e-mail: Andrew Raibeck/Tucson/IBM < at > IBMUS
Internet e-mail: storman < at > us.ibm.com

IBM Tivoli Storage Manager support web page:
http://www.ibm.com/software/sysmgmt/products/support/IBMTivoliStorageManager.
html

The only dumb question is the one that goes unasked.
The command line is your friend.
"Good enough" is the enemy of excellence.

"ADSM: Dist Stor Manager" <ADSM-L < at > VM.MARIST.EDU> wrote on 07/09/2008
09:02:05 AM:

The way the downtime for this particular machine goes, there should be
no
file system changes occurring.

Sooo...with all of these issues with jbb, are they being worked
on/fine-tuned?

-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L < at > VM.MARIST.EDU] On Behalf Of
Andrew Raibeck
Sent: Wednesday, July 09, 2008 10:40 AM
To: ADSM-L < at > VM.MARIST.EDU
Subject: Re: [ADSM-L] Journaling database retaining upon reboot?

While the journal service is down, any file system changes are not
tracked. If a file changes while the journal service is down, it will
not
be backed up until either another full (non-journaled) backup runs, OR
the
file changes again and the journal service is able to track it at that
time.

If the journal service is not shut down cleanly (e.g., a cluster
failover
in a cluster setup, or something else that causes the journal to not
shut
down cleanly), the journal database might be in an inconsistent
("corrupt") state, after which behavior is unpredictable.
"Unpredictable"
could run the gamut of files not getting backed up when they should or a

journal service crash when the corruption is encountered.

So PreserveDbOnExit=1 should only be used when you know you can stop the

journal service cleanly and you expect it to start shortly thereafter
(like in a controlled reboot scenario).

Best regards,

Andy

Andy Raibeck
IBM Software Group
Tivoli Storage Manager Client Product Development
Level 3 Team Lead
Internal Notes e-mail: Andrew Raibeck/Tucson/IBM < at > IBMUS
Internet e-mail: storman < at > us.ibm.com

IBM Tivoli Storage Manager support web page:

http://www.ibm.com/software/sysmgmt/products/support/IBMTivoliStorageManager.
html

The only dumb question is the one that goes unasked.
The command line is your friend.
"Good enough" is the enemy of excellence.

"ADSM: Dist Stor Manager" <ADSM-L < at > VM.MARIST.EDU> wrote on 07/09/2008
08:09:54 AM:

Yes, that's it. You set it to 1 if you want it to retain/preserve
after
going
offline.

I wonder why IBM recommended I keep it to zero in my situation?

-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L < at > VM.MARIST.EDU] On Behalf
Of
Erwann Simon
Sent: Wednesday, July 09, 2008 9:38 AM
To: ADSM-L < at > VM.MARIST.EDU
Subject: Re: [ADSM-L] Journaling database retaining upon reboot?

Hi Charles,

Look at the PreserveDbOnExit parameter of the tsmjbbd.ini
configuration
file.

As a general reference about JBB, read the FAQ :
http://www-1.ibm.com/support/docview.wss?uid=swg21155524

--
Best regards / Cordialement / ãÚ ÊÍíÇÊí
Erwann SIMON

Bell, Charles (Chip) a écrit :
I have a machine that has 36 NTFS Mount Points, each with millions
of
files.
Since they are all hosted on the same machines, TSM Support highly
recommended setting up a node for each, and to have TSM journal
each.
Now I
am running into a situation where every time that machine is
rebooted
for
patches/etc., the incrementals have to rebuild the journal database.

Is
there
a way to avoid that, as I am only 1/3 of the way through the Mount
Point
Usage (They have only used 12 of the 36 thus far). Please help. J



God bless you!!!

Chip Bell
Network Engineer I
IBM Tivoli Certified Deployment Professional (ITSM)
Baptist Health System
Birmingham, AL










-----------------------------------------
Confidentiality Notice:
The information contained in this email message is privileged and
confidential information and intended only for the use of the
individual or entity named in the address. If you are not the
intended recipient, you are hereby notified that any dissemination,
distribution, or copying of this information is strictly
prohibited. If you received this information in error, please
notify the sender and delete this information from your computer
and retain no copies of any of this information.

--------------------------------------------------------------
Ce message ne contient aucun virus connu. http://www.astaro.fr


--------------------------------------------------------------
Ce message ne contient aucun virus connu. http://www.astaro.fr

They did very early on in the process, and to them it screamed too expensive.
They may have to swallow their wallet eventually, and I agree with you. We
may revisit...

-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L < at > VM.MARIST.EDU] On Behalf Of
Orville Lantto
Sent: Wednesday, July 09, 2008 12:42 PM
To: ADSM-L < at > VM.MARIST.EDU
Subject: Re: [ADSM-L] Journaling database retaining upon reboot?

Your situation is screaming for hardware snapshots instead of TSM. Try to
get management to look at something like NetApp vfilers and remote mirrors.

Orville L. Lantto


________________________________

From: ADSM: Dist Stor Manager on behalf of Bell, Charles (Chip)
Sent: Wed 7/9/2008 09:26
To: ADSM-L < at > VM.MARIST.EDU
Subject: [ADSM-L] Journaling database retaining upon reboot?



I have a machine that has 36 NTFS Mount Points, each with millions of files.
Since they are all hosted on the same machines, TSM Support highly
recommended setting up a node for each, and to have TSM journal each. Now I
am running into a situation where every time that machine is rebooted for
patches/etc., the incrementals have to rebuild the journal database. Is there
a way to avoid that, as I am only 1/3 of the way through the Mount Point
Usage (They have only used 12 of the 36 thus far). Please help. J



God bless you!!!

Chip Bell
Network Engineer I
IBM Tivoli Certified Deployment Professional (ITSM)
Baptist Health System
Birmingham, AL










-----------------------------------------
Confidentiality Notice:
The information contained in this email message is privileged and
confidential information and intended only for the use of the
individual or entity named in the address. If you are not the
intended recipient, you are hereby notified that any dissemination,
distribution, or copying of this information is strictly
prohibited. If you received this information in error, please
notify the sender and delete this information from your computer
and retain no copies of any of this information.







This email and any files transmitted with it are confidential and intended
solely for the use of the individual or entity to whom they are addressed. If
you have received this email in error please notify the system manager. This
message contains confidential information and is intended only for the
individual named. If you are not the named addressee you should not
disseminate, distribute or copy this e-mail.

Can the PreserveDbOnExit be changed on the fly so that it can be
turned on as part of controlled reboot, then left off in case of a
crash?

-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L < at > VM.MARIST.EDU]On Behalf Of Bell, Charles (Chip)
Sent: Wednesday, July 09, 2008 10:48 AM
To: ADSM-L < at > VM.MARIST.EDU
Subject: Re: Journaling database retaining upon reboot?


Well, I have to have some relief here, considering there's nothing that I can
do about the setup. That means after they are using all 36 mount point
(millions of files each), and there's a reboot, 36 different backups with
journaling are going to have to start from scratch. Not only is that
terrible, but there's nothing I can do about it! So for those reading this,
what would you do in my situation, set PreserveDbOnExit to 1 or 0?

The TSM client host in question has 36 NTFS mount points, each will 5
million+ files. Per IBM TSM Support recommendations, I set the host machine
up with a TSM node for each mount point (therefore, 36), and configured
journaling for each mount point as well. All per their guidelines and
instruction. The application (an imaging app) on the host machine writes to a
mount point until reaching a 80% utilized threshold, and then begins writing
to the next numerical mount point. Last February, they started with mount
point 1, and now they are on 12. That means when they patched the machine
last night and rebooted, several backups have failed with memory errors, and
there are 4 running right now, each just spinning their wheels on directories
that mostly unchanged. I am really trying to remain calm, though it's
sickening to think that there is simply nothing that I can do about it.

-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L < at > VM.MARIST.EDU] On Behalf Of
Andrew Raibeck
Sent: Wednesday, July 09, 2008 11:42 AM
To: ADSM-L < at > VM.MARIST.EDU
Subject: Re: [ADSM-L] Journaling database retaining upon reboot?

Sooo...with all of these issues with jbb, are they being worked
on/fine-tuned?

Yes and no.

For the case where you use PreserveDbOnExit=1 and the journal service is
shut down for some period of time, the behavior will remain unchanged. Use
at your own risk. If the journal service is down, it cannot effectively
monitor file system changes so there is no known solution for this.

For the journal corruption issue, we are currently targeting an
enhancement that allows the journal service to know whether the database
was closed cleanly (by a clean journal service shutdown). Upon restart, if
the journal service does not detect the "I was closed cleanly" flag in the
journal database, it will effectively ignore PreserveDbOnExit=1 and reset
the journal database, as if PreserveDbOnExit=0 was in effect. This would
be in a future release of the product. Note that this does not represent a
formal announcement or commitment, so this is subject to change.

A THEORETICALLY cleaner solution could include implementation of of a
journal database with transactional capabilities, including rollback, more
thorough structural integrity checking, etc. Even in a rollback scenario,
in-flight file system changes would be lost (like the case where you use
PreserveDbOnExit=1 and journal service is down for some period of time),
leaving a less than perfect solution. But this is all hypothetical, and
there are no plans in place.

Best regards,

Andy

Andy Raibeck
IBM Software Group
Tivoli Storage Manager Client Product Development
Level 3 Team Lead
Internal Notes e-mail: Andrew Raibeck/Tucson/IBM < at > IBMUS
Internet e-mail: storman < at > us.ibm.com

IBM Tivoli Storage Manager support web page:
http://www.ibm.com/software/sysmgmt/products/support/IBMTivoliStorageManager.
html

The only dumb question is the one that goes unasked.
The command line is your friend.
"Good enough" is the enemy of excellence.

"ADSM: Dist Stor Manager" <ADSM-L < at > VM.MARIST.EDU> wrote on 07/09/2008
09:02:05 AM:

The way the downtime for this particular machine goes, there should be
no
file system changes occurring.

Sooo...with all of these issues with jbb, are they being worked
on/fine-tuned?

-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L < at > VM.MARIST.EDU] On Behalf Of
Andrew Raibeck
Sent: Wednesday, July 09, 2008 10:40 AM
To: ADSM-L < at > VM.MARIST.EDU
Subject: Re: [ADSM-L] Journaling database retaining upon reboot?

While the journal service is down, any file system changes are not
tracked. If a file changes while the journal service is down, it will
not
be backed up until either another full (non-journaled) backup runs, OR
the
file changes again and the journal service is able to track it at that
time.

If the journal service is not shut down cleanly (e.g., a cluster
failover
in a cluster setup, or something else that causes the journal to not
shut
down cleanly), the journal database might be in an inconsistent
("corrupt") state, after which behavior is unpredictable.
"Unpredictable"
could run the gamut of files not getting backed up when they should or a

journal service crash when the corruption is encountered.

So PreserveDbOnExit=1 should only be used when you know you can stop the

journal service cleanly and you expect it to start shortly thereafter
(like in a controlled reboot scenario).

Best regards,

Andy

Andy Raibeck
IBM Software Group
Tivoli Storage Manager Client Product Development
Level 3 Team Lead
Internal Notes e-mail: Andrew Raibeck/Tucson/IBM < at > IBMUS
Internet e-mail: storman < at > us.ibm.com

IBM Tivoli Storage Manager support web page:

http://www.ibm.com/software/sysmgmt/products/support/IBMTivoliStorageManager.
html

The only dumb question is the one that goes unasked.
The command line is your friend.
"Good enough" is the enemy of excellence.

"ADSM: Dist Stor Manager" <ADSM-L < at > VM.MARIST.EDU> wrote on 07/09/2008
08:09:54 AM:

Yes, that's it. You set it to 1 if you want it to retain/preserve
after
going
offline.

I wonder why IBM recommended I keep it to zero in my situation?

-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L < at > VM.MARIST.EDU] On Behalf
Of
Erwann Simon
Sent: Wednesday, July 09, 2008 9:38 AM
To: ADSM-L < at > VM.MARIST.EDU
Subject: Re: [ADSM-L] Journaling database retaining upon reboot?

Hi Charles,

Look at the PreserveDbOnExit parameter of the tsmjbbd.ini
configuration
file.

As a general reference about JBB, read the FAQ :
http://www-1.ibm.com/support/docview.wss?uid=swg21155524

--
Best regards / Cordialement / ãÚ ÊÍíÇÊí
Erwann SIMON

Bell, Charles (Chip) a écrit :
I have a machine that has 36 NTFS Mount Points, each with millions
of
files.
Since they are all hosted on the same machines, TSM Support highly
recommended setting up a node for each, and to have TSM journal
each.
Now I
am running into a situation where every time that machine is
rebooted
for
patches/etc., the incrementals have to rebuild the journal database.

Is
there
a way to avoid that, as I am only 1/3 of the way through the Mount
Point
Usage (They have only used 12 of the 36 thus far). Please help. J



God bless you!!!

Chip Bell
Network Engineer I
IBM Tivoli Certified Deployment Professional (ITSM)
Baptist Health System
Birmingham, AL










-----------------------------------------
Confidentiality Notice:
The information contained in this email message is privileged and
confidential information and intended only for the use of the
individual or entity named in the address. If you are not the
intended recipient, you are hereby notified that any dissemination,
distribution, or copying of this information is strictly
prohibited. If you received this information in error, please
notify the sender and delete this information from your computer
and retain no copies of any of this information.

--------------------------------------------------------------
Ce message ne contient aucun virus connu. http://www.astaro.fr


--------------------------------------------------------------
Ce message ne contient aucun virus connu. http://www.astaro.fr

On Jul 9, 2008, at 1:48 PM, Bell, Charles (Chip) wrote:

... The application (an imaging app) on the host machine writes to a
mount point until reaching a 80% utilized threshold, and then begins
writing
to the next numerical mount point. Last February, they started with
mount
point 1, and now they are on 12. ...

I would approach a situation like this on a logical basis...
In that the app is doing the Mad Hatter's Tea Party thing (abandon the
current position and move on to the next clean place at the table), I
would have the backup initiate from a script on the client, which
would run a backup of only the latest volume, and there use Incrbydate
in that the list of Active files is pointless in a case like this.
You could run an infrequent, staggered Incremental of the earlier
volumes if you felt compelled, just in case. Repeatedly running a
backup scan against static volumes is just waste.

Richard Sims

Does that plan apply, considering (something I don't think I mentioned) that
only the most recent numerical mount point is getting changed data? Most of
the other mount point backups are getting few to no files changed. A
co-worker and I were discussing a similar script...

-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L < at > VM.MARIST.EDU] On Behalf Of
Richard Sims
Sent: Wednesday, July 09, 2008 1:03 PM
To: ADSM-L < at > VM.MARIST.EDU
Subject: Re: [ADSM-L] Journaling database retaining upon reboot?

On Jul 9, 2008, at 1:48 PM, Bell, Charles (Chip) wrote:

... The application (an imaging app) on the host machine writes to a
mount point until reaching a 80% utilized threshold, and then begins
writing
to the next numerical mount point. Last February, they started with
mount
point 1, and now they are on 12. ...

I would approach a situation like this on a logical basis...
In that the app is doing the Mad Hatter's Tea Party thing (abandon the
current position and move on to the next clean place at the table), I
would have the backup initiate from a script on the client, which
would run a backup of only the latest volume, and there use Incrbydate
in that the list of Active files is pointless in a case like this.
You could run an infrequent, staggered Incremental of the earlier
volumes if you felt compelled, just in case. Repeatedly running a
backup scan against static volumes is just waste.

Richard Sims

-----------------------------------------
Confidentiality Notice:
The information contained in this email message is privileged and
confidential information and intended only for the use of the
individual or entity named in the address. If you are not the
intended recipient, you are hereby notified that any dissemination,
distribution, or copying of this information is strictly
prohibited. If you received this information in error, please
notify the sender and delete this information from your computer
and retain no copies of any of this information.