SearchFAQMemberlist Log in
Reply to topic Page 1 of 1
To many open files
Author Message
Post To many open files 
My backup server is running real slow and the daemon log is showing to many
open files.






Note: To sign off this list, send a "signoff networker" command via email
should be sent to stan < at > temple.edu

Post To many open files 
Ken,

Are you running it on a UNIX or Linux OS? If so then you have too many open
files on the system that it can

A. Either not handle that many open files
Or
B. You have to increase the number of open files allowed on your kernel.

Let us know if you have any more questions.


Chad Smykay, RHCE, LCNA
Systems Storage Administrator
Rackspace Managed Hosting (TM) - The Managed Hosting Specialist (TM)

-----Original Message-----
From: Legato NetWorker discussion [mailto:NETWORKER < at > LISTMAIL.TEMPLE.EDU] On
Behalf Of Ciolek, Ken
Sent: Wednesday, January 05, 2005 12:33 PM
To: NETWORKER < at > LISTMAIL.TEMPLE.EDU
Subject: [Networker] To many open files

My backup server is running real slow and the daemon log is showing to many
open files.






Note: To sign off this list, send a "signoff networker" command via email
should be sent to stan < at > temple.edu

Note: To sign off this list, send a "signoff networker" command via email
should be sent to stan < at > temple.edu

Post To many open files 
Running on a Solaris 2.8 platform.


-----Original Message-----
From: Chad Smykay [mailto:csmykay < at > rackspace.com]
Sent: Wednesday, January 05, 2005 4:01 PM
To: 'Legato NetWorker discussion'; Ciolek, Ken
Subject: RE: [Networker] To many open files

Ken,

Are you running it on a UNIX or Linux OS? If so then you have too many open
files on the system that it can

A. Either not handle that many open files
Or
B. You have to increase the number of open files allowed on your kernel.

Let us know if you have any more questions.


Chad Smykay, RHCE, LCNA
Systems Storage Administrator
Rackspace Managed Hosting (TM) - The Managed Hosting Specialist (TM)

-----Original Message-----
From: Legato NetWorker discussion [mailto:NETWORKER < at > LISTMAIL.TEMPLE.EDU] On
Behalf Of Ciolek, Ken
Sent: Wednesday, January 05, 2005 12:33 PM
To: NETWORKER < at > LISTMAIL.TEMPLE.EDU
Subject: [Networker] To many open files

My backup server is running real slow and the daemon log is showing to many
open files.






Note: To sign off this list, send a "signoff networker" command via email
should be sent to stan < at > temple.edu

Note: To sign off this list, send a "signoff networker" command via email
should be sent to stan < at > temple.edu

Post To many open files 
Ken,

When you have "Too many open files" errors in ANY of your logs this is
usually indicative of an OS problem. I am not that familiar with Solaris so
I am not sure how the underlying kernel allows for more or less processes
along with files to be opened and closed for such processes. You really
should check with someone that knows SUN fairly well that can look at this
issue. But I can tell you that it is not a Legato issue.

Regards,

Chad Smykay, RHCE, LCNA
Systems Storage Administrator
Rackspace Managed Hosting (TM) - The Managed Hosting Specialist (TM)

-----Original Message-----
From: Ciolek, Ken [mailto:Ken.Ciolek < at > aig.com]
Sent: Wednesday, January 05, 2005 3:09 PM
To: 'Chad Smykay'; 'Legato NetWorker discussion'
Subject: RE: [Networker] To many open files

Running on a Solaris 2.8 platform.


-----Original Message-----
From: Chad Smykay [mailto:csmykay < at > rackspace.com]
Sent: Wednesday, January 05, 2005 4:01 PM
To: 'Legato NetWorker discussion'; Ciolek, Ken
Subject: RE: [Networker] To many open files

Ken,

Are you running it on a UNIX or Linux OS? If so then you have too many open
files on the system that it can

A. Either not handle that many open files
Or
B. You have to increase the number of open files allowed on your kernel.

Let us know if you have any more questions.


Chad Smykay, RHCE, LCNA
Systems Storage Administrator
Rackspace Managed Hosting (TM) - The Managed Hosting Specialist (TM)

-----Original Message-----
From: Legato NetWorker discussion [mailto:NETWORKER < at > LISTMAIL.TEMPLE.EDU] On
Behalf Of Ciolek, Ken
Sent: Wednesday, January 05, 2005 12:33 PM
To: NETWORKER < at > LISTMAIL.TEMPLE.EDU
Subject: [Networker] To many open files

My backup server is running real slow and the daemon log is showing to many
open files.






Note: To sign off this list, send a "signoff networker" command via email
should be sent to stan < at > temple.edu

Note: To sign off this list, send a "signoff networker" command via email
should be sent to stan < at > temple.edu

Post To many open files 
This parameter is changed in /etc/system on a sun 2.8 box, which then
requires a reboot to apply it. In my experience, I've generally had to bump
some of those default system parameters up in Solaris, as any number of
programs I've worked with seem to run into those and similar issues.

As far as what you should set it to, I would do some research on it.
Sunmanagers.org's listmail would be a good spot to check, I'm sure someone's
asked for recommendations on it before, and there's a lot of very
knowledgeable people in that group.

Andrew Harkin



-----Original Message-----
From: Chad Smykay [mailto:csmykay < at > RACKSPACE.COM]
Sent: Wednesday, January 05, 2005 3:16 PM
To: NETWORKER < at > LISTMAIL.TEMPLE.EDU
Subject: Re: [Networker] To many open files

Ken,

When you have "Too many open files" errors in ANY of your logs this is
usually indicative of an OS problem. I am not that familiar with Solaris so
I am not sure how the underlying kernel allows for more or less processes
along with files to be opened and closed for such processes. You really
should check with someone that knows SUN fairly well that can look at this
issue. But I can tell you that it is not a Legato issue.

Regards,

Chad Smykay, RHCE, LCNA
Systems Storage Administrator
Rackspace Managed Hosting (TM) - The Managed Hosting Specialist (TM)

____________________
Confidentiality Notice: This e-mail message, including any attachments, is
for the sole use of the intended recipient(s) and may contain confidential
and privileged information. Any unauthorized review, use, disclosure, or
distribution is prohibited. If you are not the intended recipient, please
contact the sender by reply e-mail and destroy all copies of the original
message.

Note: To sign off this list, send a "signoff networker" command via email
should be sent to stan < at > temple.edu

Post To many open files 
Hello,

Where exactly do you have the message "too many open files" ?
To verify this issue, if you can isolate the process that displays this message, use the following
command to verify the number of open files
pfiles $(pgrep nsrexecd) <-- change nsrexecd by save or savefs or your backup command.

Updating the file /etc/system will change the maximum number of open for the overall system, which
may be is not the best practice.
The second option is to modify the startup script of the Networker daemons and adding at the top
of the script the line "ulimit -n 1024".
By default a process is started with a maximum of 256 file descriptors.
After changing the startup script of networker /etc/init.d/networker, you'll have to restart the
daemons.
( cd / && sh /etc/init.d/networker stop && sh /etc/init.d/networker start)
The client daemons nsrexecd will be restart, you can verify that the file descriptor limit has
been updated by typing
pfiles $(pgrep nsrexecd)
As all the backup commands (save, savefs, nsrnmo,...) are initated by the nsrexecd daemons, then
will use the same file descriptot limit.


The example here will increase the number of open files from 256 to 1024, if you want more you'll
have to update the file /etc/system first.

I've never seen such an issue before...

Regards.

--- Andrew Harkin <andrew.harkin_networker < at > MCKENNAN.ORG> a écrit :
This parameter is changed in /etc/system on a sun 2.8 box, which then
requires a reboot to apply it. In my experience, I've generally had to bump
some of those default system parameters up in Solaris, as any number of
programs I've worked with seem to run into those and similar issues.

As far as what you should set it to, I would do some research on it.
Sunmanagers.org's listmail would be a good spot to check, I'm sure someone's
asked for recommendations on it before, and there's a lot of very
knowledgeable people in that group.

Andrew Harkin



-----Original Message-----
From: Chad Smykay [mailto:csmykay < at > RACKSPACE.COM]
Sent: Wednesday, January 05, 2005 3:16 PM
To: NETWORKER < at > LISTMAIL.TEMPLE.EDU
Subject: Re: [Networker] To many open files

Ken,

When you have "Too many open files" errors in ANY of your logs this is
usually indicative of an OS problem. I am not that familiar with Solaris so
I am not sure how the underlying kernel allows for more or less processes
along with files to be opened and closed for such processes. You really
should check with someone that knows SUN fairly well that can look at this
issue. But I can tell you that it is not a Legato issue.

Regards,

Chad Smykay, RHCE, LCNA
Systems Storage Administrator
Rackspace Managed Hosting (TM) - The Managed Hosting Specialist (TM)

____________________
Confidentiality Notice: This e-mail message, including any attachments, is
for the sole use of the intended recipient(s) and may contain confidential
and privileged information. Any unauthorized review, use, disclosure, or
distribution is prohibited. If you are not the intended recipient, please
contact the sender by reply e-mail and destroy all copies of the original
message.

--
Note: To sign off this list, send a "signoff networker" command via email
should be sent to stan < at > temple.edu







Découvrez le nouveau Yahoo! Mail : 250 Mo d'espace de stockage pour vos mails !
Créez votre Yahoo! Mail sur http://fr.mail.yahoo.com/

Note: To sign off this list, send a "signoff networker" command via email
should be sent to stan < at > temple.edu

Post To many open files 
Andrew Harkin wrote:
This parameter is changed in /etc/system on a sun 2.8 box, which then
requires a reboot to apply it. In my experience, I've generally had to bump
some of those default system parameters up in Solaris, as any number of
programs I've worked with seem to run into those and similar issues.

The problem is not on the system running NetWorker, but on Legato's
build system, where NetWorker was compiled. So this won't help, although
you would think it should.

Note: To sign off this list, send a "signoff networker" command via email
should be sent to stan < at > temple.edu

Post To many open files 
Ah yes, Legato should be well informed with this. You didn't
post what version of Networker. I can guess it is 6.1.x or the distributed
7.1.1. A recap for everyone..

To reproduce the problem:

a.) Run one of the versions mentioned above
b.) Disable all tape drives
c.) Run many groups and load the server up as much as possible
d.) Watch Networker crumble and post this error in the daemon.log.

Solution to the problem:

I. Immediate solution;

a.) Get the server as unloaded as possible.
b.) See if the stuck jukebox or jukebox out of writable media can be
addressed.
c.) If the condition persist for too long and the backup load cannot
be reduced only option is to shutdown networker (show over).

Other have reported success with savegroup parallelism and load to keep
open file count under the limit (1024).

II. Long term cause and solution;

This condition exist because of the way Networker is compiled for Solaris
(32 bit). Solution to scale is to:

a.) Run a 64 bit build of Networker for Solaris (7.1.2 on or request a
7.1.1 build from Legato).

b.) Raise the open file limit in /etc/system as described by others. We
saw the 32bit 7.1.1 hit the 1024 limit despite larger values in /etc/system.

c.) Of course, the root cause that generated the open files will
need to be found; ie. tapes not mounting for whatever reason.

In short later versions should scale better.

Let me know if you need an open file counter to see how far away from
meltdown you are :)


Robert Maiello
Pioneer Data Systems

On Wed, 5 Jan 2005 16:08:49 -0500, Ciolek, Ken <Ken.Ciolek < at > AIG.COM> wrote:

Running on a Solaris 2.8 platform.


-----Original Message-----
From: Chad Smykay [mailto:csmykay < at > rackspace.com]
Sent: Wednesday, January 05, 2005 4:01 PM
To: 'Legato NetWorker discussion'; Ciolek, Ken
Subject: RE: [Networker] To many open files

Ken,

Are you running it on a UNIX or Linux OS? If so then you have too many open
files on the system that it can

A. Either not handle that many open files
Or
B. You have to increase the number of open files allowed on your kernel.

Let us know if you have any more questions.


Chad Smykay, RHCE, LCNA
Systems Storage Administrator
Rackspace Managed Hosting (TM) - The Managed Hosting Specialist (TM)

-----Original Message-----
From: Legato NetWorker discussion [mailto:NETWORKER < at > LISTMAIL.TEMPLE.EDU] On
Behalf Of Ciolek, Ken
Sent: Wednesday, January 05, 2005 12:33 PM
To: NETWORKER < at > LISTMAIL.TEMPLE.EDU
Subject: [Networker] To many open files

My backup server is running real slow and the daemon log is showing to many
open files.






--
Note: To sign off this list, send a "signoff networker" command via email
should be sent to stan < at > temple.edu

--
Note: To sign off this list, send a "signoff networker" command via email
should be sent to stan < at > temple.edu

Note: To sign off this list, send a "signoff networker" command via email
should be sent to stan < at > temple.edu

Display posts from previous:
Reply to topic Page 1 of 1
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
  


Magic SEO URL for phpBB