6:57 a.m.
Hi,
I just ran into this yesterday on CentOS 6 / OpenLdap 2.4.36, my own build against bdb
5.1.29 and OpenSSL.
Dec 2 16:55:40 animal slapd[13735]: bdb(dc=clarku,dc=edu): /var/lib/ldap/log.0000032796:
log file unreadable: Too many open files
Dec 2 16:55:40 animal slapd[13735]: bdb(dc=clarku,dc=edu): PANIC: Too many open files
Dec 2 16:55:40 animal slapd[13735]: bdb(dc=clarku,dc=edu): PANIC: fatal region error
detected; run recovery
Dec 2 16:55:40 animal slapd[13735]: bdb(dc=clarku,dc=edu): PANIC: fatal region error
detected; run recovery
Dec 2 16:55:40 animal slapd[13735]: null_callback : error code 0x50
I never did anything to change the default process limit, looks like it's set to 1024
in /proc/$(pidof slapd)/limits.
Is there anything I can do short of an automatic restart to prevent that from cropping up
again? I have an MMR via Syncrepl setup with nothing too complicated; can't think of
any reason offhand that slapd would need to have more than 1024 files open.
Best,
Aaron
---
Aaron Bennett
Manager of Systems Administration
Clark University ITS
W:508.793.7315
7:25 a.m.
On Dec 3, 2013, at 3:57 PM, Aaron Bennett wrote:
Dec 2 16:55:40 animal slapd[13735]: bdb(dc=clarku,dc=edu):
/var/lib/ldap/log.0000032796: log file unreadable: Too many open files
Try setting fs.file-max (with sysctl or similar).
celia(SSD1):~# cat /proc/sys/fs/file-max
65536
celia(SSD1):~# grep file-max /etc/sysctl.conf
fs.file-max = 65536
8:20 a.m.
On 03/12/2013 14:57, Aaron Bennett wrote:
can't think of any reason offhand that slapd would need to have
more than 1024 files open.
Each client connection to slapd uses a file handle. I've hit this
problem in the past when a cluster of 300 systems came online for the
first time, all of which had five threads configured in nslcd.conf. That
was 1500 file handles straight away.
--
Liam Gretton liam.gretton(a)le.ac.uk
Systems Specialist http://www.le.ac.uk/its
IT Services Tel: +44 (0)116 2522254
University of Leicester, University Road
Leicestershire LE1 7RH, United Kingdom
11:07 a.m.
On Tue, 3 Dec 2013, Aaron Bennett wrote:
Dec 2 16:55:40 animal slapd[13735]: bdb(dc=clarku,dc=edu):
/var/lib/ldap/log.0000032796: log file unreadable: Too many open files
[...]
Is there anything I can do short of an automatic restart to prevent
that
from cropping up again? I have an MMR via Syncrepl setup with nothing
too complicated; can?t think of any reason offhand that slapd would need
to have more than 1024 files open.
If you're not expecting this load, perhaps you should verify it with
lsof(8) / pfiles(1) / similar tools ... then you can modify the limit
and/or rectify the behavior as appropriate. If the behavior involves
"unnecessary" long-lived connections, please read slapd.conf(5) paying
particular attention to the idletimeout and writetimeout directives. I
don't see any reason why restarts should be necessary.
1:28 a.m.
New subject: Antw: 2.4.26 : too many open files
>> Aaron Bennett <abennett(a)clarku.edu> schrieb am
03.12.2013 um 15:57 in Nachricht
<A3F98FF07EAAD840ADBD429BEA17559026DA34E6(a)PERCH.ad.clarku.edu>:
Hi,
I just ran into this yesterday on CentOS 6 / OpenLdap 2.4.36, my own build
against bdb 5.1.29 and OpenSSL.
did you do an "lsof -p 13735"? I wonder what files are open...
Dec 2 16:55:40 animal slapd[13735]: bdb(dc=clarku,dc=edu):
/var/lib/ldap/log.0000032796: log file unreadable: Too many open files
Dec 2 16:55:40 animal slapd[13735]: bdb(dc=clarku,dc=edu): PANIC: Too many
open files
Dec 2 16:55:40 animal slapd[13735]: bdb(dc=clarku,dc=edu): PANIC: fatal
region error detected; run recovery
Dec 2 16:55:40 animal slapd[13735]: bdb(dc=clarku,dc=edu): PANIC: fatal
region error detected; run recovery
Dec 2 16:55:40 animal slapd[13735]: null_callback : error code 0x50
I never did anything to change the default process limit, looks like it's
set to 1024 in /proc/$(pidof slapd)/limits.
Is there anything I can do short of an automatic restart to prevent that
from cropping up again? I have an MMR via Syncrepl setup with nothing too
complicated; can't think of any reason offhand that slapd would need to have
more than 1024 files open.
Best,
Aaron
---
Aaron Bennett
Manager of Systems Administration
Clark University ITS
W:508.793.7315
1:32 a.m.
New subject: Antw: Re: 2.4.26 : too many open files
>> Liam Gretton <liam.gretton(a)leicester.ac.uk> schrieb am
03.12.2013 um 17:20 in
Nachricht <529E04BD.5000503(a)leicester.ac.uk>:
On 03/12/2013 14:57, Aaron Bennett wrote:
> can't think of any reason offhand that slapd would need to have more than
1024 files open.
Each client connection to slapd uses a file handle. I've hit this
problem in the past when a cluster of 300 systems came online for the
first time, all of which had five threads configured in nslcd.conf. That
was 1500 file handles straight away.
With recent help from this list, I managed to get these values from my (older) server (a
small landscape):
# Max File Descriptors, Connections, Monitor
dn: cn=Max File Descriptors,cn=Connections,cn=Monitor
monitorCounter: 1024
# Total, Connections, Monitor
dn: cn=Total,cn=Connections,cn=Monitor
monitorCounter: 26972
# Current, Connections, Monitor
dn: cn=Current,cn=Connections,cn=Monitor
monitorCounter: 119
Maybe compare with your values.
I was using a query like this (you'll get more output): "ldapsearch -Y EXTERNAL
-H ldapi:/// -b 'cn=monitor' -s sub '(objectClass=*)' description
monitoredInfo monitorCounter monitorOpInitiated monitorOpCompleted
monitorConnectionPeerAddress monitorConnectionOpsReceived monitorConnectionActivityTime
monitorTimestamp"
Regards,
Ulrich
--
Liam Gretton liam.gretton(a)le.ac.uk
Systems Specialist http://www.le.ac.uk/its
IT Services Tel: +44 (0)116 2522254
University of Leicester, University Road
Leicestershire LE1 7RH, United Kingdom
3468
days inactive
3469
days old
openldap-technical@openldap.org
5 comments
5 participants
participants (5)
-
Aaron Bennett -
Aaron Richton -
Liam Gretton -
Turbo Fredriksson -
Ulrich Windl