Re: (ITS#7216) Recursive function call of "include"
by hyc＠symas.com
tixu(a)cs.ucsd.edu wrote:
> Full_Name: Tianyin Xu
> Version: 2.4.30
> OS: Ubuntu Linux 10.04
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (128.54.167.68)
>
>
> 1. Problem
>
> In OpenLDAP-2.4.30, the recursive "include" directive in the configuration files
> cannot be detected but only terminated by the OS. The message cannot pinpoint
> the root cause (it's quite misleading). Here, the recursive "include" is
> referred to the following case:
>
> In slapd.conf: include sub.conf
> In sub.conf: include slapd.conf
Don't do that.
Closing this ITS.
>
> The message will be like:
>
> Mar 22 11:39:03 tianyin-desktop start_slapd[30825]: could not open config file
> "/home/tianyin/openldap-2.4.30/etc/openldap/slapd.conf": Too many open files
> (24)
>
>
> 2. Root Cause
>
> (1) The bomb point is in read_config_file() in servers/slapd/config.c
>
> if ( stat( fname,&s ) != 0 ) {
> ldap_syslog = 1;
> Debug(LDAP_DEBUG_ANY,
> "could not stat config file \"%s\": %s (%d)\n",
> fname, strerror(errno), errno);
> ch_free( c );
> return(1);
> }
>
> (2) The recursive call chain: notice #0, #4, and #8
>
> #0 read_config_file (fname=0x94ebc8
> "/home/tianyin/openldap-2.4.30/etc/openldap/slapd.conf", depth=1, cf=0x94c790,
> cft=0x845de0) at config.c:704
> #1 0x00000000004124e9 in config_include (c=0x94c790) at bconfig.c:3754
> #2 0x000000000041f0cc in config_set_vals (Conf=0x8463a0, c=0x94c790) at
> config.c:345
> #3 0x000000000041f62b in config_add_vals (Conf=0x8463a0, c=0x94c790) at
> config.c:418
> #4 0x000000000042087f in read_config_file (fname=0x9493f0
> "/home/tianyin/openldap-2.4.30/sub.conf", depth=1, cf=0x949ad0, cft=0x845de0) at
> config.c:783
> #5 0x00000000004124e9 in config_include (c=0x949ad0) at bconfig.c:3754
> #6 0x000000000041f0cc in config_set_vals (Conf=0x8463a0, c=0x949ad0) at
> config.c:345
> #7 0x000000000041f62b in config_add_vals (Conf=0x8463a0, c=0x949ad0) at
> config.c:418
> #8 0x000000000042087f in read_config_file (fname=0x5c5198
> "/home/tianyin/openldap-2.4.30/etc/openldap/slapd.conf", depth=0, cf=0x0,
> cft=0x845de0) at config.c:783
> #9 0x0000000000413c9b in read_config (fname=0x0, dir=0x0) at bconfig.c:4213
> #10 0x0000000000407fc7 in main (argc=1, argv=0x7fffffffe2f8) at main.c:792
>
>
> Thanks!
> Tianyin
>
>
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
11 years, 5 months
(ITS#7216) Recursive function call of "include"
by tixu＠cs.ucsd.edu
Full_Name: Tianyin Xu
Version: 2.4.30
OS: Ubuntu Linux 10.04
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (128.54.167.68)
1. Problem
In OpenLDAP-2.4.30, the recursive "include" directive in the configuration files
cannot be detected but only terminated by the OS. The message cannot pinpoint
the root cause (it's quite misleading). Here, the recursive "include" is
referred to the following case:
In slapd.conf: include sub.conf
In sub.conf: include slapd.conf
The message will be like:
Mar 22 11:39:03 tianyin-desktop start_slapd[30825]: could not open config file
"/home/tianyin/openldap-2.4.30/etc/openldap/slapd.conf": Too many open files
(24)
2. Root Cause
(1) The bomb point is in read_config_file() in servers/slapd/config.c
if ( stat( fname, &s ) != 0 ) {
ldap_syslog = 1;
Debug(LDAP_DEBUG_ANY,
"could not stat config file \"%s\": %s (%d)\n",
fname, strerror(errno), errno);
ch_free( c );
return(1);
}
(2) The recursive call chain: notice #0, #4, and #8
#0 read_config_file (fname=0x94ebc8
"/home/tianyin/openldap-2.4.30/etc/openldap/slapd.conf", depth=1, cf=0x94c790,
cft=0x845de0) at config.c:704
#1 0x00000000004124e9 in config_include (c=0x94c790) at bconfig.c:3754
#2 0x000000000041f0cc in config_set_vals (Conf=0x8463a0, c=0x94c790) at
config.c:345
#3 0x000000000041f62b in config_add_vals (Conf=0x8463a0, c=0x94c790) at
config.c:418
#4 0x000000000042087f in read_config_file (fname=0x9493f0
"/home/tianyin/openldap-2.4.30/sub.conf", depth=1, cf=0x949ad0, cft=0x845de0) at
config.c:783
#5 0x00000000004124e9 in config_include (c=0x949ad0) at bconfig.c:3754
#6 0x000000000041f0cc in config_set_vals (Conf=0x8463a0, c=0x949ad0) at
config.c:345
#7 0x000000000041f62b in config_add_vals (Conf=0x8463a0, c=0x949ad0) at
config.c:418
#8 0x000000000042087f in read_config_file (fname=0x5c5198
"/home/tianyin/openldap-2.4.30/etc/openldap/slapd.conf", depth=0, cf=0x0,
cft=0x845de0) at config.c:783
#9 0x0000000000413c9b in read_config (fname=0x0, dir=0x0) at bconfig.c:4213
#10 0x0000000000407fc7 in main (argc=1, argv=0x7fffffffe2f8) at main.c:792
Thanks!
Tianyin
11 years, 5 months
(ITS#7215) Configuration Bug of index_substr_if_{max, min}len
by tixu＠cs.ucsd.edu
Full_Name: Tianyin Xu
Version: 2.4.30
OS: Ubuntu Linux 10.04
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (128.54.167.68)
1. Symptom:
In the main configuration file of OpenLDAP (slapd.conf), set the following
configuration:
index_substr_if_minlen 100
index_substr_if_maxlen 5
Then, everything is fine. The system behaves as expectation. However, if reverse
the order of the two configuration line as follows:
index_substr_if_maxlen 5
index_substr_if_minlen 100
Then, the system refused to start, with the following misleading message:
Mar 25 21:29:18 tianyin-desktop slapd[12700]:
/home/tianyin/openldap-2.4.30/etc/openldap/slapd.conf: line 1:
<index_substr_if_minlen> invalid value (5)
---------------------------------------------------------
2. Root cause in terms of code snippet
(1) servers/slapd/slap.h
#define SLAP_INDEX_SUBSTR_IF_MINLEN_DEFAULT 2
#define SLAP_INDEX_SUBSTR_IF_MAXLEN_DEFAULT 4
(2) servers/slapd/schema_init.c
unsigned int index_substr_if_minlen = SLAP_INDEX_SUBSTR_IF_MINLEN_DEFAULT;
unsigned int index_substr_if_maxlen = SLAP_INDEX_SUBSTR_IF_MAXLEN_DEFAULT;
(3) servers/slapd/bconfig.c
case CFG_SSTR_IF_MAX:
if (c->value_uint < index_substr_if_minlen) {
snprintf(c->cr_msg, sizeof(c->cr_msg), "<%s> invalid value",
c->argv[0]);
Debug(LDAP_DEBUG_ANY, "%s: %s (%d)\n", c->log, c->cr_msg,
c->value_int);
return(1);
}
index_substr_if_maxlen = c->value_uint;
break;
case CFG_SSTR_IF_MIN:
if (c->value_uint > index_substr_if_maxlen) {
snprintf(c->cr_msg, sizeof( c->cr_msg ), "<%s> invalid value",
c->argv[0]);
Debug(LDAP_DEBUG_ANY, "%s: %s (%d)\n", c->log, c->cr_msg, c->value_int
);
return(1);
}
index_substr_if_minlen = c->value_uint;
break;
(4) backtrace
#0 config_generic (c=0x915760) at bconfig.c:1996
#1 0x0000000000433a00 in config_set_vals (Conf=0x8146c0, c=0x915760) at
config.c:345
#2 0x0000000000433f5f in config_add_vals (Conf=0x8146c0, c=0x915760) at
config.c:418
#3 0x00000000004351b3 in read_config_file (fname=0x5a0a98
"/home/tianyin/openldap-2.4.30/etc/openldap/slapd.conf", depth=0, cf=0x0,
cft=0x8140c0) at config.c:783
#4 0x000000000042846b in read_config (fname=0x0, dir=0x0) at bconfig.c:4213
#5 0x000000000041c217 in main (argc=1, argv=0x7fffffffe2f8) at main.c:792
---------------------------------------------------------
It's quite clear that the order matters, and this should be a bug related to
configuration.
Thanks,
Tianyin
11 years, 5 months
(ITS#7214) Openldap contextCSN issue on seconday ldap server
by pawank.kamboj＠gmail.com
Full_Name: Pawan Kumar
Version: openldap-2.3.32
OS: CentOS release 4.5
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (14.140.116.135)
Hi,
We are using openldap-2.3.32 on CentOS release 4.5(2.6.9-89.0.25.ELsmp).
We have master-slave setup and we have two secondary ldap servers on which we
are only replicating the one of our OU. We are not able see the contextCSN on
secondary ldap servers. we are using the syncreplca(delta sync replication using
accesslog.
Below are the secondary slapd.conf replication section.
syncrepl rid=111 provider=ldap://masterldapserver type=refreshAndPersist
retry="60 10 300 +" searchbase="ou=people,dc=example,dc=co,dc=in"
schemachecking=off bindmethod=simple binddn="cn=Manager,dc=example,dc=co,dc=in"
credentials=password logbase="cn=accesslog"
logfilter="(&(objectClass=auditWriteObject)(reqResult=0))" syncdata=accesslog
And how we can monitor ldap replication in that case where we are replicating
only one OU from master ldap server?
any quick response will be appreciated.
11 years, 5 months
Re: (ITS#7213) back-mdb corruption with quickmode
by quanah＠zimbra.com
--On Wednesday, March 21, 2012 1:41 AM +0000 quanah(a)OpenLDAP.org wrote:
> Full_Name: Quanah Gibson-Mount
> Version: 2.4.30
> OS: Linux 2.6
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (75.108.184.39)
>
>
> Using quickmode with slapadd can result in a corrupt database.
>
zimbra@zqa-067:~$ /opt/zimbra/openldap/sbin/slapadd -q -F
/opt/zimbra/data/ldap/config -b '' -l /opt/zimbra/data/ldap/ldap.80
_#################### 100.00% eta none elapsed none fast!
Closing DB...
zimbra@zqa-067:~$ ldap start
Started slapd: pid 27998
zimbra@zqa-067:~$ ./libexec/zmslapcat /tmp
# no data for entry id=00000024
No issue when you don't use -q:
/opt/zimbra/openldap/sbin/slapadd -F /opt/zimbra/data/ldap/config -b '' -l
/opt/zimbra/data/ldap/ldap.80
_#################### 100.00% eta none elapsed none fast!
Closing DB...
zimbra@zqa-067:~$ ldap start
Started slapd: pid 30573
zimbra@zqa-067:~$ ./libexec/zmslapcat /tmp
zimbra@zqa-067:~$
--Quanah
--
Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
11 years, 6 months
Re: (ITS#7210) back-mdb endless database growth
by quanah＠zimbra.com
--On Tuesday, March 20, 2012 2:16 PM +0000 hyc(a)symas.com wrote:
> hyc(a)symas.com wrote:
>> quanah(a)OpenLDAP.org wrote:
>>> Full_Name: Quanah Gibson-Mount
>>> Version: 2.4.30
>>> OS: Linux 2.6
>>> URL: ftp://ftp.openldap.org/incoming/
>>> Submission from: (NULL) (75.108.184.39)
>>>
>>>
>>> If you create a script to add& delete the same 50,000 users, and then
>>> re-run that script multiple times, the MDB database on disk continues
>>> to grow, despite the fact that we are only adding and deleting the same
>>> data repeatedly. There should be zero growth to the database.
>>>
>>> Instead, the data.mdb file went from 108MB in size to over 1.2GB after
>>> 5-6 runs. Time to add/delete all the users went from 3 minutes 32
>>> seconds on the first run to over 90 minutes on the final run (5th or
>>> 6th, I lost count).
>>
>> At present, MDB is working as intended. In your test a number of overflow
>> pages are being used. Currently libmdb always allocates overflow pages
>> from new space, because it's not smart enough to search the free list for
>> contiguous pages. Can treat this ITS as an enhancement request for that
>> purpose, but this is not a new issue nor is it unexpected in the current
>> version of MDB.
>>
> Hm, I take this back. The overflow pages are not enough to account for
> the difference. Still looking.
If this were truly the expected behavior, it would make using back-mdb with
delta-syncrepl useless, as the accesslog would simply grow without bound.
;) Even back-bdb/hdb don't do that. While the DB will never be smaller
than the maximum size ever reached without reloading, it won't
unnecessarily grow past that either. :P
--Quanah
--
Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
11 years, 6 months
Re: (ITS#7210) back-mdb endless database growth
by hyc＠symas.com
hyc(a)symas.com wrote:
> quanah(a)OpenLDAP.org wrote:
>> Full_Name: Quanah Gibson-Mount
>> Version: 2.4.30
>> OS: Linux 2.6
>> URL: ftp://ftp.openldap.org/incoming/
>> Submission from: (NULL) (75.108.184.39)
>>
>>
>> If you create a script to add& delete the same 50,000 users, and then re-run
>> that script multiple times, the MDB database on disk continues to grow, despite
>> the fact that we are only adding and deleting the same data repeatedly. There
>> should be zero growth to the database.
>>
>> Instead, the data.mdb file went from 108MB in size to over 1.2GB after 5-6 runs.
>> Time to add/delete all the users went from 3 minutes 32 seconds on the first
>> run to over 90 minutes on the final run (5th or 6th, I lost count).
>
> At present, MDB is working as intended. In your test a number of overflow
> pages are being used. Currently libmdb always allocates overflow pages from
> new space, because it's not smart enough to search the free list for
> contiguous pages. Can treat this ITS as an enhancement request for that
> purpose, but this is not a new issue nor is it unexpected in the current
> version of MDB.
>
Hm, I take this back. The overflow pages are not enough to account for the
difference. Still looking.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
11 years, 6 months
Re: (ITS#7210) back-mdb endless database growth
by hyc＠symas.com
quanah(a)OpenLDAP.org wrote:
> Full_Name: Quanah Gibson-Mount
> Version: 2.4.30
> OS: Linux 2.6
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (75.108.184.39)
>
>
> If you create a script to add& delete the same 50,000 users, and then re-run
> that script multiple times, the MDB database on disk continues to grow, despite
> the fact that we are only adding and deleting the same data repeatedly. There
> should be zero growth to the database.
>
> Instead, the data.mdb file went from 108MB in size to over 1.2GB after 5-6 runs.
> Time to add/delete all the users went from 3 minutes 32 seconds on the first
> run to over 90 minutes on the final run (5th or 6th, I lost count).
At present, MDB is working as intended. In your test a number of overflow
pages are being used. Currently libmdb always allocates overflow pages from
new space, because it's not smart enough to search the free list for
contiguous pages. Can treat this ITS as an enhancement request for that
purpose, but this is not a new issue nor is it unexpected in the current
version of MDB.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
11 years, 6 months
(ITS#7212) slapmodify support for back-config
by ondrej.kuznik＠acision.com
Full_Name: Ondrej Kuznik
Version: HEAD
OS: Linux
URL: ftp://ftp.openldap.org/incoming/Ondrej-Kuznik-20120319-back-config-slapmo...
Submission from: (NULL) (62.168.56.1)
For slapmodify to be usable on a database, that database needs to
implement the bi_tool_dn2id_get and bi_tool_entry_modify functions,
which is not the case of the back-config. I've put together a minimal
(and maybe too naive) implementation of these for back-config and
back-ldif at the above link.
Currently, there is no validation whether the modification passes
back-config checks since config_modify_internal takes a modify
operation, while bi_tool_entry_modify receives only the new version of
the entry to be modified.
Also, for slapmodify to be really useful, it would have to allow entry
deletion, for which there is no tool mode callback. To implement
deletes, do you think it is better that deletion of an entry with
children fail or delete the entire subtree? While the former seems both
easier to implement and more sensible to me, I would like your know opinion.
The attached file is derived from OpenLDAP Software. All of the
modifications to OpenLDAP Software represented in the following
patch(es) were developed by Acision. Acision has not assigned rights
and/or interest in this work to any party. I, Ondrej Kuznik am
authorized by Acision, my employer, to release this work under the
following terms.
The attached modifications to OpenLDAP Software are subject to the
following notice:
Copyright 2012 Acision
Redistribution and use in source and binary forms, with or without
modification, are permitted only as authorized by the OpenLDAP Public
License.
11 years, 6 months
