openldap-bugs March 2012

openldap-bugs@openldap.org

28 participants
78 discussions

Re: (ITS#7216) Recursive function call of "include"
by hyc＠symas.com 26 Mar '12

26 Mar '12

tixu(a)cs.ucsd.edu wrote: > Full_Name: Tianyin Xu > Version: 2.4.30 > OS: Ubuntu Linux 10.04 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (128.54.167.68) > > > 1. Problem > > In OpenLDAP-2.4.30, the recursive "include" directive in the configuration files > cannot be detected but only terminated by the OS. The message cannot pinpoint > the root cause (it's quite misleading). Here, the recursive "include" is > referred to the following case: > > In slapd.conf: include sub.conf > In sub.conf: include slapd.conf Don't do that. Closing this ITS. > > The message will be like: > > Mar 22 11:39:03 tianyin-desktop start_slapd[30825]: could not open config file > "/home/tianyin/openldap-2.4.30/etc/openldap/slapd.conf": Too many open files > (24) > > > 2. Root Cause > > (1) The bomb point is in read_config_file() in servers/slapd/config.c > > if ( stat( fname,&s ) != 0 ) { > ldap_syslog = 1; > Debug(LDAP_DEBUG_ANY, > "could not stat config file \"%s\": %s (%d)\n", > fname, strerror(errno), errno); > ch_free( c ); > return(1); > } > > (2) The recursive call chain: notice #0, #4, and #8 > > #0 read_config_file (fname=0x94ebc8 > "/home/tianyin/openldap-2.4.30/etc/openldap/slapd.conf", depth=1, cf=0x94c790, > cft=0x845de0) at config.c:704 > #1 0x00000000004124e9 in config_include (c=0x94c790) at bconfig.c:3754 > #2 0x000000000041f0cc in config_set_vals (Conf=0x8463a0, c=0x94c790) at > config.c:345 > #3 0x000000000041f62b in config_add_vals (Conf=0x8463a0, c=0x94c790) at > config.c:418 > #4 0x000000000042087f in read_config_file (fname=0x9493f0 > "/home/tianyin/openldap-2.4.30/sub.conf", depth=1, cf=0x949ad0, cft=0x845de0) at > config.c:783 > #5 0x00000000004124e9 in config_include (c=0x949ad0) at bconfig.c:3754 > #6 0x000000000041f0cc in config_set_vals (Conf=0x8463a0, c=0x949ad0) at > config.c:345 > #7 0x000000000041f62b in config_add_vals (Conf=0x8463a0, c=0x949ad0) at > config.c:418 > #8 0x000000000042087f in read_config_file (fname=0x5c5198 > "/home/tianyin/openldap-2.4.30/etc/openldap/slapd.conf", depth=0, cf=0x0, > cft=0x845de0) at config.c:783 > #9 0x0000000000413c9b in read_config (fname=0x0, dir=0x0) at bconfig.c:4213 > #10 0x0000000000407fc7 in main (argc=1, argv=0x7fffffffe2f8) at main.c:792 > > > Thanks! > Tianyin > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7216) Recursive function call of "include"
by tixu＠cs.ucsd.edu 26 Mar '12

26 Mar '12

Full_Name: Tianyin Xu Version: 2.4.30 OS: Ubuntu Linux 10.04 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (128.54.167.68) 1. Problem In OpenLDAP-2.4.30, the recursive "include" directive in the configuration files cannot be detected but only terminated by the OS. The message cannot pinpoint the root cause (it's quite misleading). Here, the recursive "include" is referred to the following case: In slapd.conf: include sub.conf In sub.conf: include slapd.conf The message will be like: Mar 22 11:39:03 tianyin-desktop start_slapd[30825]: could not open config file "/home/tianyin/openldap-2.4.30/etc/openldap/slapd.conf": Too many open files (24) 2. Root Cause (1) The bomb point is in read_config_file() in servers/slapd/config.c if ( stat( fname, &s ) != 0 ) { ldap_syslog = 1; Debug(LDAP_DEBUG_ANY, "could not stat config file \"%s\": %s (%d)\n", fname, strerror(errno), errno); ch_free( c ); return(1); } (2) The recursive call chain: notice #0, #4, and #8 #0 read_config_file (fname=0x94ebc8 "/home/tianyin/openldap-2.4.30/etc/openldap/slapd.conf", depth=1, cf=0x94c790, cft=0x845de0) at config.c:704 #1 0x00000000004124e9 in config_include (c=0x94c790) at bconfig.c:3754 #2 0x000000000041f0cc in config_set_vals (Conf=0x8463a0, c=0x94c790) at config.c:345 #3 0x000000000041f62b in config_add_vals (Conf=0x8463a0, c=0x94c790) at config.c:418 #4 0x000000000042087f in read_config_file (fname=0x9493f0 "/home/tianyin/openldap-2.4.30/sub.conf", depth=1, cf=0x949ad0, cft=0x845de0) at config.c:783 #5 0x00000000004124e9 in config_include (c=0x949ad0) at bconfig.c:3754 #6 0x000000000041f0cc in config_set_vals (Conf=0x8463a0, c=0x949ad0) at config.c:345 #7 0x000000000041f62b in config_add_vals (Conf=0x8463a0, c=0x949ad0) at config.c:418 #8 0x000000000042087f in read_config_file (fname=0x5c5198 "/home/tianyin/openldap-2.4.30/etc/openldap/slapd.conf", depth=0, cf=0x0, cft=0x845de0) at config.c:783 #9 0x0000000000413c9b in read_config (fname=0x0, dir=0x0) at bconfig.c:4213 #10 0x0000000000407fc7 in main (argc=1, argv=0x7fffffffe2f8) at main.c:792 Thanks! Tianyin

1 0

(ITS#7215) Configuration Bug of index_substr_if_{max, min}len
by tixu＠cs.ucsd.edu 26 Mar '12

26 Mar '12

Full_Name: Tianyin Xu Version: 2.4.30 OS: Ubuntu Linux 10.04 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (128.54.167.68) 1. Symptom: In the main configuration file of OpenLDAP (slapd.conf), set the following configuration: index_substr_if_minlen 100 index_substr_if_maxlen 5 Then, everything is fine. The system behaves as expectation. However, if reverse the order of the two configuration line as follows: index_substr_if_maxlen 5 index_substr_if_minlen 100 Then, the system refused to start, with the following misleading message: Mar 25 21:29:18 tianyin-desktop slapd[12700]: /home/tianyin/openldap-2.4.30/etc/openldap/slapd.conf: line 1: <index_substr_if_minlen> invalid value (5) --------------------------------------------------------- 2. Root cause in terms of code snippet (1) servers/slapd/slap.h #define SLAP_INDEX_SUBSTR_IF_MINLEN_DEFAULT 2 #define SLAP_INDEX_SUBSTR_IF_MAXLEN_DEFAULT 4 (2) servers/slapd/schema_init.c unsigned int index_substr_if_minlen = SLAP_INDEX_SUBSTR_IF_MINLEN_DEFAULT; unsigned int index_substr_if_maxlen = SLAP_INDEX_SUBSTR_IF_MAXLEN_DEFAULT; (3) servers/slapd/bconfig.c case CFG_SSTR_IF_MAX: if (c->value_uint < index_substr_if_minlen) { snprintf(c->cr_msg, sizeof(c->cr_msg), "<%s> invalid value", c->argv[0]); Debug(LDAP_DEBUG_ANY, "%s: %s (%d)\n", c->log, c->cr_msg, c->value_int); return(1); } index_substr_if_maxlen = c->value_uint; break; case CFG_SSTR_IF_MIN: if (c->value_uint > index_substr_if_maxlen) { snprintf(c->cr_msg, sizeof( c->cr_msg ), "<%s> invalid value", c->argv[0]); Debug(LDAP_DEBUG_ANY, "%s: %s (%d)\n", c->log, c->cr_msg, c->value_int ); return(1); } index_substr_if_minlen = c->value_uint; break; (4) backtrace #0 config_generic (c=0x915760) at bconfig.c:1996 #1 0x0000000000433a00 in config_set_vals (Conf=0x8146c0, c=0x915760) at config.c:345 #2 0x0000000000433f5f in config_add_vals (Conf=0x8146c0, c=0x915760) at config.c:418 #3 0x00000000004351b3 in read_config_file (fname=0x5a0a98 "/home/tianyin/openldap-2.4.30/etc/openldap/slapd.conf", depth=0, cf=0x0, cft=0x8140c0) at config.c:783 #4 0x000000000042846b in read_config (fname=0x0, dir=0x0) at bconfig.c:4213 #5 0x000000000041c217 in main (argc=1, argv=0x7fffffffe2f8) at main.c:792 --------------------------------------------------------- It's quite clear that the order matters, and this should be a bug related to configuration. Thanks, Tianyin

1 0

(ITS#7214) Openldap contextCSN issue on seconday ldap server
by pawank.kamboj＠gmail.com 25 Mar '12

25 Mar '12

Full_Name: Pawan Kumar Version: openldap-2.3.32 OS: CentOS release 4.5 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (14.140.116.135) Hi, We are using openldap-2.3.32 on CentOS release 4.5(2.6.9-89.0.25.ELsmp). We have master-slave setup and we have two secondary ldap servers on which we are only replicating the one of our OU. We are not able see the contextCSN on secondary ldap servers. we are using the syncreplca(delta sync replication using accesslog. Below are the secondary slapd.conf replication section. syncrepl rid=111 provider=ldap://masterldapserver type=refreshAndPersist retry="60 10 300 +" searchbase="ou=people,dc=example,dc=co,dc=in" schemachecking=off bindmethod=simple binddn="cn=Manager,dc=example,dc=co,dc=in" credentials=password logbase="cn=accesslog" logfilter="(&(objectClass=auditWriteObject)(reqResult=0))" syncdata=accesslog And how we can monitor ldap replication in that case where we are replicating only one OU from master ldap server? any quick response will be appreciated.

1 0

Re: (ITS#7213) back-mdb corruption with quickmode
by quanah＠zimbra.com 20 Mar '12

20 Mar '12

--On Wednesday, March 21, 2012 1:41 AM +0000 quanah(a)OpenLDAP.org wrote: > Full_Name: Quanah Gibson-Mount > Version: 2.4.30 > OS: Linux 2.6 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (75.108.184.39) > > > Using quickmode with slapadd can result in a corrupt database. > zimbra@zqa-067:~$ /opt/zimbra/openldap/sbin/slapadd -q -F /opt/zimbra/data/ldap/config -b '' -l /opt/zimbra/data/ldap/ldap.80 _#################### 100.00% eta none elapsed none fast! Closing DB... zimbra@zqa-067:~$ ldap start Started slapd: pid 27998 zimbra@zqa-067:~$ ./libexec/zmslapcat /tmp # no data for entry id=00000024 No issue when you don't use -q: /opt/zimbra/openldap/sbin/slapadd -F /opt/zimbra/data/ldap/config -b '' -l /opt/zimbra/data/ldap/ldap.80 _#################### 100.00% eta none elapsed none fast! Closing DB... zimbra@zqa-067:~$ ldap start Started slapd: pid 30573 zimbra@zqa-067:~$ ./libexec/zmslapcat /tmp zimbra@zqa-067:~$ --Quanah -- Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

(ITS#7213) back-mdb corruption with quickmode
by quanah＠OpenLDAP.org 20 Mar '12

20 Mar '12

Full_Name: Quanah Gibson-Mount Version: 2.4.30 OS: Linux 2.6 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (75.108.184.39) Using quickmode with slapadd can result in a corrupt database.

1 0

Re: (ITS#7210) back-mdb endless database growth
by quanah＠zimbra.com 20 Mar '12

20 Mar '12

--On Tuesday, March 20, 2012 2:16 PM +0000 hyc(a)symas.com wrote: > hyc(a)symas.com wrote: >> quanah(a)OpenLDAP.org wrote: >>> Full_Name: Quanah Gibson-Mount >>> Version: 2.4.30 >>> OS: Linux 2.6 >>> URL: ftp://ftp.openldap.org/incoming/ >>> Submission from: (NULL) (75.108.184.39) >>> >>> >>> If you create a script to add& delete the same 50,000 users, and then >>> re-run that script multiple times, the MDB database on disk continues >>> to grow, despite the fact that we are only adding and deleting the same >>> data repeatedly. There should be zero growth to the database. >>> >>> Instead, the data.mdb file went from 108MB in size to over 1.2GB after >>> 5-6 runs. Time to add/delete all the users went from 3 minutes 32 >>> seconds on the first run to over 90 minutes on the final run (5th or >>> 6th, I lost count). >> >> At present, MDB is working as intended. In your test a number of overflow >> pages are being used. Currently libmdb always allocates overflow pages >> from new space, because it's not smart enough to search the free list for >> contiguous pages. Can treat this ITS as an enhancement request for that >> purpose, but this is not a new issue nor is it unexpected in the current >> version of MDB. >> > Hm, I take this back. The overflow pages are not enough to account for > the difference. Still looking. If this were truly the expected behavior, it would make using back-mdb with delta-syncrepl useless, as the accesslog would simply grow without bound. ;) Even back-bdb/hdb don't do that. While the DB will never be smaller than the maximum size ever reached without reloading, it won't unnecessarily grow past that either. :P --Quanah -- Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#7210) back-mdb endless database growth
by hyc＠symas.com 20 Mar '12

20 Mar '12

hyc(a)symas.com wrote: > quanah(a)OpenLDAP.org wrote: >> Full_Name: Quanah Gibson-Mount >> Version: 2.4.30 >> OS: Linux 2.6 >> URL: ftp://ftp.openldap.org/incoming/ >> Submission from: (NULL) (75.108.184.39) >> >> >> If you create a script to add& delete the same 50,000 users, and then re-run >> that script multiple times, the MDB database on disk continues to grow, despite >> the fact that we are only adding and deleting the same data repeatedly. There >> should be zero growth to the database. >> >> Instead, the data.mdb file went from 108MB in size to over 1.2GB after 5-6 runs. >> Time to add/delete all the users went from 3 minutes 32 seconds on the first >> run to over 90 minutes on the final run (5th or 6th, I lost count). > > At present, MDB is working as intended. In your test a number of overflow > pages are being used. Currently libmdb always allocates overflow pages from > new space, because it's not smart enough to search the free list for > contiguous pages. Can treat this ITS as an enhancement request for that > purpose, but this is not a new issue nor is it unexpected in the current > version of MDB. > Hm, I take this back. The overflow pages are not enough to account for the difference. Still looking. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7210) back-mdb endless database growth
by hyc＠symas.com 20 Mar '12

20 Mar '12

quanah(a)OpenLDAP.org wrote: > Full_Name: Quanah Gibson-Mount > Version: 2.4.30 > OS: Linux 2.6 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (75.108.184.39) > > > If you create a script to add& delete the same 50,000 users, and then re-run > that script multiple times, the MDB database on disk continues to grow, despite > the fact that we are only adding and deleting the same data repeatedly. There > should be zero growth to the database. > > Instead, the data.mdb file went from 108MB in size to over 1.2GB after 5-6 runs. > Time to add/delete all the users went from 3 minutes 32 seconds on the first > run to over 90 minutes on the final run (5th or 6th, I lost count). At present, MDB is working as intended. In your test a number of overflow pages are being used. Currently libmdb always allocates overflow pages from new space, because it's not smart enough to search the free list for contiguous pages. Can treat this ITS as an enhancement request for that purpose, but this is not a new issue nor is it unexpected in the current version of MDB. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7212) slapmodify support for back-config
by ondrej.kuznik＠acision.com 20 Mar '12

20 Mar '12

Full_Name: Ondrej Kuznik Version: HEAD OS: Linux URL: ftp://ftp.openldap.org/incoming/Ondrej-Kuznik-20120319-back-config-slapmodi… Submission from: (NULL) (62.168.56.1) For slapmodify to be usable on a database, that database needs to implement the bi_tool_dn2id_get and bi_tool_entry_modify functions, which is not the case of the back-config. I've put together a minimal (and maybe too naive) implementation of these for back-config and back-ldif at the above link. Currently, there is no validation whether the modification passes back-config checks since config_modify_internal takes a modify operation, while bi_tool_entry_modify receives only the new version of the entry to be modified. Also, for slapmodify to be really useful, it would have to allow entry deletion, for which there is no tool mode callback. To implement deletes, do you think it is better that deletion of an entry with children fail or delete the entire subtree? While the former seems both easier to implement and more sensible to me, I would like your know opinion. The attached file is derived from OpenLDAP Software. All of the modifications to OpenLDAP Software represented in the following patch(es) were developed by Acision. Acision has not assigned rights and/or interest in this work to any party. I, Ondrej Kuznik am authorized by Acision, my employer, to release this work under the following terms. The attached modifications to OpenLDAP Software are subject to the following notice: Copyright 2012 Acision Redistribution and use in source and binary forms, with or without modification, are permitted only as authorized by the OpenLDAP Public License.

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs March 2012