openldap-bugs

openldap-bugs@openldap.org

1 participants
20963 discussions

(ITS#7214) Openldap contextCSN issue on seconday ldap server
by pawank.kamboj＠gmail.com 25 Mar '12

25 Mar '12

Full_Name: Pawan Kumar Version: openldap-2.3.32 OS: CentOS release 4.5 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (14.140.116.135) Hi, We are using openldap-2.3.32 on CentOS release 4.5(2.6.9-89.0.25.ELsmp). We have master-slave setup and we have two secondary ldap servers on which we are only replicating the one of our OU. We are not able see the contextCSN on secondary ldap servers. we are using the syncreplca(delta sync replication using accesslog. Below are the secondary slapd.conf replication section. syncrepl rid=111 provider=ldap://masterldapserver type=refreshAndPersist retry="60 10 300 +" searchbase="ou=people,dc=example,dc=co,dc=in" schemachecking=off bindmethod=simple binddn="cn=Manager,dc=example,dc=co,dc=in" credentials=password logbase="cn=accesslog" logfilter="(&(objectClass=auditWriteObject)(reqResult=0))" syncdata=accesslog And how we can monitor ldap replication in that case where we are replicating only one OU from master ldap server? any quick response will be appreciated.

1 0

Re: (ITS#7213) back-mdb corruption with quickmode
by quanah＠zimbra.com 20 Mar '12

20 Mar '12

--On Wednesday, March 21, 2012 1:41 AM +0000 quanah(a)OpenLDAP.org wrote: > Full_Name: Quanah Gibson-Mount > Version: 2.4.30 > OS: Linux 2.6 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (75.108.184.39) > > > Using quickmode with slapadd can result in a corrupt database. > zimbra@zqa-067:~$ /opt/zimbra/openldap/sbin/slapadd -q -F /opt/zimbra/data/ldap/config -b '' -l /opt/zimbra/data/ldap/ldap.80 _#################### 100.00% eta none elapsed none fast! Closing DB... zimbra@zqa-067:~$ ldap start Started slapd: pid 27998 zimbra@zqa-067:~$ ./libexec/zmslapcat /tmp # no data for entry id=00000024 No issue when you don't use -q: /opt/zimbra/openldap/sbin/slapadd -F /opt/zimbra/data/ldap/config -b '' -l /opt/zimbra/data/ldap/ldap.80 _#################### 100.00% eta none elapsed none fast! Closing DB... zimbra@zqa-067:~$ ldap start Started slapd: pid 30573 zimbra@zqa-067:~$ ./libexec/zmslapcat /tmp zimbra@zqa-067:~$ --Quanah -- Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

(ITS#7213) back-mdb corruption with quickmode
by quanah＠OpenLDAP.org 20 Mar '12

20 Mar '12

Full_Name: Quanah Gibson-Mount Version: 2.4.30 OS: Linux 2.6 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (75.108.184.39) Using quickmode with slapadd can result in a corrupt database.

1 0

Re: (ITS#7210) back-mdb endless database growth
by quanah＠zimbra.com 20 Mar '12

20 Mar '12

--On Tuesday, March 20, 2012 2:16 PM +0000 hyc(a)symas.com wrote: > hyc(a)symas.com wrote: >> quanah(a)OpenLDAP.org wrote: >>> Full_Name: Quanah Gibson-Mount >>> Version: 2.4.30 >>> OS: Linux 2.6 >>> URL: ftp://ftp.openldap.org/incoming/ >>> Submission from: (NULL) (75.108.184.39) >>> >>> >>> If you create a script to add& delete the same 50,000 users, and then >>> re-run that script multiple times, the MDB database on disk continues >>> to grow, despite the fact that we are only adding and deleting the same >>> data repeatedly. There should be zero growth to the database. >>> >>> Instead, the data.mdb file went from 108MB in size to over 1.2GB after >>> 5-6 runs. Time to add/delete all the users went from 3 minutes 32 >>> seconds on the first run to over 90 minutes on the final run (5th or >>> 6th, I lost count). >> >> At present, MDB is working as intended. In your test a number of overflow >> pages are being used. Currently libmdb always allocates overflow pages >> from new space, because it's not smart enough to search the free list for >> contiguous pages. Can treat this ITS as an enhancement request for that >> purpose, but this is not a new issue nor is it unexpected in the current >> version of MDB. >> > Hm, I take this back. The overflow pages are not enough to account for > the difference. Still looking. If this were truly the expected behavior, it would make using back-mdb with delta-syncrepl useless, as the accesslog would simply grow without bound. ;) Even back-bdb/hdb don't do that. While the DB will never be smaller than the maximum size ever reached without reloading, it won't unnecessarily grow past that either. :P --Quanah -- Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#7210) back-mdb endless database growth
by hyc＠symas.com 20 Mar '12

20 Mar '12

hyc(a)symas.com wrote: > quanah(a)OpenLDAP.org wrote: >> Full_Name: Quanah Gibson-Mount >> Version: 2.4.30 >> OS: Linux 2.6 >> URL: ftp://ftp.openldap.org/incoming/ >> Submission from: (NULL) (75.108.184.39) >> >> >> If you create a script to add& delete the same 50,000 users, and then re-run >> that script multiple times, the MDB database on disk continues to grow, despite >> the fact that we are only adding and deleting the same data repeatedly. There >> should be zero growth to the database. >> >> Instead, the data.mdb file went from 108MB in size to over 1.2GB after 5-6 runs. >> Time to add/delete all the users went from 3 minutes 32 seconds on the first >> run to over 90 minutes on the final run (5th or 6th, I lost count). > > At present, MDB is working as intended. In your test a number of overflow > pages are being used. Currently libmdb always allocates overflow pages from > new space, because it's not smart enough to search the free list for > contiguous pages. Can treat this ITS as an enhancement request for that > purpose, but this is not a new issue nor is it unexpected in the current > version of MDB. > Hm, I take this back. The overflow pages are not enough to account for the difference. Still looking. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7210) back-mdb endless database growth
by hyc＠symas.com 20 Mar '12

20 Mar '12

quanah(a)OpenLDAP.org wrote: > Full_Name: Quanah Gibson-Mount > Version: 2.4.30 > OS: Linux 2.6 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (75.108.184.39) > > > If you create a script to add& delete the same 50,000 users, and then re-run > that script multiple times, the MDB database on disk continues to grow, despite > the fact that we are only adding and deleting the same data repeatedly. There > should be zero growth to the database. > > Instead, the data.mdb file went from 108MB in size to over 1.2GB after 5-6 runs. > Time to add/delete all the users went from 3 minutes 32 seconds on the first > run to over 90 minutes on the final run (5th or 6th, I lost count). At present, MDB is working as intended. In your test a number of overflow pages are being used. Currently libmdb always allocates overflow pages from new space, because it's not smart enough to search the free list for contiguous pages. Can treat this ITS as an enhancement request for that purpose, but this is not a new issue nor is it unexpected in the current version of MDB. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7212) slapmodify support for back-config
by ondrej.kuznik＠acision.com 20 Mar '12

20 Mar '12

Full_Name: Ondrej Kuznik Version: HEAD OS: Linux URL: ftp://ftp.openldap.org/incoming/Ondrej-Kuznik-20120319-back-config-slapmodi… Submission from: (NULL) (62.168.56.1) For slapmodify to be usable on a database, that database needs to implement the bi_tool_dn2id_get and bi_tool_entry_modify functions, which is not the case of the back-config. I've put together a minimal (and maybe too naive) implementation of these for back-config and back-ldif at the above link. Currently, there is no validation whether the modification passes back-config checks since config_modify_internal takes a modify operation, while bi_tool_entry_modify receives only the new version of the entry to be modified. Also, for slapmodify to be really useful, it would have to allow entry deletion, for which there is no tool mode callback. To implement deletes, do you think it is better that deletion of an entry with children fail or delete the entire subtree? While the former seems both easier to implement and more sensible to me, I would like your know opinion. The attached file is derived from OpenLDAP Software. All of the modifications to OpenLDAP Software represented in the following patch(es) were developed by Acision. Acision has not assigned rights and/or interest in this work to any party. I, Ondrej Kuznik am authorized by Acision, my employer, to release this work under the following terms. The attached modifications to OpenLDAP Software are subject to the following notice: Copyright 2012 Acision Redistribution and use in source and binary forms, with or without modification, are permitted only as authorized by the OpenLDAP Public License.

1 0

Re: (ITS#7211) mdb inconsistency without LAZY_LOCKS
by h.b.furuseth＠usit.uio.no 19 Mar '12

19 Mar '12

I wrote: > I'm sleeping for write transactions only since that's when the > inconsistent copy is written back to disk. Please ignore that, that was another too-paranoid patch -- Hallvard

1 0

(ITS#7211) mdb inconsistency without LAZY_LOCKS
by h.b.furuseth＠usit.uio.no 19 Mar '12

19 Mar '12

Full_Name: Hallvard B Furuseth Version: master, af3a23a1ec93220be392a8ea6372475b7357c171 OS: Linux x86_64 URL: Submission from: (NULL) (195.1.106.125) Submitted by: hallvard A write transaction can overwrite env->me_metas[]->mm_dbs, me_numdbs and me_dbxs while another txn is copying them in mdb_txn_renew0(), resulting in an inconsistent copy. '#define LAZY_LOCKS 0' fixes it. Without it, test054 dumps core with this patch: diff --git a/libraries/libmdb/mdb.c b/libraries/libmdb/mdb.c index b714119..5e22827 100644 --- a/libraries/libmdb/mdb.c +++ b/libraries/libmdb/mdb.c @@ -1519,2 +1519,4 @@ mdb_txn_renew0(MDB_txn *txn) memcpy(txn->mt_dbs, env->me_metas[txn->mt_toggle]->mm_dbs, 2 * sizeof(MDB_db)); + if ((rand() & 1)) usleep(100000); + assert(!memcmp(txn->mt_dbs, env->me_metas[txn->mt_toggle]->mm_dbs, 2 * sizeof(MDB_db))); if (txn->mt_numdbs > 2) I'm sleeping for write transactions only since that's when the inconsistent copy is written back to disk.

1 0

(ITS#7210) back-mdb endless database growth
by quanah＠OpenLDAP.org 19 Mar '12

19 Mar '12

Full_Name: Quanah Gibson-Mount Version: 2.4.30 OS: Linux 2.6 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (75.108.184.39) If you create a script to add & delete the same 50,000 users, and then re-run that script multiple times, the MDB database on disk continues to grow, despite the fact that we are only adding and deleting the same data repeatedly. There should be zero growth to the database. Instead, the data.mdb file went from 108MB in size to over 1.2GB after 5-6 runs. Time to add/delete all the users went from 3 minutes 32 seconds on the first run to over 90 minutes on the final run (5th or 6th, I lost count).

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs