openldap-technical

openldap-technical@openldap.org

7 participants
9901 discussions

Re: Very slow ldapserach
by Saša-Stjepan Bakša 09 Apr '15

09 Apr '15

On 9 April 2015 at 17:57, Quanah Gibson-Mount <quanah(a)zimbra.com> wrote: > --On Thursday, April 09, 2015 10:21 AM +0200 Saša-Stjepan Bakša < > ssbaksa(a)gmail.com> wrote: > >> And search for all still crashes openldap server >> >> ldapsearch -h 10.14.252.103 -p 389 -D cn=admin,dc=spr -w siemens -s sub >> -a always -b dc=SPR objectClass=* >> >> >> I must wait for 2.4.41 release then. In the mean time I will use LTB >> project build with HDB to see can I have that version in working state >> for my colleagues test suite. >> > > Hi Sasa, > > Thanks for checking. Looks like there is some more work to do in this > area before 2.4.41 releases. I wil be asking you to do further testing in > the near future. If it were possible for you to provide me an example DB > and your slapd configuration that replicates this behavior, it would help > with a resolution. > > --Quanah Hi Quanah, I will be more than happy to provide help with testing. I am at home now but tomorrow morning (8:00 CET) I will provide as much data as I can to you. As some of data can be of sensitive nature, may be I will be obliged to send it directly to you. Is that Ok with you? Best regards, Sasa

1 0

Re: Very slow ldapserach
by Saša-Stjepan Bakša 09 Apr '15

09 Apr '15

On 9 April 2015 at 17:57, Quanah Gibson-Mount <quanah(a)zimbra.com> wrote: > --On Thursday, April 09, 2015 10:21 AM +0200 Saša-Stjepan Bakša < > ssbaksa(a)gmail.com> wrote: > >> And search for all still crashes openldap server >> >> >> >> I must wait for 2.4.41 release then. In the mean time I will use LTB >> project build with HDB to see can I have that version in working state >> for my colleagues test suite. >> > > Hi Sasa, > > Thanks for checking. Looks like there is some more work to do in this > area before 2.4.41 releases. I wil be asking you to do further testing in > the near future. If it were possible for you to provide me an example DB > and your slapd configuration that replicates this behavior, it would help > with a resolution. > > --Quanah > > And again I did stupid thing to send withoth checking recipient. I hate this Gmail web interface. Sorry.

1 0

Bug with MDB backend and onelevel scope
by Côme BERNIGAUD 09 Apr '15

09 Apr '15

Hello, I got this weird behavior: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=782212 Summary: # LDAPv3 # base <ou=wheezy,ou=debian,ou=fai,ou=configs,ou=systems,dc=mcmic,dc=test> with scope oneLevel # filter: objectClass=FAIbranch # requesting: ALL # # wheezy, debian, fai, configs, systems, mcmic.test dn: ou=wheezy,ou=debian,ou=fai,ou=configs,ou=systems,dc=mcmic,dc=test This dn should not be in the results as it’s the base and the scope is oneLevel. As I managed to build slapd from http://www.openldap.org/software/download/ and reproduce the problem I thought I should ask here if anyone already encountered this kind of problems. It only happens with MDB, with this particular base and filter (filter "ou=wheezy" for instance works correctly). Any idea what the root of this problem could be?

3 2

Re: Very slow ldapserach
by Saša-Stjepan Bakša 09 Apr '15

09 Apr '15

On 9 April 2015 at 02:37, Quanah Gibson-Mount <quanah(a)zimbra.com> wrote: > --On Wednesday, April 08, 2015 8:51 PM +0200 Saša-Stjepan Bakša < > ssbaksa(a)gmail.com> wrote: > > I am sorry for this mistake with answering to you and not to the group. >> It was unintentional mistake. >> >> >> Ok, I will check ITS#7657. >> > > Please try current RE24 code. It is believed the fixes for ITS#8011 which > are scheduled to be part of the 2.4.41 relase should resolve the issue. > > Thanks, > > Quanah > > At 8:00 local time: git checkout OPENLDAP_REL_ENG_2_4 git pull Build + test root@test:~# date; ldapsearch -h 10.14.252.103 -p 389 -D cn=admin,dc=spr -w siemens -s sub -a always -b msisdn=385212000009,dc=MSISDN,dc=SPR objectClass=*; date *Thu Apr 9 08:22:46 CEST 2015* # extended LDIF # # LDAPv3 # base <msisdn=385212000009,dc=MSISDN,dc=SPR> with scope subtree # filter: objectClass=* # requesting: ALL # ----- cut------ # search result search: 2 result: 0 Success # numResponses: 9 # numEntries: 8 *Thu Apr 9 08:23:38 CEST 2015* Nothing changed yet - 52 seconds And search for all still crashes openldap server ldapsearch -h 10.14.252.103 -p 389 -D cn=admin,dc=spr -w siemens -s sub -a always -b dc=SPR objectClass=* 552641e7 >>> dnPrettyNormal: <dc=SPR> 552641e7 <<< dnPrettyNormal: <dc=SPR>, <dc=spr> ber_scanf fmt (m) ber: ber_scanf fmt ({M}}) ber: 552641e7 => mdb_search 552641e7 mdb_dn2entry("dc=spr") 552641e7 => mdb_dn2id("dc=spr") 552641e7 <= mdb_dn2id: got id=0x1 552641e7 => mdb_entry_decode: 552641e7 <= mdb_entry_decode 552641e7 search_candidates: base="dc=spr" (0x00000001) scope=2 552641e7 => mdb_equality_candidates (objectClass) 552641e7 => key_read 552641e7 <= mdb_index_read 10000000 candidates 552641e7 <= mdb_equality_candidates: id=-1, first=16, last=10000015 Segmentation fault I must wait for 2.4.41 release then. In the mean time I will use LTB project build with HDB to see can I have that version in working state for my colleagues test suite. Br Sasa

2 1

LMDB and HP-UX Itanium
by Kristian Amlie 08 Apr '15

08 Apr '15

I'm wondering if the LMDB database has been tested on HP-UX with the Itanium processor? We are trying to use the database there and are seeing strange errors that don't occur on other platforms. Examples are assertions when trying to do cross process access and values not making it into the database correctly. I can provide more details, but I thought I would ask about the current status first. -- Kristian

3 11

LMDB error
by Shu, Xinxin 08 Apr '15

08 Apr '15

Hi list , I am working on intergrating LMDB with ceph, I met follow error "Invalid argument" when I commit transaction to LMDB, details are descripted as following. Two threads operate on LMDB, each thread is responsible for create write transaction and read transaction , the write transaction will update LMDB, and read transaction is for iterator that read values from LMDB, the two threads share same MDB_env. I check mdb_txn_commit source code , there are two places to return "EINVAL", 1. if (txn == NULL || txn->mt_env == NULL) return EINVAL; 2. if (txn != env->me_txn) { DPUTS("attempt to commit unknown transaction"); rc = EINVAL; goto fail; } I check txn and txn->mt_env and rule out the first case , so only the second case cause "EINVAL", and variable env->me_txn is only initialized when you start a write transaction with mdb_txn_open, and there is a lock to prevent several writer from operating on LMDB. Btw , I tried to add lock to synchronize the two threads , everything seems ok. My question is in what situation can cause "txn != env-> me_txn",. I'm very new to LMDB, if there is anything wrong, please correct me. Cheers, xinxin

2 1

Re: Very slow ldapserach
by Saša-Stjepan Bakša 08 Apr '15

08 Apr '15

On 8 April 2015 at 18:05, Quanah Gibson-Mount <quanah(a)zimbra.com> wrote: > --On Wednesday, April 08, 2015 11:58 AM +0200 Saša-Stjepan Bakša < > ssbaksa(a)gmail.com> wrote: > > >> >> I have tested search with your suggestion. No crash - slapd crashes only >> when deref is on or in best case it is slow (10 to 20 seconds for >> answer). As I have said, when database is small (relative size) it is >> quick as we expected. >> >> >> So, which alternatives do I have? Back to hdb? Uh, I don't like it to >> much. >> >> Is this long standing issue a big problem to solve? I am not developer so >> maybe this was a rude question (sorry!). >> > > Please keep replies on the list if you want help. > > My guess is this is ITS#7657. > > > --Quanah > > I am sorry for this mistake with answering to you and not to the group. It was unintentional mistake. Ok, I will check ITS#7657. Sasa

2 1

two namingContexts problem
by Wang, Hui 08 Apr '15

08 Apr '15

Hi, All, I got two nameContexts, but I only set up one in slapd.conf which is o=csun. Is there any way to find out where the other nameContexts coming from? Is this normal? I also searched all the files on Linux, I didn’t find any file contains “cn=changelog” in the file content. [root@oldaptest01 init.d]# ldapsearch -x -b '' -s base '(objectclass=*)' namingContexts # extended LDIF # # LDAPv3 # base <> with scope baseObject # filter: (objectclass=*) # requesting: namingContexts # # dn: namingContexts: cn=changelog namingContexts: o=csun # search result search: 2 result: 0 Success # numResponses: 2 # numEntries: 1 # find -name "*.conf" -print | xargs grep -i "cn=change" No result Thank you Hui

1 0

[lmdb] MDB_BAD_DBI error upon mdb_txn_commit (newbie question)
by HLaw 08 Apr '15

08 Apr '15

Dear list, I am new to lmdb and my apologies if this is on something obvious. I encountered the MDB_BAD_DBI error in a simple single process, multi-threaded program using lmdb. The scenario is : * One thread doing a relatively long write transaction mdb_txn_begin -> mdb_dbi_open on several database -> (write) -> mdb_txn_commit * Another thread (or other threads) of the same process doing short read-only transactions mdb_txn_begin -> mdb_dbi_open on some of the database above -> (read) -> mdb_txn_commit As per the advice in the document I do not close the database directly. Also the environment created / opened is shared by all threads, being in the same process. All is fine when these threads execute separately, but if the read transaction thread(s) open the database and commit within the time span of the write transaction in another thread, the above error occurs upon attempt to commit the write transaction. I have done a search on the MDB_BAD_DBI error and noticed the commit below. https://gitorious.org/mdb/mdb/commit/0401f2deed75a83d2de790b8a1313e1792e5a0… Upon a brief look into this and the source of lmdb, my understanding is that the error is due to the increment of lmdb's internal sequence numbers of the database kept by the environment (by the read thread when the read transaction commits). When the write thread subsequently tries to commit, the database sequence number kept privately by the write transaction (copied from the environment when the transaction began) does not match that of the environment, causing the error. To avoid this in a single process multi-threaded environment, it seems that no read transactions could commit on databases while they are involved in a write transaction, i.e. I need to complete all read transactions before a write transaction on the same database could begin. Alternatively, to have concurrent read transactions with a write transaction, the read transactions have to take place in a separate process (such that they would not share the same environment and caught by the above error). Grateful if anyone could kindly enlighten me on whether the understanding is correct, or if I am doing something contrary to the how lmdb should be used. Thank you very much. - H Law

4 5

Re: Very slow ldapserach
by Quanah Gibson-Mount 08 Apr '15

08 Apr '15

--On Wednesday, April 08, 2015 11:58 AM +0200 Saša-Stjepan Bakša <ssbaksa(a)gmail.com> wrote: > > > > > > On 7 April 2015 at 21:14, Quanah Gibson-Mount <quanah(a)zimbra.com> wrote: > > --On Tuesday, April 07, 2015 1:10 PM -0700 Quanah Gibson-Mount > <quanah(a)zimbra.com> wrote: > > > > ldapsearch -h 10.14.252.104 -p 389 -D cn=admin,dc=spr -w password -s sub > -a always -b dc=SPR objectClass=* > > > Sounds like there's something wrong with your openldap build or server. > Zimbra has clients with DBs of various sizes and entry counts, often in > the multi-GBs, and we're not seeing any such issues. > > > Actually, you are doing alias deref. I believe there's a known issue > with back-mdb and alias deref. Do you see the same behavior if you > select -a never instead? > > > > --Quanah > > > > > Yes, we are depending on aliases. Without them we will need at least 2 > searches to get data we need. In new version even more because we have > data which is similar to many users and we use another aliases inside > each user to fetch that data for particular user. Let we say that each > user have 3 aliases to 3 groups of data which can have at leas 30 > different possibilities for each user. Without aliases data size for each > user will be to big and in summary database will be huge. > > > I have tested search with your suggestion. No crash - slapd crashes only > when deref is on or in best case it is slow (10 to 20 seconds for > answer). As I have said, when database is small (relative size) it is > quick as we expected. > > > So, which alternatives do I have? Back to hdb? Uh, I don't like it to > much. > > Is this long standing issue a big problem to solve? I am not developer so > maybe this was a rude question (sorry!). Please keep replies on the list if you want help. My guess is this is ITS#7657. --Quanah -- Quanah Gibson-Mount Platform Architect Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical