openldap-technical

openldap-technical@openldap.org

14 participants
9796 discussions

When is it ok to call MDB_NEXT_MULTIPLE?
by Bryan Matsuo 01 Nov '15

01 Nov '15

The docs imply that the MDB_NEXT_MULTIPLE cursor operation may only be used following MDB_GET_MULTIPLE (or another MDB_NEXT_MULTIPLE). Is that the case? In my experience MDB_NEXT_MULTIPLE has behaved the same as MDB_GET_MULTIPLE following MDB_NEXT_NODUP. That was a desirable thing in my case. I am wondering if that behavior can be depended on. The documentation for MDB_NEXT_MULTIPLE is probably technically correct. But it is vague. I don't understand what "preparing the cursor" entails.

1 0

Binding mdb_set_compare with Go
by Bryan Matsuo 01 Nov '15

01 Nov '15

openldap-technical, I am working on some Go (golang) bindings[1] for the LMDB library and I have some interest in exposing the functionality of mdb_set_compare (and mdb_set_dupsort). But it is proving difficult and I have a question about the function(s). Calling mdb_set_compare from the Go runtime is challenging. Using C APIs with callbacks comes with restrictions[2][3]. I believe it impossible to bind these functions way that is flexible, as one would expect. A potential change to LMDB that would make binding drastically easier is having MDB_cmp_func to take a third "context" argument with type void*. Then a binding could safely use an arbitrary Go function for comparisons. Is it possible for future versions of LMDB to add a third argument to the MDB_cmp_func signature? Otherwise would it be acceptable for a variant API to be added using a different function type, one accepting three arguments? Thanks for the consideration. Cheers, - Bryan [1] Go bindings -- https://github.com/bmatsuo/lmdb-go [2] Cgo pointer restrictions -- https://github.com/golang/proposal/blob/master/design/12416-cgo-pointers.md [3] Cgo documentation -- https://golang.org/cmd/cgo/

4 10

Re: Unable to build LMDB on SUSE Linux: undefined reference to pthread_mutexattr_setrobust / pthread_mutex_consistent
by Howard Chu 31 Oct '15

31 Oct '15

> Hello, > > I am trying to build LMDB libraries from source on SUSE Linux using gcc 4.7.2 to try for an experimental project at work. I am getting some build errors. I see pthread.h and libpthread.{a/so} installed in /usr/include and /usr/lib64, respectively. But I do not see these functions in the header file or in the library. Maybe the pthread library version is old. I am not sure. I tried searching on the web, but I was not able to understand how to install a different version of the pthread library (it seems to come packaged with the OS?). I do not have admin rights so I would try installing in my local area. > > Can someone please advise on things I can try to resolve this issue? Thank you for your time and help. You're not using a released branch of LMDB code. When you use a development branch you're expected to actually know how your development environment works. > (P.S. I had built LMDB successfully on a Mac OS X machine (home computer) without any issues) > > $ make > gcc -pthread -O2 -g -W -Wall -Wno-unused-parameter -Wbad-function-cast -Wuninitialized -c mdb.c > mdb.c: In function ‘mdb_mutex_failed’: > mdb.c:9827:4: warning: implicit declaration of function ‘pthread_mutex_consistent’ [-Wimplicit-function-declaration] > gcc -pthread -O2 -g -W -Wall -Wno-unused-parameter -Wbad-function-cast -Wuninitialized -c midl.c > ar rs liblmdb.a mdb.o midl.o > ar: creating liblmdb.a > gcc -pthread -O2 -g -W -Wall -Wno-unused-parameter -Wbad-function-cast -Wuninitialized -fPIC -c mdb.c -o mdb.lo > mdb.c: In function ‘mdb_mutex_failed’: > mdb.c:9827:4: warning: implicit declaration of function ‘pthread_mutex_consistent’ [-Wimplicit-function-declaration] > gcc -pthread -O2 -g -W -Wall -Wno-unused-parameter -Wbad-function-cast -Wuninitialized -fPIC -c midl.c -o midl.lo > gcc -pthread -shared -o liblmdb.so mdb.lo midl.lo > gcc -pthread -O2 -g -W -Wall -Wno-unused-parameter -Wbad-function-cast -Wuninitialized -c mdb_stat.c > gcc -pthread -O2 -g -W -Wall -Wno-unused-parameter -Wbad-function-cast -Wuninitialized mdb_stat.o liblmdb.a -o mdb_stat > liblmdb.a(mdb.o): In function `mdb_env_setup_locks': > /nfs/pdx/proj/dt/tctg03/pgupta6/lmdb-mdb.master/libraries/liblmdb/mdb.c:4642: undefined reference to `pthread_mutexattr_setrobust' > liblmdb.a(mdb.o): In function `mdb_mutex_failed': > /nfs/pdx/proj/dt/tctg03/pgupta6/lmdb-mdb.master/libraries/liblmdb/mdb.c:9827: undefined reference to `pthread_mutex_consistent' > collect2: error: ld returned 1 exit status > make: *** [mdb_stat] Error 1 > > My environment is as follows: > > $ more /etc/SuSE-release > SUSE Linux Enterprise Server 11 (x86_64) > VERSION = 11 > PATCHLEVEL = 2 > > $ uname -a > Linux 3.0.101-0.7.29.1.8482.4.PTF-default #1 SMP Tue May 5 16:00:34 UTC 2015 () x86_64 x86_64 x86_64 GNU/Linux SUSE Linux Enterprise Server 11 definitely supports robust mutexes, since they actually list bug fixes for that feature. https://www.suse.com/support/update/announcement/2013/suse-su-20131852-1.ht… But it's possible that the function definitions are hidden or renamed. Look for them in /usr/include/pthread.h and then supply the appropriate preprocessor flags to expose their definitions. You may need to adjust the LMDB code to use pthread_mutexattr_setrobust_np and pthread_mutex_consistent_np. If you don't know how to do this, stick with the actual release branch. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Ch

1 0

Attribute syntax and search filters
by Michael 31 Oct '15

31 Oct '15

I originally had a request from developers to allow substr searches on the "drawId" integer attribute you see below. As you know, substr searches on integers are not supported so I changed the schema and modified the attribute type to a numeric string. This worked as expected but now they are coming back to me asking why negative numbers are not being matched in their search filters. After some reading I understand the numeric string syntax is used to hold strings comprised of one or more ASCII numeric digits digits and/or space characters. The hyphen is not allowed, so it is not possible to represent negative values. Is there any alternative here? Original schema entry attributetype ( 3.0.0.3 NAME 'drawId' DESC 'ID of the retailer a user is assigned to' EQUALITY integerMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.27 SINGLE-VALUE ) Current schema entry. attributetype ( 3.0.0.3 NAME 'drawId' DESC 'ID of the retailer a user is assigned to' EQUALITY numericStringMatch SUBSTR numericStringSubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.36 SINGLE-VALUE )

2 2

Migrate from openldap 2.2 to 2.4 issue
by DEVARIEUX Alain 30 Oct '15

30 Oct '15

Hi! First excuse me for my approximative english. I'm trying to migrate from an old Redhat server running openldap 2.2 to a brand new one using Centos 7 and openldap 2.4. Using slapcat / sldapadd I can't have my new server running with an olc config. I'd like to know what I'm doing wrong during this process : # To remove entryUUID lines because they're not usable with openldap 2.4 sed -i -e "/entryUUID/d" /root/myslapcat.ldif # Running slapadd with a 'cleaned' version of my old slapd.conf slapadd -f /tpm/oldserver/slapd.conf -F /etc/openldap/slapd.d/ -c -u -o schema-check=yes -l /root/myslapcat.ldif # moving from file configuration to olc : slaptest -f /tmp/oldserver/slapdb.conf -F /etc/openldap/slapd.d/ # now, I can start the service without problem : systemctl start slapd But, when I try to access the diectory, here is the error messages I have : Oct 29 16:02:57 ldap01-qualif slapd[12351]: conn=1002 fd=11 ACCEPT from IP=10.35.100.87:49238 (IP=0.0.0.0:389) Oct 29 16:02:57 ldap01-qualif slapd[12351]: conn=1002 op=0 BIND dn="cn=Manager,dc=mydomain,dc=fr" method=128 Oct 29 16:02:57 ldap01-qualif slapd[12351]: conn=1002 op=0 BIND dn="cn=Manager,dc=mydomain,dc=fr" mech=SIMPLE ssf=0 Oct 29 16:02:57 ldap01-qualif slapd[12351]: conn=1002 op=0 RESULT tag=97 err=0 text= Oct 29 16:02:57 ldap01-qualif slapd[12351]: conn=1002 op=1 SRCH base="dc=mydomain,dc=fr" scope=1 deref=0 filter="(objectClass=*)" Oct 29 16:02:57 ldap01-qualif slapd[12351]: conn=1002 op=1 SRCH attr=objectclass Oct 29 16:02:57 ldap01-qualif slapd[12351]: bdb(dc=mydomain,dc=fr): BDB0060 PANIC: fatal region error detected; run recovery Oct 29 16:02:57 ldap01-qualif slapd[12351]: bdb(dc=mydomain,dc=fr): BDB0060 PANIC: fatal region error detected; run recovery Oct 29 16:02:57 ldap01-qualif slapd[12351]: bdb(dc=mydomain,dc=fr): BDB0060 PANIC: fatal region error detected; run recovery Oct 29 16:02:57 ldap01-qualif slapd[12351]: bdb(dc=mydomain,dc=fr): BDB0060 PANIC: fatal region error detected; run recovery Oct 29 16:02:57 ldap01-qualif slapd[12351]: conn=1002 op=1 SEARCH RESULT tag=101 err=80 nentries=0 text=internal error But, if a launch slapd like telling it the configuration file to use, everything works well (ie : I can bind to server and view all entries) slapd -u ldap -f /tmp/oldserver/slapd.conf Am I missing something obvious? I'm new to openldap... Regards, -- Alain Devarieux Pôle Infrastructures GIP SIB

3 3

Re: [LMDB] What pointer is returned with combination of MDB_WRITEMAP and MDB_RESERVE?
by Victor Baybekov 30 Oct '15

30 Oct '15

Thanks a lot! My proof-of-concept code works OK. I do not understand all subtle details of mmap reliability, could you please help with these two: If I write data to a pointer to an opaque blob as discussed above, and my process crashes before mdb_env_sync, but OS doesn't crash - will that data be secure in the mmap file? Also, am I correct that mdb_env_sync synchronizes all dirty pages in the mmap file as seen by a file system, regardless how they were modified - either via LMDB API or via a direct pointer writes? As for "you could at least set a callback to notify you that a block has moved" - if that is implemented, it would be nice to have a notification *before* a block is moved (with old and new address, so that right after the callback it is OK to use the new address), otherwise this non-intended but convenient use of LMDB won't work anymore. Best regards, Victor On Sat, Oct 3, 2015 at 1:27 AM, Howard Chu <hyc(a)symas.com> wrote: > Howard Chu wrote: > >> Victor Baybekov wrote: >> >>> Thank you! I understand this copy-on-write behavior, but am interested >>> if I >>> could control it a little. What if I use records that are always much >>> bigger >>> than a single page, e.g. 100 kb with 4kb pages, and make sure that a >>> record is >>> never updated (via LMDB means) during a lifetime of an environment, - is >>> there >>> any scenario that the location of such a big record could be changed >>> during a >>> lifetime of an environment, without updating the record? >>> >> >> At this point in time, no, if you don't update a large record there is no >> reason that it will move. That is not to say that this won't change in the >> future. The documentation tells you what promises we are willing to make. >> Relying on any non-documented behavior is your own responsibility. >> > > Note that the relocation functions in LMDB are intended to accommodate > blocks being moved around. The actual guts of that API haven't been > implemented, but probably in 1.x we'll flesh them out. Given that support, > you could at least set a callback to notify you that a block has moved. But > currently, overflow pages don't move if they're not modified. > > >>> >>> >>> On Fri, Oct 2, 2015 at 4:38 PM, Howard Chu <hyc(a)symas.com >>> <mailto:hyc@symas.com>> wrote: >>> >>> Victor Baybekov wrote: >>> >>> Hi, >>> >>> Docs for MDB_RESERVE say that a returned pointer to the reserved >>> space is >>> valid "before the next update operation or the transaction >>> ends." Docs >>> for MDB_WRITEMAP say that it "writes directly to the mmap >>> instead of >>> using >>> malloc for pages." Does combining the two options return a >>> pointer >>> directly to >>> a place in a mmap >>> >>> >>> Yes. >>> >>> so that this pointer could be used after a transaction ends >>> or after the next update? >>> >>> >>> No. >>> >>> Longer answer: maybe. >>> >>> Full answer: LMDB is copy-on-write. If you update another record on >>> the >>> same page, in a later transaction, the contents of that page will be >>> copied to a new page and the original page will go onto the >>> freelist. In >>> that case, the pointer you got must not be used again. >>> >>> If you don't directly update that page and cause it to be copied, >>> then you >>> might get lucky and be able to use the pointer for a while. It all >>> depends >>> on what other modifications you do and how they affect that node or >>> neighboring nodes. >>> >>> >>> I have a use case where I want to somewhat abuse LMDB safety for >>> convenience. >>> If I could get a pointer to a place inside a mmap I could work >>> with >>> LMDB value >>> as opaque blob or as a region inside the single big mmap. This >>> could >>> be more >>> convenient than creating and opening hundreds of temporary memory >>> mapped files >>> and keeping open handles to them. For example, Aeron terms could >>> be >>> stored >>> like this: a stream id per an LMDB db and a term id for a key in >>> the >>> db. >>> >>> >>> Thanks! >>> Victor >>> >>> >>> >>> -- >>> -- Howard Chu >>> CTO, Symas Corp. http://www.symas.com >>> Director, Highland Sun http://highlandsun.com/hyc/ >>> Chief Architect, OpenLDAP http://www.openldap.org/project/ >>> >>> >>> >> >> > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/ >

2 1

Re: (ITS#8290) Our system has occured "Server is unwilling to perform" from slapd when syncrepl rid was changed.
by Andrew Findlay 30 Oct '15

30 Oct '15

On Tue, Oct 27, 2015 at 02:55:11AM +0000, t-ogura(a)az.jp.nec.com wrote: > Subject: (ITS#8290) Our system has occured "Server is unwilling to perform" This does not look like a bug report, so you should direct it to the openldap-technical mailing-list rather than the ITS. > Master server connect two slave nodes , A and B. And, LDAP syncronization > configure as below. > > B server <---> Master server <---> A server > rid=27 rid=30 rid=32 > > Master server is set for LDAP syncronization in slapd.conf. Remember that the syncrepl directive only configures a replication consumer. Servers A and B also need syncrepl directives. All three servers also need the syncprov overlay, and I suspect that this is missing in your case leading to the error that you mentioned. Andrew -- ----------------------------------------------------------------------- | From Andrew Findlay, Skills 1st Ltd | | Consultant in large-scale systems, networks, and directory services | | http://www.skills-1st.co.uk/ +44 1628 782565 | -----------------------------------------------------------------------

1 0

pwdHistory, encrypted passwords and selfservice application
by Bogdan Rudas 30 Oct '15

30 Oct '15

Hello all, I'm working on Self-service application and want to prevent user from re-using old passwords. What is correct way to chage password takin in mind password history? I guess it is: 1. Bind with special user and check if specified uid exists 2. Bind using user-supplied uid and password 3. Get password policy, history etc. and validate on selfservice-side 4. Execute LDAP modifyRequest with single item: userPassword and value of new hashed password. In my case same password gives same hash. Are there any way to force encrypted password history validation on server side? Thank you. -- Bogdan Rudas Head of Minsk IT Support Department Exadel Inc. http://www.exadel.com/ E-mail: brudas(a)exadel.com <mandrushkevich(a)exadel.com> Skype ID: bogdan.rudas -- CONFIDENTIALITY NOTICE: This email and files attached to it are confidential. If you are not the intended recipient you are hereby notified that using, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error please notify the sender and delete this email.

4 3

Millions of users linked into a single group
by Alessandro Lasmar Mourão 28 Oct '15

28 Oct '15

Hello, I wonder if there is any limitation on the number of users linked to a group groupOfUniqueNames type? We will provide an application on the Internet for more than 10 million users, and all these users belong (uniqueMember) to a single group. Our support reported that it is recommended that the user group should not have more than 16,000 members, this information accurate? Regards, Alessandro Lasmar Mourão

6 6

syncrepl without cleartext password.
by Prakash Padadune 27 Oct '15

27 Oct '15

I want to implement syncrepl without having cleartext password in the slapd.conf. How this can be achieved? ~~~ *Prakash*

3 2

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical