openldap-devel

openldap-devel@openldap.org

1 participants
1421 discussions

Update/drop cruft: LDAP_DEPRECATED, CLDAP, C version
by Hallvard Breien Furuseth 21 Jun '15

21 Jun '15

Some wishes for RE25: Expose most LDAP_DEPRECATED functions again. This code is not going away, or it'd be gone by now after 16 years of deprecation. I usually #define LDAP_DEPRECATED anyway. What's the point of trying to make me pass 2 extra NULL args to a bunch of functions? And making a "SASL bind" function the proper way to do Simple Bind is silly and confusing. OTOH some depreciation make sense, like ldap_get_values() which returns char** instead of berval**. Add --enable-cldap as "experimental". Add tests and some fixes. The CLDAP code isn't going away either, so i seems better to support it properly. People are submitting libldap ITSes, but slapd and ldapsearch usage remain broken. slapd looks easy to fix, ldapsearch depends on vestiges of libldap code to ignore Bind requests. Dunno if I want to bother tracking that down in libldap or just omit the Bind in ldapsearch. Maybe RedHat has a fix. ...or do the opposite, kill LDAP_DEPRECATED and CLDAP. I hope not, but that's better than eternally deprecated/code-rotting features. Kill K&R1 support, i.e. the LDAP_P(()) stuff. Supposedly the headers support K&R1, but OpenLDAP-2.0 broke this (ITS#6171). Nobody has cared enough to fix it, and I refuse to fix it. See if we can require C99 to build OpenLDAP, but keep the headers C90-compliant. In particular, replace Debug() with a varargs macro, initially without changing the calls to Debug(). If we get away with that, we can kill the extra ,0,0,0 arguments to Debug() later so gcc -Wformat will catch missing/spurious args. -- Hallvard

3 4

slapd-sock: extensions always UTF-8
by Michael Ströder 19 Jun '15

19 Jun '15

HI! Are these extensions guaranteed to be always sent by back-sock as raw UTF-8 in one line? Or does the listener also has to handle base64 encoding and line-folding like in LDIF for these extension lines? binddn: <bound DN> peername: IP=<address>:<port> ssf: <SSF value> connid: <connection ID> Ciao, Michael.

1 0

Relaxed mode for delta-sync MMR
by Quanah Gibson-Mount 10 Jun '15

10 Jun '15

After having deployed delta-sync MMR at several customer sites, the general handling of conflict resolution in MMR mode is significantly sub optimal, and routinely causes the MMR nodes to get further out of sync, worsening things significantly (Mainly due to ITS#8125). The main issues I see are the following: a) Two masters get different change requests at approximately the same time to add a value X to an attribute. b) Two masters get different change requests at approximately the same time to delete a value X from an attribute. In these two specific cases, in relaxed mode, rather than falling back and re-syncing the entire database, I think the conflict should be discarded (skipped), and logged as such. I.e., there is no actual discrepancy in the object. It still has X present in the add case, and X gone in the delete case. At best, if we're going to do fallback, then we should only see about resyncing the specific entry. The overall behavior I'm seeing from OpenLDAP is the masters get in an endless cycle of re-sync, and the more they do so, the more out of sync they become, leading to a point at which you have to stop all masters, export all their DBs, sort them, find missing entries between all sets of masters, and build a brand new DB with which to reload them, until they get massively out of sync again. I.e., the current strategy of resync is doing no favors to anyone. It may work OK on very small DBs, where a resync only takes seconds, but on larger dbs were such syncs take 30+ minutes to hours, it is not a useful methodology. --Quanah -- Quanah Gibson-Mount Platform Architect Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Infinite loop with "mdb_search: 124xxxx1 scope not okay" while adding an entry
by Benjamin Dauvergne 02 Jun '15

02 Jun '15

Hi, I'm using OpenLDAP debian package from wheezy-backports (version 2.4.31+really2.4.40+dfsg) which is a 2.4.40 but backported I think under another version number to allow the jessie package to execute its migration when upgrade time will come. When trying to initialize a new DB by loading an LDIF file using ldapvi, looking like that: add dc=coin2,dc=fr objectClass: organization objectClass: dcObject objectClass: eduOrg objectClass: supannOrg dc: coin2 o: whatever supannEtablissement: {UAI}ccc add ou=people,dc=coin2,dc=fr objectClass: organizationalUnit ou: people add uid=admin,ou=people,dc=coin2,dc=fr objectClass: inetOrgPerson objectClass: eduPerson objectClass: supannPerson uid: admin cn: Administrateur annuaire displayName: Administrateur annuaire givenName: Administrateur sn: annuaire supannListeRouge: TRUE userPassword: xxxx supannEtablissement: {UAI}COIN It blocks on adding the ou=people. After setting loglever to 255 I got my syslog filled with such messages, soon filling the virtual machine virtual disk: Jun 2 02:34:26 ldap1-psl slapd[12159]: mdb_search: 1 scope not okay Jun 2 02:34:26 ldap1-psl slapd[12159]: mdb_search: 2 scope not okay Jun 2 02:34:26 ldap1-psl slapd[12159]: mdb_search: 3 scope not okay etc... The log of the query finishing like that is: Jun 2 02:34:26 ldap1-psl slapd[12159]: connection_get(15) Jun 2 02:34:26 ldap1-psl slapd[12159]: connection_get(15): got connid=1001 Jun 2 02:34:26 ldap1-psl slapd[12159]: connection_read(15): checking for input on id=1001 Jun 2 02:34:26 ldap1-psl slapd[12159]: op tag 0x68, time 1433205266 Jun 2 02:34:26 ldap1-psl slapd[12159]: conn=1001 op=9 do_add Jun 2 02:34:26 ldap1-psl slapd[12159]: conn=1001 op=9 do_add: dn (ou=people,dc=coin2,dc=fr) Jun 2 02:34:26 ldap1-psl slapd[12159]: >>> dnPrettyNormal: <ou=people,dc=coin2,dc=fr> Jun 2 02:34:26 ldap1-psl slapd[12159]: <<< dnPrettyNormal: <ou=people,dc=coin2,dc=fr>, <ou=people,dc=coin2,dc=fr> Jun 2 02:34:26 ldap1-psl slapd[12159]: ==> unique_add <ou=people,dc=coin2,dc=fr> Jun 2 02:34:26 ldap1-psl slapd[12159]: ==> unique_search (|(objectClass=organizationalUnit)(ou=people)) Jun 2 02:34:26 ldap1-psl slapd[12159]: str2filter "(|(objectClass=organizationalUnit)(ou=people))" Jun 2 02:34:26 ldap1-psl slapd[12159]: begin get_filter Jun 2 02:34:26 ldap1-psl slapd[12159]: OR Jun 2 02:34:26 ldap1-psl slapd[12159]: begin get_filter_list Jun 2 02:34:26 ldap1-psl slapd[12159]: begin get_filter Jun 2 02:34:26 ldap1-psl slapd[12159]: EQUALITY Jun 2 02:34:26 ldap1-psl slapd[12159]: end get_filter 0 Jun 2 02:34:26 ldap1-psl slapd[12159]: begin get_filter Jun 2 02:34:26 ldap1-psl slapd[12159]: EQUALITY Jun 2 02:34:26 ldap1-psl slapd[12159]: end get_filter 0 Jun 2 02:34:26 ldap1-psl slapd[12159]: end get_filter_list Jun 2 02:34:26 ldap1-psl slapd[12159]: end get_filter 0 Jun 2 02:34:26 ldap1-psl slapd[12159]: => mdb_search Jun 2 02:34:26 ldap1-psl slapd[12159]: mdb_dn2entry("dc=coin2,dc=fr") Jun 2 02:34:26 ldap1-psl slapd[12159]: => mdb_dn2id("dc=coin2,dc=fr") Jun 2 02:34:26 ldap1-psl slapd[12159]: <= mdb_dn2id: got id=0x1 Jun 2 02:34:26 ldap1-psl slapd[12159]: => mdb_entry_decode: Jun 2 02:34:26 ldap1-psl slapd[12159]: <= mdb_entry_decode Jun 2 02:34:26 ldap1-psl slapd[12159]: => access_allowed: search access to "dc=coin2,dc=fr" "entry" requested Jun 2 02:34:26 ldap1-psl slapd[12159]: <= root access granted Jun 2 02:34:26 ldap1-psl slapd[12159]: => access_allowed: search access granted by manage(=mwrscxd) Jun 2 02:34:26 ldap1-psl slapd[12159]: search_candidates: base="dc=coin2,dc=fr" (0x00000001) scope=-1 Jun 2 02:34:26 ldap1-psl slapd[12159]: => mdb_filter_candidates Jun 2 02:34:26 ldap1-psl slapd[12159]: #011OR Jun 2 02:34:26 ldap1-psl slapd[12159]: => mdb_list_candidates 0xa1 Jun 2 02:34:26 ldap1-psl slapd[12159]: => mdb_filter_candidates Jun 2 02:34:26 ldap1-psl slapd[12159]: #011EQUALITY Jun 2 02:34:26 ldap1-psl slapd[12159]: => mdb_equality_candidates (objectClass) Jun 2 02:34:26 ldap1-psl slapd[12159]: => key_read Jun 2 02:34:26 ldap1-psl slapd[12159]: mdb_idl_fetch_key: [b49d1940] Jun 2 02:34:26 ldap1-psl slapd[12159]: <= mdb_index_read: failed (-30798) Jun 2 02:34:26 ldap1-psl slapd[12159]: <= mdb_equality_candidates: id=0, first=0, last=0 Jun 2 02:34:26 ldap1-psl slapd[12159]: <= mdb_filter_candidates: id=0 first=0 last=0 Jun 2 02:34:26 ldap1-psl slapd[12159]: => mdb_filter_candidates Jun 2 02:34:26 ldap1-psl slapd[12159]: #011OR Jun 2 02:34:26 ldap1-psl slapd[12159]: => mdb_list_candidates 0xa1 Jun 2 02:34:26 ldap1-psl slapd[12159]: => mdb_filter_candidates Jun 2 02:34:26 ldap1-psl slapd[12159]: #011EQUALITY Jun 2 02:34:26 ldap1-psl slapd[12159]: => mdb_equality_candidates (objectClass) Jun 2 02:34:26 ldap1-psl slapd[12159]: => key_read Jun 2 02:34:26 ldap1-psl slapd[12159]: mdb_idl_fetch_key: [9bee355f] Jun 2 02:34:26 ldap1-psl slapd[12159]: <= mdb_index_read: failed (-30798) Jun 2 02:34:26 ldap1-psl slapd[12159]: <= mdb_equality_candidates: id=0, first=0, last=0 Jun 2 02:34:26 ldap1-psl slapd[12159]: <= mdb_filter_candidates: id=0 first=0 last=0 Jun 2 02:34:26 ldap1-psl slapd[12159]: => mdb_filter_candidates Jun 2 02:34:26 ldap1-psl slapd[12159]: #011EQUALITY Jun 2 02:34:26 ldap1-psl slapd[12159]: => mdb_equality_candidates (ou) Jun 2 02:34:26 ldap1-psl slapd[12159]: <= mdb_equality_candidates: (ou) not indexed Jun 2 02:34:26 ldap1-psl slapd[12159]: <= mdb_filter_candidates: id=-1 first=1 last=-1 Jun 2 02:34:26 ldap1-psl slapd[12159]: <= mdb_list_candidates: id=-1 first=1 last=-1 Jun 2 02:34:26 ldap1-psl slapd[12159]: <= mdb_filter_candidates: id=-1 first=1 last=-1 Jun 2 02:34:26 ldap1-psl slapd[12159]: <= mdb_list_candidates: id=-1 first=1 last=-1 Jun 2 02:34:26 ldap1-psl slapd[12159]: <= mdb_filter_candidates: id=-1 first=1 last=-1 Jun 2 02:34:26 ldap1-psl slapd[12159]: mdb_search_candidates: id=-1 first=1 last=-1 Jun 2 02:34:26 ldap1-psl slapd[12159]: mdb_search: 1 scope not okay Jun 2 02:34:26 ldap1-psl slapd[12159]: mdb_search: 2 scope not okay Jun 2 02:34:26 ldap1-psl slapd[12159]: mdb_search: 3 scope not okay Jun 2 02:34:26 ldap1-psl slapd[12159]: mdb_search: 4 scope not okay Jun 2 02:34:26 ldap1-psl slapd[12159]: mdb_search: 5 scope not okay etc... I don't know why it's doing a search on an add but seeing the message "(ou) not indexed" I though that maybe adding an equality index on this attribute would help, and effectively it did. Now the add ou=people passed, but it started looping again when adding the uid=admin entry. The infinite loop happen in server/slapd/back-mdb/search.c in mdb_search(). If you have any idea I can continue investigating or add debug logs. The debian package has the following patches applied over openldap 2.4.40: add-tlscacert-option-to-ldap-conf autogroup-makefile contrib-modules-use-dpkg-buildflags do-not-second-guess-sonames evolution-ntlm fix-build-top-mk getaddrinfo-is-threadsafe heimdal-fix index-files-created-as-root ITS6035-olcauthzregex-needs-restart.patch ITS7975-fix-mdb-onelevel-search.patch ITS8027-deref-reject-empty-attr-list.patch ITS8046-fix-vrFilter_free-crash.patch lastbind-makefile ldap-conf-tls-cacertdir ldapi-socket-place libldap-symbol-versions man-slapd no-AM_INIT_AUTOMAKE no-bdb-ABI-second-guessing pw-sha2-makefile sasl-default-path slapi-errorlog-file smbk5pwd-makefile switch-to-lt_dlopenadvise-to-get-RTLD_GLOBAL-set.diff wrong-database-location

1 2

Does a LMDB database file ever shrink, after row deletion, if so when?
by Tinker 23 May '15

23 May '15

Hi! Is there any mechanism in LMDB to give back disk space after some particular amount of row deletion, how does this work, if not are there any plans for it now? If there's not, as a user I need 2x disk space and some mechanism to be able to migrate, so automatic shrinking would be a great thing. Thanks!! Tinker

2 1

*prio* SQLightning/LMDB OpenBSD problem: ./sqlite3 fails with "unknown error" on start and sqlite3_open() fails with 2 = SQLITE_INTERNAL. Did I build it incorrectly or do we need to resolve a bug?
by Tinker 23 May '15

23 May '15

Hi! On OpenBSD 5.5 AMD64, I compile SQLightning as follows: > git clone https://github.com/LMDB/lmdb.git > cd lmdb > git checkout mdb.RE/0.9 > cd libraries/liblmdb > make > sudo make install > (On Linux: sudo ldconfig) > cd ../../.. > > git clone https://github.com/LMDB/sqlightning.git > cd sqlightning > git checkout mdb > ./configure (trying "make" here complained that "mdb.c" doesn't exist on Linux and that "mdb.h" doesn't exist on OpenBSD, so I fixed this by "cp ../lmdb/libraries/liblmdb/* src/" and then): > make -j2 > sudo make install > (On Linux: sudo ldconfig) I then try the newly built sqlite3 binary: > ./sqlite3 /tmp/zz > SQLite version 3.7.17 2013-05-20 00:56:22 > Enter ".help" for instructions > Enter SQL statements terminated with a ";" > sqlite> create table a(b); > Error: unable to open database "/tmp/zz": UNKNOWN ERROR Equally so, sqlite3_open() fails with 2 = SQLITE_INTERNAL , http://www.sqlite.org/rescode.html#internal [1] . Both SQLightning, LMDB and SQLite are built specifically to work on OpenBSD (except for in the read-only mmap+fwrite()+fsync() mode, how to switch that off or on is beyond my current knowledge). How fix? Thank you very much! Tinker Links: ------ [1] http://www.sqlite.org/rescode.html#internal

1 0

lmdb: is read operation permitted inside the write transaction?
by Dmytro Milinevskyy 19 May '15

19 May '15

Hello, this might be a trivial question for you but I didn't find anything in the documentation nor any mention in the examples. Basically I need to have an opportunity to call mdb_get while in write transaction. The sequence is smth like: - txn = mdb_txn_begin(flags=0) for i in 0..x: - v = mdb_get(txn, i) - mdb_put(txn, x+i, v) - mdb_put(txn, i, v+1) - mdb_txn_commit(txn) Will it be always valid data? I'm running a stress test and didn't spot any issues so far. Though I might be extremely lucky today. The DB is used in multi-threaded environment but the code illustrated above is protected with a mutex. I tried to dig a bit into the lmdb internals but unfortunately didn't get any glue. Thanks for your help, Dmytro

4 5

LMDB-Patches and SQLightning-Issues
by Johannes Munk (P5) 17 May '15

17 May '15

Dear Howard and OpenLDAP-team! First of all: thanks a lot for the great LMDB and all the effort you have put into it. In the last couple of weeks we have been trying to get LMDB/SQLightning to work as backend for our semantic network software. On that road we made LMDB compatible with Win8.1/MSVC2013 (see patches here <https://gitlab.com/lukas-w/lmdb/commits/p5>), fixing scrambled filenames <https://gitlab.com/lukas-w/lmdb/commit/e10837272788f87ce43049bbbd3c9bcc0071…> and made SQLightning work with files greater than 4 GB <https://gitlab.com/lukas-w/sqlightning/commit/e52a9949fd5c30b7ac45b0909259c…>. But after some simple tests that worked, we ran into some issues, which we can't get a handle on. We have reported most of our issues here: https://gitlab.com/mdb/sqlightning/issues It would be great, if you could take a peek! Thx a lot in advance, Johannes

2 4

ITS#8102, syncrepl concurrency
by Howard Chu 13 May '15

13 May '15

We should probably change the consumer such that when multiple consumers are configured on a context, only one of them can be active at a time. In particular, in refresh phase, you'll get multiple redundant copies of the refresh data. All providers should be equivalent, so whichever one we connect to first should be used and any others should be left idle. Or, each session should be established completely sequentially - do a complete refresh from provider #1, then do the same for provider #2, etc... In a quiescent setup, the refreshes after provider #1 will be no-ops because the consumer is already up to date. In a busy setup, each refresh will still have work to do but it will be a smaller volume after provider #1's refresh completes. Simplest would be to just establish the first connection and ignore other providers unless the first connection breaks. Unfortunately this wouldn't be reliable in MMR - you could have two nodes pointed at each other, ignoring any other nodes in the configuration. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

3 2

syncrepl consumer is slow
by Howard Chu 12 May '15

12 May '15

One thing I just noticed, while testing replication with 3 servers on my laptop - during a refresh, the provider gets blocked waiting to write to the consumers after writing about 4000 entries. I.e., the consumers aren't processing fast enough to keep up with the search running on the provider. (That's actually not too surprising since reads are usually faster than writes anyway.) The consumer code has lots of problems as it is, just adding this note to the pile. I'm considering adding an option to the consumer to write its entries with dbnosync during the refresh phase. The rationale being, there's nothing to lose anyway if the refresh is interrupted. I.e., the consumer can't update its contextCSN until the very end of the refresh, so any partial refresh that gets interrupted is wasted effort - the consumer will always have to start over from the beginning on its next refresh attempt. As such, there's no point in safely/synchronously writing any of the received entries - they're useless until the final contextCSN update. The implementation approach would be to define a new control e.g. "fast write" for the consumer to pass to the underlying backend on any write op. We would also have to e.g. add an MDB_TXN_NOSYNC flag to mdb_txn_begin() (BDB already has the equivalent flag). This would only be used for writes that are part of a refresh phase. In persist mode the provider and consumers' write speeds should be more closely matched so it wouldn't be necessary or useful. Comments? -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

5 16

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-devel