openldap-technical February 2017

openldap-technical@openldap.org

34 participants
62 discussions

Re: MDB_CORRUPTED: Located page was wrong type
by Howard Chu 09 Feb '17

09 Feb '17

opensource(a)gmx-topmail.de wrote: > > Hello, > > We build an Android app with LMDB and are now seeing an error report with the > following info: > error -30796: MDB_CORRUPTED: Located page was wrong type > > We also logged some general info about the DB: > entries=25381 depth=3 branch-pages=43 leaf-pages=891 overflow-pages=10 > page-size=4096 last-page-number=4464 last-tx-id=3752 size=536870912 > max-readers=126 readers=4 > > It is an Android 6.0.1 device, and that's about all the info we have. > Unfortunately we do not have the DB files etc. > > The app runs a LMDB version from mdb.master (45a8827 Howard Chu on 9/1/16 at > 1:41 AM, ITS#8489 reset cursor EOF flag in cursor_set). Generally you should not use mdb.master in production, as it's a development branch. > > DB curruption sounds like a rather bad issue. Any idea how this could have > happend? Not offhand. We've seen a few corruptions on Windows due to running without fully synchronous transactions, with OS crashes and/or faulty hard drives. It's pretty rare for phones to crash though. > Also, is there any chance to repair this DB (automatically)? Nothing automated at present. If you had the DB files you could try to scan the DB using the previous meta-page to see if it's still intact. > Thanks for your help, > Markus > > > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

MDB_CORRUPTED: Located page was wrong type
by opensource＠gmx-topmail.de 09 Feb '17

09 Feb '17

1 0

Transform accesslog database to LDIF for ldapmodify or other way
by Michael Wandel 07 Feb '17

07 Feb '17

Hey, I'm searching for a tool which is able to transform an accesslog Database to an ldif file, what can be used for ldapmodify. Or is there an alternative way to use the accesslog to rebuild an ldap database from a backup time to actual ? Every hint is welcome best regards Michael

2 4

Re: help troubleshooting
by Quanah Gibson-Mount 07 Feb '17

07 Feb '17

--On Monday, January 30, 2017 7:08 PM -0700 scar <scar(a)drigon.com> wrote: > However, this brings me to the next problem: the contents of slapd.conf > do not match the slapd.d/cn\=config.ldif file, so it seems the fixes i am > trying to the ACL's don't have any effect, even when i restart slapd. > If i try "ldapmodify -nv" it just hangs. When i try to stop slapd and > remove slapd.d/* and then start slapd, the contents are recreated > according to the config file, but then users can't login (all i see in > the logfile is access_allowed and slap_access_allowed but no conn lines) If you are using the configuration backend for slapd, then you can ignore the slapd.conf file entirely, and simply use the ldapmodify command to modify your access rules. I suggest reading the ldapmodify manual page for information on how to properly execute it. If you are using a distribution provided build of OpenLDAP, the necessary steps may depend on how they configured things. I would note that the rootdn is never subject to ACLs (as documented in the slapd.access(5) man page). So there is no point in listing it in ACLs. I would note that your final acl: "access to * by dn="uid=ldapadmin,dc=X,dc=Y,dc=Z" read" will never be applied, since ACL processing stops on the first matching acl (As noted in the slapd.access(5) man page), and the ACL immediately preceeding it already covers "access to *". I would note that your next to last ACL has also has items that would never be processed, specifically the "by * auth", since the "by * read" takes precedence. You don't provide any information on what identit(y/ies) you want to be able to modify the userPassword attribute, so it's difficult to help you further. Hopefully this is enough information to help you have forward progress. :) Regards, Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

3 2

Script for mass updates
by Nick Milas 06 Feb '17

06 Feb '17

Hello, Does anyone have a ready-made script (e.g. bash) that would do the following: Loop on all entries in the ou=people branch where ou <> "system" { If attribute DisplayName does not exist{ Set DisplayName to the value of attibute cn } } I could do it with a bit of work, but it's urgent. Any help will be appreciated! Thanks in advance, Nick

7 14

Re: RE24 testing call (2.4.45) LMDB RE0.9 testing call (0.9.20)
by Quanah Gibson-Mount 05 Feb '17

05 Feb '17

--On Tuesday, January 31, 2017 12:37 PM -0800 Quanah Gibson-Mount <quanah(a)symas.com> wrote: > --On Tuesday, January 31, 2017 12:05 PM -0800 Quanah Gibson-Mount > <quanah(a)symas.com> wrote: > >>> ERROR: Entry 21 not replicated to ldap://localhost:9012/! (32)! >>> Error found after 1 of 1 iterations >>>>>>>> test061-syncreplication-initiation failed for mdb >>> (exit 1) >>> Makefile:310: recipe for target 'mdb-yes' failed >>> make: *** [mdb-yes] Error 1 >> >> Interesting -- Can you tar up the testrun directory, ftp it up, and file >> an ITS with a pointer to the upload? > > I was able to reproduce this as well with back-mdb. Are you sure you're testing RE24 and not HEAD? I can reproduce this failure in HEAD easily, but test061 has passed for me approximately 1000 times so far in RE24. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

3 2

Re: Script for mass updates
by Ulrich Windl 05 Feb '17

05 Feb '17

(forgot to reply to the list) >>> Ulrich Windl <Ulrich.Windl(a)rz.uni-regensburg.de> schrieb am 06.02.2017 um 08:14 in Nachricht <58983099.ED38.00A1.0(a)rz.uni-regensburg.de>: >>>> "Ralf Mattes" <rm(a)mh-freiburg.de> schrieb am 01.02.2017 um 17:23 in Nachricht > <45d3-58920b80-2b-2963e90@255443456>: > > > Am Mittwoch, 01. Februar 2017 16:52 CET, Jephte Clain > > <jephte.clain(a)univ-reunion.fr> schrieb: > > > >> using michaël's filter, you could try this: > >> > >> ldapsearch [options] '(&(ou:dn:=people)(!(ou=system))(!(displayName=*)))' > >> cn | awk ' > >> /^dn:/ { > >> print > >> print "changetype: modify" > >> print "replace: displayName" > >> next > >> } > >> /^cn:/ { > >> sub(/^cn/, "displayName") > >> } > >> { print } > >> ' | ldapmodify [options] > >> > >> we aren't doing your homework, are we? :-) > > > > Danger, Will Robinson! > > This will only work for dn values that aren't encoded. > > That's a trivial job for perl or python (or whatever). AWK operates > > on character streams and that's a bad fit for LDIF. > > Zwo questions: > > 1: Doesn't /^dn:/ also match "dn::" (likewise /^cn:/ and "cn::"? > 2: If /^cn:/ matches "cn::", wouldn't the sub() replace "cn::" with > "displayName::"? > > So everything would still work IMHO. > > We don't use displayName like that; we only us it if someone with an > academic title insists on having his "Prof." or "Dr." (or any collection of > that in his/her name). > > Regards, > Ulrich > > Thought of the day: Trump vs. America; who will win? > > > > > Cheers, Ralf Mattes > > > >> regards, > >> Jephté > >> > > > >

1 0

RE: RE24 testing call (2.4.45) LMDB RE0.9 testing call (0.9.20)
by Quanah Gibson-Mount 05 Feb '17

05 Feb '17

--On Friday, February 03, 2017 12:38 PM -0800 "Paul B. Henson" <henson(a)acm.org> wrote: >> From: Quanah Gibson-Mount >> Sent: Thursday, February 02, 2017 1:26 PM >> >> quanah@ub16:~/openldap-2.4.45RC/tests$ ./run -l 500 -b mdb test061 >> >> Will run test061 500 times, using back-mdb as the backend. Instead of a >> specific test, one can do "all" to run everything through a loop X times. > > Ok; to clarify then, if a test fails, make will abort? Yes. >> Interesting... it may be worthwhile to look in the testrun directory and >> see why slapd failed to launch. I'll see if I can reproduce it as well. > > Ah, I bet it is this: > > Unrecognized database type (ldap) > 5894e87d > /user/henson/tmp/openldap-OPENLDAP_REL_ENG_2_4-ab6f04c/tests/testrun/slap > d.2 .conf: line 16: <database> failed init (ldap) > > I configured it the way I run it in production; this test must be using a > module I don't enable. Sorry for the false alarm. For testing purposes, > should I just run ./configure with no options? Or a subset of the options > I would normally use on a production system? It means I need to fix the regression test to ignore back-ldap. ;) >> I've not been able to reproduce it. > > Hmm, I must have misunderstood you; I thought in this message: > > http://www.openldap.org/lists/openldap-technical/201612/msg00108.html > > You had indicated you were able to reproduce it. It turned out to not be related. :/ --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

2 1

Re: RE24 testing call (2.4.45) LMDB RE0.9 testing call (0.9.20)
by Quanah Gibson-Mount 05 Feb '17

05 Feb '17

--On Friday, February 03, 2017 2:34 PM +0100 "A. Schulze" <sca(a)andreasschulze.de> wrote: > > > Am 31.01.2017 um 22:21 schrieb A. Schulze: > >> * but last: make test failed >> ( attached make_test_result.txt ) > > the failing test was test059 Ah, whoops. That I was able to reproduce at 25/500 runs. Thanks! --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

RE: RE24 testing call (2.4.45) LMDB RE0.9 testing call (0.9.20)
by Quanah Gibson-Mount 03 Feb '17

03 Feb '17

--On Thursday, February 02, 2017 1:01 PM -0800 "Paul B. Henson" <henson(a)acm.org> wrote: >> From: Quanah Gibson-Mount >> Subject: RE24 testing call (2.4.45) LMDB RE0.9 testing call (0.9.20) >> >> For this testing call, we particularly need folks to test OpenLDAP with >> startTLS/LDAPS when compiled against OpenSSL (both pre 1.1 series and >> with the 1.1 series). > > Compiled successfully with Gentoo linux and openSSL 1.02j/cyrus-sasl > 2.1.26, configured as: > > --prefix=/usr --build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu > --mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share > --sysconfdir=/etc --localstatedir=/var/lib --disable-dependency-tracking > --disable-silent-rules --libdir=/usr/lib64 > --libexecdir=/usr/lib64/openldap --disable-static --enable-ldap > --enable-slapd --enable-bdb --enable-hdb --enable-dnssrv=mod > --enable-ldap=mod --enable-mdb=mod --enable-meta=mod --enable-monitor=mod > --enable-null=mod --enable-passwd=mod > --enable-relay=mod --enable-shell=mod --enable-sock=mod --disable-perl > --disable-sql --disable-crypt --disable-slp --disable-lmpasswd > --enable-syslog --enable-aci --enable-cleartext --enable-modules > --enable-rewrite --enable-rlookups --enable-slapi --enable-syncprov=yes > --enable-overlays=mod --enable-ipv6 --with-cyrus-sasl --enable-spasswd > --disable-wrappers --with-tls=openssl --enable-dynamic --enable-local > --enable-proctitle --enable-shared > > make test completed successfully, is there any particular way to verify > all the tests were okay? Does the make itself fail if any of the tests > do, I did not see a summary at the end. make its was not as happy: "make test" will do a single pass through the entire test suite, for each backend. Unfortunately, that does not always catch issues. For example, the issue Dieter reported against HEAD with test061 only happens to me occassionaly. To help with that, there is the abilty to run the test suite in a loop, for a given backend. Like: quanah@ub16:~/openldap-2.4.45RC/tests$ ./run -l 500 -b mdb test061 Will run test061 500 times, using back-mdb as the backend. Instead of a specific test, one can do "all" to run everything through a loop X times. >>>>>> Starting its4326 ... > running defines.sh > Running slapadd to build slapd database... > Starting slapd on TCP/IP port 9011... > Using ldapsearch to check that slapd is running... > Starting proxy slapd on TCP/IP port 9012... > Using ldapsearch to check that proxy slapd is running... > Waiting 5 seconds for slapd to start... > Waiting 5 seconds for slapd to start... > Waiting 5 seconds for slapd to start... > Waiting 5 seconds for slapd to start... > Waiting 5 seconds for slapd to start... > Waiting 5 seconds for slapd to start... > ldapsearch failed (255)! > ./data/regressions/its4326/its4326: line 93: kill: (28780) - No such > process >>>>>> ./data/regressions/its4326/its4326 failed (exit 255) Interesting... it may be worthwhile to look in the testrun directory and see why slapd failed to launch. I'll see if I can reproduce it as well. > I see the fix for ITS8432 is included in this release (yay); I was > wondering if you've had any luck tracking down the underlying issue > behind ITS8444? So far I still haven't seen any corruption or operational > issues from it, but the rampant noise in the logs and errors being > generated are quite disconcerting :). Plus they will potentially mask any > errors that are actually indicative of a real problem. I've not been able to reproduce it. There's a regression test I wrote for it included in the RE24 source, but it's never really triggered for me. Please feel free to look it over and see if I've missed anything obvious in my reproduction attempts. :) You can execute it directly with ./run -b mdb its8444 --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

2 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical February 2017