openldap-technical

openldap-technical@openldap.org

9 participants
9902 discussions

Sync replication halted
by espeake＠oreillyauto.com 25 Nov '15

25 Nov '15

We are run a 3 node MMR with Openldap 2.4.39 on Ubuntu 14.04 (trusty) Everything runs great with an occasional hiccup like this. I see the following in the logs that appear to be searches on the RID numbers that are used for config replicationL Nov 25 04:31:22 tn-ldap-a-2 slapd[5931]: syncprov_matchops: skipping relayed sid 003 Nov 25 04:31:22 tn-ldap-a-2 slapd[5931]: syncprov_matchops: skipping original sid 001 Nov 25 04:31:22 tn-ldap-a-2 slapd[5931]: syncprov_matchops: skipping relayed sid 003 Nov 25 04:31:22 tn-ldap-a-2 slapd[5931]: syncprov_matchops: skipping original sid 001 Imeadiately after this all changes show csn_too_old errors: Nov 25 04:31:22 tn-ldap-a-2 slapd[5931]: do_syncrep2: rid=004 cookie=rid=004,sid=001,csn=20151125103121.701210Z#000000#001#000000 Nov 25 04:31:22 tn-ldap-a-2 slapd[5931]: do_syncrep2: rid=004 CSN too old, ignoring 20151125103121.701210Z#000000#001#000000 (uid=JRIVAS21,ou=Users,dc=oreillyauto,dc=com) Nov 25 04:31:22 tn-ldap-a-2 slapd[5931]: do_syncrep2: rid=004 cookie=rid=004,sid=001,csn=20151125103121.705110Z#000000#001#000000 Nov 25 04:31:22 tn-ldap-a-2 slapd[5931]: do_syncrep2: rid=004 CSN too old, ignoring 20151125103121.705110Z#000000#001#000000 (uid=JRIVAS21,ou=Users,dc=oreillyauto,dc=com) Nov 25 04:31:22 tn-ldap-a-2 slapd[5931]: do_syncrep2: rid=004 cookie=rid=004,sid=001,csn=20151125103121.713760Z#000000#001#000000 Nov 25 04:31:22 tn-ldap-a-2 slapd[5931]: do_syncrep2: rid=004 CSN too old, ignoring 20151125103121.713760Z#000000#001#000000 (uid=ACERVANT4,ou=Users,dc=oreillyauto,dc=com) Nov 25 04:31:22 tn-ldap-a-2 slapd[5931]: do_syncrep2: rid=004 cookie=rid=004,sid=001,csn=20151125103121.718018Z#000000#001#000000 Nov 25 04:31:22 tn-ldap-a-2 slapd[5931]: do_syncrep2: rid=004 CSN too old, ignoring 20151125103121.718018Z#000000#001#000000 (uid=ACERVANT4,ou=Users,dc=oreillyauto,dc=com) Nov 25 04:31:22 tn-ldap-a-2 slapd[5931]: do_syncrep2: rid=004 cookie=rid=004,sid=001,csn=20151125103121.726847Z#000000#001#000000 Nov 25 04:31:22 tn-ldap-a-2 slapd[5931]: do_syncrep2: rid=004 CSN too old, ignoring 20151125103121.726847Z#000000#001#000000 (uid=KCOOPER10,ou=Users,dc=oreillyauto,dc=com) This happened on one node. The third node had the same replication error for a few miliseconds and then recovered. I am attaching the database config file as well. Thank you, Eric Speake Senior Systems Administrator Information Systems O'Reilly Auto Parts (417) 862-2674 Ext. 1975 (See attached file: olcDatabase={1}mdb.ldif)This communication and any attachments are confidential, protected by Communications Privacy Act 18 USCS � 2510, solely for the use of the intended recipient, and may contain legally privileged material. If you are not the intended recipient, please return or destroy it immediately. Thank you.

1 0

LMDB: successive fillups and drops break the DB after a while
by Dominik Taborsky 25 Nov '15

25 Nov '15

Hello, I've tried stress-testing LMDB in our use case and I've discovered something I consider a bug. In our use case we need to keep inserting to the DB until it's full, flush it and repeat. We've designed the insert function to fail if there are not enough headroom pages for safe delete / DB drop. During the tests the DB works correctly for some amount of repeats, but then it suddenly refuses to insert anything. It seems LMDB doesn't clear or reuse all of its pages. I would like to get anyone's opinion on this. I have attached the source file containing the test. The DB size is set to 20MB, the inserted values have by default 25kB. In the first run the DB accepts 365 inserts, which decreases over time and stabilizes at 280. After 18 repeats the DB does not accept any inserts at all: ok 1 - pass #1 fillup run (365 inserts) ok 2 - pass #2 fillup run (300 inserts) ok 3 - pass #3 fillup run (286 inserts) ok 4 - pass #4 fillup run (283 inserts) ok 5 - pass #5 fillup run (281 inserts) ok 6 - pass #6 fillup run (280 inserts) ok 7 - pass #7 fillup run (280 inserts) ok 8 - pass #8 fillup run (280 inserts) ok 9 - pass #9 fillup run (280 inserts) ok 10 - pass #10 fillup run (280 inserts) ok 11 - pass #11 fillup run (280 inserts) ok 12 - pass #12 fillup run (280 inserts) ok 13 - pass #13 fillup run (280 inserts) ok 14 - pass #14 fillup run (280 inserts) ok 15 - pass #15 fillup run (280 inserts) ok 16 - pass #16 fillup run (280 inserts) ok 17 - pass #17 fillup run (280 inserts) ok 18 - pass #18 fillup run (108 inserts) not ok 19 - pass #19 fillup run (0 inserts) not ok 20 - pass #20 fillup run (0 inserts) The test is run 4 times with different approaches that we thought could have had some impact. These approaches combine opening the DB per insert or per single fillup, and reopening the DB just for the DB drop. In this case these options have no effect and the results are the same, but originally a unit test that this originates from had some differences so I kept them in. Note that I also tried calling mdb_drop with "1" as the last parameter, but it had no effect on the result. I am linking against the latest LMDB from sources. I also tried adding a constant number of pages to the reserve before the insert function fails. This only changes the amount of inserted items but doesn't change the functionality. If you compile the program you can supply it with an argument to change the default size of the item value size. Lower values break the DB sooner (20kB requires only 12 repeated fillups and drops). If anyone has any idea what might be the issue, please let me know. I know deleting the DB from filesystem and creating a new one would work, but that's a hack, not a fix. Thanks. Dominik

2 4

rwm overlay
by BÖSCH Christian 25 Nov '15

25 Nov '15

Hi, I configured rwm overlay like the example in the man page to allow binds with the email address. dn: olcOverlay={0}rwm,olcDatabase={-1}frontend,cn=config objectClass: olcOverlayConfig objectClass: olcRwmConfig olcOverlay: {0}rwm olcRwmRewrite: {0}rwm-rewriteEngine "on" olcRwmRewrite: {1}rwm-rewriteMap "ldap" "attr2dn" "ldap://localhost/dc=abcd,dc =net?dn?sub" olcRwmRewrite: {2}rwm-rewriteContext "bindDN" olcRwmRewrite: {3}rwm-rewriteRule "^mail=[^,]+@[^,]+$" "${attr2dn($0)}" ":@I" olcRwmTFSupport: false olcRwmNormalizeMapped: FALSE But I get the error message the the DN is invalid: ldapsearch -x -D "cb(a)abcd.net" -W -b 'dc=abcd,dc=net' -H ldap://openldap1.abcd.net/ 'uid=cb' Enter LDAP Password: ldap_bind: Invalid DN syntax (34) additional info: invalid DN Is there something missing or wrong? Thanks, Chris

2 2

LMDB overflow pages
by Christian Sell 24 Nov '15

24 Nov '15

Hello, something wrong with the question below? I am trying to use LMDB to store large (huge) amounts of binary data which, for the reason of limiting memory footprint, are split into chunks. Each chunk ist stored under a separate key, made up of [collectionId, chunkId], so that I can later iterate the chunks using a LMDB cursor. Chunk size is configurable. During my tests, I encountered a strange scenario where, after inserting some 2000 chunks consisting of 512KB each, the database size had grown to a value that was roughly 135 times the calculated size of the data. I ran the stat utility over the db and saw that there were > 12000 overflow pages vs. approx. 2000 data pages. When I reduced the chunk size to 4060 bytes, the number of overflow pages went down to 1000, and the database size went down to the expected number (I experimented with different sizes, this was the best result). I did not find any documentation to explain this behaviour, or how to deal with it. Of course it makes me worry about database bloat and the consequences. Can anyone shed light on this? thanks, Christian

1 0

Re: ERR_employeeadd {'info': 'modifications require authentication', 'desc': 'Strong(er) authentication required'}
by Andrei Valoshyn 24 Nov '15

24 Nov '15

Le 19/11/2015 19:43, Andrei Valoshyn a écrit : Hello! I have slapd 2.4.39 and python 2.6 I tried to create an user via python when I tried do that with rootpermission - it's OK. But when I did this with config in slapd.conf"access to * bygroup.exact="cn=LDAP_admins,ou=Roles,ou=Groups,dc=exadel,dc=com" write"I have an error " ERR_employeeadd {'info': 'modifications requireauthentication', 'desc': 'Strong(er) authentication required'} " I tried to use " l.protocol_version = ldap.VERSION{2,3} " via 389 port My function for adding ldif is : l = ldap.initialize(server) l.simple_bind(username, ldapsrvpassword) def employeeadd(): ldif = modlist.addModlist(attrs) l.add_s(dn,ldif) Will be very appreciate for any help Hello Andrei, I suppose that the username you use is a member ofcn=LDAP_admins,ou=Roles,ou=Groups,dc=exadel,dc=com, but whichobjectClass did you use in your group? By default, the OpenLDAP ACLsystem will use groupOfNames, searching user in the member attribute. Ifyou have for example a groupOfUnixNames, you need to set your ACL likethis:access to * bygroup/groupOfUniqueNames/uniqueMember.exact="cn=LDAP_admins,ou=Roles,ou=Groups,dc=exadel,dc=com"write Hello *Clément*, Thank you for reply! I have groupOfNames and posixGroup objectClass. But at that moment this problem was fixed. The matter of fact Python had some problem. It was reinstalled. Thank you! -- CONFIDENTIALITY NOTICE: This email and files attached to it are confidential. If you are not the intended recipient you are hereby notified that using, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error please notify the sender and delete this email.

1 0

Multiple certificates in slapd
by Olivier Nicole 23 Nov '15

23 Nov '15

Hi, I am planing a transition of the certificate I use in OpenLDAP for LDAP over SSL (port 636). My selft signed certificate is quite old and has become obsolete/not recognized on some systems (for example Mac OS 10.11) so it is time to update. But I have many systems that use LDAP and updating all of them cannot be done at once. So I was wondering if it is possible to have one single slapd process running with several posut open over SSL, keeping port 636 with the old certificate and opening port 637 with the new certificate. That way, i can transition the clients at my onw pace, not needing to do all at same time. I know that I could set-up a slave server, but that would be not as transparent s0 I'd prefer my idea of havingslapd -h ldaps://192.168.10.1:636/ ldaps:/192.168.10.1:637/ each using a different certificate. Thanks in advance, Olivier --

2 1

Re: Trying to set up multimaster syncrepl, error attribute 'olcTLSCertificateFile' not allowed , why?
by Betsy Schwartz 23 Nov '15

23 Nov '15

On Sat, Nov 21, 2015 at 2:00 PM, Quanah Gibson-Mount <quanah(a)zimbra.com> wrote: > > I would suggest using slapcat to export the config database and clean up > the invalid attribute values that were incorrectly added to the bdb > database. > > Thank you very much! I have made progress but am now in an entertaining new place (having *overwritten* my database with my overlay files). But I'm going to go RTFM and see if I can't dig myself out of that one. > a) Upgrading to a current openldap release > b) Switching to back-mdb, assuming a 64-bit OS. > Thank you! Sigh, 2.4.40 seems to be the latest in the Oracle repo (we are using Oracle's version of RHEL6). It looks like there have been some good bug fixes and I'll make the case for going outside the repo. Sometime in the next couple months these two new servers will become the primary production servers and then we'll be able to do what we want with them. The two old servers, current production, are running 2.4.39 and 2.4.23 (and not syncing with each other!) . I've been hoping that I could sync data between at least these two new servers and the 2.4.39 server, is that possible or a foolish hope? Again, once these new ones become primary I'll be able to keep them identical to each other, but I don't really 'own' the old ones and don't want to break them. I'm still working through the manual and the configuration settings. One more question - still trying to understand what was done on these old servers - on these servers the config database is 0, the monitor database is 1 and the bdb database is 2. I can't slapcat the monitor database, is that normal? I get slapcat -n 1 slapcat: database doesn't support necessary operations. I had a little trouble with this particular section of the manual which reads simply: 20.1. Monitor configuration via cn=config(5) This section has yet to be written. thanks again Betsy

2 1

questions about memberof-refint option
by M. P. 22 Nov '15

22 Nov '15

Hi, I'm playing with memberof overlay. For my tests, I use the default database (numbered 1) from slapd installation with suffix dc=nodomain. The tests are running on debian jessie 8.2 and slapd version 2.4.40+dfsg-1 Activating the module in cn=module entry and activating the overlay for the database, I have something that works like (I think) it should. I mean adding a user (attribute member) in a group creates an attribute memberOf for the user and deleting a user from the group deletes the user's memberOf attribute. That's great. There is nothing special configured. # Entrée 1: olcOverlay={0}memberof,olcDatabase={1}mdb,cn=config dn: olcOverlay={0}memberof,olcDatabase={1}mdb,cn=config objectclass: olcConfig objectclass: olcOverlayConfig objectclass: olcMemberOf objectclass: top olcoverlay: {0}memberof Reading the man page, I saw memberof-refint option. From what I understand, when set to true, you can alter the user's "is member of" attribute and that would be reflected in the group's "member" attribute. Right ? But, the member attribute is an operational attribute and can't be modified. So I started to search for an alternative and found the eduMember schema from here https://spaces.internet2.edu/display/macedir/OpenLDAP+eduMember. Once added to the installation I could use it for objects. It adds isMemberOf and hasMember attributes that can be setable for users and groups. But can't make it work with memberof overlay. When trying to add isMemberOf as memberof-memberof-ad it was rejected with member attribute=”isMemberOf” must either have DN (1.3.6.1.4.1.1466.115.121.1.12) or nameUID (1.3.6.1.4.1.1466.115.121.1.34) syntax And the same error was reported with hasMember as memberof-member-ad. To make it work together I modified the attribute's definitions and reimported them to openldap. So I can now set isMemberOf as memberof-memberof-ad and the same for hasMember as memberof-member-ad. The configuration then was like this # Entrée 1: olcOverlay={0}memberof,olcDatabase={1}mdb,cn=config dn: olcOverlay={0}memberof,olcDatabase={1}mdb,cn=config objectclass: olcConfig objectclass: olcOverlayConfig objectclass: olcMemberOf objectclass: top olcmemberofmemberofad: isMemberOf olcoverlay: {0}memberof Now that works like (I think) it should. I mean adding a user (attribute member) in a group creates an attribute isMemberOf for the user and deleting a user from the group deletes the user's isMemberOf attribute. That's great. isMemberOf is a modifiable attribute so it's time to test the memberof-refint and set it to TRUE # Entrée 1: olcOverlay={0}memberof,olcDatabase={1}mdb,cn=config dn: olcOverlay={0}memberof,olcDatabase={1}mdb,cn=config objectclass: olcConfig objectclass: olcOverlayConfig objectclass: olcMemberOf objectclass: top olcmemberofmemberofad: isMemberOf olcmemberofrefint: TRUE olcoverlay: {0}memberof And this is where things do not work. I mean what was working before is still working. If I add a member in a group an atttribute isMemberOf is created for the user. But if I add a second attribute isMemberOf with a second group, no new member is created on the second group. And if I delete the attribute isMemberOf from the user's entry, it is still visible on the group. Does anybody have any idea why the modifications made on the user (with the deletion of isMemberOf) are not applied to the group ? Is there something I'm doing wrong ? Thanks. -- ------------ M. P.

4 11

LMDB data growth - overflow pages
by Christian Sell 22 Nov '15

22 Nov '15

Hello, I am trying to use LMDB to store large (huge) amounts of binary data which, for the reason of limiting memory footprint, are split into chunks. Each chunk ist stored under a separate key, made up of [collectionId, chunkId], so that I can later iterate the chunks using a LMDB cursor. Chunk size is configurable. During my tests, I encountered a strange scenario where, after inserting some 2000 chunks consisting of 512KB each, the database size had grown to a value that was roughly 135 times the calculated size of the data. I ran stat over the db and saw that there were > 12000 overflow pages vs. approx. 2000 data pages. When I reduced the chunk size to 4060 bytes, the number of overflow pages went down to 1000, and the database size went down to the expected number (I experimented with different sizes, this was the best result). I did not find any documentation that would explain this behavior, or how to deal with it. Of course it makes me worry about database bloat and the consequences. Can anyone shed light on this? thanks, Christian

1 0

Human-friendly olcAccess management
by Bogdan Rudas 22 Nov '15

22 Nov '15

Hello all, I would like to start use of olcAccess rules, are there human-friendly editor for that ACLs? I can't even use line breaks in ldif file to make my restrictions a bit more readable! I strongly dislike very long string values, one day this will cause mistake and access violation. I've tried with Apache DS, ldif import and few puppet modules, everything require huge line ACL. Any help will be welcome. -- Bogdan Rudas Head of Minsk IT Support Department Exadel Inc. http://www.exadel.com/ E-mail: brudas(a)exadel.com <mandrushkevich(a)exadel.com> Skype ID: bogdan.rudas -- CONFIDENTIALITY NOTICE: This email and files attached to it are confidential. If you are not the intended recipient you are hereby notified that using, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error please notify the sender and delete this email.

4 3

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical