openldap-technical

openldap-technical@openldap.org

9 participants
9868 discussions

OpenLDAP 2.4.43 packages for RedHat/CentOS
by Clément OUDOT 02 Dec '15

02 Dec '15

Hi all, you can find RPM packages for OpenLDAP 2.4.43 on LTB project : http://ltb-project.org/wiki/download?&#packages_for_red_hatcentos Debian packages should be available soon. -- Clément OUDOT Consultant en logiciels libres, Expert infrastructure et sécurité Savoir-faire Linux

1 0

updated my openSUSE packages (was: OpenLDAP 2.4.43 available)
by Michael Ströder 01 Dec '15

01 Dec '15

OpenLDAP Project wrote: > OpenLDAP 2.4.43 is now available for download as detailed on our download page: > > http://www.openldap.org/software/download/ Thanks to OpenLDAP team for the new release. I've updated my private openSUSE packages to this release. The OBS source: https://build.opensuse.org/package/show/home:stroeder:branches:network:ldap… Repos for various SUSE versions: http://download.opensuse.org/repositories/home:/stroeder:/branches:/network… Please test. Ciao, Michael.

2 2

OpenLdap Clear-text Password in Debug Mode
by Rich Alford 01 Dec '15

01 Dec '15

Hi All: I'm not sure if this issue results from my ignorance of OpenLdap, or it's not capable of resolving. Regardless, any direction you can provide would be greatly appreciated: I have a basic OpenLdap installation with TLS encryption. Passwords are hashed in the ldap directory. The user password travels from client to server encrypted as it should, then gets unencrypted by slapd, and IF IN DEBUG MODE gets displayed in *clear-text*. Theoretically, the password should be hashed on the client, sent across the network, to be compared against the hashed passwords in the database. What am I missing?? Thank you, Rich

3 3

playing with ldap protocol
by Friedrich Locke 28 Nov '15

28 Nov '15

i am trying to understand ldap protocol! i have read the RFC 4511 (i believe) about it. In order to understand it better, i wrote a program that reads from internet and writes to an output file. I have issued the following ldapsearch command: ldapsearch -x -h localhost -p 2000 -D ou=ufv,dc=br -w 123456 What i got on the output file was: sioux@scallop$ hexdump -C o 00000000 30 1e 02 01 01 60 19 02 01 03 04 0c 6f 75 3d 75 |0....`......ou=u| 00000010 66 76 2c 64 63 3d 62 72 80 06 31 32 33 34 35 36 |fv,dc=br..123456| 00000020 I have the following understanding o f the protocol: 60 19 [02 01 [3] 04 0c [ou=ufv,dc=br] 80 06 [123456]] What about the first "30 1e 02 01 01" ? 1e means the size is bigger than 30 and 2 bytes is specified ? Is it for the message id ? What about the rest of 27 bytes of message what is not accounted ? Shoud it not account for the 27 bytes length ? Thanks in advance. BTW: what is the message id for the message sent ?

4 3

LMDB overflow pages - MDB_RESERVE
by Christian Sell 27 Nov '15

27 Nov '15

I forgot to mention: I am using MDB_RESERVE to avoid extra memcpy. Could it be that this is the cause for the extreme database bloat? > Hello, > > something wrong with the question below? > > I am trying to use LMDB to store large (huge) amounts of binary data which, > for > the reason of limiting memory footprint, are split into chunks. Each chunk ist > stored under a separate key, made up of [collectionId, chunkId], so that I can > later iterate the chunks using a LMDB cursor. Chunk size is configurable. > > During my tests, I encountered a strange scenario where, after inserting some > 2000 chunks consisting of 512KB each, the database size had grown to a value > that was roughly 135 times the calculated size of the data. I ran the stat > utility over the db and saw that there were > 12000 overflow pages vs. approx. > 2000 data pages. When I reduced the chunk size to 4060 bytes, the number of > overflow pages went down to 1000, and the database size went down to the > expected number (I experimented with different sizes, this was the best > result). > I did not find any documentation to explain this behaviour, or how to deal > with > it. Of course it makes me worry about database bloat and the consequences. Can > anyone shed light on this? > > thanks, > Christian > Christian Sell

1 1

Getting around the single-threaded syncrepl model?
by Bannister, Mark 25 Nov '15

25 Nov '15

I was speaking with Howard in person last week at LDAPCon about the problems I might hit if I wanted to run hundreds of replicas off a single master. If I heard right, Howard told me that OpenLDAP uses a single thread for replication, and therefore processes each replica in a serial fashion, one at a time. If any replica is going slowly (high network latency for example), it would have a knock-on effect to the time it takes to get changes replicated across the whole estate. We have an aggressive target to meet - updates out in 5 minutes. Updates are small, but alarm bells are now ringing in my head about OpenLDAP's ability to deliver this. This leads to a couple of questions: 1 - How easy would it be to patch OpenLDAP 2.4 to get the master to use multiple threads for replication? Is that a reasonably straightforward fix, or is this quite a sizeable architectural change? 2 - If my master server has 40 cores, would there be mileage in setting up a number of slapd processes (say 10) on the same host, bound to different sockets, all acting as first-level replicas, and then all the replicas fanning out across the WAN synchronise from these processes rather than from the master process? That would be a way, I suppose, of getting the master server to make better use of the available cores to get updates out quicker? Thanks, Mark. ________________________________ NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies; do not disclose, use or act upon the information; and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers If you cannot access these links, please notify us by reply message and we will send the contents to you. By messaging with Morgan Stanley you consent to the foregoing.

8 16

Sync replication halted
by espeake＠oreillyauto.com 25 Nov '15

25 Nov '15

We are run a 3 node MMR with Openldap 2.4.39 on Ubuntu 14.04 (trusty) Everything runs great with an occasional hiccup like this. I see the following in the logs that appear to be searches on the RID numbers that are used for config replicationL Nov 25 04:31:22 tn-ldap-a-2 slapd[5931]: syncprov_matchops: skipping relayed sid 003 Nov 25 04:31:22 tn-ldap-a-2 slapd[5931]: syncprov_matchops: skipping original sid 001 Nov 25 04:31:22 tn-ldap-a-2 slapd[5931]: syncprov_matchops: skipping relayed sid 003 Nov 25 04:31:22 tn-ldap-a-2 slapd[5931]: syncprov_matchops: skipping original sid 001 Imeadiately after this all changes show csn_too_old errors: Nov 25 04:31:22 tn-ldap-a-2 slapd[5931]: do_syncrep2: rid=004 cookie=rid=004,sid=001,csn=20151125103121.701210Z#000000#001#000000 Nov 25 04:31:22 tn-ldap-a-2 slapd[5931]: do_syncrep2: rid=004 CSN too old, ignoring 20151125103121.701210Z#000000#001#000000 (uid=JRIVAS21,ou=Users,dc=oreillyauto,dc=com) Nov 25 04:31:22 tn-ldap-a-2 slapd[5931]: do_syncrep2: rid=004 cookie=rid=004,sid=001,csn=20151125103121.705110Z#000000#001#000000 Nov 25 04:31:22 tn-ldap-a-2 slapd[5931]: do_syncrep2: rid=004 CSN too old, ignoring 20151125103121.705110Z#000000#001#000000 (uid=JRIVAS21,ou=Users,dc=oreillyauto,dc=com) Nov 25 04:31:22 tn-ldap-a-2 slapd[5931]: do_syncrep2: rid=004 cookie=rid=004,sid=001,csn=20151125103121.713760Z#000000#001#000000 Nov 25 04:31:22 tn-ldap-a-2 slapd[5931]: do_syncrep2: rid=004 CSN too old, ignoring 20151125103121.713760Z#000000#001#000000 (uid=ACERVANT4,ou=Users,dc=oreillyauto,dc=com) Nov 25 04:31:22 tn-ldap-a-2 slapd[5931]: do_syncrep2: rid=004 cookie=rid=004,sid=001,csn=20151125103121.718018Z#000000#001#000000 Nov 25 04:31:22 tn-ldap-a-2 slapd[5931]: do_syncrep2: rid=004 CSN too old, ignoring 20151125103121.718018Z#000000#001#000000 (uid=ACERVANT4,ou=Users,dc=oreillyauto,dc=com) Nov 25 04:31:22 tn-ldap-a-2 slapd[5931]: do_syncrep2: rid=004 cookie=rid=004,sid=001,csn=20151125103121.726847Z#000000#001#000000 Nov 25 04:31:22 tn-ldap-a-2 slapd[5931]: do_syncrep2: rid=004 CSN too old, ignoring 20151125103121.726847Z#000000#001#000000 (uid=KCOOPER10,ou=Users,dc=oreillyauto,dc=com) This happened on one node. The third node had the same replication error for a few miliseconds and then recovered. I am attaching the database config file as well. Thank you, Eric Speake Senior Systems Administrator Information Systems O'Reilly Auto Parts (417) 862-2674 Ext. 1975 (See attached file: olcDatabase={1}mdb.ldif)This communication and any attachments are confidential, protected by Communications Privacy Act 18 USCS � 2510, solely for the use of the intended recipient, and may contain legally privileged material. If you are not the intended recipient, please return or destroy it immediately. Thank you.

1 0

LMDB: successive fillups and drops break the DB after a while
by Dominik Taborsky 25 Nov '15

25 Nov '15

Hello, I've tried stress-testing LMDB in our use case and I've discovered something I consider a bug. In our use case we need to keep inserting to the DB until it's full, flush it and repeat. We've designed the insert function to fail if there are not enough headroom pages for safe delete / DB drop. During the tests the DB works correctly for some amount of repeats, but then it suddenly refuses to insert anything. It seems LMDB doesn't clear or reuse all of its pages. I would like to get anyone's opinion on this. I have attached the source file containing the test. The DB size is set to 20MB, the inserted values have by default 25kB. In the first run the DB accepts 365 inserts, which decreases over time and stabilizes at 280. After 18 repeats the DB does not accept any inserts at all: ok 1 - pass #1 fillup run (365 inserts) ok 2 - pass #2 fillup run (300 inserts) ok 3 - pass #3 fillup run (286 inserts) ok 4 - pass #4 fillup run (283 inserts) ok 5 - pass #5 fillup run (281 inserts) ok 6 - pass #6 fillup run (280 inserts) ok 7 - pass #7 fillup run (280 inserts) ok 8 - pass #8 fillup run (280 inserts) ok 9 - pass #9 fillup run (280 inserts) ok 10 - pass #10 fillup run (280 inserts) ok 11 - pass #11 fillup run (280 inserts) ok 12 - pass #12 fillup run (280 inserts) ok 13 - pass #13 fillup run (280 inserts) ok 14 - pass #14 fillup run (280 inserts) ok 15 - pass #15 fillup run (280 inserts) ok 16 - pass #16 fillup run (280 inserts) ok 17 - pass #17 fillup run (280 inserts) ok 18 - pass #18 fillup run (108 inserts) not ok 19 - pass #19 fillup run (0 inserts) not ok 20 - pass #20 fillup run (0 inserts) The test is run 4 times with different approaches that we thought could have had some impact. These approaches combine opening the DB per insert or per single fillup, and reopening the DB just for the DB drop. In this case these options have no effect and the results are the same, but originally a unit test that this originates from had some differences so I kept them in. Note that I also tried calling mdb_drop with "1" as the last parameter, but it had no effect on the result. I am linking against the latest LMDB from sources. I also tried adding a constant number of pages to the reserve before the insert function fails. This only changes the amount of inserted items but doesn't change the functionality. If you compile the program you can supply it with an argument to change the default size of the item value size. Lower values break the DB sooner (20kB requires only 12 repeated fillups and drops). If anyone has any idea what might be the issue, please let me know. I know deleting the DB from filesystem and creating a new one would work, but that's a hack, not a fix. Thanks. Dominik

2 4

rwm overlay
by BÖSCH Christian 25 Nov '15

25 Nov '15

Hi, I configured rwm overlay like the example in the man page to allow binds with the email address. dn: olcOverlay={0}rwm,olcDatabase={-1}frontend,cn=config objectClass: olcOverlayConfig objectClass: olcRwmConfig olcOverlay: {0}rwm olcRwmRewrite: {0}rwm-rewriteEngine "on" olcRwmRewrite: {1}rwm-rewriteMap "ldap" "attr2dn" "ldap://localhost/dc=abcd,dc =net?dn?sub" olcRwmRewrite: {2}rwm-rewriteContext "bindDN" olcRwmRewrite: {3}rwm-rewriteRule "^mail=[^,]+@[^,]+$" "${attr2dn($0)}" ":@I" olcRwmTFSupport: false olcRwmNormalizeMapped: FALSE But I get the error message the the DN is invalid: ldapsearch -x -D "cb(a)abcd.net" -W -b 'dc=abcd,dc=net' -H ldap://openldap1.abcd.net/ 'uid=cb' Enter LDAP Password: ldap_bind: Invalid DN syntax (34) additional info: invalid DN Is there something missing or wrong? Thanks, Chris

2 2

LMDB overflow pages
by Christian Sell 24 Nov '15

24 Nov '15

Hello, something wrong with the question below? I am trying to use LMDB to store large (huge) amounts of binary data which, for the reason of limiting memory footprint, are split into chunks. Each chunk ist stored under a separate key, made up of [collectionId, chunkId], so that I can later iterate the chunks using a LMDB cursor. Chunk size is configurable. During my tests, I encountered a strange scenario where, after inserting some 2000 chunks consisting of 512KB each, the database size had grown to a value that was roughly 135 times the calculated size of the data. I ran the stat utility over the db and saw that there were > 12000 overflow pages vs. approx. 2000 data pages. When I reduced the chunk size to 4060 bytes, the number of overflow pages went down to 1000, and the database size went down to the expected number (I experimented with different sizes, this was the best result). I did not find any documentation to explain this behaviour, or how to deal with it. Of course it makes me worry about database bloat and the consequences. Can anyone shed light on this? thanks, Christian

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical