openldap-technical

openldap-technical@openldap.org

7 participants
9901 discussions

Re: Different contextCSN on multi-master replication servers
by NICOLAS KOVAC NEUMANN 24 Apr '15

24 Apr '15

Thank you both Howard and Leonid. Yes, you're right, it happened the other way around; the modification was made on the second server and propagated back to the first one. However, I don't know why the change was in turn returned to the second server - SIDs are ok as far as I know. However, as Leonid mentioned, both servers weren't synchronized correctly anyway. Turns out that yesterday we upgraded to 2.4.39-6 (as I stated in my first mail). Previously, we were using 2.4.39-3 and it seemed to work fine. We also noted that 2.4.39-6 produced some additional issues (like client and syncrepl sockets dying without any apparent reason), so today we downgraded back to 2.4.39-3 and everything seems to work just fine again. We had a look at the changelog from 2.4.39-3 to 2.4.39-6 and no change seems to be explicitly syncrepl related, but rather related to LDAPS (strange, as we use the LDAP protocol for syncrepl instead of LDAPS). Anyway, we'll keep version 2.4.39-3 as far as it works well. Thanks. Regards. 2015-04-21 23:53 GMT+01:00 Леонид Юрьев <leo(a)yuriev.ru>: > Hi Nicolás, > > 1) If contextCSN(s) are differs on servers, then are still not > syncronized (or has a glitches). > http://www.openldap.org/lists/openldap-technical/201108/threads.html#00001 > > 2) Replication takes a some time. Therefore contextCSN(s) may be equals > only when some time was no any changes. > > 3) Make sure that the time is synchronized on servers (e.g. by using > ntpdate). > > 4) Unfortunatelly, all current releases (include 2.4.39 and 2.4.40) have > enough bugs in replication code. > For example, by ITS#8081 ( > http://www.openldap.org/its/index.cgi/Software%20Bugs?id=8081) you could > get segfault, but also lost (like undo) some changes by a replication. > > 5) We made a fork of OpenLDAP project for our usecase (highload > TELCO-aware multi-master), it called ReOpenLDAP. > If you decide to build slapd from sources, I recommend use our ReOpenLDAP > ;) > > New features yet not documented in english man-pages, by you can translate > by Google: > https://github.com/ReOpen/ReOpenLDAP/releases/tag/ReOpenLDAP-2.4.41-rc > > https://github.com/ReOpen/ReOpenLDAP/commit/4fc4bc18dd4bd80909aa80700c5c19b… > > https://github.com/ReOpen/ReOpenLDAP/commit/95808b156ee36a886523b7096a75d50… > > https://github.com/ReOpen/ReOpenLDAP/commit/1c94bc17ec285388e8a8299399ed537… > > Leonid. > > 2015-04-21 16:01 GMT+03:00 Nicolás Kovac Neumann <nkovacne(a)ull.edu.es>: > >> Hi, >> >> We're currently using N-way multimaster replication on two servers for >> our LDAP directory, both for the config and the hdb databases. It's working >> fine mostly, but we've run into a possible issue with the syncrepl engine >> which we would like to cast light on. We're using CentOS 7 with >> openldap-servers version 2.4.39-6. >> >> We made an update on one of the entries (server1, in this case), so >> server2 replicated that change (as shown below in the log): >> >> ==> server1/ldap.log <== >> Apr 21 13:38:55 server1 slapd[1835]: do_syncrep2: rid=002 >> cookie=rid=002,sid=002,csn=20150421123855.643239Z#000000#002#000000 >> Apr 21 13:38:55 server1 slapd[1835]: syncrepl_message_to_entry: >> rid=002 DN: uid=user1,cn=subtree,dc=example,dc=org, UUID: >> 18a2436c-73ce-1030-95dd-b52dc05ced13 >> Apr 21 13:38:55 server1 slapd[1835]: syncrepl_entry: rid=002 >> LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_MODIFY) >> Apr 21 13:38:55 server1 slapd[1835]: syncrepl_entry: rid=002 >> be_search (0) >> Apr 21 13:38:55 server1 slapd[1835]: syncrepl_entry: rid=002 >> uid=user1,cn=subtree,dc=example,dc=org >> Apr 21 13:38:55 server1 slapd[1835]: slap_queue_csn: queing >> 0x7ff8f42789f0 20150421123855.643239Z#000000#002#000000 >> Apr 21 13:38:55 server1 slapd[1835]: slap_graduate_commit_csn: >> removing 0x7ff8f435e770 20150421123855.643239Z#000000#002#000000 >> Apr 21 13:38:55 server1 slapd[1835]: syncrepl_entry: rid=002 >> be_modify uid=user1,cn=subtree,dc=example,dc=org (0) >> Apr 21 13:38:55 server1 slapd[1835]: syncprov_sendresp: >> cookie=rid=001,sid=001,csn=20150421123855.643239Z#000000#002#000000 >> Apr 21 13:38:55 server1 slapd[1835]: slap_queue_csn: queing >> 0x7ff8f42789f0 20150421123855.643239Z#000000#002#000000 >> Apr 21 13:38:55 server1 slapd[1835]: slap_graduate_commit_csn: >> removing 0x7ff8f41b7b90 20150421123855.643239Z#000000#002#000000 >> >> ==> server2/ldap.log <== >> Apr 21 13:38:55 server2 slapd[1948]: slap_queue_csn: queing >> 0x7f897affb220 20150421123855.643239Z#000000#002#000000 >> Apr 21 13:38:55 server2 slapd[1948]: syncprov_sendresp: to=001, >> cookie=rid=002,sid=002,csn=20150421123855.643239Z#000000#002#000000 >> Apr 21 13:38:55 server2 slapd[1948]: slap_graduate_commit_csn: >> removing 0x7f89307f42a0 20150421123855.643239Z#000000#002#000000 >> >> Nothing strange up to now, however, if we query the contextCSN, it >> differs on both servers. >> >> For server1, we have: >> >> contextCSN: 20150421123523.281736Z#000000#001#000000 >> contextCSN: 20150421123417.889477Z#000000#002#000000 >> >> For server2, the value for server ID 001 differs: >> >> contextCSN: 20150421115324.003502Z#000000#001#000000 >> contextCSN: 20150421123417.889477Z#000000#002#000000 >> >> However, the entry seems to replicate the entryCSN correctly on both >> servers: >> >> entryCSN: 20150421123417.889477Z#000000#002#000000 >> >> Is this the expected behavior? Shouldn't both contextCSN values match on >> both servers? >> >> Thanks! >> >> Regards, >> >> Nicolás >> >> >

2 1

LMDB usage with long or variable length keys
by Alex V. 24 Apr '15

24 Apr '15

I am new to key-value stores. I would like to use it like big persistent associative array and I want to be able to use long length keys. For example file path or URL. What technique should I use to implement such dictionary on top of LMDB? Probably some kind of hashing and collision resolution? Or just recompile with MDB_MAXKEYSIZE=2047 or something like this? Or LMDB is a wrong tool for this job?

5 7

ldap configuration question
by Ross, Daniel B. 24 Apr '15

24 Apr '15

I have looked a couple options, and read the openldap documentation, but I really dont know how to accomplish what I need to do. Here is what I am trying to do. I have a greater organization that is stuck on using Microsoft products namely Microsoft LDS. To make matters worse they present the data to my linux servers in a completely non-standard way. Its driving my solaris and linux box nuts and they simply dont want to work with it. What i need to do is continue to use the campus usernames and passwords but present the Data in a format that my linux/unix hosts can use. Is this possible? i.e. userid would still be samwise but instead of a bizzarre OU=monkeypeople,dc=example,dc=com I want it to present as people,dc=example,dc=com. The mismatches are far more and I need to add things like nfs mounts and it would be nice to allow users on only certain hosts, none of which their LDS can do, or even if it can they are not willing to do. I looked at referral and aliasing but it does not seem to be doing what I am trying to do. Passthrough authentication looks close but I cant find sufficient documentation to actually configure a system to use it. So what I want to do is use campus for the username and password authentication, but the local ldap for everything else, groups, dns, etc, etc. Thanks Daniel

1 0

Re: Antw: Re: Different contextCSN on multi-master replication servers
by Nicolás Kovac Neumann 24 Apr '15

24 Apr '15

Hi Ulrich, El 24/04/15 a las 07:07, Ulrich Windl escribió: > [...] > After haveing had such problems, I recommand exporting the databases of both > servers and compare the contents: If the mismatch, I'd import one database from > the other server's dump... > > Regards, > Ulrich That's the approach we've been using years ago but as this is a patchy workaround we rather preferred get syncrepl working correctly. After downgrading the package to 2.4.39-3 again it seems to work again, so I think we'll apply the old and good advise "If it works, don't fix it". At least not until a certain time. Thanks. Regards, Nicolás.

1 0

moznss, gnutls, openssl
by Chuck Theobald 24 Apr '15

24 Apr '15

What is the current wisdom regarding which tls library to use? I've got a version 2.4.39 installation on RHEL 6.6 for which I cannot get tls to work. I end up with the "TLS: can't connect: TLS error -5938:Encountered end of file." error. Likely a misconfiguration of moznss, though I followed one set of directions using certutil, but lack the proper setting for my ldap TLSCACertificateFile. My Debian-based ldap servers run with either openssl or gnutls. Thanks, -- Chuck Theobald System Administrator The Robert and Beverly Lewis Center for Neuroimaging University of Oregon P: 541-346-0343 F: 541-346-0345

3 2

solaris ssh key passwordless login openldap,
by Csaba Dobo 24 Apr '15

24 Apr '15

Hi, I am curerntly putting together a system to create a central authentication and have a problem with solaris machines to log in to openldap with sshkey that are stored in openldap. I have found a thread: http://www.openldap.org/lists/openldap-software/200710/msg00041.html that talks about the issue: it seems as if this control is needed in the rootdse request control has an OID of 1.3.6.1.4.1.42.2.27.9.5.8 Problem is I have no idea how to put that in? Do I have to create a special ppolicy? If yes, could you please help? Got any idea? many thanks, Csaba http://man7.org/linux/man-pages/man5/slapo-ppolicy.5.html

2 1

Re: Antw: LMDB usage with long or variable length keys
by Alex V. 23 Apr '15

23 Apr '15

On 4/24/2015 9:02 AM, Ulrich Windl wrote: > > I think this depends very much on your access patterns (which you didn't explain). > > Regards, > Ulrich What required is `get(key)` and `set(key, val)`. And, optional but not unwanted, range lookup.

1 0

mis-identified self-signed cert
by Chuck Theobald 23 Apr '15

23 Apr '15

Built openldap 2.4.40 from source, trying to replicate the directory structure used by RHEL, but using openssl instead of nss. Various dir-placement options to configure got me to a standard RHEL (and typical Linux) structure. I am now trying to start using the /etc/init.d/slapd script from a mostly-working (sans TLS) RHEL installation, but startup fails. Silently. This may be because slapd cannot read the private server key file, but should this not be read before changing the effective running user to ldap? I would like my slapd to be running as something other than user 0. Anyway, I managed to prop up a server from the command line: slapd -F ./slapd.d but now cannot talk to it with TLS enabled: TLS trace: SSL_connect:before/connect initialization TLS trace: SSL_connect:SSLv2/v3 write client hello A TLS trace: SSL_connect:SSLv3 read server hello A TLS certificate verification: depth: 3, err: 19, subject: /C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root, issuer: /C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root TLS certificate verification: Error, self signed certificate in certificate chain TLS trace: SSL3 alert write:fatal:unknown CA TLS trace: SSL_connect:error in SSLv3 read server certificate B TLS trace: SSL_connect:error in SSLv3 read server certificate B TLS: can't connect: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed (self signed certificate in certificate chain). ldap_err2string ldap_start_tls: Connect error (-11) additional info: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed (self signed certificate in certificate chain) Enter LDAP Password: The server cert and ca certs I am using are not self-signed, at least by me, and were obtained through Internet2 via our University's central IS department. The same certs are working fine with the web server on my machine. I think the key clue is the "unknown CA" in the messages above. But, how to solve? -- Chuck Theobald System Administrator The Robert and Beverly Lewis Center for Neuroimaging University of Oregon P: 541-346-0343 F: 541-346-0345

2 1

Re: Antw: Re: Slapd running very slow
by Geoff Swan 22 Apr '15

22 Apr '15

On 2015-04-22 11:45 PM, Ulrich Windl wrote: >>>> Geoff Swan <gswan3(a)bigpond.net.au> schrieb am 22.04.2015 um 14:06 in Nachricht > <55378EE0.40103(a)bigpond.net.au>: > >> On 2015-04-22 4:49 PM, Ulrich Windl wrote: > [...] >>>> Are there any clues about key factors affecting this? Linux, in this >>>> case, has vm.swappiness set to 10, vm.dirty_ratio at 12 and >>>> vm.dirty_background at 3. However I've noticed that when dirty pages are >>>> flushed to disc, the system stalls. And that operation appears to take a >>>> relatively long time. Disc write speed should be close to 130MB/s (file >>>> copy, dd test etc) however it appears to be much slower than this with >>>> the page flush. >>> Did you try NOT tuning those? A swapped in-memory database is not the thing >> you usually want. >> Swappiness for an out-of-the-box kernel was 60, which sounds way too >> high. So I reduced it to 10. > Did we see your "free" stats? On a server running SLES11 SP3 (kernel 3.0.101) with similar memory (no slapd, but a huge Oracle and a NFS Server) I have: > # free > total used free shared buffers cached > Mem: 132156276 131202644 953632 40317620 5219804 95059508 > -/+ buffers/cache: 30923332 101232944 > Swap: 20956756 165852 20790904 > > I believe what the system pages out is really "dead meat". According to sar there is little actual paging: > > 00:00:01 pgpgin/s pgpgout/s fault/s majflt/s pgfree/s pgscank/s pgscand/s pgsteal/s %vmeff > 11:00:01 12671.34 1979.03 4469.13 0.00 3412.08 0.00 0.00 0.00 0.00 > 11:10:01 11598.11 1693.74 4949.54 0.01 3466.14 0.00 0.00 0.00 0.00 > 11:20:01 12141.05 4759.01 4219.09 0.02 3753.33 0.00 0.00 0.00 0.00 > 11:30:01 14619.11 1562.84 4185.04 0.00 3134.25 0.00 0.00 0.00 0.00 > > (Most "fage faults" are due to processes staring and ending with demand paging, I guess) > > Regards, > Ulrich > Free stats look fine to me. No swap is being used, or has been used yet on this system. total used free shared buffers cached Mem: 132005400 21928192 110077208 10612 363064 19230072 -/+ buffers/cache: 2335056 129670344 Swap: 8388604 0 8388604 The effect I am seeing is that despite a reasonably fast disc system, the kernel writing of dirty pages is painfully slow.

1 0

Re: Antw: Re: Slapd running very slow
by Geoff Swan 22 Apr '15

22 Apr '15

On 2015-04-22 4:49 PM, Ulrich Windl wrote: >>>> Geoff Swan <gswan3(a)bigpond.net.au> schrieb am 21.04.2015 um 23:19 in Nachricht > <5536BEC9.3040902(a)bigpond.net.au>: > >> On 2015-04-22 6:04 AM, Howard Chu wrote: >>> Brian Reichert wrote: >>>> On Tue, Apr 21, 2015 at 08:23:31AM -0700, Quanah Gibson-Mount wrote: >>>>> --On Tuesday, April 21, 2015 11:54 AM -0400 Brian Reichert >>>>> <reichert(a)numachi.com> wrote: >>>>>> What does your config file look like? >>>>>> >>>>>> In particular, what does this setting look like for you: >>>>>> >>>>>> # Threads - four per CPU >>>>>> threads 8 >>>>> According to his summary, he's using 48 threads. >>>> Thanks for pointing that out; I should finish my coffee before >>>> posting. :) >>>> >>>>> 4 per CPU/core was a good >>>>> rule of thumb with bdb/hdb. So far in playing with back-mdb, it's >>>>> seemed >>>>> closer to 2 per CPU/core for me in benchmarking. >> Interesting. What is the relationship between the number of threads and >> the number of concurrent bind operations? >> If I have, say, 500 clients wanting access to perform simple >> authentication/bind and perform some read/write operation, how is this >> usually handled within slapd? >> >>>> Useful to note. Has this detail ended up in any docs yet? >>> No, nor should it. Performance depends on system environment and >>> workload - the right value is one that each site must discover for >>> themselves in their own deployment. >>> >> Are there any clues about key factors affecting this? Linux, in this >> case, has vm.swappiness set to 10, vm.dirty_ratio at 12 and >> vm.dirty_background at 3. However I've noticed that when dirty pages are >> flushed to disc, the system stalls. And that operation appears to take a >> relatively long time. Disc write speed should be close to 130MB/s (file >> copy, dd test etc) however it appears to be much slower than this with >> the page flush. > Did you try NOT tuning those? A swapped in-memory database is not the thing you usually want. > Swappiness for an out-of-the-box kernel was 60, which sounds way too high. So I reduced it to 10. >

2 2

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical