openldap-technical December 2018

openldap-technical@openldap.org

13 participants
15 discussions

speed up parallel add/delete
by Derek Zhou 12 Dec '18

12 Dec '18

all I have a patch that can speed up parallel ldap add/delete a lot; in my testing about 2.5X. The idea is described here: the mdb backend does not really support parallelism in rw transactions, so add/delete are serialized by a mutex inside the mdb backend as small transactions. In my patch, all add/delete are put in a queue and executed in a dedicated worker thread. It may seem pointless at first, but by doing them all in the same thread, now I can merge concurrent ops from different clients into larger transactions, and reduce the number of expensive txn_commit calls (fsyncs). my test results are ( your mileage may vary) 10 thread parallel add: 1, unpatched slapd + default config, ~ 2800 op/s (performance stable over time) 2, unpatched slapd + dbnosync ~9000 op/s (lots of fluctuation due to background dirty page flush) 3, unpatched slapd + dpnosync + checkpoint every minute ~8000 op/s, with reduced fluctuation 4, patched slapd ~7300 op/s (fluctuation nearly gone) 2 is not a recommended config, because crashing or power lost will cause massive data lost. 3 is better, but still up to 1 minutes of data can be lost at crash. With this patch, I can achieve >90% of the performance of 3, with data durability as good as 1; or 2.5X performance of 1 with same data durability guaranty. The patch still need some tidy up; is this email list the place to send patches? -- Derek Zhou Shannon Systems http://www.shannon-sys.com

2 1

not be able to initialize ldap
by Eggo why 06 Dec '18

06 Dec '18

Hi All, I have install openldap, openldap-servers and openldap-clients and systemctl start slapd without anyproblem but when try ti do an initial ldapadd I got error. My slapd.conf and oidbase.ldif as below. Please help and thank you very much Garry slapd.conf # # See slapd.conf(5) for details on configuration options. # This file should NOT be world readable. # include /etc/openldap/schema/core.schema include /etc/openldap/schema/cosine.schema include /etc/openldap/schema/inetorgperson.schema include /etc/openldap/schema/nis.schema include /etc/openldap/oidschema/oidbase.schema include /etc/openldap/oidschema/oidnet.schema include /etc/openldap/oidschema/oidrdbms.schema include /etc/openldap/oidschema/alias.schema # Allow LDAPv2 client connections. This is NOT the default. allow bind_v2 # Do not enable referrals until AFTER you have a working directory # service AND an understanding of referrals. #referral ldap://root.openldap.org pidfile /var/run/openldap/slapd.pid argsfile /var/run/openldap/slapd.args #default searchbase defaultsearchbase "dc=cornell,dc=edu" ####################################################################### # ldbm and/or bdb database definitions ####################################################################### database bdb suffix "dc=cornell,dc=edu" rootdn "cn=admin,dc=cornell,dc=edu" rootpw {SSHA}gZp2hnEA/xq/FfxMgPTjDmSPo78PuZAs # The database directory MUST exist prior to running slapd AND # should only be accessible by the slapd and slap tools. # Mode 700 recommended. directory /var/lib/ldap/cornell # Indices to maintain for this database index objectClass eq,pres index ou,cn,mail,surname,givenname eq,pres,sub index uidNumber,gidNumber,loginShell eq,pres index uid,memberUid eq,pres,sub index nisMapName,nisMapEntry eq,pres,sub ### for root oraclecontext # database bdb suffix "" rootdn "cn=admin" rootpw {SSHA}gZp2hnEA/xq/FfxMgPTjDmSPo78PuZAs # The database directory MUST exist prior to running slapd AND # should only be accessible by the slapd and slap tools. # Mode 700 recommended. directory /var/lib/ldap/rootoraclecontext # Indices to maintain for this database index objectClass eq,pres index ou,cn,mail,surname,givenname eq,pres,sub index uidNumber,gidNumber,loginShell eq,pres index uid,memberUid eq,pres,sub index nisMapName,nisMapEntry eq,pres,sub oidbase.ldif dn: dc=cornell,dc=edu objectClass: top objectClass: dcObject objectClass: organization o: cornell.edu dc: cornell dn: cn=OracleContext,dc=cornell,dc=edu objectclass: orclContext cn: OracleContext l$ ldapadd -vc -x -D “cn=admin,dc=cornell,dc=edu” -w $passwd -f oidbase.ldif ldap_initialize( <DEFAULT> ) ldap_bind: Invalid DN syntax (34) additional info: invalid DN

3 3

Re: On removing a duplicated ppolicy overlay
by Derek Zhou 05 Dec '18

05 Dec '18

On Monday, December 03, 2018 02:11:27 PM Quanah Gibson-Mount wrote: > --On Monday, December 03, 2018 1:57 PM -0800 Daniel Howard > <dannyman(a)toldme.com> wrote: > > > > A potentially minor improvement along these lines could be a very nice > > feature enhancement for OpenLDAP. Thank you for your consideration. > > You're talking about two things: a) slapmodify, which is slated for > release with OpenLDAP 2.5, which allows offline modifications of cn=config, > and b) delete support in cn=config, also slated for release with OpenLDAP > 2.5. slapmodify is nice, however modifying schema is still hard. The old schema file format is still much better than the ldif format for viewing and editing. It would be a nice to have a tool that can parse and diff schemas in the old format and generate ldif modifications for slapmodify or ldapmodify to handle. -- Derek Zhou Shannon Systems http://www.shannon-sys.com

1 0

Re: On removing a duplicated ppolicy overlay
by Daniel Howard 04 Dec '18

04 Dec '18

On Wed, Nov 28, 2018 at 11:05 AM Quanah Gibson-Mount <quanah(a)symas.com> wrote: > --On Wednesday, November 28, 2018 10:16 AM -0800 Daniel Howard > <dannyman(a)toldme.com> wrote: > > ># This file MUST be edited with the 'visudo' command as root. > > > > Perhaps this is a consideration that is already on the roadmap? > > You mean like it already does? :) > > head -1 cn\=config.ldif > # AUTO-GENERATED FILE - DO NOT EDIT!! Use ldapmodify. > Quanah, (and Howard) That is certainly a warning. It would be more easily noticed with a bit more whitespace around it, as you find in the sudoers file. This warning is not as helpful as the visudo warning, as it does not give the user a way to edit the file. A wrapper script, or a link to a good document on managing configurations, would be more useful. For example, if I follow the advice in sudoers, and I type "visudo" then I get to edit /etc/sudoers, with a validation before the file can go live. This is fantastic! If I follow the advice here, and type "ldapmodify" then I get an error message. If I research the correct command line, then, in my experience, I manage to import multiple conflicting configs into the system that crash the server. I then research some more, and find that ldapmodify can not delete the conflicting configs. I research some more, and learn that I could just remove them from the filesystem. As I wish to be a good citizen, I share this knowledge, and I am told that this is wrong, and I need to use slapcat to export, delete my config files, then slapadd to import, using a different set of flags than ldapmodify. Perhaps, you can spare a moment of empathy to acknowledge how frustrating this must be for a user. I appreciate your warnings, but given the cumbersome and, in my experience, dangerous nature of managing config files through ldapmodify, I am inclined to very carefully tweak the config files in the config directory. If, however, there was a convenient, safe wrapper, like visudo, or a reference to a reassuring doc that explains the right way to do things, then I would preach the good news. A potentially minor improvement along these lines could be a very nice feature enhancement for OpenLDAP. Thank you for your consideration. Sincerely, -danny -- http://dannyman.toldme.com

3 2

Re: On removing a duplicated ppolicy overlay
by Daniel Howard 04 Dec '18

04 Dec '18

On Mon, Dec 3, 2018 at 2:11 PM Quanah Gibson-Mount <quanah(a)symas.com> wrote: > --On Monday, December 03, 2018 1:57 PM -0800 Daniel Howard > <dannyman(a)toldme.com> wrote: > > > > A potentially minor improvement along these lines could be a very nice > > feature enhancement for OpenLDAP. Thank you for your consideration. > > You're talking about two things: a) slapmodify, which is slated for > release with OpenLDAP 2.5, which allows offline modifications of > cn=config, > and b) delete support in cn=config, also slated for release with OpenLDAP > 2.5. > I love these features already. ;) -- http://dannyman.toldme.com

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical December 2018