openldap-technical

openldap-technical@openldap.org

14 participants
9796 discussions

RE: Make slapadd faster?
by Marc Roos 16 Aug '19

16 Aug '19

Ok ok I will look at this mdb again. -----Original Message----- From: Quanah Gibson-Mount [mailto:quanah@symas.com] Subject: Re: Make slapadd faster? --On Friday, August 16, 2019 10:14 AM +0200 Marc Roos <M.Roos(a)f1-outsourcing.eu> wrote: > > I know you can disable some checks to make slapadd faster. But I think > in my test vm with limited disk iops, it looks like this disk io is > the problem. > I am not sure how slapadd adds entries, I guess one at a time? You > could get a significant improvement by reading more entries and > writing more entries at once? And if such a batch transaction fails > you can always go back to submitting the transactions of the batch one > by one, to identify which one is failing. Stop using back-hdb, it's signficantly slower than back-mdb. If you're going to insist on using back-hdb, then you *must* have a well tuned DB_CONFIG before you import. For any backend, ensure you have tool-threads set appropriately (For back-mdb, values > 2 are ignored). Use the -q flag if you know the LDIF is good. slapadd has been heavily profiled and tested. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Make slapadd faster?
by Marc Roos 16 Aug '19

16 Aug '19

I know you can disable some checks to make slapadd faster. But I think in my test vm with limited disk iops, it looks like this disk io is the problem. I am not sure how slapadd adds entries, I guess one at a time? You could get a significant improvement by reading more entries and writing more entries at once? And if such a batch transaction fails you can always go back to submitting the transactions of the batch one by one, to identify which one is failing.

2 1

recommended ppolicy pwdCheckModule
by Dave Macias 16 Aug '19

16 Aug '19

Hello, Currently we use ppm https://github.com/ltb-project/ppm which works quite well as a quality checker. Wanted to see what others use. Was looking around and found some interesting ones: pam_pwquality ( i believe these are only for users in passwd file ) ppchecker http://www.meddeb.net/pqchecker/?Idx=0 Any input is appreciated. Thanks! Dave

2 1

Openldap in container advice, how have you done it?
by Marc Roos 16 Aug '19

16 Aug '19

I was thinking of putting read-only slapd('s) in a container environment so other tasks can query their data. Up until now I have had replication only between vm's. To be more flexible I thought of using stateless containers. Things that could be caveats - replication id's say I spawn another instance, I need to have a new replication id to get updates from the master. But what if the tasks is killed, should I keep this replication id? Or better just always use a random unique replication id whenever a slapd container is launched? Maybe use launch date/time (date +'%g%H%M%S%2N') as repid? Is this giving issues with the master? What if I test with launching instances and the master will think there are a hundred slaves that are not connecting anymore? - updating of a newly spawned slapd instance When the new task is launched, it is not up to date with its database, can I prevent connections to the slapd until it is fully synced? Say I have user id's in slapd, it could be that when launching a new instance, this user is not available yet. When clients are requesting this data, they do not get it, and this user could be 'offline' until that specific instance of slapd is fully updated. - to prevent lots of records syncing Can I just copy the data of /var/lib/ldap of any running instance to the container default image? Or does it have some unique id's that will prevent this data to be run multiple times? Is there some advice on how to do this? - doing some /var/lib/ldap cleanup I am cleaning with db_checkpoint -1 -h /var/lib/ldap, and db_archive -d. Is there an option slapd can initiate this? - keep uniform configuration environment, or better a few different slapd instances? In my current environment vm slave slapd's only sync data from the master that the masters acls allow access to. That results in that on some vm's the ldap database is quite small and on other it is larger. I think for the container slapd instances to have all data, and just limit client access via the acls. But this means a lot more indexes on the slapd What else am I missing?

4 5

RE: Antw: RE: Openldap in container advice, how have you done it?
by Marc Roos 16 Aug '19

16 Aug '19

The ip address is known when I start the container that would mean I need to sed some ready ldif and import it into the slapd at runtime. That would also require the the availability of some secret to be able to import it. Although I have prepared the container for ldif fetching. Nicer would be if I could specify something like an environment variable in olcSyncrepl -----Original Message----- From: Ulrich Windl [mailto:Ulrich.Windl@rz.uni-regensburg.de] Sent: maandag 12 augustus 2019 8:56 To: Marc Roos Subject: Antw: RE: Openldap in container advice, how have you done it? >>> "Marc Roos" <M.Roos(a)f1-outsourcing.eu> schrieb am 10.08.2019 um 14:07 in Nachricht <"H00000710014b895.1565438831.sx.f1-outsourcing.eu*"@MHS>: > Ok so long rep id is not going to work modifying entry > "olcDatabase={2}hdb,cn=config" > ldap_modify: Other (e.g., implementation specific) error (80) > additional info: Error: parse_syncrepl_line: syncrepl id > 1911533132 is out of range [0..999] Why not derive the ID from some container ID or from the container's IP address? > > > > > ‑‑‑‑‑Original Message‑‑‑‑‑ > From: Marc Roos > Sent: zaterdag 10 augustus 2019 1:24 > To: openldap‑technical(a)openldap.org > Subject: Openldap in container advice, how have you done it? > > > > I was thinking of putting read‑only slapd('s) in a container > environment so other tasks can query their data. Up until now I have > had replication only between vm's. > > To be more flexible I thought of using stateless containers. Things > that could be caveats > > ‑ replication id's > say I spawn another instance, I need to have a new replication id to > get updates from the master. But what if the tasks is killed, should I > keep this replication id? Or better just always use a random unique > replication id whenever a slapd container is launched? Maybe use > launch date/time (date +'%g%H%M%S%2N') as repid? Is this giving issues > with the master? What if I test with launching instances and the > master will think there are a hundred slaves that are not connecting anymore? > > ‑ updating of a newly spawned slapd instance When the new task is > launched, it is not up to date with its database, can I prevent > connections to the slapd until it is fully synced? > Say I have user id's in slapd, it could be that when launching a new > instance, this user is not available yet. When clients are requesting > this data, they do not get it, and this user could be 'offline' until > that specific instance of slapd is fully updated. > > ‑ to prevent lots of records syncing > Can I just copy the data of /var/lib/ldap of any running instance to > the container default image? Or does it have some unique id's that > will prevent this data to be run multiple times? Is there some advice > on how to do this? > > ‑ doing some /var/lib/ldap cleanup > I am cleaning with db_checkpoint ‑1 ‑h /var/lib/ldap, and db_archive ‑d. > Is there an option slapd can initiate this? > > ‑ keep uniform configuration environment, or better a few different > slapd instances? > In my current environment vm slave slapd's only sync data from the > master that the masters acls allow access to. That results in that on > some vm's the ldap database is quite small and on other it is larger. > I think for the container slapd instances to have all data, and just > limit client access via the acls. But this means a lot more indexes on > the slapd > > What else am I missing?

1 0

PID file /var/run/openldap/slapd.pid not readable (yet?) after start
by Paul Pathiakis 14 Aug '19

14 Aug '19

Hi.... After the previous issue... I went to startup slapd and got the error above. I don't even know how to address that. Slapd won't even start. I'm on CentOS 7. :( systemctl status slapd.service ● slapd.service - OpenLDAP Server Daemon Loaded: loaded (/usr/lib/systemd/system/slapd.service; enabled; vendor preset: disabled) Active: failed (Result: timeout) since Wed 2019-08-14 11:34:15 EDT; 2min 7s ago Docs: man:slapd man:slapd-config man:slapd-hdb man:slapd-mdb file:///usr/share/doc/openldap-servers/guide.html Process: 15117 ExecStart=/usr/sbin/slapd -u ldap -h ${SLAPD_URLS} $SLAPD_OPTIONS (code=exited, status=0/SUCCESS) Process: 15102 ExecStartPre=/usr/libexec/openldap/check-config.sh (code=exited, status=0/SUCCESS) Main PID: 14277 (code=exited, status=0/SUCCESS) Aug 14 11:32:45 NewLDAP.hq.boston-engineering.com systemd[1]: Starting OpenLDAP Server Daemon... Aug 14 11:32:45 NewLDAP.hq.boston-engineering.com runuser[15105]: pam_unix(runuser:session): session opened for user ldap by (uid=0) Aug 14 11:32:45 NewLDAP.hq.boston-engineering.com runuser[15105]: pam_unix(runuser:session): session closed for user ldap Aug 14 11:32:45 NewLDAP.hq.boston-engineering.com slapd[15117]: @(#) $OpenLDAP: slapd 2.4.44 (Jan 29 2019 17:42:45) $ mockbuild@x86-01.bsys.centos.org:/builddir/build/BUILD/openldap-2.4.44/openlda...s/slapd Aug 14 11:32:45 NewLDAP.hq.boston-engineering.com systemd[1]: PID file /var/run/openldap/slapd.pid not readable (yet?) after start. Aug 14 11:34:15 NewLDAP.hq.boston-engineering.com systemd[1]: slapd.service start operation timed out. Terminating. Aug 14 11:34:15 NewLDAP.hq.boston-engineering.com systemd[1]: Failed to start OpenLDAP Server Daemon. Aug 14 11:34:15 NewLDAP.hq.boston-engineering.com systemd[1]: Unit slapd.service entered failed state. Aug 14 11:34:15 NewLDAP.hq.boston-engineering.com systemd[1]: slapd.service failed. Hint: Some lines were ellipsized, use -l to show in full. [root@NewLDAP openldap]# systemctl start slapd.service Job for slapd.service failed because a timeout was exceeded. See "systemctl status slapd.service" and "journalctl -xe" for details. Thank you, P.

3 2

mdb_idl_insert_keys: c_put id failed: MDB_MAP_FULL: Environment mapsize limit reached (-30792)
by Paul Pathiakis 14 Aug '19

14 Aug '19

Hi, I'm reloading a DB into OpenLDAP after an upgrade. It starts off fine with: ldapadd -f ./20190812-200001.ldif -v -D "cn=root,dc=hq,dc=example,dc=com" -h newldap.hq.example.com -W -c However, during the load, it gets through a few hundred entries and then: adding new entry "cn=ZRO001-INTERNAL,ou=ZRO001,ou=Projects,dc=hq,dc=example,dc=com"ldap_add: Other (e.g., implementation specific) error (80) additional info: index generation failed During the load, it seems that the indexes are failing as well. Could I have what the issue is?How do I correct it?What are the commands I run properly load this (and generate the indexes)?How do I make sure enough space for the DB and the indexes is allocated? @(#) $OpenLDAP: slapd 2.4.44 (Jan 29 2019 17:42:45) $ mockbuild@x86-01.bsys.centos.org:/builddir/build/BUILD/openldap-2.4.44/openldap-2.4.44/servers/slapd Thank you! P.

2 1

Re: Multi-master replication : read_config: no serverID / URL match found. Check slapd -h arguments.
by Quanah Gibson-Mount 14 Aug '19

14 Aug '19

--On Wednesday, August 14, 2019 11:23 AM +0200 HG <hanspeter.sloot(a)gmail.com> wrote: > > olcServerID: 1 ldap://linux1014.ts.aa-srv.com > olcServerID: 2 ldap://linux1015.ts.aa-srv.com Neither of those has a trailing slash. You noted: SLAPD_URLS="ldapi:/// ldap://linux1014.ts.aa-srv.com/" doesn't start, but SLAPD_URLS="ldapi:/// ldap://linux1014.ts.aa-srv.com" starts. I would expect this. As I said before, it's a MATCH between what's provided in the olcServerID field and the -h option to slapd (SLAPD_URLS in your case). I.e., the software is functioning as documented. If you want it to match on the trailing slash, adjust your olcServerID values to include a trailing slash. Regards, Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Username case
by JC 14 Aug '19

14 Aug '19

I have an OpenLDAP 2.4.42 server in which I have loaded an LDIF file that contains (among other entries) the following: # james, staff, yoyodyne.com dn: uid=james,ou=staff,dc=yoyodyne,dc=com uid: james cn: james objectClass: account objectClass: posixAccount loginShell: /bin/bash uidNumber: 1010 gidNumber: 100 homeDirectory: /home/james Separately, I have a Centos 7 box that retrieves user information from the OpenLDAP server above, by means of the Name Service Switch framework in the Centos 7 system. Users to be authenticated in this box may not be defined locally, but in the OpenLDAP server alone; PAM has been configured in the Centos system so that, for such users, and when attempting to log in over SSH, a directory is created in the Centos box on the fly as necessary, using the path retrieved from the OpenLDAP server in the homeDirectory attribute for the user. The authentication itself is carried out by a remote RADIUS server; PAM is correctly configured for this too. This works fine as described, in that when one attempts to log in over SSH into the Centos system as james, provided that the right password is supplied, the '/home/james' directory is created in the Centos system and an interactive SSH session is established. What I would be interested in next is an OpenLDAP setup so that the entry for a given user can be retrieved from the OpenLDAP server in a case-insensitive manner. The goal is to be able to authenticate username, password pairs over the SSH interface in the Centos7 system, with the username being cases-insensitive. As far as I can tell, in the process of establishing an SSH session, the OpenSSH daemon in the Centos 7 system will attempt to verify that the username is valid by invoking the gepwnam() Linux system call. This function will attempt to retrieve information for its argument - a username string - in a case-sensitive way. It first looks into the relevant local files and, if nothing is found and the NSS framework is appropriately configured, it will next attempt to obtain the data from an LDAP server. PAM helps me here by providing a PAM module (pam_regex) which enables me to normalize incoming usernames - in essence, this module makes is easy to recast all incoming usernames to strings where all uppercase letters are transformed into their lowercase versions. The problem that I have is that OpenSSH invokes getpwnam() both before and after the pam_regex module has been invoked. Thus, if the username received from the client is, say, James, OpenSSH will invoke getpwnam("James") first, and getpwnam("james") later, after pam_regex has normalized "James". The upshot here seems to be that what I would need is an OpenLDAP setup such that when getpwnam("james") and getpwnam("James") - or any other case combination of "James" - are invoked, the OpenLDAP server would return the same user data. Now it seems to be the case that, with a user entry in OpenLDAP as described above, getpwnam("james") will look for an entry such that the its uid attribute takes the value "james". I.e. if the value of uid is, say, "James" then it will be ignored. Which, following the discussion above, doesn't fit my goal. I guess that the question would be the following: Is it possible to configure OpenLDAP so that getpwnam() can retrieve a given entry from the OpenLDAP server in a case-insensitive way? How would one create an entry for a user identified as James so that getpwnam("James"), getpwnam("james"), getpwnam("JAMES"), etc. would all cause the OpenLDAP server to return exactly the same Linux user data (uidNumber, gidNumber, homeDirectory)? Other than the obvious and impractical approach consisting of creating identical entries for all the case combinations of the target name, that is.

4 5

OpenLDAP 2.5 plans and community engagement
by Ondřej Kuzník 13 Aug '19

13 Aug '19

Hi all, as 2.4 has finally stabilised and the ITS list is getting shorter not longer (thanks to Quanah's tireless efforts), the project can finally tackle some of the long-standing pain points. So this post aims to outline where we are/want to move to as well as to start a discussion and hopefully get you, the community, more involved on the road to 2.5 and beyond. We would love to welcome more people to participate in the project and make it more active and resilient. People of various skills and experience can make a difference, I am personally happy to assist anyone who wants to contribute with getting up to speed and start helping out, and for sure I'm not the only one. Also OpenLDAP is about more than just C programming and I'll try to outline the main areas where we want to focus our work in the short term here: One of the pain points in the 2.4 release cycle was the level of testing done on the code before we have released. There are issues with the existing test suite that make testing hard and I will come to how this could be tackled in a later email. In the short term however, we could benefit from having the tests we have run more often and on more diverse environments. If you have a chance to run it regularly, in a loop and report issues picked up, that would be a definite help. If you could help extend the test suite to cover scenarios that are of interest to you and are not appropriately covered yet, even better. Another issue frustrating both users and contributors to the project has been the existing jitterbugs bug tracker (a.k.a. ITS), which has by now outlived its usefulness. Plans to move to a project-hosted Gitlab instance + Bugzilla (via Gitlab's built in Bugzilla integration) are being made and this should make the issue tracker searchable again, help with triage as well as greatly improve the visibility into our release process. Especially after the migration, this would be another opportunity for anyone with just a bit of spare time to help by triaging open issues so we can make timely releases of better quality. I'm sure everyone agrees our website could do with a redesign. We've started looking into this but it has been a slow process so far and if you can contribute here, that might speed things up. It doesn't need much, keeping it a collection of static html pages, just a slight reorganisation of the content that is more friendly to anyone visiting it for the first time plus a simplistic design along the lines of many other open source projects (openssl.org for example) would go a long way. This leads to documentation, much of which is already hosted on the website. While I believe there is good work to be done on the admin guide, it is one of the better parts of our documentation. This is where user feedback on its usefulness is more important, please read it, have people on your team read it and show it people just getting started and report which parts are confusing, how they could be improved. We also intend to document our contributor guidelines in a more readable way. The FAQ site however is on the opposite side of the spectrum and will be removed at some later point. There have been two suggestions how to replace it. We could use Gitlab internal wiki or a static webpage site based on Gitlab pages and we want to transfer over articles that are still relevant. In general, moving to Gitlab should let us integrate CI much better, especially if there are people willing to integrate and manage external runners. As mentioned above, we could definitely do with more regular testing on non-Linux platforms, e.g. *BSD or Windows+MSYS2. Pull requests might then also be automatically tested. Let us know what the pain points have been with OpenLDAP when you were just starting, right now and if you have a suggestion how to make it easier to start using it. Or if you wanted to contribute, has anything discouraged you? There are things we might not be able to influence easily (LDAP itself can be complex), but a fresh look might help direct efforts in the right direction. Thanks ever so much for reading this far. This email is long enough already so I will follow up with another one about my long term plans to overhaul the test infrastructure and other tools that might be worth introducing to help with setting up and managing OpenLDAP deployments. Regards, -- Ondřej Kuzník Senior Software Engineer Symas Corporation http://www.symas.com Packaged, certified, and supported LDAP solutions powered by OpenLDAP

11 39

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical