openldap-technical August 2019

openldap-technical@openldap.org

33 participants
47 discussions

LDAPCon 2019 CFP closimg today
by Howard Chu 18 Aug '19

18 Aug '19

Just a reminder that the Call for Papers is closing today. If you've been thinking about submitting a talk, this is your last chance! https://ldapcon.org/2019 -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 1

Fresh install changing the hdb to mdb
by Marc Roos 18 Aug '19

18 Aug '19

This is the default file that rhel/centos have in their slapd.d dir for the database. I thought I would just remove this one and place the one for mdb, seems to work, don't know about this entryUUID? Or can I do this with ldapmodify? [@53386e4b0025 cn=config]# cat /tmp/olcDatabase\=\{2\}hdb.ldif # AUTO-GENERATED FILE - DO NOT EDIT!! Use ldapmodify. # CRC32 4f2ac1fc dn: olcDatabase={2}hdb objectClass: olcDatabaseConfig objectClass: olcHdbConfig olcDatabase: {2}hdb olcDbDirectory: /var/lib/ldap olcSuffix: dc=my-domain,dc=com olcRootDN: cn=Manager,dc=my-domain,dc=com olcDbIndex: objectClass eq,pres olcDbIndex: ou,cn,mail,surname,givenname eq,pres,sub structuralObjectClass: olcHdbConfig entryUUID: 537b0adc-5476-1039-9bf9-1dc025e1859d creatorsName: cn=config createTimestamp: 20190816133433Z entryCSN: 20190816133433.095410Z#000000#000#000000 modifiersName: cn=config modifyTimestamp: 20190816133433Z [@53386e4b0025 cn=config]# cat olcDatabase\=\{2\}mdb.ldif # AUTO-GENERATED FILE - DO NOT EDIT!! Use ldapmodify. # CRC32 b6a274bd dn: olcDatabase={2}mdb objectClass: olcDatabaseConfig objectClass: olcMdbConfig olcDatabase: {2}mdb olcDbDirectory: /var/lib/ldap olcSuffix: dc=my-domain,dc=com olcRootDN: cn=Manager,dc=my-domain,dc=com olcDbIndex: objectClass eq,pres olcDbIndex: ou,cn,mail,surname,givenname eq,pres,sub structuralObjectClass: olcMdbConfig entryUUID: 537b0adc-5476-1039-9bf9-1dc025e1859d creatorsName: cn=config createTimestamp: 20190816133433Z entryCSN: 20190816133433.095410Z#000000#000#000000 modifiersName: cn=config modifyTimestamp: 20190816133433Z

4 4

Initial syncreplication details
by Marc Roos 17 Aug '19

17 Aug '19

Am I correct to understand from this page[0] that the consumer gets its 'new' contextCSN from the slapcat import. (I saw it in the file). And will get all data since that date at startup? The replication id is of no influence. So if I would stop slapd import again the same old slapcat file. It would again receive all new data since that contextCSN? Regardless of the same rid being used with the provider. The idea behind this is that if you create a container with a default imported slapcat/slapadd file. Every time the task is stopped and started. It will go back to it initial container state. (unless you make it statefull of course) I am not adding that much data so I could make a new default image every week or so, this would be used every time a container is launched. (which only happens in a failover situation) [0] https://www.openldap.org/doc/admin24/replication.html

2 2

Environment variable in slapd config
by Marc Roos 16 Aug '19

16 Aug '19

Is it possible to reference an environment variable in olcSyncrepl: {0}rid= ? --On Saturday, August 10, 2019 6:54 PM +0200 Michael Ströder <michael(a)stroeder.com> wrote: > Are you talking about the serverID? > > serverID is not needed on a read-only consumer. Just leave it out. He's talking about replication ID (rid), and it's clearly out of bounds in his post. The slapd.conf/slapd-config man pages clearly document the allowed range that can be used for a RID. rid identifies the current syncrepl directive within the replication consumer site. It is a non-negative integer not greater than 999 (limited to three decimal digits --Quanah --

6 16

RE: Environment variable in slapd config
by Marc Roos 16 Aug '19

16 Aug '19

I guess I am missing some info??? Is there any animosity between openldap and redhat? My general perspective is RedHat has made a core business of supporting a linux enterprise os. So I do hope that, if the shit hits the fan, you can rely on the paid services based on the knowledge of their 10k+ employees or so? These guys that work on ceph are doing a great job, looks like very competent people. And now they are owned by IBM I like them even more. :) Always have had IBM in high regards with their r&d. -----Original Message----- Subject: RE: Environment variable in slapd config --On Friday, August 16, 2019 5:17 PM +0200 Marc Roos <M.Roos(a)f1-outsourcing.eu> wrote: > I am more fan of Centos because then I can fall back on RedHat > support, especially for production environments. That's the most laughable statement (in relation to OpenLDAP at least) that I've heard in years. Thanks for the morning chuckle. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

2 1

RE: Make slapadd faster?
by Marc Roos 16 Aug '19

16 Aug '19

Ok ok I will look at this mdb again. -----Original Message----- From: Quanah Gibson-Mount [mailto:quanah@symas.com] Subject: Re: Make slapadd faster? --On Friday, August 16, 2019 10:14 AM +0200 Marc Roos <M.Roos(a)f1-outsourcing.eu> wrote: > > I know you can disable some checks to make slapadd faster. But I think > in my test vm with limited disk iops, it looks like this disk io is > the problem. > I am not sure how slapadd adds entries, I guess one at a time? You > could get a significant improvement by reading more entries and > writing more entries at once? And if such a batch transaction fails > you can always go back to submitting the transactions of the batch one > by one, to identify which one is failing. Stop using back-hdb, it's signficantly slower than back-mdb. If you're going to insist on using back-hdb, then you *must* have a well tuned DB_CONFIG before you import. For any backend, ensure you have tool-threads set appropriately (For back-mdb, values > 2 are ignored). Use the -q flag if you know the LDIF is good. slapadd has been heavily profiled and tested. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Make slapadd faster?
by Marc Roos 16 Aug '19

16 Aug '19

I know you can disable some checks to make slapadd faster. But I think in my test vm with limited disk iops, it looks like this disk io is the problem. I am not sure how slapadd adds entries, I guess one at a time? You could get a significant improvement by reading more entries and writing more entries at once? And if such a batch transaction fails you can always go back to submitting the transactions of the batch one by one, to identify which one is failing.

2 1

recommended ppolicy pwdCheckModule
by Dave Macias 16 Aug '19

16 Aug '19

Hello, Currently we use ppm https://github.com/ltb-project/ppm which works quite well as a quality checker. Wanted to see what others use. Was looking around and found some interesting ones: pam_pwquality ( i believe these are only for users in passwd file ) ppchecker http://www.meddeb.net/pqchecker/?Idx=0 Any input is appreciated. Thanks! Dave

2 1

Openldap in container advice, how have you done it?
by Marc Roos 16 Aug '19

16 Aug '19

I was thinking of putting read-only slapd('s) in a container environment so other tasks can query their data. Up until now I have had replication only between vm's. To be more flexible I thought of using stateless containers. Things that could be caveats - replication id's say I spawn another instance, I need to have a new replication id to get updates from the master. But what if the tasks is killed, should I keep this replication id? Or better just always use a random unique replication id whenever a slapd container is launched? Maybe use launch date/time (date +'%g%H%M%S%2N') as repid? Is this giving issues with the master? What if I test with launching instances and the master will think there are a hundred slaves that are not connecting anymore? - updating of a newly spawned slapd instance When the new task is launched, it is not up to date with its database, can I prevent connections to the slapd until it is fully synced? Say I have user id's in slapd, it could be that when launching a new instance, this user is not available yet. When clients are requesting this data, they do not get it, and this user could be 'offline' until that specific instance of slapd is fully updated. - to prevent lots of records syncing Can I just copy the data of /var/lib/ldap of any running instance to the container default image? Or does it have some unique id's that will prevent this data to be run multiple times? Is there some advice on how to do this? - doing some /var/lib/ldap cleanup I am cleaning with db_checkpoint -1 -h /var/lib/ldap, and db_archive -d. Is there an option slapd can initiate this? - keep uniform configuration environment, or better a few different slapd instances? In my current environment vm slave slapd's only sync data from the master that the masters acls allow access to. That results in that on some vm's the ldap database is quite small and on other it is larger. I think for the container slapd instances to have all data, and just limit client access via the acls. But this means a lot more indexes on the slapd What else am I missing?

4 5

RE: Antw: RE: Openldap in container advice, how have you done it?
by Marc Roos 16 Aug '19

16 Aug '19

The ip address is known when I start the container that would mean I need to sed some ready ldif and import it into the slapd at runtime. That would also require the the availability of some secret to be able to import it. Although I have prepared the container for ldif fetching. Nicer would be if I could specify something like an environment variable in olcSyncrepl -----Original Message----- From: Ulrich Windl [mailto:Ulrich.Windl@rz.uni-regensburg.de] Sent: maandag 12 augustus 2019 8:56 To: Marc Roos Subject: Antw: RE: Openldap in container advice, how have you done it? >>> "Marc Roos" <M.Roos(a)f1-outsourcing.eu> schrieb am 10.08.2019 um 14:07 in Nachricht <"H00000710014b895.1565438831.sx.f1-outsourcing.eu*"@MHS>: > Ok so long rep id is not going to work modifying entry > "olcDatabase={2}hdb,cn=config" > ldap_modify: Other (e.g., implementation specific) error (80) > additional info: Error: parse_syncrepl_line: syncrepl id > 1911533132 is out of range [0..999] Why not derive the ID from some container ID or from the container's IP address? > > > > > ‑‑‑‑‑Original Message‑‑‑‑‑ > From: Marc Roos > Sent: zaterdag 10 augustus 2019 1:24 > To: openldap‑technical(a)openldap.org > Subject: Openldap in container advice, how have you done it? > > > > I was thinking of putting read‑only slapd('s) in a container > environment so other tasks can query their data. Up until now I have > had replication only between vm's. > > To be more flexible I thought of using stateless containers. Things > that could be caveats > > ‑ replication id's > say I spawn another instance, I need to have a new replication id to > get updates from the master. But what if the tasks is killed, should I > keep this replication id? Or better just always use a random unique > replication id whenever a slapd container is launched? Maybe use > launch date/time (date +'%g%H%M%S%2N') as repid? Is this giving issues > with the master? What if I test with launching instances and the > master will think there are a hundred slaves that are not connecting anymore? > > ‑ updating of a newly spawned slapd instance When the new task is > launched, it is not up to date with its database, can I prevent > connections to the slapd until it is fully synced? > Say I have user id's in slapd, it could be that when launching a new > instance, this user is not available yet. When clients are requesting > this data, they do not get it, and this user could be 'offline' until > that specific instance of slapd is fully updated. > > ‑ to prevent lots of records syncing > Can I just copy the data of /var/lib/ldap of any running instance to > the container default image? Or does it have some unique id's that > will prevent this data to be run multiple times? Is there some advice > on how to do this? > > ‑ doing some /var/lib/ldap cleanup > I am cleaning with db_checkpoint ‑1 ‑h /var/lib/ldap, and db_archive ‑d. > Is there an option slapd can initiate this? > > ‑ keep uniform configuration environment, or better a few different > slapd instances? > In my current environment vm slave slapd's only sync data from the > master that the masters acls allow access to. That results in that on > some vm's the ldap database is quite small and on other it is larger. > I think for the container slapd instances to have all data, and just > limit client access via the acls. But this means a lot more indexes on > the slapd > > What else am I missing?

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical August 2019