openldap-technical September 2016

openldap-technical@openldap.org

50 participants
44 discussions

Ldap Replication getting delayed for 20 seconds.
by Prashanth P.Nair 12 Sep '16

12 Sep '16

Hello All I am facing a performance issue on my LDAP nodes. I have two LDAP nodes ,which is syncrepl enabled. When i create new record ,its taking 18-20 second to replicate on other node. So the application which is reading the records from the second node is unable to find those records. 1.Is this normal behavior? 2.Is there any way to findout how much time took a record to be replicated? 3.Any other paramter needs to be added to fasten this replication? Below is my configuration details. ------------------------------------------------------------------------- # This is the main slapd configuration file. See slapd.conf(5) for more # info on the configuration options. ####################################################################### # Global Directives: # Features to permit #allow bind_v2 TLSCACertificateFile /etc/ssl/ldap.pem TLSCertificateKeyFile /etc/ssl/ldap.pem TLSCertificateFile /etc/ssl/ldap.pem include /etc/ldap/schema/core.schema include /etc/ldap/schema/cosine.schema include /etc/ldap/schema/nis.schema include /etc/ldap/schema/inetorgperson.schema include /etc/ldap/schema/sncds.schema pidfile /var/run/slapd/slapd.pid Server 002 argsfile /var/run/slapd/slapd.args loglevel sync stats modulepath /usr/lib/ldap moduleload back_hdb moduleload syncprov SizeLimit 500 tool-threads 8 threads 16 backend hdb database hdb monitoring on cachesize 50000 idlcachesize 50000 suffix "dc=xx,dc=xxx,dc=xx" rootdn "cn=xx,dc=xx,dc=xx,dc=xx" rootpw xx directory "/var/lib/ldap/account" dbconfig set_cachesize 0 167772160 0 dbconfig set_lk_max_locks 1500 dbconfig set_lk_max_lockers 1500 index objectClass, snAccount, snEnabled, entryCSN, entryUUID eq index cn eq,pres,subany lastmod on checkpoint 5120 30 access to attrs=userPassword,shadowLastChange by dn="cn=xx,ou=xx,dc=xx,dc=xx" write by anonymous auth by self write by * none access to dn.base="" by * read access to * by dn="cn=xx,ou=xx,dc=xx,dc=xx" write by self read by * none limits dn.exact="cn=xx,ou=xx,dc=xx,dc=xx" size=unlimited syncrepl rid=001 provider=ldap://IP bindmethod=simple binddn="cn=xx,ou=xx,dc=xx,dc=xx" credentials=xx searchbase="dc=xx,dc=xx,dc=xx" schemachecking=on type=refreshAndPersist retry="10 +" mirrormode on overlay syncprov syncprov-checkpoint 100 10 syncprov-sessionlog 10000

2 1

LMDB: Ignore SIGPIPE in mdb_env_copythr
by Lorenz Bauer 07 Sep '16

07 Sep '16

Hello Hallvard, List, I'd like to propose explicitly ignoring SIGPIPE in the copy thread via pthread_sigmask, and returning EPIPE to the caller instead. This is a change that makes little sense in the C world, since the caller can simply adjust the sigmask before calling the function (which is indeed what the utilities do). However, this amount of control is not always given, e.g. when using LMDB from Go (which is what I am doing). Due to the way the Go runtime works, using mdb_env_copyfd2 with CP_COMPACT on a pipe or a network connection will abort the whole process with SIGPIPE if the reading end of the fd is closed prematurely. The details why this happens are in [1]. Go has some additional trickery when dealing with stdin and stdout, so ignoring SIGPIPE for the whole process is undesirable (details in [2]). I understand that you might be reluctant to change the behaviour due to the constraints of a different runtime. However, I think that even in the C world it is surprising that calling a library function can abort your process. What are your thoughts regarding this? I think the code changes would be small, and the API would not change for consumers of the library. 1: https://golang.org/pkg/os/signal/#hdr-Go_programs_that_use_cgo_or_SWIG 2: https://golang.org/pkg/os/signal/#hdr-SIGPIPE Best, Lorenz

1 0

UNKNOWN attributeDescription "ATTRIBUTE1" inserted
by PenguinWhispererThe 07 Sep '16

07 Sep '16

Hi all, I've been trying to update a schema with new olcAttributeTypes and modified an existing objectClass (X) so these attributesTypes may be used with it. However after adding these attributes to objectClass X I did a "rollback" (replacing the current olcObjectClasses (replace: olcObjectClasses) with the old definition and the same for the olcAttributes (replace: olcAttributeTypes). I think somewhere along the way something wasn't cleaned up properly. Now when I start slapd service I get: slapd[21335]: UNKNOWN attributeDescription "ATTRIBUTE1 " inserted. slapd[21335]: UNKNOWN attributeDescription "ATTRIBUTE2" inserted. slapd[21335]: [66B blob data] With slapschema: UNKNOWN attributeDescription "ATTRIBUTE1 " inserted. UNKNOWN attributeDescription "ATTRIBUTE2" inserted. UNKNOWN attributeDescription "ATTRIBUTE3 With slapcat I can't find anything that's matching these names. When I grep in the mdb file grep says the binary file is matching. However when I ldapsearch for the attribute I can't find it. Neither with slapcat. Note that I replaced the actual attribute names in the posted output. So I think I might have some data in that mdb that's not linked to anything anymore. So my question is: how can I clean this up? And/or what did/went wrong? Thanks in advance for your help.

3 8

Default LDIF Configuration
by Trent Dierking 07 Sep '16

07 Sep '16

When running slapd with -f etc/openldap/slapd.ldif, I receive the following line 5: unknown directive <dn:> outside backend info and database definitions. By default slapd is using the .conf file instead, which works fine. This is the default ldif file, and line 5 seems to be the same as line 2 (first relevant line) from the configuration guide. I have tried using no options to configure ldap, with the same issues. I understand what the message is saying, but why does this conflict with the guide/default ldif file?

2 1

Re: UNKNOWN attributeDescription "ATTRIBUTE1" inserted
by Howard Chu 06 Sep '16

06 Sep '16

PenguinWhispererThe . wrote: > Once again thanks for taking your time to followup on this. > > I've tried slapcat before to dump to a file and couldn't find the attributes > that were mentioned. > So today I did it again (just to make sure) and the dump doesn't seem to hold > the attributes. You're still not paying attention to Michael's advice. Wipe out your original DB and feed the slapcat output to slapadd. Your current DB files still contain entries with the offending attributes. > What I did get is : The first database does not allow slapcat; using the first > available one (2). Apparently you have more than one database configured. You're probably not slapcat'ing the DB with the offending attributes. > > So then I went to the directory and just grepped for the attribute name and it > seems that it's matching on a binary file: cn=accesslog/log.000000123123 > (changed the number to post). > > This seems a good and a bad thing: good as in the actual data has no issues. > Bad as in when I restart the service we get the message about the attributes. > Is there something slapd does with those log files when it starts that > explains it being printed on the screen/syslog? > > I only have some "spare" time once in a while to further investigate this as > it's not a high priority but I'd like to get rid of it. > > > > 2016-09-04 14:07 GMT+02:00 Michael Ströder <michael(a)stroeder.com > <mailto:michael@stroeder.com>>: > > PenguinWhispererThe . wrote: > > Is "cleaning" my data somewhere like a slapcat and slapadd? This is a > > replicated environment. Will those changes be propagated? I'd think not as > > the import probably keeps the same timestamps. > > You have to start from scratch like after a full restore (cleaning the > databases > before slapadd-ing the sanitized data). > > Ciao, Michael. > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Should I use OpenLDAP or PostgreSQL for this?
by John Lewis 06 Sep '16

06 Sep '16

I want to start a project to document my local government starting at the municipal level and going upwards from there. I want to build an interface to allow people to look up their representatives and their public servants by issue and geographic area or issue and get their contact information back. I want it to to have fast lookups so community organizers will come to my site first they want to find out who does what. I would also like it to be able to scale geographically so I can get local activist could have their own copy of the database on a local server that they can also delegate access to. I would like to delegate updates to a development version of the database to other people similar to Open Street Map, but I would still like to verify changes so there won't be a flood of bad data. The presentation and the data storage will be separate components. Knowing this information could you tell me what data management engine would be more appropriate for the task?

2 1

approximate/phonetic/accent-insensitive search configuration
by clauds2x＠gmail.com 06 Sep '16

06 Sep '16

Hi, I'm trying to figure out the configuration options available for approximate/phonetic/accent-insensitive search on OpenLDAP (2.4). First off, is there any documentation that I might have missed? I only found mention of an approx index without further details... nothing about what needs to be enabled etc. In some older questions (~2002) an option "--enable-phonetic" was mentioned... was this removed respectively is it obsolete (e.g. the default)? I did some tests using the approximate match operator ~= once on a fresh LDAP with no approx index and once with the approx index set and I got the same result... is this expected behaviour that approximate matching also works without defining an index? If it's not necessary, what's the advantage of setting the approx index, improved performance? (Mind that the attributes I search do have other indices set e.g. sub. Maybe approximate search implementation also considers that index?) Since data in our LDAP is mostly French, Italian and German I was wondering if any language specific settings can be made? I came across a related question from 2015 which then was answered with no, is that still true? > Is there any possibility to configure phontic search specially for Austria? ITS is for bug reports/enhancements. Use the openldap-technical mailinglist for help reqests. Anyway: No, soundex/metaphone is hardcoded in slapd approx match. See <openldap source>/servers/slapd/phonetic.c. Clients can do a little of this "by hand" by inserting many variants of a name (e.g. o instead of ö) as invisible attribute values in the directory, see the x-hidden examples in man slapd.conf. http://www.openldap.org/lists/openldap-bugs/201506/msg00002.html Could matching rules be used in any way to influence the search behaviour with regard to localization e.g. using collation? I came across "caseIgnoreOrderingMatch" (RFC 4517) but as far as I can see it's only good for sorting... Is there a way to search only ignoring the order or in other words performing an accent-insensitive search? Any other hints/best practices for configuring OpenLDAP to support approximate/phonetic/accent-insensitive search in general and specifically for languages other than English? Thanks for your consideration. Clauds

1 0

NTLM hashes support for OpenLDAP
by Andrei Valoshyn 06 Sep '16

06 Sep '16

Hello! I have openldap-2.4.31 on ubuntu 14.04 with slapd.conf configuration. Is it possible to store NTLM hashes into it? -- With Best Wishes Andrei Valoshyn Exadel Inc. System Administrator avaloshyn(a)exadel.com -- CONFIDENTIALITY NOTICE: This email and files attached to it are confidential. If you are not the intended recipient you are hereby notified that using, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error please notify the sender and delete this email.

2 1

Documentation Request
by Tom Jay 05 Sep '16

05 Sep '16

Hello, Can I make a request that certain features of the access control documentation are emphasized? I've wasted quite a lot of time on this and some simple rules (which already exist in the documentation) would have been really helpful. These are: 8. Access Control 8.2. Access Control via Static Configuration 8.2.5. Access Control Examples To all attributes except homePhone, an entry can write to itself, entries under example.com entries can search by them, anybody else has no access (implicit by * none) excepting for authentication/authorization (which is always done anonymously). The fact that authentication is always done anonymously, even if anonymous binds are disabled in the configuration, is very important. 8.2.4. Access Control Evaluation Slapd stops with the first <what> selector that matches the entry and/or attribute. This is also very important, as it explains exactly how the access rules are processed. The order of evaluation of access directives makes their placement in the configuration file important. I don't think this is emphasized enough, as it is critical to how the access rules are processed. Also, some mention of the ACL log level would be useful! Thanks. Tom

3 4

Re: Should I use OpenLDAP or PostgreSQL for this?
by Howard Chu 05 Sep '16

05 Sep '16

John Lewis wrote: > I want to start a project to document my local government starting at > the municipal level and going upwards from there. I want to build an > interface to allow people to look up their representatives and their > public servants by issue and geographic area or issue and get their > contact information back. > > I want it to to have fast lookups so community organizers will come to > my site first they want to find out who does what. I would also like it > to be able to scale geographically so I can get local activist could > have their own copy of the database on a local server that they can also > delegate access to. I would like to delegate updates to a development > version of the database to other people similar to Open Street Map, but > I would still like to verify changes so there won't be a flood of bad > data. The presentation and the data storage will be separate components. > > Knowing this information could you tell me what data management engine > would be more appropriate for the task? I have a strong suspicion that OpenLDAP will be the superior solution for this problem, but you can't say definitively, based on a brief description. I would suggest you start to draw out on paper the types of information you plan to store, and group the related items together. This will give you some idea of what your basic DB records will contain. Then draw out what kinds of search operations you plan to support, or what kinds of data you expect users to be able to retrieve. Figure out what natural interconnections and relationships exist between your records. Most real world problem sets don't break down into 100% pure tabular data structures, nor do they break down into 100% pure hierarchical structures. But if you find there's a natural hierarchy like geographic area county municipality as I said before, I would suspect that the hierarchical structure will be a more natural fit. Also, given your requirement to scale out geographically - this is trivial to do with OpenLDAP; it is quite cumbersome to do with SQL servers. > > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

3 2

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical September 2016