openldap-technical

openldap-technical@openldap.org

9 participants
9868 discussions

Re: openldap 2.4.43 keeps crashing
by Jephte Clain 30 Dec '15

30 Dec '15

2015-12-29 20:47 GMT+04:00 Quanah Gibson-Mount <quanah(a)zimbra.com>: > --On Monday, December 28, 2015 10:18 PM -0800 Quanah Gibson-Mount > <quanah(a)zimbra.com> wrote: > >> David, >> >> Why haven't you reported this to the ITS? > > > Discussed with Howard. It isn't in 0.9 yet because it hasn't been confirmed > if the fix is good or not. David, if that fix has resolved the issue, > *please* follow up with the ITS noting that to be the case. > Hello, I can confirm this is the problem I have. Starting slapd in debug mode, it dies after some time: /tmp/buildd/openldap-2.4.43/servers/slapd/back-mdb/../../../libraries/liblmdb/mdb.c:5276: Assertion 'NUMKEYS(mp) > 1' failed in mdb_page_search_root() /usr/local/slaptools/lib/functions: line 44: 30232 Abandon /usr/sbin/slapd -h "$SLAPD_SERVICES" -g "$SLAPD_GROUP" -u "$SLAPD_USER" -F "$SLAPDD" -d "$dlevel" "$@" I didn't notice it because it doesn't dump core. And somehow, the initscript I use eats the stderr output of slapd (I'll fix it for future errors) For now, I'll revert to the previous working version, but I'm willing to test any fix The funny part is only the replica dies regularly, not the master although it is the one to receive the writes Thanks to all for your precious help. Very best regards, Jephté -- Jephté CLAIN | Développeur, Intégrateur d'applications Service Système d'Information Direction des Systèmes d'Information Tél: +262 262 93 86 31 || Gsm: +262 692 29 58 24

2 1

SASL Passthrough no request
by Küchler, Simon 30 Dec '15

30 Dec '15

Hello, I need help with the following problem. Our password authetication should use SASL but we don't see any requests in our Logs or by tcpdump. The password authentication should work as follows - userPassword-Attribute: {SASL}User@Domain - saslauthd -> use PAM - PAM -> use kerberos - kerberos -> send request to Active-Directory Server RPM list: --------------------- lshxx0693:~ # rpm -qa | grep sasl cyrus-sasl-gssapi-32bit-2.1.22-182.20.1 cyrus-sasl-gssapi-2.1.22-182.20.1 cyrus-sasl-2.1.22-182.20.1 cyrus-sasl-32bit-2.1.22-182.20.1 cyrus-sasl-digestmd5-2.1.22-182.20.1 cyrus-sasl-digestmd5-32bit-2.1.22-182.20.1 cyrus-sasl-devel-2.1.22-182.20.1 cyrus-sasl-saslauthd-2.1.22-182.19 lshxx0693:~ # rpm -qa | grep krb krb5-1.6.3-133.49.64.1 krb5-32bit-1.6.3-133.49.64.1 pam_krb5-2.3.1-47.12.1 pam_krb5-32bit-2.3.1-47.12.1 krb5-doc-1.6.3-133.49.64.1 krb5-plugin-kdb-ldap-1.6.3-133.49.64.1 krb5-server-1.6.3-133.49.64.1 krb5-client-1.6.3-133.49.64.1 lshxx0693:~ # rpm -qa | grep ldap openldap2-2.4.26-0.28.5 openldap2-client-2.4.26-0.28.5 openldap2-devel-2.4.26-0.28.5 pam_ldap-184-147.20 pam_ldap-32bit-184-147.20 nss_ldap-262-11.32.39.1 nss_ldap-32bit-262-11.32.39.1 libldap-2_4-2-2.4.26-0.28.5 libldap-2_4-2-32bit-2.4.26-0.28.5 libldapcpp1-0.3.0-0.9.29 libevoldap-2_4-2-2.4.12-4.19 yast2-ldap-2.17.8-0.7.61 yast2-ldap-client-2.17.38-0.7.2 yast2-ldap-server-2.17.44-0.5.1 lshxx0693:~ # rpm -qa | grep cyrus cyrus-sasl-gssapi-2.1.22-182.20.1 cyrus-sasl-gssapi-32bit-2.1.22-182.20.1 cyrus-sasl-saslauthd-2.1.22-182.19 cyrus-sasl-devel-2.1.22-182.20.1 cyrus-sasl-2.1.22-182.20.1 cyrus-sasl-32bit-2.1.22-182.20.1 cyrus-sasl-digestmd5-2.1.22-182.20.1 cyrus-sasl-digestmd5-32bit-2.1.22-182.20.1 Configuration files: ---------------------------- lshxx0693:~ # cat /etc/sasl2/slapd.conf mech_list: plain login pwcheck_method: saslauthd lshxx0693:~ # cat /etc/sysconfig/saslauthd SASLAUTHD_AUTHMECH=pam SASLAUTHD_THREADS=5 SASLAUTHD_PARAMS="-r" lshxx0693:~ # cat /etc/pam.d/ldap auth required pam_krb5.so no_user_check account required pam_permit.so lshxx0693:/etc/pam.d/ # cat common-account | egrep -v "^#" account requisite pam_unix2.so account sufficient pam_localuser.so account required pam_ldap.so use_first_pass lshxx0693:/etc/pam.d/ # cat common-account-pc | egrep -v "^#" account requisite pam_unix2.so account sufficient pam_localuser.so account required pam_ldap.so use_first_pass lshxx0693:/etc/pam.d/ # cat common-auth | egrep -v "^#" auth required pam_env.so auth sufficient pam_unix2.so auth required pam_ldap.so use_first_pass lshxx0693:/etc/pam.d/ # cat common-auth-pc | egrep -v "^#" auth required pam_env.so auth sufficient pam_unix2.so auth required pam_ldap.so use_first_pass shxx0693:/etc/pam.d/ # cat common-password | egrep -v "^#" password requisite pam_pwcheck.so nullok cracklib password sufficient pam_unix2.so use_authtok nullok password required pam_ldap.so try_first_pass use_authtok lshxx0693:/etc/pam.d/ # cat common-session | egrep -v "^#" session optional pam_mkhomedir.so session required pam_limits.so session required pam_unix2.so session optional pam_ldap.so session optional pam_umask.so lshxx0693:/etc/pam.d/ # cat common-session-pc | egrep -v "^#" session optional pam_mkhomedir.so session required pam_limits.so session required pam_unix2.so session optional pam_ldap.so session optional pam_umask.so lshxx0693:/etc/pam.d/ # cat common-password-pc | egrep -v "^#" password requisite pam_pwcheck.so nullok cracklib password sufficient pam_unix2.so use_authtok nullok password required pam_ldap.so try_first_pass use_authtok lshxx0693:~ # pam-config --verify lshxx0693:~ # lshxx0693:~ # cat /etc/krb5.conf [libdefaults] default_realm = INT.IT.DPP dns_lookup_kdc = true [realms] INT.IT.DPP = { kdc = 10.150.10.10 kdc = 10.150.10.10 } [logging] default = SYSLOG:NOTICE:DAEMON lshxx0693:~ # cat /etc/nsswitch.conf | egrep -v "#" passwd: compat group: files ldap hosts: files dns networks: files dns services: files ldap protocols: files rpc: files ethers: files netmasks: files netgroup: files ldap publickey: files bootparams: files automount: files nis aliases: files ldap passwd_compat: ldap Tell me, if you need more informations, please. I would like to thank you in advance for your help. Best wishes S. Kuechler

2 1

Re: Can't open env with required mapsize on windows .
by Howard Chu 30 Dec '15

30 Dec '15

Mohammed Muneer wrote: > > "Windows is like that." > > I know. Please keep your replies on the mailing list. > "Have no idea what you're actually doing, so it's impossible to answer. > Learn to ask better questions and provide better information." > > Sorry that I didn't sound intelligent. I was in a hurry when I wrote it. > I will now try to give you a detailed account of what I am doing. > > > "What revision of LMDB?" > > The latest hosted at https://github.com/LMDB/lmdb. "The latest" in which branch? Note that mdb.master is the main development branch and contains work-in-progress code that is probably not stable. As a non-LMDB-developer you should only be using actual tagged releases. You may occasionally want to use the mdb.RE/xx release engineering branches if they contain a fix for a specific issue that you encountered. > "Which version of Windows?" > > All are Windows 7 Ultimate x64 > > > "How much free disk space is there?" > > Machine 1 has around 26 GB free. Machine 2 has around 40 GB free. > Virtualbox guest has only has around 10 GB free. > > > "What else is your application code doing?" > > This is silly. My application was trying to open an env on windows. Here > is the code for it. > > > #define MB * 1024 * 1024 > #define GB * 1024 * 1024 * 1024 > > MDB_env *env = 0; > > bool openEnv() { > mdb_size_t mapSize = 390 MB; > MDB_dbi maxNamedDatabases = 100; > > > if (mdb_env_create(&env) != MDB_SUCCESS) > return false; > > if (mdb_env_set_mapsize(env, mapSize) != MDB_SUCCESS) > return false; > > if (mdb_env_set_maxdbs(env, maxNamedDatabases) != MDB_SUCCESS) > return false; > > int err = mdb_env_open(env, "test", 0, 0664); > > if (err != MDB_SUCCESS) { > fprintf(stderr, "Error opening env %i", err); > return false; > } > fprintf(stderr, "Success opening env %i", err); > > return true; > } > > int main() { > > if (!openEnv()) > return 1; > > } > > > > > "How do you know that only XX MB were "opened" > > Trial and error on windows showed that mdb_env_open returns success when > I give a mapSize of 390 MB on Machine 1, etc... > But on further testing, we can open 2 GB sometimes. But after a system > restart, situation is still the same. > > > "what stats are you checking? " > > Well as you can see from the above code, I am checking the return value > of mdb_env_open(). > > > > "What other differences exist between machine 1 and machine 2?" > > Not much. One being a core-i3 and the other core-i5. Both have 4 GB RAM. > Both are similiar in that none of them have > so many applications installed and all have used memory around 500 MB > > > > "What other processes are running," > > As I stated its mostly a default installation of Windows x64 on all > machines. > > > > "how much memory are they consuming on each machine?" > > Are you referring to other processes or my application. Assuming you are > talking about my application, it is just a > main function. So that might be irrelevant > > > > "Note that just because you create a 1GB memory map of a file doesn't > mean that it will actually consume 1GB of RAM. > That's not how memory mapping works." > > Never said that. Didn't even imply it. Don't know where you are getting > this. I know how memory map works. Having implemented > one in a custom hobby OS 5 years ago. Your implementation leveraging COW > along with performance and MVCC was the whole reason I was > interested in your project. > > > > "Read your Windows developer documentation, MSDN is always there for you > to read. We're not here to teach you how your OS works." > > You don't have to teach me how my OS works. Sorry if I gave you that > impression. Apparently we do. See below. > "Hangs where, exactly? Have you gotten a stack trace from such a hang? > That's the obvious first step that any application developer should know to do." > > On windows, if I dont set env_set_mapsize(env, 0) just before beginning > a write transaction, then first insert will complete successfully. On the second > write transaction while the application is running, and we try to for > eg: mdb_cursor_get(), it doesn't return. This can be reproduced with the below > stated toolchain. > > > ----------- > > For further notes, I am using official QtSDK-mingw on windows which is a > 32 bit compiler. But sometimes I can get LMDB to open the env > succesffully for 2 gb, etc. But on a system restart, the situation is > still the same. You can't address more than 2GB of memory in 32 bit code. You generally won't be able to create a 2GB mmap in a 32 bit process because other DLLs will already be occupying large chunks of the 32 bit address space. > But I downloaded an unofficial QtSDK-mingw-64 with the same result. But > when I change > mdb_size_t mapSize = 2147483648; // 2 GB > to > mdb_size_t mapSize = 2147483648LL; > > it works. > > > But setting 2147483648LL does not work (meaning does not return > MDB_SUCCESS) most of the times on mingw-32-bit That is expected. Again, you should already know how your OS works. > So the quesstion really is how to get it working when compiled on a > 32-bit compiler and executed on both 32 and 64 bit Windows? > > Note: I am not referring to vl32 branch because I don't want to have to > use more than 2GB as the size of my env file. It can't be done. > > > On Mon, Dec 28, 2015 at 2:17 PM, Howard Chu <hyc(a)symas.com > <mailto:hyc@symas.com>> wrote: > > Mohammed Muneer wrote: > > > I can env_set_mapsize(env, 1 GB) without any problems in linux. > > But on windows, it is a different story > > > Windows is like that. > > Machine 1) Windows 64 only opens with upto 10 MB (apporx) > Machine 2 Windows 64 only opens with upto 500 MB (apporx) > Machine 3) Virtual box Windows 64 running on linux works with 1 GB > > Q1) So why is this happening. Am I doing anything wrong. > > > Have no idea what you're actually doing, so it's impossible to answer. > Learn to ask better questions and provide better information. > > What revision of LMDB? Which version of Windows? How much free disk space > is there? What else is your application code doing? How do you know that > only XX MB were "opened", what stats are you checking? What other > differences exist between machine 1 and machine 2? > > What other processes are running, how much memory are they consuming on > each machine? Note that just because you create a 1GB memory map of a file > doesn't mean that it will actually consume 1GB of RAM. That's not how > memory mapping works. Read your Windows developer documentation, MSDN is > always there for you to read. We're not here to teach you how your OS works. > > Q2) And setting env_set_mapsize before calling open is not enough. I > need to set > env_set_mapsize(env, 0) just before beginning a write transaction > inorder to > insert something or otherwise the application indefinitely > hangs on the > second insert transaction. > > > Hangs where, exactly? Have you gotten a stack trace from such a hang? > That's the obvious first step that any application developer should know > to do. > > > Both the above questions concerns just Windows. Everything is fine > with linux. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

2 3

Re: Regarding ITS 8268 slapd-ldap quarantine
by Nikos Voutsinas 29 Dec '15

29 Dec '15

On Tuesday, 29 December 2015, Quanah Gibson-Mount <quanah(a)zimbra.com> wrote: > --On Tuesday, December 29, 2015 5:22 PM +0200 Nikos Voutsinas < > nvoutsin(a)gmail.com> wrote: > > >> >> >> Hi, >> >> >> May I ask how come and ITS 8268 never made it to software bugs? Isn't the >> case well described? >> > > OpenLDAP is a volunteer project. People work on items as they have > time/availability. If you feel an issue is urgent, you may wish to provide > a fix/patch as per the contribution guidelines, or engage support with a > company such as Symas. > > --Quanah > > > Understood, no need to clarify this. Obviously not so many from the community are using slapd-ldap quarantine. Anyway, let's say that the first step has been made and a bug report has been filed. That would have saved me some time, if someone else had done it before. Best Regards, Nikos

1 0

Re: openldap 2.4.43 keeps crashing
by Jephte Clain 29 Dec '15

29 Dec '15

Le 29/12/2015 00:55, David Hawes a écrit : > On 28 December 2015 at 01:31, Jephte Clain <jephte.clain(a)univ-reunion.fr> wrote: >> hello all, >> >> I upgraded to OpenLDAP 2.4.43 three weeks ago. It runs on debian squeeze >> 64bits lts >> it is built from sources with the fix from ITS#8330 >> >> Since the upgrade, it keeps crashing regularly (I would say once every 2 to >> 5 days) >> However, there isn't anything in the logs: no message, no error, no nothing >> >> slapd no longer runs, I restart it, and it runs ok until the next crash >> >> now I don't know how to debug that. >> I could downgrade to the previous version (2.4.41) which worked fine, but as >> our university is closed until 01/11, this is the perfect time to >> investigate. I still don't know if it's a bug in 2.4.43 or a >> misconfiguration for which previous versions was forgiving >> >> What are your advice to track down the cause for the crash? >> >> PS: here is a bit of context: >> one master with 2 mdb databases: one accesslog, one for the data >> two slaves (I mean complete replication, even cn=config) >> the master has crashed only once >> the slaves keep crashing regularly >> the slaves are complete replicas of the master, and I wonder if it could be >> a source of problems: I'll give details in another thread > Do you have accesslog purge configured? > > I ran into this on an accesslog purge with 2.4.43: > > http://www.openldap.org/its/index.cgi/Software%20Bugs?id=8336;selectid=8336 > > If you start slapd with '-d 1', you should be able to see if it is the issue. > > The fix referenced in that ITS worked for me. hello, thanks for the response. no, purge is not configured, but this is a nice suggestion... without purge, I guess the accesslog database would keep growing forever :-( thanks again, regards, Jephté CLAIN -- *Jephté CLAIN | Développeur, Intégrateur d'applications* Service Systèmes d'Information Direction des Systèmes d'Information <http://dsi.univ-reunion.fr> Tél: +262 262 93 86 31 <tel:+262262938631> || Gsm: +262 692 29 58 24 <tel:+262692295824> www.univ-reunion.fr <http://www.univ-reunion.fr> || Facebook <http://www.facebook.com/pages/Universit%C3%A9-de-La-R%C3%A9union-OFFICIEL/1…> || Twitter <http://twitter.com/univ_reunion>

2 3

Regarding ITS 8268 slapd-ldap quarantine
by Nikos Voutsinas 29 Dec '15

29 Dec '15

Hi, May I ask how come and ITS 8268 never made it to software bugs? Isn't the case well described? Anyway, while we have already worked around the problem, actually by disabling the feature, I was curious if the dev team or anyone else has tested slapd-ldap quarantine with a recent Openldap version ( we have noticed similar behavior both with 2.4.41 and 2.4.42) http://www.openldap.org/its/index.cgi/Incoming?id=8268 Best Regards, Nikos

2 1

what databases are to be replicated for a slave?
by Jephte Clain 29 Dec '15

29 Dec '15

hello, I've had problems with an upgrade to openldap 2.4.43, and I reviewed my configuration to check for errors I have two slaves which replicate all databases from the master, including cn=config. My reasoning is in case of a severe crash, I'd like one of the slaves to be able to become the new master. To allow such a scenario, each database in the master have a syncrepl directive to replicate from itself. It obviously have no effect on the master, but the slave that replicate this configuration get the right directives to replicate from the master. so far so good... now this is my question: should/could the accesslog configured this way? or should the accesslog be strictly local to a server? I mean, should I remove the syncrepl directive from the accesslog database? This configuration have run without errors until 2.4.41, but I got a weird error about accesslog corruption when I upgraded to 2.4.43 (sorry I didn't write the error down), hence my question Thanks in advance for any response. best regards, Jephté CLAIN PS: I hope it's clear enough. English is not my native language -- *Jephté CLAIN | Développeur, Intégrateur d'applications* Service Systèmes d'Information Direction des Systèmes d'Information <http://dsi.univ-reunion.fr> Tél: +262 262 93 86 31 <tel:+262262938631> || Gsm: +262 692 29 58 24 <tel:+262692295824> www.univ-reunion.fr <http://www.univ-reunion.fr> || Facebook <http://www.facebook.com/pages/Universit%C3%A9-de-La-R%C3%A9union-OFFICIEL/1…> || Twitter <http://twitter.com/univ_reunion>

2 5

openldap 2.4.43 keeps crashing
by Jephte Clain 29 Dec '15

29 Dec '15

hello all, I upgraded to OpenLDAP 2.4.43 three weeks ago. It runs on debian squeeze 64bits lts it is built from sources with the fix from ITS#8330 Since the upgrade, it keeps crashing regularly (I would say once every 2 to 5 days) However, there isn't anything in the logs: no message, no error, no nothing slapd no longer runs, I restart it, and it runs ok until the next crash now I don't know how to debug that. I could downgrade to the previous version (2.4.41) which worked fine, but as our university is closed until 01/11, this is the perfect time to investigate. I still don't know if it's a bug in 2.4.43 or a misconfiguration for which previous versions was forgiving What are your advice to track down the cause for the crash? PS: here is a bit of context: one master with 2 mdb databases: one accesslog, one for the data two slaves (I mean complete replication, even cn=config) the master has crashed only once the slaves keep crashing regularly the slaves are complete replicas of the master, and I wonder if it could be a source of problems: I'll give details in another thread -- *Jephté CLAIN | Développeur, Intégrateur d'applications* Service Systèmes d'Information Direction des Systèmes d'Information <http://dsi.univ-reunion.fr> Tél: +262 262 93 86 31 <tel:+262262938631> || Gsm: +262 692 29 58 24 <tel:+262692295824> www.univ-reunion.fr <http://www.univ-reunion.fr> || Facebook <http://www.facebook.com/pages/Universit%C3%A9-de-La-R%C3%A9union-OFFICIEL/1…> || Twitter <http://twitter.com/univ_reunion>

4 6

RE: lmdb Handle Leak with VL32 on Windows
by Jeremiah Morrill 28 Dec '15

28 Dec '15

Submitted (seems like #8342). Apologies for not doing this before. I will go through ITS with reports in the future. -Jer From: Howard Chu<mailto:hyc@symas.com> Sent: Monday, December 28, 2015 12:12 AM To: Jeremiah Morrill<mailto:Jeremiah.Morrill@econnect.tv>; openldap-technical(a)openldap.org<mailto:openldap-technical@openldap.org> Subject: Re: lmdb Handle Leak with VL32 on Windows Jeremiah Morrill wrote: > I noticed what I believe may be a handle leak against the latest (commit > 7b9928c) on github, with the MDB_VL32 on Windows. After some poking around, > it seems to be the file mapping handle, env->me_fmh. I threw it to a > CloseHandle in the env_close0(…) and it seemed to resolve it. Please submit this to the ITS. > > Here’s a quick diff: > > diff --git a/src/lmdb.c b/src/mdb.c > > index a564d40..190c860 100644 > > --- a/src/mdb.c > > +++ b/src/mdb.c > > @@ -5350,6 +5350,7 @@ mdb_env_close0(MDB_env *env, int excl) > > } > > #ifdef MDB_VL32 > > #ifdef _WIN32 > > + if (env->me_fmh) CloseHandle(env->me_fmh); > > if (env->me_rpmutex) CloseHandle(env->me_rpmutex); > > #else > > pthread_mutex_destroy(&env->me_rpmutex); > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Can't open env with required mapsize on windows .
by Mohammed Muneer 28 Dec '15

28 Dec '15

I can env_set_mapsize(env, 1 GB) without any problems in linux. But on windows, it is a different story Machine 1) Windows 64 only opens with upto 10 MB (apporx) Machine 2 Windows 64 only opens with upto 500 MB (apporx) Machine 3) Virtual box Windows 64 running on linux works with 1 GB Q1) So why is this happening. Am I doing anything wrong. Q2) And setting env_set_mapsize before calling open is not enough. I need to set env_set_mapsize(env, 0) just before beginning a write transaction inorder to insert something or otherwise the application indefinitely hangs on the second insert transaction. Both the above questions concerns just Windows. Everything is fine with linux.

2 2

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical