Re: LMBD - If passing NULL to data parameter is legal in mdb_cursor_get()
by Howard Chu
Kaho Ng wrote:
> Hi,
>
> When reading the documentation of lmdb, I wonder if the behavior in regards
> to passing NULL as data parameter to mdb_cursor_get() is subject to changes.
No. Unless there's an obvious bug, or the documentation specifically says
"subject to change", existing behavior will not change.
> When reading the source code of the library, I found that for some of the
> operations, if NULL is passed as data parameter EINVAL will be returned, and
> for the other operations passing will lead to different behavior
> (for instance, MDB_SET).
>
> I am not sure if the behavior of future versions of lmdb will stay in
> line with the
> current version. Or is it generally illegal to pass NULL as data parameter?
>
> Could you give me some hints on that? Thank you.
Ask yourself why you would ever pass NULL in each case and what that would
mean. This is basic logic.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
4 years, 11 months
Re: slapd segfault in openldap 2.4.45
by Scott Koch
Correction, my openldap version is 2.4.45(typo'ed in previous post).
On Wed, Dec 13, 2017 at 11:23 AM Scott Koch <scottkoch(a)gmail.com> wrote:
> This is an RPM of openldap I built from the latest upstream source. This
> slapd server is running on RHEL 7.4 x86_64.
>
> Error message:
> 2017-12-13T00:13:34.944152-05:00 ldap1.example.com kernel: slapd[983]:
> segfault at 766f6730 ip 00007f0272e2549b sp 00007f02017f55b8 error 4 in
> libc-2.17.so[7f0272ce8000+1b8000]
>
> We have seen 15 or so instances of this issue and in all cases the last
> LDAP operations follow the same pattern where there is an ABANDON and
> UNBIND, then there is a SRCH operation. See log output below of full
> connection for the client that performs the last operation.
>
> Let me know if there is any other information I can provide to help
> troubleshoot this problem.
>
> Thanks in advance for the help!
> -Scott
>
> 2017-12-13T00:13:03.560693-05:00 ldap1.example.com slapd[26514]:
> conn=873638 fd=105 ACCEPT from IP=10.0.4.37:48520 (IP=0.0.0.0:389)
>
> 2017-12-13T00:13:03.560869-05:00 ldap1.example.com slapd[26514]:
> conn=873638 op=0 EXT oid=1.3.6.1.4.1.1466.20037
>
> 2017-12-13T00:13:03.561012-05:00 ldap1.example.com slapd[26514]:
> conn=873638 op=0 STARTTLS
>
> 2017-12-13T00:13:03.561211-05:00 ldap1.example.com slapd[26514]:
> conn=873638 op=0 RESULT oid= err=0 text=
>
> 2017-12-13T00:13:03.569367-05:00 ldap1.example.com slapd[26514]:
> conn=873638 fd=105 TLS established tls_ssf=256 ssf=256
>
> 2017-12-13T00:13:03.569853-05:00 ldap1.example.com slapd[26514]:
> conn=873638 op=1 SRCH base="" scope=0 deref=0 filter="(objectClass=*)"
>
> 2017-12-13T00:13:03.570014-05:00 ldap1.example.com slapd[26514]:
> conn=873638 op=1 SRCH attr=* altServer namingContexts supportedControl
> supportedExtension supportedFeatures supportedLDAPVersion
> supportedSASLMechanisms domainControllerFunctionality defaultNamingContext
> lastUSN highestCommittedUSN
>
> 2017-12-13T00:13:03.570215-05:00 ldap1.example.com slapd[26514]:
> conn=873638 op=1 SEARCH RESULT tag=101 err=0 nentries=1 text=
>
> 2017-12-13T00:13:03.571953-05:00 ldap1.example.com slapd[26514]:
> conn=873638 op=2 SRCH base="dc=example,dc=com" scope=2 deref=0
> filter="(&(?objectClass=sudoRole)(|(!(?sudoHost=*))(?sudoHost=ALL)(?sudoHost=
> node1713.example.com)(?sudoHost=node1713)(?sudoHost=10.0.4.37)(?sudoHost=
> 10.134.0.0/18)(?sudoHost=ffff::ffff:ffff:fff:ffff)(?sudoHost=fe80::/64)(?sudoHost=+*)(|(?sudoHost=*\5C*)(?sudoHost=*?*)(?sudoHost=*\2A*)(?sudoHost=*[*]*)))
> <http://10.134.0.0/18)(?sudoHost=ffff::ffff:ffff:fff:ffff)(?sudoHost=fe80:...>
> )"
>
> 2017-12-13T00:13:03.572214-05:00 ldap1.example.com slapd[26514]:
> conn=873638 op=2 SRCH attr=objectClass cn sudoCommand sudoHost sudoUser
> sudoOption sudoRunAs sudoRunAsUser sudoRunAsGroup sudoNotBefore
> sudoNotAfter sudoOrder modifyTimestamp
>
> 2017-12-13T00:13:03.573488-05:00 ldap1.example.com slapd[26514]:
> conn=873638 op=2 SEARCH RESULT tag=101 err=0 nentries=0 text=
>
> 2017-12-13T00:13:34.943439-05:00 ldap1.example.com slapd[26514]:
> conn=873638 op=4 ABANDON msg=4
>
> 2017-12-13T00:13:34.943694-05:00 ldap1.example.com slapd[26514]:
> conn=873638 op=3 SRCH base="dc=example,dc=com" scope=2 deref=0
> filter="(&(uid=ntp)(objectClass=posixAccount)(&(uidNumber=*)(!(uidNumber=0))))"
>
> 2017-12-13T00:13:34.943885-05:00 ldap1.example.com slapd[26514]:
> conn=873638 op=5 UNBIND
>
> 2017-12-13T00:13:34.944092-05:00 ldap1.example.com slapd[26514]:
> conn=873638 op=3 SRCH attr=objectClass uid userPassword uidNumber gidNumber
> gecos homeDirectory loginShell krbPrincipalName cn modifyTimestamp
> modifyTimestamp shadowLastChange shadowMin shadowMax shadowWarning
> shadowInactive shadowExpire shadowFlag krbLastPwdChange
> krbPasswordExpiration pwdAttribute authorizedService accountExpires
> userAccountControl nsAccountLock host loginDisabled loginExpirationTime
> loginAllowedTimeMap sshPublicKey mail
>
4 years, 11 months
Openldap Password Reset Portal
by Douglas Duckworth
Hi
I am looking for password reset portal where users can reset their
credentials. We already have a legacy app written in php but it needs to
be retired.
I came across PWM which lists a number of schema changes that are required
before it can work with openldap.
https://github.com/pwm-project/pwm/wiki/General-Directory-Setup
I have a few questions:
1) Has anyone used PWM and if so can you describe how stable it operates?
I find the documentation lacking though it seems there's not a ton of
issues on their Github site.
2) Has anyone found other solutions besides PWM that do the same thing?
4 years, 11 months
ldap_sasl_interactive_bind_s: Can't contact LDAP server
by Turbo Fredriksson
[I’ve posted this on the OpenStack list as well, but maybe someone
here knows more]
I’m setting up (Open)LDAP (v2.4.40) on my old Newton installation,
with the LDAP servers behind a HAProxy LB.
I’m trying to have one at a time enabled to see if I can get them
working individually before I try them as a whole/group..
I tried all day yesterday, and I could do the initial connection, but
not get any results:
ldap_sasl_interactive_bind_s: Can't contact LDAP server (-1)
I see the connection in syslog on the LDAP server, but don’t get any
results back.
Now, first thing I did this morning was to just run the exact same
command (kinit && ldapwhoami) that I did last night.
AND IT WORKED!!
No idea why! It shouldn’t have. Glad it did, but since I can’t explain
WHY it worked, it’s annoying!! :)
So I then disabled that (working) LDAP server in the LB member list
and enabled the second. And now that is experiencing the same
problem as the first yesterday…
I didn’t change anything else - last thing I did before I went to bed
last night was try the ldapwhoami command -> “can’t contact ldap
server”. And the very first thing I did this morning was kdestroy
my ticket, get a new one and then run ldapwhoami.
I’ve run with multiple types of debugging, but there’s nothing obvious
that I can see, either from ‘-d -1’ or with KRB5_TRACE set).
So … “something” internally in OS changed. Any suggestions to what
or how to debug this?
What is ldap_sasl_interactive_bind_s() actually doing? Why does the
ldap_bind() earlier seem to work, but not the SASL bind?
See http://bayour.com/misc/ldapwhoami_output.txt <http://bayour.com/misc/ldapwhoami_output.txt> for full output from
KRB5_TRACE=/dev/stdout ldapwhoami -YGSSAPI -H ldaps://ldap.bayour.net -d -1
and while this is happening, this is the output from slapd in the logs
(running with “loglevel sync stats):
Nov 19 12:42:40 admin-auth-ldap-31 slapd[26613]: conn=1015 fd=29 ACCEPT from IP=10.0.17.34:53451 (IP=10.0.17.31:636)
Nov 19 12:42:40 admin-auth-ldap-31 slapd[26613]: conn=1015 fd=29 TLS established tls_ssf=256 ssf=256
Nov 19 12:42:40 admin-auth-ldap-31 slapd[26613]: conn=1015 op=0 BIND dn="" method=163
Nov 19 12:43:09 admin-auth-ldap-31 slapd[26613]: conn=1013 fd=22 closed (connection lost)
With ‘loglevel -1’ (and filtering out 'daemon: epoll: listen|daemon: activity on’
because it ends up filling the screen), I get:
Nov 19 12:49:28 admin-auth-ldap-31 slapd[27043]:
Nov 19 12:49:28 admin-auth-ldap-31 slapd[27043]: slap_listener_activate(12):
Nov 19 12:49:28 admin-auth-ldap-31 slapd[27043]: >>> slap_listener(ldaps://admin-auth-ldap-31.bayour.net:636/)
Nov 19 12:49:28 admin-auth-ldap-31 slapd[27043]: daemon: listen=12, new connection on 25
Nov 19 12:49:29 admin-auth-ldap-31 slapd[27043]:
Nov 19 12:49:33 admin-auth-ldap-31 slapd[27043]: daemon: added 25r (active) listener=(nil)
Nov 19 12:49:33 admin-auth-ldap-31 slapd[27043]: conn=1001 fd=25 ACCEPT from IP=10.0.17.34:54740 (IP=10.0.17.31:636)
Nov 19 12:49:34 admin-auth-ldap-31 slapd[27043]: 25r
Nov 19 12:49:34 admin-auth-ldap-31 slapd[27043]:
Nov 19 12:49:34 admin-auth-ldap-31 slapd[27043]: daemon: read active on 25
Nov 19 12:49:34 admin-auth-ldap-31 slapd[27043]: connection_get(25)
Nov 19 12:49:34 admin-auth-ldap-31 slapd[27043]: connection_get(25): got connid=1001
Nov 19 12:49:34 admin-auth-ldap-31 slapd[27043]: connection_read(25): checking for input on id=1001
Nov 19 12:49:35 admin-auth-ldap-31 slapd[27043]: 25r
Nov 19 12:49:35 admin-auth-ldap-31 slapd[27043]:
Nov 19 12:49:35 admin-auth-ldap-31 slapd[27043]: daemon: read active on 25
Nov 19 12:49:35 admin-auth-ldap-31 slapd[27043]: connection_get(25)
Nov 19 12:49:35 admin-auth-ldap-31 slapd[27043]: connection_get(25): got connid=1001
Nov 19 12:49:35 admin-auth-ldap-31 slapd[27043]: connection_read(25): checking for input on id=1001
Nov 19 12:49:35 admin-auth-ldap-31 slapd[27043]: connection_read(25): unable to get TLS client DN, error=49 id=1001
Nov 19 12:49:35 admin-auth-ldap-31 slapd[27043]: conn=1001 fd=25 TLS established tls_ssf=256 ssf=256
Nov 19 12:49:36 admin-auth-ldap-31 slapd[27043]: 25r
Nov 19 12:49:36 admin-auth-ldap-31 slapd[27043]:
Nov 19 12:49:36 admin-auth-ldap-31 slapd[27043]: daemon: read active on 25
Nov 19 12:49:36 admin-auth-ldap-31 slapd[27043]: connection_get(25)
Nov 19 12:49:36 admin-auth-ldap-31 slapd[27043]: connection_get(25): got connid=1001
Nov 19 12:49:36 admin-auth-ldap-31 slapd[27043]: connection_read(25): checking for input on id=1001
Nov 19 12:49:36 admin-auth-ldap-31 slapd[27043]: op tag 0x60, time 1511095776
Nov 19 12:49:36 admin-auth-ldap-31 slapd[27043]: conn=1001 op=0 do_bind
Nov 19 12:49:36 admin-auth-ldap-31 slapd[27043]: >>> dnPrettyNormal: <>
Nov 19 12:49:36 admin-auth-ldap-31 slapd[27043]: <<< dnPrettyNormal: <>, <>
Nov 19 12:49:36 admin-auth-ldap-31 slapd[27043]: conn=1001 op=0 BIND dn="" method=163
Nov 19 12:49:36 admin-auth-ldap-31 slapd[27043]: do_bind: dn () SASL mech GSSAPI
Nov 19 12:49:36 admin-auth-ldap-31 slapd[27043]: ==> sasl_bind: dn="" mech=GSSAPI datalen=617
Nov 19 12:49:37 admin-auth-ldap-31 slapd[27043]:
Nov 19 12:49:54 admin-auth-ldap-31 slapd[27043]:
Nov 19 12:49:55 admin-auth-ldap-31 slapd[27043]:
Nov 19 12:50:26 admin-auth-ldap-31 slapd[27043]: 25r
Nov 19 12:50:26 admin-auth-ldap-31 slapd[27043]:
Nov 19 12:50:26 admin-auth-ldap-31 slapd[27043]: daemon: read active on 25
Nov 19 12:50:26 admin-auth-ldap-31 slapd[27043]: connection_get(25)
Nov 19 12:50:26 admin-auth-ldap-31 slapd[27043]: connection_get(25): got connid=1001
Nov 19 12:50:26 admin-auth-ldap-31 slapd[27043]: connection_read(25): checking for input on id=1001
Nov 19 12:50:26 admin-auth-ldap-31 slapd[27043]: ber_get_next on fd 25 failed errno=0 (Success)
Nov 19 12:50:26 admin-auth-ldap-31 slapd[27043]: connection_read(25): input error=-2 id=1001, closing.
Nov 19 12:50:26 admin-auth-ldap-31 slapd[27043]: connection_closing: readying conn=1001 sd=25 for close
Nov 19 12:50:26 admin-auth-ldap-31 slapd[27043]: connection_close: deferring conn=1001 sd=25
Nov 19 12:50:27 admin-auth-ldap-31 slapd[27043]:
Nov 19 12:50:28 admin-auth-ldap-31 slapd[27043]:
So nothing obvious that I can see. Which is reasonable, because
“eventually” it worked on the previous LDAP server, so can’t be
a slapd problem. But I was hoping someone that have tried this
on OS or behind a HAProxy setup might be able to shed some
light on this.
PS. I’ve done the exact same thing at work, in AWS and there it
works just fine. So I’m fairly certain it’s something with OS/HAProxy,
but I don’t know how to debug that bit..
4 years, 11 months
Re: [LMDB] Large transactions
by Howard Chu
Jürgen Baier wrote:
> Hi,
>
> I have a question about LMDB (I hope this is the right mailing list for such a
> question).
>
> I'm running a benchmark (which is similar to my intended use case) which does
> not behave as I hoped. I store 1 billion key/value pairs in a single LMDB
> database. _In a single transaction._ The keys are MD5 hash codes from random
> data (16 bytes) and the value is the string "test".
> The documentation about mdb_page_spill says (as far as I understand) that this
> function is called to prevent MDB_TXN_FULL situations. Does this mean that my
> transaction is simply too large to be handled efficiently by LMDB?
Yes.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
4 years, 11 months
slapd-crash in web directory?
by Dave Horsfall
Not a problem as such, but I happened to notice this in my Apache log:
164.132.162.164 - - [03/Dec/2017:11:57:00 +1100] "GET /slapd-crash/ HTTP/1.1" 404 210 "-" "Mozilla/5.0 (compatible; AhrefsBot/5.2; +http://ahrefs.com/robot/)"
AhrefsBot, for the uninitiated, is a web spider that tends to ignore such
bothersome conventions as "content=noindex" and "robots.txt" etc, but
anyway...
Is it common practice to store SLAPD coredumps where every man and his dog
can retrieve them? Note that I am not suggesting that OpenLDAP does this
(but somebody obviously is), but I'm curious to know how widespread this
silly practice is.
--
Dave Horsfall DTM (VK2KFU) "Those who don't understand security will suffer."
4 years, 11 months
Re: [LMDB] Large transactions
by Howard Chu
Jürgen Baier wrote:
> Hi,
>
> thanks for the answer. However, I still have a follow-up question on this
> benchmark.
>
> When I add 1 billion key/value pairs (16 byte MD5) to the LMDB database (in a
> single transaction (but I also get similar results when I add the same data in
> multiple transactions)) I get the following results:
>
> Windows, without MDB_WRITEMAP: 46h
> Windows, with MDB_WRITEMAP: 6h (!)
> Linux (ext4), without MDB_WRITEMAP: 75h
> Linux (ext4), with MDB_WRITEMAP: 73h
>
> MDB_WRITEMAP seems to have a huge impact on write performance on Windows, but
> on Linux I do not see similar improvements.
>
> So I have two questions:
>
> 1) Could the the difference between Linux and Windows performance regarding
> the MDB_WRITEMAP option be related to the fact that LMDB currently uses sparse
> files on Linux, but not on Windows?
Unlikely.
> 2) Is there a way to speed up Linux? Is there a way to pre-allocate the
> data.mdb on startup?
Try it and see. Use the env fd with fallocate(2).
> Thanks,
>
> Jürgen
>
>
> On 21.11.17 21:17, Howard Chu wrote:
>> Jürgen Baier wrote:
>>> Hi,
>>>
>>> I have a question about LMDB (I hope this is the right mailing list for
>>> such a question).
>>>
>>> I'm running a benchmark (which is similar to my intended use case) which
>>> does not behave as I hoped. I store 1 billion key/value pairs in a single
>>> LMDB database. _In a single transaction._ The keys are MD5 hash codes from
>>> random data (16 bytes) and the value is the string "test".
>>
>>> The documentation about mdb_page_spill says (as far as I understand) that
>>> this function is called to prevent MDB_TXN_FULL situations. Does this mean
>>> that my transaction is simply too large to be handled efficiently by LMDB?
>>
>> Yes.
>>
>
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
4 years, 11 months
