openldap-bugs June 2015

openldap-bugs@openldap.org

21 participants
71 discussions

Re: (ITS#8048) back-sock/slapo-sock issue
by michael＠stroeder.com 30 Jun '15

30 Jun '15

This is a multi-part message in MIME format. --------------090704060507030702010600 Content-Type: text/plain; charset=ISO-8859-2 Content-Transfer-Encoding: quoted-printable hyc(a)symas.com wrote: > hyc(a)symas.com wrote: >> michael(a)stroeder.com wrote: >=20 >>> Can I raise the debug level? Log level trace does not really show the= ori=3D >>> ginal >>> byte sequence received by the external listener. >> >> Currently there is no debug code for this. >=20 > Actually, if you use debug level SHELL you'll get a plaintext dump of t= he=20 > received text. If you direct this into a file you can examine it with a= hex=20 > editor. I tested this again running as overlay with -d SHELL. See attachment of t= he log excerpt as Python string representation read with open(file,'rb') bef= ore. You can find *exactly* the lines from this response string generated by t= he external listener: 'RESULT\nmsgid: 1\ncode: 49\ninfo: HOTP value wrong\n\n' All lines starting with sock search reading line: (RESULT\n) (msgid: 1\n) (code: 49\n) (info: HOTP value wrong\n) Excerpt of slapd's log repeated here for better readability in the ITS. P= lease note the extra space after "text=3D" which is not what the external liste= ner returned. ------------------------------------------------ 5592d039 sock search reading line (RESULT ) 5592d039 sock search reading line (msgid: 1 ) 5592d039 sock search reading line (code: 49 ) 5592d039 sock search reading line (info: HOTP value wrong ) 5592d039 sock search reading line ( ) 5592d039 str2result (msgid: 1 code: 49 info: HOTP value wrong ) unknown 5592d039 str2result ( ) unknown 5592d039 conn=3D1000 op=3D0 RESULT tag=3D97 err=3D49 text=3D HOTP value w= rong ------------------------------------------------ Ciao, Michael. --------------090704060507030702010600 Content-Type: text/plain; charset=UTF-8; name="string" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="string" NTU5MmQwMzkgY29ubj0xMDAwIGZkPTEyIEFDQ0VQVCBmcm9tIFBBVEg9L29wdC91aS92YXIv b3BlbmxkYXAvcnVuL2xkYXBpLXAgKFBBVEg9L29wdC91aS92YXIvb3BlbmxkYXAvcnVuL2xk YXBpLXApXG41NTkyZDAzOSBjb25uPTEwMDAgb3A9MCBCSU5EIGRuPSJ1aWQ9c2VvZmosY249 dGVzdCxvdT1leGFtcGxlIiBtZXRob2Q9MTI4XG41NTkyZDAzOSBjb25uPTEwMDEgZmQ9MTQg QUNDRVBUIGZyb20gUEFUSD0vb3B0L3VpL3Zhci9vcGVubGRhcC9ydW4vbGRhcGktcCAoUEFU SD0vb3B0L3VpL3Zhci9vcGVubGRhcC9ydW4vbGRhcGktcClcbjU1OTJkMDM5IGNvbm49MTAw MSBvcD0wIEJJTkQgZG49IiIgbWV0aG9kPTE2M1xuNTU5MmQwMzkgY29ubj0xMDAxIG9wPTAg QklORCBhdXRoY2lkPSJnaWROdW1iZXI9MTAwK3VpZE51bWJlcj0xMDAwLGNuPXBlZXJjcmVk LGNuPWV4dGVybmFsLGNuPWF1dGgiIGF1dGh6aWQ9ImdpZE51bWJlcj0xMDArdWlkTnVtYmVy PTEwMDAsY249cGVlcmNyZWQsY249ZXh0ZXJuYWwsY249YXV0aCJcbjU1OTJkMDM5IGNvbm49 MTAwMSBvcD0wIEJJTkQgZG49ImNuPXJvb3Qsb3U9ZXhhbXBsZSIgbWVjaD1FWFRFUk5BTCBz YXNsX3NzZj0wIHNzZj0yNTZcbjU1OTJkMDM5IGNvbm49MTAwMSBvcD0wIFJFU1VMVCB0YWc9 OTcgZXJyPTAgdGV4dD1cbjU1OTJkMDM5IGNvbm49MTAwMSBvcD0xIFNSQ0ggYmFzZT0idWlk PXNlb2ZqLGNuPXRlc3Qsb3U9ZXhhbXBsZSIgc2NvcGU9MCBkZXJlZj0wIGZpbHRlcj0iKCYo b2JqZWN0Q2xhc3M9b2F0aEhPVFBVc2VyKShvYXRoSE9UUENvdW50ZXI9Kikob2F0aFNlY3Jl dD0qKSh8KCEodWlOb3RCZWZvcmU9KikpKHVpTm90QmVmb3JlPD0yMDE1MDYzMDE3MjIwMVop KSh8KCEodWlOb3RBZnRlcj0qKSkodWlOb3RBZnRlcj49MjAxNTA2MzAxNzIyMDFaKSkpIlxu NTU5MmQwMzkgY29ubj0xMDAxIG9wPTEgU1JDSCBhdHRyPW9hdGhIT1RQQ291bnRlciBvYXRo SE9UUFBhcmFtcyBvYXRoU2VjcmV0IG9hdGhUb2tlblNlcmlhbE51bWJlciBvYmplY3RDbGFz cyBwd2RQb2xpY3lTdWJlbnRyeSB1c2VyUGFzc3dvcmRcbjU1OTJkMDM5IGNvbm49MTAwMSBv cD0xIFNFQVJDSCBSRVNVTFQgdGFnPTEwMSBlcnI9MCBuZW50cmllcz0xIHRleHQ9XG41NTky ZDAzOSBjb25uPTEwMDEgb3A9MiBTUkNIIGJhc2U9ImNuPW9hdGgtcG9saWN5LWhvdHAtdXNl cnMsY249ZXhhbXBsZSxvdT1leGFtcGxlIiBzY29wZT0wIGRlcmVmPTAgZmlsdGVyPSIob2Jq ZWN0Q2xhc3M9b2F0aEhPVFBQYXJhbXMpIlxuNTU5MmQwMzkgY29ubj0xMDAxIG9wPTIgU1JD SCBhdHRyPW9hdGhIT1RQQ291bnRlck1heCBvYXRoSE9UUExvb2tBaGVhZCBvYXRoT1RQTGVu Z3RoXG41NTkyZDAzOSBjb25uPTEwMDEgb3A9MiBTRUFSQ0ggUkVTVUxUIHRhZz0xMDEgZXJy PTAgbmVudHJpZXM9MSB0ZXh0PVxuNTU5MmQwMzkgc29jayBzZWFyY2ggcmVhZGluZyBsaW5l IChSRVNVTFRcbilcbjU1OTJkMDM5IHNvY2sgc2VhcmNoIHJlYWRpbmcgbGluZSAobXNnaWQ6 IDFcbilcbjU1OTJkMDM5IHNvY2sgc2VhcmNoIHJlYWRpbmcgbGluZSAoY29kZTogNDlcbilc bjU1OTJkMDM5IHNvY2sgc2VhcmNoIHJlYWRpbmcgbGluZSAoaW5mbzogSE9UUCB2YWx1ZSB3 cm9uZ1xuKVxuNTU5MmQwMzkgc29jayBzZWFyY2ggcmVhZGluZyBsaW5lIChcbilcbjU1OTJk MDM5IHN0cjJyZXN1bHQgKG1zZ2lkOiAxXG5jb2RlOiA0OVxuaW5mbzogSE9UUCB2YWx1ZSB3 cm9uZ1xuXG4pIHVua25vd25cbjU1OTJkMDM5IHN0cjJyZXN1bHQgKFxuKSB1bmtub3duXG41 NTkyZDAzOSBjb25uPTEwMDAgb3A9MCBSRVNVTFQgdGFnPTk3IGVycj00OSB0ZXh0PSBIT1RQ IHZhbHVlIHdyb25nXG41NTkyZDAzOSBjb25uPTEwMDAgb3A9MSBVTkJJTkRcbjU1OTJkMDM5 IGNvbm49MTAwMCBmZD0xMiBjbG9zZWRcbg== --------------090704060507030702010600--

1 0

Re: (ITS#8181) LMDB page leaks etc when treating DBs as data
by h.b.furuseth＠usit.uio.no 30 Jun '15

30 Jun '15

On 30. juni 2015 14:52, hyc(a)symas.com wrote: > We have already documented this. "Database names are kept as keys in the > unnamed database." "Hey, that may mean I can drop a DB without having to open it and mess with LMDB's DBI handling! Let me try this..." > It should be obvious that if you muck with the record of an > existing key, things will change, therefore you should not muck with those keys. Lots of people think differently than you or me. We're not going to agree. Not on this, and not on what you said in your next mail. I don't see the point of rehashing those arguments yet again, and I didn't intend to. I was just ITSing a commit before pushing, _after_ asking you about it. I thought. -- Hallvard

1 0

Re: (ITS#8181) LMDB page leaks etc when treating DBs as data
by hyc＠symas.com 30 Jun '15

30 Jun '15

1 0

Re: (ITS#8181) LMDB page leaks etc when treating DBs as data
by hyc＠symas.com 30 Jun '15

30 Jun '15

Hallvard Breien Furuseth wrote: > On 29/06/15 23:05, hyc(a)symas.com wrote: >> Seriously, why aren't we just saying "don't do this" and moving on? > > OK, then document it. Maybe keeping it simple, call > mixing mainDB data and named DBs a user error and warn this > can break the DB. We have already documented this. "Database names are kept as keys in the unnamed database." It should be obvious that if you muck with the record of an existing key, things will change, therefore you should not muck with those keys. >> There are >> lots of stupid things you can do with software. It's a waste of time and >> energy to prevent them all. > > We totally disagree, starting with what is stupid. But we knew > that. That's I asked which parts of "mdb/bundle" to push and > which you rejected, we must have misunderstood each other. I > thought I was just ITSing this commit properly before pushing. I just don't see this happening in practice. Most applications will never use subDBs. It's not like some random person is going to come along and open an arbitrary LMDB file created by some other application and start mucking with it, without knowing the meaning of what's inside. Embedded database libraries are used in specific contexts by their enclosing application. They don't just get random ops thrown at them from 3rd party code. The designer of the enclosing app will design a schema/whatever set of tables to use and that scheme will be fixed for the life of the application. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8181) LMDB page leaks etc when treating DBs as data
by h.b.furuseth＠usit.uio.no 30 Jun '15

30 Jun '15

On 29/06/15 23:05, hyc(a)symas.com wrote: > Seriously, why aren't we just saying "don't do this" and moving on? OK, then document it. Maybe keeping it simple, call mixing mainDB data and named DBs a user error and warn this can break the DB. > There are > lots of stupid things you can do with software. It's a waste of time and > energy to prevent them all. We totally disagree, starting with what is stupid. But we knew that. That's I asked which parts of "mdb/bundle" to push and which you rejected, we must have misunderstood each other. I thought I was just ITSing this commit properly before pushing.

1 0

Re: (ITS#8048) back-sock/slapo-sock issue
by hyc＠symas.com 30 Jun '15

30 Jun '15

hyc(a)symas.com wrote: > michael(a)stroeder.com wrote: >> Can I raise the debug level? Log level trace does not really show the ori= >> ginal >> byte sequence received by the external listener. > > Currently there is no debug code for this. Actually, if you use debug level SHELL you'll get a plaintext dump of the received text. If you direct this into a file you can examine it with a hex editor. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8048) back-sock/slapo-sock issue
by hyc＠symas.com 30 Jun '15

30 Jun '15

michael(a)stroeder.com wrote: > Howard Chu wrote: >> michael(a)stroeder.com wrote: >>> michael(a)stroeder.com wrote: >>>> Null-bytes really needed? >>> >>> Hmm, with null-bytes the wrong result code is returned to the LDAP cli= > ent. >>> >>> Without null-bytes the correct result code is returned to the client b= > ut slapd >>> complains about unknown string: >>> >>> 'RESULT\nmsgid: 1\ncode: 49\nmatched: uid=3Dwuqww,cn=3Dampua,ou=3Dampu= > a\ninfo: >>> NOK\n\n' >>> >>> results in log message: >>> >>> 559174df str2result (msgid: 1 >>> code: 0 >>> >>> ) unknown >>> 559174df str2result ( >>> ) unknown >> =20 >> I would almost suspect you were running on Windows. None of the behavio= > r >> you're reporting occurs for me using the searchexample.pl script. > > Your script does not return any real 'RESULT..' responses. Sure it does: print "dn: cn=test, dc=example, dc=com\n"; print "cn: test\n"; print "objectclass: cnobject\n"; print "\n"; print "RESULT\n"; print "code: 0\n"; print "info: answered by process $$\n"; And the result at the client: violino:~/OD/hobj/tests> ../clients/tools/ldapsearch -x -H ldap://:9011 -b dc=example,dc=com # extended LDIF # # LDAPv3 # base <dc=example,dc=com> with scope subtree # filter: (objectclass=*) # requesting: ALL # # test, example.com dn: cn=test,dc=example,dc=com cn: test objectClass: cnobject # search result search: 2 result: 0 Success text: answered by process 3077 # numResponses: 2 # numEntries: 1 Notice there are no embedded NUL characters. Only newlines. > I also have less issues when just returning 'CONTINUE\n'. > >> Sounds like your problem is a python runtime configuration, not a slapd= > >> bug. > > Not on Windows and no run-time configuration involved. > > Look at the raw string representations I've posted. That string goes dire= > ctly > to SocketServer.BaseRequestHandler.request.sendall(response_str). > > Can I raise the debug level? Log level trace does not really show the ori= > ginal > byte sequence received by the external listener. Currently there is no debug code for this. At any rate, the code uses fgets(3) which stops reading at a newline character. The fact that your string isn't being parsed correctly implies either libc fgets is broken inside your slapd process (pretty unlikely) or your socket sender isn't really sending newline characters. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8181) LMDB page leaks etc when treating DBs as data
by hyc＠symas.com 29 Jun '15

29 Jun '15

h.b.furuseth(a)usit.uio.no wrote: > Full_Name: Hallvard B Furuseth > Version: LMDB_0.9.15 > OS: > URL: > Submission from: (NULL) (81.191.45.5) > Submitted by: hallvard > > > The user can overwrite or delete a named DB record > with normal data. This leaks the pages of the DB. > Also the new non-DB can then be treated as a DB. > A fix is coming (from my branch mdb/bundle). Seriously, why aren't we just saying "don't do this" and moving on? There are lots of stupid things you can do with software. It's a waste of time and energy to prevent them all. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8048) back-sock/slapo-sock issue
by michael＠stroeder.com 29 Jun '15

29 Jun '15

Howard Chu wrote: > michael(a)stroeder.com wrote: >> michael(a)stroeder.com wrote: >>> Null-bytes really needed? >> >> Hmm, with null-bytes the wrong result code is returned to the LDAP cli= ent. >> >> Without null-bytes the correct result code is returned to the client b= ut slapd >> complains about unknown string: >> >> 'RESULT\nmsgid: 1\ncode: 49\nmatched: uid=3Dwuqww,cn=3Dampua,ou=3Dampu= a\ninfo: >> NOK\n\n' >> >> results in log message: >> >> 559174df str2result (msgid: 1 >> code: 0 >> >> ) unknown >> 559174df str2result ( >> ) unknown >=20 > I would almost suspect you were running on Windows. None of the behavio= r > you're reporting occurs for me using the searchexample.pl script. Your script does not return any real 'RESULT..' responses. I also have less issues when just returning 'CONTINUE\n'. > Sounds like your problem is a python runtime configuration, not a slapd= > bug. Not on Windows and no run-time configuration involved. Look at the raw string representations I've posted. That string goes dire= ctly to SocketServer.BaseRequestHandler.request.sendall(response_str). Can I raise the debug level? Log level trace does not really show the ori= ginal byte sequence received by the external listener. Ciao, Michael.

1 0

(ITS#8181) LMDB page leaks etc when treating DBs as data
by h.b.furuseth＠usit.uio.no 29 Jun '15

29 Jun '15

Full_Name: Hallvard B Furuseth Version: LMDB_0.9.15 OS: URL: Submission from: (NULL) (81.191.45.5) Submitted by: hallvard The user can overwrite or delete a named DB record with normal data. This leaks the pages of the DB. Also the new non-DB can then be treated as a DB. A fix is coming (from my branch mdb/bundle).

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs June 2015