openldap-bugs

openldap-bugs@openldap.org

1 participants
21003 discussions

Re: (ITS#7844) LMDB Delete Cursor inconsistencies
by armon.dadgar＠gmail.com 13 May '14

13 May '14

--5372ac85_8edbdab_1271 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline =46or now, we have application code to retry the delete until no further = rows are removed. Still, it would be nice to have this resolved (and tested) in master=21 Best Regards, Armon Dadgar =46rom:=C2=A0David Wilson dw=40hmmz.org Reply:=C2=A0David Wilson dw=40hmmz.org Date:=C2=A0May 13, 2014 at 3:41:08 PM To:=C2=A0armon.dadgar=40gmail.com armon.dadgar=40gmail.com Cc:=C2=A0openldap-its=40openldap.org openldap-its=40openldap.org Subject:=C2=A0 Re: (ITS=237844) LMDB Delete Cursor inconsistencies =20 This sounds very much like a known regression in mdb.master in the past =20 few months. If you are still affected, please retest against LMDB 0.9.11 = =20 which at least was not affected by the regression. =20 David =20 --5372ac85_8edbdab_1271 Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline <html><head><style>body=7Bfont-family:Helvetica,Arial;font-size:13px=7D</= style></head><body style=3D=22word-wrap: break-word; -webkit-nbsp-mode: s= pace; -webkit-line-break: after-white-space;=22><div id=3D=22bloop=5Fcust= omfont=22 style=3D=22font-family:Helvetica,Arial;font-size:13px; color: r= gba(0,0,0,1.0); margin: 0px; line-height: auto;=22>=46or now, we have app= lication code to retry the delete until no further rows are removed.</div= ><div id=3D=22bloop=5Fcustomfont=22 style=3D=22font-family:Helvetica,Aria= l;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;= =22>Still, it would be nice to have this resolved (and tested) in master=21= </div><div id=3D=22bloop=5Fcustomfont=22 style=3D=22font-family:Helvetica= ,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: = auto;=22><br></div> <div id=3D=22bloop=5Fsign=5F1400024102388026112=22 cl= ass=3D=22bloop=5Fsign=22><div style=3D=22font-family:helvetica,arial;font= -size:13px=22>Best Regards,</div><div style=3D=22font-family:helvetica,ar= ial;font-size:13px=22>Armon Dadgar<br></div></div> <div style=3D=22color:= black=22><br>=46rom: <span style=3D=22color:black=22>David Wilson</s= pan> <a href=3D=22mailto:dw=40hmmz.org=22>dw=40hmmz.org</a><br>Reply:&nbs= p;<span style=3D=22color:black=22>David Wilson</span> <a href=3D=22mailto= :dw=40hmmz.org=22>dw=40hmmz.org</a><br>Date: <span style=3D=22color:= black=22>May 13, 2014 at 3:41:08 PM</span><br>To: <span style=3D=22c= olor:black=22>armon.dadgar=40gmail.com</span> <a href=3D=22mailto:armon.d= adgar=40gmail.com=22>armon.dadgar=40gmail.com</a><br>Cc: <span style= =3D=22color:black=22>openldap-its=40openldap.org</span> <a href=3D=22mail= to:openldap-its=40openldap.org=22>openldap-its=40openldap.org</a><br>Subj= ect: <span style=3D=22color:black=22> Re: (ITS=237844) LMDB Delete C= ursor inconsistencies <br></span></div><br> <blockquote type=3D=22cite=22= class=3D=22clean=5Fbq=22><span><div><div></div><div>This sounds very muc= h like a known regression in mdb.master in the past <br>few months. If you are still affected, please retest against LMDB 0.9= .11 <br>which at least was not affected by the regression. <br> <br> <br>David <br></div></div></span></blockquote></body></html> --5372ac85_8edbdab_1271--

1 0

Re: (ITS#7853) Improve safety of mdb_env_close
by hyc＠symas.com 13 May '14

13 May '14

dw(a)botanicus.net wrote: > Full_Name: David Wilson > Version: LMDB 0.9.11 > OS: > URL: https://raw.githubusercontent.com/dw/py-lmdb/master/misc/readers_mrb_env.pa… > Submission from: (NULL) (178.238.153.20) > > > Currently if a user (wilfully or accidentally, say, through composition of > third party libs) opens the same LMDB environment twice within a process, and > maintains active read transactions at the time one mdb_env_close() is called, > all reader slots will be deallocated in all environments due to the logic > around line 4253 that unconditionally clears reader slots based on PID. > > I'd like to avoid this in py-lmdb, and seems there are a few alternatives to > fix it: > > > 1) Tell users not to do that (doc fix, nobody reads docs) This is already in the doc - don't open the same env twice within one process. > 2) Externally maintain a set of (st_dev, st_ino) keys representing already > opened environments, and check any attempt to open an environment against this > list, failing if data.mdb's key already appears. That would require global library state, which is definitely unwanted. > 3) Disable the mdb_env_close() cleanup loop when MDB_NOTLS is enabled. Since no > reader slot will ever exist in TLS in this mode, the existing mdb_env_close() > doc "All transactions, databases, and cursors must already be closed before > calling this function." ensures no readers exist at close time, and the loop > need not run. Could do this, but since it's an incomplete solution, not sure there's any point. > 4) Modify lock.mdb to include MDB_env* address within the process, and update > mdb_env_close() to invalidate only readers associated with the environment > being closed. I dislike using the MDB_env* as an opaque publicly visible > cookie, but perhaps storing this field might be reused for other things in > future. Hmm. I guess this could work. I think storing the MDB_env* is harmless since the pointer is meaningless to any other process. > Option 3 lets the user wilfully mix py-lmdb Environment objects within the > process, since py-lmdb always enables NOTLS, but it does not fix the case where > the user is integrating a Python application with a C library with some opaque > interface, and both the application and the library independently have need to > access the environment. Allowing multiple independent libraries to open the same environment would consume redundant chunks of address space. For large enough DBs this may simply fail. Unfortunately the semantics of fcntl locks prevents us from even detecting that this problem has occurred. > This kind of scenario regularly crops up in huge software projects, the user > may not even be aware the library uses LMDB internally. In that case, the user > is again exposed to having their reader slots silently become deallocated > through no fault of their own. > > > Option 4 is the least aesthetic, but has all the benefits of option 3 in > addition to preventing the "accidental integration" scenario. I've attached a > patch for it (against 0.9.11), although I completely understand if it does not > get applied. :) > > One alternative would be instead of using MDB_env* address, using a > monotonically increasing counter based on a static global, say, "mrb_token", > but this might break in the case two copies of LMDB were somehow linked in the > application. (It's easy to argue the user is an idiot in this case ;) Global state is verboten. > > > David > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7844) LMDB Delete Cursor inconsistencies
by dw＠hmmz.org 13 May '14

13 May '14

This sounds very much like a known regression in mdb.master in the past few months. If you are still affected, please retest against LMDB 0.9.11 which at least was not affected by the regression. David

1 0

Re: (ITS#7853) Improve safety of mdb_env_close
by dw＠botanicus.net 13 May '14

13 May '14

It's worth mentioning that this is not a theoretical issue, I've already hit it while writing unit tests.

1 0

(ITS#7853) Improve safety of mdb_env_close
by dw＠botanicus.net 13 May '14

13 May '14

Full_Name: David Wilson Version: LMDB 0.9.11 OS: URL: https://raw.githubusercontent.com/dw/py-lmdb/master/misc/readers_mrb_env.pa… Submission from: (NULL) (178.238.153.20) Currently if a user (wilfully or accidentally, say, through composition of third party libs) opens the same LMDB environment twice within a process, and maintains active read transactions at the time one mdb_env_close() is called, all reader slots will be deallocated in all environments due to the logic around line 4253 that unconditionally clears reader slots based on PID. I'd like to avoid this in py-lmdb, and seems there are a few alternatives to fix it: 1) Tell users not to do that (doc fix, nobody reads docs) 2) Externally maintain a set of (st_dev, st_ino) keys representing already opened environments, and check any attempt to open an environment against this list, failing if data.mdb's key already appears. 3) Disable the mdb_env_close() cleanup loop when MDB_NOTLS is enabled. Since no reader slot will ever exist in TLS in this mode, the existing mdb_env_close() doc "All transactions, databases, and cursors must already be closed before calling this function." ensures no readers exist at close time, and the loop need not run. 4) Modify lock.mdb to include MDB_env* address within the process, and update mdb_env_close() to invalidate only readers associated with the environment being closed. I dislike using the MDB_env* as an opaque publicly visible cookie, but perhaps storing this field might be reused for other things in future. Option 3 lets the user wilfully mix py-lmdb Environment objects within the process, since py-lmdb always enables NOTLS, but it does not fix the case where the user is integrating a Python application with a C library with some opaque interface, and both the application and the library independently have need to access the environment. This kind of scenario regularly crops up in huge software projects, the user may not even be aware the library uses LMDB internally. In that case, the user is again exposed to having their reader slots silently become deallocated through no fault of their own. Option 4 is the least aesthetic, but has all the benefits of option 3 in addition to preventing the "accidental integration" scenario. I've attached a patch for it (against 0.9.11), although I completely understand if it does not get applied. :) One alternative would be instead of using MDB_env* address, using a monotonically increasing counter based on a static global, say, "mrb_token", but this might break in the case two copies of LMDB were somehow linked in the application. (It's easy to argue the user is an idiot in this case ;) David

1 0

Re: (ITS#7852) problems in memberof overlay
by quanah＠zimbra.com 12 May '14

12 May '14

--On May 12, 2014 at 10:20:24 AM +0000 efbt(a)hotmail.com wrote: > Full_Name: jeff.gao > Version: 2.4.23 The ITS system is for reporting software bugs, not for asking usage questions. Please redirect your question to the openldap-technical(a)openldap.org list. I would note that the OpenLDAP 2.4.23 build shipped by RHEL is utterly broken. I strongly advise you obtain a working build. If you are unable to build OpenLDAP yourself, you can obtain a working build from Symas or the LTB project. This ITS will be closed. --Quanah -- Quanah Gibson-Mount Server Architect Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

(ITS#7852) problems in memberof overlay
by efbt＠hotmail.com 12 May '14

12 May '14

Full_Name: jeff.gao Version: 2.4.23 OS: centos URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (114.242.123.100) hi. I configured the function of 'memberof overlay' these days, but At first I used the methods on the webpage of http://www.openldap.org/doc/admin24/overlays.html, I add the codes of "overlay memeberof" in the file of slapd.conf,then I started the slapd service, the system give me an error "overlay "memberof" not found" . second,I googled the internet and found an article on this webpage:http://www.redmine.org/projects/redmine/wiki/RedmineLDAP,so I create two LDIF files as follow: the first file : dn: cn=module,cn=config objectClass: olcModuleList cn: module olcModulePath: /usr/lib/ldap olcModuleLoad: memberof the second file: dn: olcOverlay=memberof,olcDatabase={1}hdb,cn=config objectClass: olcMemberOf objectClass: olcOverlayConfig objectClass: olcConfig objectClass: top olcOverlay: memberof olcMemberOfDangling: ignore olcMemberOfRefInt: TRUE olcMemberOfGroupOC: groupOfNames olcMemberOfMemberAD: member olcMemberOfMemberOfAD: memberOf after I entered the command "ldapadd -x -D 'cn=Manager,dc=example,dc=com' -W -f 1.ldif",the system give an error:"ldap_add: Insufficient access (50)" please help me ,thanks.

1 0

Re: (ITS#7851) [PATCH] buffer overrun in password checkers with malformed hash
by ryan＠nardis.ca 11 May '14

11 May '14

Sorry, I missed a line from the patch. Corrected one: ftp://ftp.openldap.org/incoming/rtandy_20140511_fix-passwd-b64-buffer_v2.pa…

1 0

(ITS#7851) [PATCH] buffer overrun in password checkers with malformed hash
by ryan＠nardis.ca 11 May '14

11 May '14

Full_Name: Ryan Tandy Version: master, commit 141f1680 OS: Ubuntu 14.04 URL: ftp://ftp.openldap.org/incoming/rtandy_20140511_fix-passwd-b64-buffer.patch Submission from: (NULL) (24.68.121.206) The password checkers call lutil_b64_pton like this: rc = lutil_b64_pton(passwd->bv_val, orig_pass, passwd->bv_len); However, the size of orig_pass is actually only 3/4 of passwd->bv_len. Normally this isn't a problem; however, if the length of the password hash is not a multiple of 4, then LUTIL_BASE64_DECODE_LEN rounds down, orig_pass gets a shorter buffer allocated than is actually needed, and it's possible for lutil_b64_pton to write past its end. (How can that happen? An admin could mess up copy-pasting from slappasswd to userPassword/olcRootPW, as could a user with write access to their own userPassword who decided to set it manually.) Example: $ slappasswd -s secret -h '{SSHA}' {SSHA}86ks4vJqfruLKer9skuEpTmgPxcaqSLe If we lose the last character ('e') and try to authenticate against the length 31 string, clang's AddressSanitizer notices: ==15335==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x603000049736 at pc 0xef0720 bp 0x7fa9b38d1c10 sp 0x7fa9b38d1c08 WRITE of size 1 at 0x603000049736 thread T2 #0 0xef071f in lutil_b64_pton (libraries/liblutil/base64.c:225) #1 0xe9547c in chk_ssha1 (libraries/liblutil/passwd.c:508) #2 0xe90a09 in lutil_passwd (libraries/liblutil/passwd.c:327) #3 0x7d5e1c in slap_passwd_check (servers/slapd/passwd.c:529) [snip ...] 0x603000049736 is located 0 bytes to the right of 22-byte region [0x603000049720,0x603000049736) allocated by thread T2 here: #0 0x43c9a4 in malloc (/opt/openldap/libexec/slapd+0x43c9a4) #1 0x112903f in ber_memalloc_x (libraries/liblber/memory.c:228) #2 0x1129396 in ber_memalloc (libraries/liblber/memory.c:244) #3 0xe952ea in chk_ssha1 (libraries/liblutil/passwd.c:503) #4 0xe90a09 in lutil_passwd (libraries/liblutil/passwd.c:327) #5 0x7d5e1c in slap_passwd_check (servers/slapd/passwd.c:529) [snip...] (Built with ./configure CC=clang CFLAGS="-O0 -g -fsanitize=address") Patch in URL. (Apparently the apr1 passwd module has CRLF endings; sorry for the noisy diff.) BTW, I noticed in the same routines that orig_pass is allocated with + 1 to the required length, as if it were to be null-terminated, but that's never done. Is that intentional? (AFAICS ber_memalloc doesn't zero memory, maybe I'm wrong.) thanks, Ryan

1 0

Re: (ITS#7850) slapd crashes on modrdn to an attr with no equality matching rule
by ryan＠nardis.ca 09 May '14

09 May '14

On Fri, May 9, 2014 at 11:33 AM, Howard Chu <hyc(a)symas.com> wrote: > Thanks for the report. Should be fixed now in git master, please test. $ ldapmodrdn -x -D cn=root,dc=example,dc=com -W 'cn=Ryan Tandy,dc=example,dc=com' 'jpegPhoto=test' Enter LDAP Password: Rename Result: Naming violation (64) Additional info: naming attribute has no equality matching rule Confirmed fixed.

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs