openldap-bugs

openldap-bugs@openldap.org

1 participants
20966 discussions

(ITS#9008) module rpath incorrect
by ondra＠openldap.org 17 Apr '19

17 Apr '19

Full_Name: Ondrej Kuznik Version: re24/master OS: Linux URL: https://github.com/mistotebe/openldap/tree/its9008 Submission from: (NULL) (82.10.24.68) Modules that link against libraries not already present in slapd will only try to look in the rpaths encoded in the module, not in slapd. And there is no point encoding $(moduledir) there, since we never install anything of substance there. All the while the libraries we need probably live in $(libdir). The linked patch fixes this and makes it possible for $(moduledir) (the path modules will be installed into) to be set at configure time. IPR notice: The linked files are derived from OpenLDAP Software. All of the modifications to OpenLDAP Software represented in the following patches were developed Symas corp. Symas corp. has not assigned rights and/or interest in this work to any party. I, Ondrej Kuznik am authorized by Symas corp. to release this work under the following terms. Symas corp. hereby place the following modifications to OpenLDAP Software (and only these modifications) into the public domain. Hence, these modifications may be freely used and/or redistributed for any purpose with or without attribution and/or other notice.

1 0

Re: (ITS#9007) In writemap mode, a loose page can end up in me_dpages, causing a segfault on mdb_env_close.
by hyc＠symas.com 16 Apr '19

16 Apr '19

kriszyp(a)gmail.com wrote: > Full_Name: Kristopher William Zyp > Version: LMDB 0.9.23 > OS: Windows > URL: fhttps://github.com/kriszyp/node-lmdb/commit/dc290553acb57fa3f2d6d88a0d5e02… > Submission from: (NULL) (71.199.6.148) > > > In writemap mode in LMDB, it seems that a loose page can end up in me_dpages, > causing a segfault on mdb_env_close or on writes after mdb_env_set_mapsize. Thanks for the report, fix applied to mdb.master > > If I understand correctly, in MDB_WRITEMAP mode, dirty pages should never enter > into me_dpages, since dirty pages directly refer to the mapped memory, and > me_dpages are allocated from the heap (for reuse). However, I think a dirty page > that has become a loose page can end up in me_dpages; it seems that when > mdb_freelist_save() is called, it iterates through the loose pages and can end > up calling mdb_dpage_free at > https://github.com/LMDB/lmdb/blob/mdb.RE/0.9/libraries/liblmdb/mdb.c#L3114, > allowing the loose page to end up in me_dpages. This means that me_dpages ends > up with a reference to data in the memory map. And this becomes apparent if you > call mdb_env_close or do a mdb_env_set_mapsize triggering a segfault when it > attempts to free a memory map'ed page or access previously unmapped page. > > Everywhere else in the code, the MDB_WRITEMAP flag prevents any calls to > mdb_dpage_free (or mdb_dlist_free) in writemap mode (which I assume is > intentional), except in mdb_freelist_save. It seems like the mdb_dpage_free call > just needs to be moved up a few lines to the else block of the MDB_TXN_WRITEMAP > conditional, so that it also won't be called in writemap mode. > > I apologize I can't provide a more isolated test case. This combination actually > seems to be pretty rare occurrence, and very difficult to reproduce, I only > occasionally observe it happening with large amounts of operations with frequent > closing/resizing of the env. And I don't understand the internals to be 100% > confident of this, and perhaps I am misunderstanding this code path. But, it > does seems like this fix prevents these crashes for us, and seems like ensuring > there are no mdb_dpage_free calls in writemap mode (only mdb_page_free calls) is > the legitimate intention of the code. > > The attached URL contains the fix as a patch (against the node-lmdb project). > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#9007) In writemap mode, a loose page can end up in me_dpages, causing a segfault on mdb_env_close.
by kriszyp＠gmail.com 15 Apr '19

15 Apr '19

Full_Name: Kristopher William Zyp Version: LMDB 0.9.23 OS: Windows URL: fhttps://github.com/kriszyp/node-lmdb/commit/dc290553acb57fa3f2d6d88a0d5e02… Submission from: (NULL) (71.199.6.148) In writemap mode in LMDB, it seems that a loose page can end up in me_dpages, causing a segfault on mdb_env_close or on writes after mdb_env_set_mapsize. If I understand correctly, in MDB_WRITEMAP mode, dirty pages should never enter into me_dpages, since dirty pages directly refer to the mapped memory, and me_dpages are allocated from the heap (for reuse). However, I think a dirty page that has become a loose page can end up in me_dpages; it seems that when mdb_freelist_save() is called, it iterates through the loose pages and can end up calling mdb_dpage_free at https://github.com/LMDB/lmdb/blob/mdb.RE/0.9/libraries/liblmdb/mdb.c#L3114, allowing the loose page to end up in me_dpages. This means that me_dpages ends up with a reference to data in the memory map. And this becomes apparent if you call mdb_env_close or do a mdb_env_set_mapsize triggering a segfault when it attempts to free a memory map'ed page or access previously unmapped page. Everywhere else in the code, the MDB_WRITEMAP flag prevents any calls to mdb_dpage_free (or mdb_dlist_free) in writemap mode (which I assume is intentional), except in mdb_freelist_save. It seems like the mdb_dpage_free call just needs to be moved up a few lines to the else block of the MDB_TXN_WRITEMAP conditional, so that it also won't be called in writemap mode. I apologize I can't provide a more isolated test case. This combination actually seems to be pretty rare occurrence, and very difficult to reproduce, I only occasionally observe it happening with large amounts of operations with frequent closing/resizing of the env. And I don't understand the internals to be 100% confident of this, and perhaps I am misunderstanding this code path. But, it does seems like this fix prevents these crashes for us, and seems like ensuring there are no mdb_dpage_free calls in writemap mode (only mdb_page_free calls) is the legitimate intention of the code. The attached URL contains the fix as a patch (against the node-lmdb project).

1 0

(ITS#9006) sha2 module is not usable
by aditya1606＠gmail.com 10 Apr '19

10 Apr '19

Full_Name: Aditya Version: 2.4.43 OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (131.228.66.13) I am not able to load the module due to this undefined symbol. $ nm pw-sha2.so | grep -i "lutil_passwd_add" U lutil_passwd_add

1 0

Re: (ITS#9004) LDAP_VENDOR_VERSION shall not be 0
by hyc＠symas.com 10 Apr '19

10 Apr '19

dpa-openldap(a)aegee.org wrote: > Full_Name: &#1044;&#1080;&#1083;&#1103;&#1085; &amp= ;#1055;&#1072;&#1083;&#1072;&#1091;&#1079;&#1086;= &#1074; > Version: git/master > OS: Linux > URL:=20 > Submission from: (NULL) (141.12.129.132) >=20 >=20 > Per https://gitlab.gnome.org/GNOME/evolution-data-server/issues/99: >=20 > =E2=80=A2 Please do not set LDAP_VENDOR_VERSION to 0, when compiling fr= om git/master, > but use reasonable default: a number bigger or equal to 20000. Like 20= 500 for > LDAP 2.5. >=20 >=20 git master is a development branch and not intended for production use. Use a tagged release or the release engineering branch. Closing this ITS. --=20 -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#9005) LDAP_VENDOR_VERSION shall not be 0
by dpa-openldap＠aegee.org 10 Apr '19

10 Apr '19

Full_Name: &#1044;&#1080;&#1083;&#1103;&#1085; &#1055;&#1072;&#1083;&#1072;&#1091;&#1079;&#1086;&#1074; Version: git/master OS: Linux URL: Submission from: (NULL) (141.12.129.132) Per https://gitlab.gnome.org/GNOME/evolution-data-server/issues/99: Please do not set LDAP_VENDOR_VERSION to 0, when compiling from git/master, but use reasonable default: a number bigger or equal to 20000. Like 20500 for LDAP 2.5.

1 0

(ITS#9004) LDAP_VENDOR_VERSION shall not be 0
by dpa-openldap＠aegee.org 10 Apr '19

10 Apr '19

1 0

Re: (ITS#8875) [Patch] Performance problems in back-mdb with large DITs and many aliases
by henrik.bohnenkamp＠ionos.com 07 Apr '19

07 Apr '19

On Thu, Apr 04, 2019 at 05:32:16PM +0100, Howard Chu wrote: Hi Howard, thanks for your comments. There is now a new version of patches available in my github repository. Apart from addressing your comments, I have rebased the patches against the current master branch (quite a lot activity in the last two months, I have noticed). In particular, the new function mdb_get_aliases now also uses the global size variables for IDL dimensions, rather than the previous CPP constants. As for your comments: > > Hi, thanks for the report and investigation. I've briefly reviewed your > patches. Please squash your #3 into #2, there's no reason to preserve that. done > > We don't use "inline" in this code. If the compiler is smart enough it will > inline automatically anyway. Please drop that from your patch. done > > No need to keep the old mdb_idscope invocation commented out in patch #2. > Just delete it, we have the git history. done > > In patch #1 delete.c your patch line #145 is incorrect. You should be > using is_entry_alias(e) instead of (op->ora_e.) The entry in question is not > part of the op request, only its DN is in the Delete request. Ah, that was a mistake when squashing my commits down to the two patches... patch 1 introduced the faulty line with op->ora_e, patch 2 changed op->ora_e to e. I have cleaned that up. > > Not sure why you dropped nsubs from the diskNode definition. Are you sure you > haven't broken the nsubs processing in this patch? Note that the nsubs declaration was always commented out and had thus only a decorative function. Anyway, I have added this comment back to reduce the foot print of the patches. The current master branch with the two patches builds without problems. I have run the mdb tests in the tests directory... the last test was not applied due to missing software. All other 74 tests passed. (ubuntu 18.4). Best regards Henrik > > > -- Henrik Bohnenkamp

1 0

Re: (ITS#8996) Please supply a pkg-config file for libldap
by hugh.mcmaster＠outlook.com 05 Apr '19

05 Apr '19

In reference to #8996, I have prepared a patchset that adds support for generating pkg-config files at configure-time. There are three patches, one each for libldap, libldap_r and liblber. The attached patch files are derived from OpenLDAP Software. All of the modifications to OpenLDAP Software represented in the following patch(es) were developed by Hugh McMaster <hugh.mcmaster at outlook.com>. I have not assigned rights and/or interest in this work to any party. I, Hugh McMaster, hereby place the following modifications to OpenLDAP Software (and only these modifications) into the public domain. Hence, these modifications may be freely used and/or redistributed for any purpose with or without attribution and/or other notice. ftp://ftp.openldap.org/incoming/hugh-mcmaster-190405.patch ftp://ftp.openldap.org/incoming/hugh-mcmaster-190405-02.patch ftp://ftp.openldap.org/incoming/hugh-mcmaster-190405-03.patch Please review the above patches and let me know if you require any changes. Thank you Hugh

1 0

Re: (ITS#8875) [Patch] Performance problems in back-mdb with large DITs and many aliases
by hyc＠symas.com 04 Apr '19

04 Apr '19

henrik.bohnenkamp(a)ionos.com wrote: > On Fri, Jan 25, 2019 at 06:31:41AM -0800, Quanah Gibson-Mount wrote: >> --On Friday, January 25, 2019 1:51 PM +0000 hbohnenkamp(a)united-internet.de >> wrote: >> Hi, thanks for the report and investigation. I've briefly reviewed your patches. Please squash your #3 into #2, there's no reason to preserve that. We don't use "inline" in this code. If the compiler is smart enough it will inline automatically anyway. Please drop that from your patch. No need to keep the old mdb_idscope invocation commented out in patch #2. Just delete it, we have the git history. In patch #1 delete.c your patch line #145 is incorrect. You should be using is_entry_alias(e) instead of (op->ora_e.) The entry in question is not part of the op request, only its DN is in the Delete request. Not sure why you dropped nsubs from the diskNode definition. Are you sure you haven't broken the nsubs processing in this patch? > > Hi, > > I have now done quite a lot of testing in the way described in my > email from January (Followup 3). I used version 2.4.47, not the HEAD > of the openldap git master branch. > > I have found one bug, the one mentioned already in the same email. I > have added a fix for that bug to the patch directory in > https://github.com/hbo/openldap-mdb-deref-problem > > I have come across the regression with the back-hdb backend again, > which I also described in my previous email already. I am now > convinced that this regression is really a problem with the back-hdb > implementation, and I can show it without using my patched > implementation of back-mdb. The demonstration requires a bit more work > to be really presentable and should in any case go into a different > bug report. > > Apart from these two issues I have not seen any regressions, and I am > now quite confident that my patch would work reliable in the > environment of my company... no guarantees for anything else :-) > > I have seen that Quanah has suggested to look at ITS#8875 for the > 2.4.48 release. Whether there will be a fix in 2.4.48 and whether it > will be based on my patch is of course up to the lead developers (the > change in the DB schema is certainly a strong point against), but this > patch can perhaps serve as something to test the actual fix against. > > Best regards > Henrik > > > -- > Henrik Bohnenkamp > > > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs