openldap-bugs

openldap-bugs@openldap.org

1 participants
20967 discussions

Re: AW: AW: (ITS#8777) [LMDB] Closing read cursor uses already freed transaction (MDB_VL32)
by hyc＠symas.com 09 Feb '18

09 Feb '18

Markus Junginger wrote: > I perfectly see that MDB_VL32 is the superior as a feature. On the other > hand I'm a bit worried that it may hold up LMDB development for use cases > you refer to as obsolete yourself. > > While there are workarounds for this particular bug (which we applied), There was never a bug here. The doc is quite explicit http://www.lmdb.tech/doc/group__mdb.html#ga73a5938ae4c3239ee11efa07eb22b882 After txn_abort the only valid call on a still-open cursor is cursor_renew, not cursor_close. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

AW: AW: (ITS#8777) [LMDB] Closing read cursor uses already freed transaction (MDB_VL32)
by markus＠greenrobot.de 09 Feb '18

09 Feb '18

I perfectly see that MDB_VL32 is the superior as a feature. On the other hand I'm a bit worried that it may hold up LMDB development for use cases you refer to as obsolete yourself. While there are workarounds for this particular bug (which we applied), I wouldn't be surprised if MDB_VL32 still needs some stabilization after it will be officially released to get real exposure in the real world. Also, for future changes, the MDB_VL32 "fork" will have to be adjusted as well.

1 0

Re: AW: (ITS#8777) [LMDB] Closing read cursor uses already freed transaction (MDB_VL32)
by hyc＠symas.com 09 Feb '18

09 Feb '18

markus(a)greenrobot.de wrote: >> When using MDB_VL32 to access a 64bit DB on a 32bit CPU, the DB cannot fit >> into a single mmap. >> [...] > > Long story short: I did not understand MDB_VL32 in the first place. I > thought it would just enable 64 bit sizes. Pointless, if you're still limited to 2GB. It would only eat up 4 extra bytes overhead in every internal integer. >> Ideally, you should quit using obsolete 32bit CPUs. > > Of course. Think about mobile and IoT however: Here, there are still 32 bit > CPUs and it's not uncommon to even make 64 bit ARM CPUs operate in 32 bit > mode to reduce binary size etc. > > I'd like to propose a different approach to 64 bits on 32 bit CPUs; > something like this: > #ifdef MDB_SIZE64 > typedef uint64_t mdb_size_t; > ... > > Why? MDB_VL32 adds complexity and it would take additional coding and > testing efforts to get it production ready. If you had used the API as intended, with txn_reset/renew as I originally suggested, there would be no problem. > > I think MDB_SIZE64 might be a better trade-off. Of course this would only > work with smaller DBs files (< 2GB/4GB). But I don't see a huge scenario for > 32 bit CPUs being able to work with huge files anyway. The reason MDB_VL32 exists at all is because of exactly this - people wanted to use 32 bit CPUs with 64 bit databases. > The main reason I see > to use 64 bits consistently across CPU architecture is binary compatibility. > This is a big thing for Android (shipping apps with prefilled DBs). Also it > enables getting DB files from 32 bit devices and open those on the desktop, > etc. > > What do you think? Bad idea. MDB_VL32 provides *full* compatibility between 32bit and 64bit CPUs *regardless of filesize*. Your suggestion will only cause confusion, as people wonder why their DB dies after reaching 2GB in size. It offers none of the benefits of 64bit capability, while still paying all of the cost in terms of storage overhead. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

AW: (ITS#8777) [LMDB] Closing read cursor uses already freed transaction (MDB_VL32)
by markus＠greenrobot.de 09 Feb '18

09 Feb '18

> When using MDB_VL32 to access a 64bit DB on a 32bit CPU, the DB cannot fit > into a single mmap. > [...] Long story short: I did not understand MDB_VL32 in the first place. I thought it would just enable 64 bit sizes. > Ideally, you should quit using obsolete 32bit CPUs. Of course. Think about mobile and IoT however: Here, there are still 32 bit CPUs and it's not uncommon to even make 64 bit ARM CPUs operate in 32 bit mode to reduce binary size etc. I'd like to propose a different approach to 64 bits on 32 bit CPUs; something like this: #ifdef MDB_SIZE64 typedef uint64_t mdb_size_t; ... Why? MDB_VL32 adds complexity and it would take additional coding and testing efforts to get it production ready. I think MDB_SIZE64 might be a better trade-off. Of course this would only work with smaller DBs files (< 2GB/4GB). But I don't see a huge scenario for 32 bit CPUs being able to work with huge files anyway. The main reason I see to use 64 bits consistently across CPU architecture is binary compatibility. This is a big thing for Android (shipping apps with prefilled DBs). Also it enables getting DB files from 32 bit devices and open those on the desktop, etc. What do you think? Thanks, Markus PS.: Here's the full patch (adding MDB_SIZE64), which runs fine at least with our test suite: lmdb.h @@ -187,7 +187,15 @@ # define MDB_FMT_Z "z" /**< printf/scanf format modifier for size_t */ #endif -#ifndef MDB_VL32 +#if defined(MDB_VL32) && defined(MDB_SIZE64) +#error either define MDB_VL32 OR MDB_SIZE64 +#endif +#ifdef MDB_SIZE64 +typedef uint64_t mdb_size_t; +# define MDB_SIZE_MAX SIZE_MAX /**< max #mdb_size_t */ +# define MDB_PRIy(t) PRI##t##64 +# define MDB_SCNy(t) SCN##t##64 +#elif !defined(MDB_VL32)) /** Unsigned type used for mapsize, entry counts and page/transaction IDs. * * It is normally size_t, hence the name. Defining MDB_VL32 makes it

1 0

(ITS#8802) ldappasswd ppolicy
by matsl＠irf.se 08 Feb '18

08 Feb '18

Full_Name: Mats Luspa Version: openldap-2.4.40+dfsg OS: 3.16.0-4-686-pae #1 SMP Debian 3.16.43-2+deb8u5 (2017-09-19) i686 GNU/Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (2001:6b0:27:cc:2740:e692:a5b1:4b0f) Hello! When you are using ppolicy password changed are recorded in pwdHistory attribute. ldappasswd can't be used due to that. It checks of some reason that pwdHistory not exists before it changes that password. If pwdHistory exists then the ldappaswd can't change the password. Here's the log file: 2018-02-08T09:42:45+01:00 mailserver slapd[725]: bdb_modify_internal: replace userPassword 2018-02-08T09:42:45+01:00 mailserver slapd[725]: bdb_modify_internal: replace pwdChangedTime 2018-02-08T09:42:45+01:00 mailserver slapd[725]: bdb_modify_internal: add pwdHistory 2018-02-08T09:42:45+01:00 mailserver slapd[725]: bdb_modify_internal: replace pwdChangedTime 2018-02-08T09:42:45+01:00 mailserver slapd[725]: bdb_modify_internal: add pwdHistory 2018-02-08T09:42:45+01:00 mailserver slapd[725]: bdb_modify_internal: 20 modify/add: pwdHistory: value #0 already exists 2018-02-08T09:42:45+01:00 mailserver slapd[725]: send_ldap_result: err=20 matched="" text="modify/add: pwdHistory: value #0 already exists" /Regards Mats

1 0

Re: (ITS#8801) Fix pending CSN queue
by Howard Chu 06 Feb '18

06 Feb '18

The problem was noticed because the accesslog overlay was invoking a spurious slap_graduate_commit_csn() on a CSN that had already been graduated by the underlying backend.

1 0

(ITS#8801) Fix pending CSN queue
by hyc＠openldap.org 06 Feb '18

06 Feb '18

Full_Name: Howard Chu Version: 2.4 OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (83.136.45.75) Submitted by: hyc The CSN queue tracks in-progress write operations. It uses connID and opID to identify a CSN queue entry that corresponds to a given operation. This is error-prone in the case of syncrepl, which always uses the same connID and opID for all of its internal operations. Should just use the Operation pointer, which is always unique and unambiguous.

1 0

Re: (ITS#8800) MMR: out of date master will ignore history of its own changes
by quanah＠symas.com 30 Jan '18

30 Jan '18

--On Tuesday, January 30, 2018 9:04 PM +0000 quanah(a)openldap.org wrote: > Full_Name: Quanah Gibson-Mount > Version: 2.4.45 > OS: Linux > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (47.208.148.239) > > > Did the following test: > > 4-way MMR setup, database populated from an initial DB that has history > to it Per Howard's suggestion, I commented out lines 2390-2396 in syncprov.c, which allows the master to get fed back its own operations on startup. On the plus side, the database indeed get all the operations sent back. There were 1187 entries in the accesslog DBs of all 4 nodes. On the minus side, while server IDs 2,3, and 4 all agreed on the final resulting contextCSN, serverID 1 did not. Which then broke the ability for the nodes to communicate with each other (err=53) Last entry on serverIDs 1/2/3/4: dn: reqStart=20180130233019.000017Z,cn=accesslog objectClass: auditModify reqStart: 20180130233019.000017Z reqEnd: 20180130233019.000018Z reqType: modify reqSession: 1 reqAuthzID: cn=ldaproot,dc=xxx,dc=yyy reqDN: dc=xxx,dc=yyy reqResult: 0 reqMod: contextCSN:= 20180130233019.035885Z#000000#001#000000 reqMod: contextCSN:= 20171130222521.056018Z#000000#002#000000 reqMod: contextCSN:= 20171130222318.939265Z#000000#003#000000 reqMod: contextCSN:= 20171203041258.811473Z#000000#004#000000 reqEntryUUID: 156eb8cc-18e9-1027-80e5-d3f2010890dc contextCSNs on 2/3/4: base contextCSN: 20180130233019.035885Z#000000#001#000000 contextCSN: 20171130222521.056018Z#000000#002#000000 contextCSN: 20171130222318.939265Z#000000#003#000000 contextCSN: 20171203041258.811473Z#000000#004#000000 accesslog contextCSN: 20180130233019.035885Z#000000#001#000000 contextCSN: 20171130222521.056018Z#000000#002#000000 contextCSN: 20171130222318.939265Z#000000#003#000000 contextCSN: 20171203041258.811473Z#000000#004#000000 contextCSNs on 1: base contextCSN: 20180130233019.035885Z#000000#001#000000 contextCSN: 20171130222521.056018Z#000000#002#000000 contextCSN: 20171130222318.939265Z#000000#003#000000 contextCSN: 20171203041258.811473Z#000000#004#000000 accesslog contextCSN: 20180130233016.137867Z#000000#001#000000 contextCSN: 20171130222521.056018Z#000000#002#000000 contextCSN: 20171130222318.939265Z#000000#003#000000 contextCSN: 20171203041258.811473Z#000000#004#000000 Note that the contextCSN is correct on the database root, but incorrect in the accesslog entry. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

(ITS#8800) MMR: out of date master will ignore history of its own changes
by quanah＠openldap.org 30 Jan '18

30 Jan '18

Full_Name: Quanah Gibson-Mount Version: 2.4.45 OS: Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (47.208.148.239) Did the following test: 4-way MMR setup, database populated from an initial DB that has history to it Make several thousand MODs to serverID 1 only Stop serverID 1 wipe its database reload serverID 1 from the initial DB Expected result: serverID 1 REFRESHes from the other servers to sync up its database Actual result: serverID 1 updates its contextCSN to the last change op, but does not sync any actual changes back, leaving it out of sync with the rest of the cluster.

1 0

Re: (ITS#8100) Empty accesslog causes issues with delta-syncrepl MMR configurations
by quanah＠symas.com 29 Jan '18

29 Jan '18

--On Monday, January 29, 2018 10:23 AM -0800 Quanah Gibson-Mount <quanah(a)symas.com> wrote: > I'll continue testing for the other half of the fix (Deleting all but the > most recent entry from the accesslog during purge) This part appears to work as desired. I set the purge interval to 10 minutes, checking every 5 minutes. Made changes. All entries but the most recent one were removed after 15 minutes went by. Made more changes, did the same wait period, and again, all entries but the most recent were removed during the next cleanup interval. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs