- openldap-bugs - openldap.org

Re: (ITS#8023) slappasswd with sha2 overlay can generate hashes but not salted hashes
by hyc＠symas.com 22 Jan '15

22 Jan '15

quanah(a)zimbra.com wrote: > --On Tuesday, January 13, 2015 7:24 PM +0000 Jonathan Price > <freebsd(a)jonathanprice.org> wrote: > >> I do apologise for the confusion, I'll try to clarify below: >> >> Here is the command you ran successfully: >> /opt/zimbra/openldap/sbin/slappasswd -h >> '{SSHA512}' -o module-path=/opt/zimbra/openldap/sbin/openldap -o >> module-load=pw-sha2 -s test >> {SSHA512}TSwAWmK3sv42RbAasugMPR8d7GLozXtKU00v5Jdd4ebmXBsOpt5We5HNkXxFfy5 >> Ptaoa/KUsmTV5484NA3UmrHrOpyUVnEh9 >> >> Here is an example of me running just a plain SHA512 >> slappasswd -h '{SHA512}' -o module-path=/usr/local/libexec/openldap -o >> module-load=pw-sha2 >> {SHA512}7iaw3Ur350mqGo7jwQrpkj9hiYB3Lkc/iBml1JQODbJ6wYX4oOHV+E+IvIh/1nsUN >> zLDBMxfqa2Ob1f1ACio/w== >> >> And here is an example of me running a salted SHA512 (SSHA512) >> slappasswd -h '{SSHA512}' -o module-path=/usr/local/libexec/openldap -o >> module-load=pw-sha2 -s test >> Password verification failed. >> >> I hope this helps to clarify. > > Yes, thank you. So I'm using 2.4.39. There were some minor changes to > slapd-sha2 in 2.4.40. I will see if I can reproduce the issue with current > RE24. I have a FreeBSD 9 VM here with 2.4.40 installed from ports. Both SHA512 and SSHA512 work fine on it. Doesn't look to me like there's any OpenLDAP bug here, this is one for the FreeBSD folks to sort out. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8023) slappasswd with sha2 overlay can generate hashes but not salted hashes
by freebsd＠jonathanprice.org 22 Jan '15

22 Jan '15

Sorry for the slow response, but I have made some progress with the issue= .=0A=0A(as an aside, I installed a build from LTB, and unfortunately it d= oes not=0Acontain this overlay)=0A=0AI have detailed my findings (includi= ng some trawling through the source) =0Aover on the FreeBSD bug tracker, = as I suspect it could well be a platform=0Arelated issue. Nonetheless, it= might be worth reading:=0Ahttps://bugs.freebsd.org/bugzilla/show_bug.cgi= ?id=3D197004=0A=0AJanuary 14 2015 4:31 PM, "Quanah Gibson-Mount" <quanah@= zimbra.com> wrote: =0A> --On Wednesday, January 14, 2015 11:00 AM +0000 f= reebsd(a)jonathanprice.org=0A> wrote:=0A> =0A>> Hi,=0A>> =0A>> I tried 2.4.= 39 under FreeBSD and still had the same issue.=0A>> =0A>> I have also tri= ed the packages for both CentOS 7 and Debian Wheezy, but=0A>> unfortunate= ly neither of them include the SHA2 overlay by default.=0A>> =0A>> Finall= y, I tried installing zimbra-core and zimbra-ldap under CentOS.=0A>> When= I used this installation, it worked successfully.=0A>> =0A>> I ran slapd= -V on the zimbra installation, and it's 2.4.39. However,=0A>> based on i= t still not working on 2.4.39 on FreeBSD it appears to have=0A>> narrowed= it down to two reasons: - An issue with the packaging under=0A>> FreeBSD= =0A>> - The functionality is specific to Zimbra=0A>> =0A>> The next step = in the process to narrow this down is to do a manual=0A>> compilation on = CentOS, including the SHA2 overlay. If this works, then it=0A>> would con= firm it to be a FreeBSD issue, and if it doesn't work that would=0A>> str= ongly suggest that Zimbra has modified something.=0A> =0A> You could simp= ly grab the LTB project builds. I'm pretty sure they build=0A> out the co= ntrib modules.=0A> =0A> In any case, I already noted that Zimbra doesn't = patch anything in OpenLDAP=0A> that would affect this area.=0A> =0A> --Qu= anah=0A> =0A> --=0A> =0A> Quanah Gibson-Mount=0A> Platform Architect=0A> = Zimbra, Inc. =0A> _______________________________=0A> =0A> Zimbra :: the = leader in open source messaging and collaboration

1 0

Re: (ITS#8018) a lot of warnings building with -Wall
by h.b.furuseth＠usit.uio.no 21 Jan '15

21 Jan '15

On 22/01/15 06:13, leo(a)yuriev.ru wrote: > I unexpectedly detected that the variadic macro feature of C99 was > already used in OpenLDAP 2.4 ;) Look a bit closer: > grep -R __VA_ARGS__ * > build/ltmain.sh:# define DEBUG(format, ...) fprintf(stderr, format, > __VA_ARGS__) > contrib/ldapc++/ltmain.sh:# define DEBUG(format, ...) fprintf(stderr, > format, __VA_ARGS__) ltmain.sh is a shell/code library generated by GNU libtool. Lots of stuff in there which we do not use. > libraries/liblmdb/mdb.c: fprintf(stderr, "%s:%d " fmt "\n", > mdb_func_, __LINE__, __VA_ARGS__) > servers/slapd/back-bdb/back-bdb.h:#define > BDB_LOG_PRINTF(env,txn,fmt,...) > (env)->log_printf((env),(txn),(fmt),__VA_ARGS__) > servers/slapd/back-bdb/back-bdb.h:#define > BDB_LOG_PRINTF(env,txn,fmt,...) > __db_logmsg((env),(txn),"DIAGNOSTIC",0,(fmt),__VA_ARGS__) Debug macros which are off by default and fall back to ((void)0) or somesuch.

1 0

Re: (ITS#8018) a lot of warnings building with -Wall
by leo＠yuriev.ru 21 Jan '15

21 Jan '15

I unexpectedly detected that the variadic macro feature of C99 was already used in OpenLDAP 2.4 ;) grep -R __VA_ARGS__ * build/ltmain.sh:# define DEBUG(format, ...) fprintf(stderr, format, __VA_ARGS__) contrib/ldapc++/ltmain.sh:# define DEBUG(format, ...) fprintf(stderr, format, __VA_ARGS__) libraries/liblmdb/mdb.c: fprintf(stderr, "%s:%d " fmt "\n", mdb_func_, __LINE__, __VA_ARGS__) servers/slapd/back-bdb/back-bdb.h:#define BDB_LOG_PRINTF(env,txn,fmt,...) (env)->log_printf((env),(txn),(fmt),__VA_ARGS__) servers/slapd/back-bdb/back-bdb.h:#define BDB_LOG_PRINTF(env,txn,fmt,...) __db_logmsg((env),(txn),"DIAGNOSTIC",0,(fmt),__VA_ARGS__)

1 0

Re: (ITS#8032) Update man page for ldap_get_option(3)
by hyc＠symas.com 21 Jan '15

21 Jan '15

gnoe(a)symas.com wrote: > Full_Name: Greg Noe > Version: 2.4.x > OS: N/A > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (63.142.209.94) > > > Additional language is needed for the LDAP_OPT_API_INFO option in the > ldap_get_option(3) man page. Fixed in git master. > > A sentence needs to be added stating that the ldapai_info_version field must be > initialized in the LDAPAPIInfo struct before using it to call > ldap_get_option(NULL,LDAP_OPT_API_INFO,&info). > > Current > ------------------------------ > LDAP_OPT_API_INFO > Fills-in a LDAPAPIInfo; outvalue must be a LDAPAPIInfo *, point- > ing to an already allocated struct. This is a read-only option. > > Suggested > ------------------------------ > LDAP_OPT_API_INFO > Fills-in a LDAPAPIInfo; outvalue must be a LDAPAPIInfo *, point- > ing to an already allocated struct. The ldapai_info_version field > must be initialized before making the call. This is a read-only > option. > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8032) Update man page for ldap_get_option(3)
by gnoe＠symas.com 21 Jan '15

21 Jan '15

Full_Name: Greg Noe Version: 2.4.x OS: N/A URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (63.142.209.94) Additional language is needed for the LDAP_OPT_API_INFO option in the ldap_get_option(3) man page. A sentence needs to be added stating that the ldapai_info_version field must be initialized in the LDAPAPIInfo struct before using it to call ldap_get_option(NULL,LDAP_OPT_API_INFO,&info). Current ------------------------------ LDAP_OPT_API_INFO Fills-in a LDAPAPIInfo; outvalue must be a LDAPAPIInfo *, point- ing to an already allocated struct. This is a read-only option. Suggested ------------------------------ LDAP_OPT_API_INFO Fills-in a LDAPAPIInfo; outvalue must be a LDAPAPIInfo *, point- ing to an already allocated struct. The ldapai_info_version field must be initialized before making the call. This is a read-only option.

1 0

Re: (ITS#8031) suffixmassage does not seem to be working with olc configuration
by quanah＠zimbra.com 21 Jan '15

21 Jan '15

--On Wednesday, January 21, 2015 4:31 PM +0000 fmgre-01(a)yahoo.fr wrote: > Full_Name: fred aka gnafou > Version: 2.4.31 This release is years old, and has numeorus known problems. Use a current release. --Quanah -- Quanah Gibson-Mount Platform Architect Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

(ITS#8031) suffixmassage does not seem to be working with olc configuration
by fmgre-01＠yahoo.fr 21 Jan '15

21 Jan '15

Full_Name: fred aka gnafou Version: 2.4.31 OS: debian 7 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (194.167.193.57) Hello I am trying to configure a meta database ; it seems that the uri ( described in the doc ) is called olcDbUri in the olc configuration ; and i cannot find the olc keyword for suffixmassage Thanks Fred

1 0

Re: (ITS#7145) cn=Connection 0, cn=Connections, cn=Monitor received twice
by hyc＠symas.com 21 Jan '15

21 Jan '15

varkoly(a)suse.com wrote: > Hallo, > > I've the same issue on SLES11-SP3. openLDAP version 2.4.26 > Was this issue fixed already? No, there was never any information provided on how to reproduce it. But 2.4.26 is quite old, do you still see this occurring using 2.4.40? -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7145) cn=Connection 0, cn=Connections, cn=Monitor received twice
by varkoly＠suse.com 21 Jan '15

21 Jan '15

Hallo, I've the same issue on SLES11-SP3. openLDAP version 2.4.26 Kernel 3.0.101-0.15-default sles11-sp3-krb5-master:~ # ldapsearch -Y EXTERNAL -H ldapi:// -b cn=Connections,cn=Monitor -s sub '(cn=Connection 0)' SASL/EXTERNAL authentication started SASL username: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth SASL SSF: 0 # extended LDIF # # LDAPv3 # base <cn=Connections,cn=Monitor> with scope subtree # filter: (cn=Connection 0) # requesting: ALL # # Connection 0, Connections, Monitor dn: cn=Connection 0,cn=Connections,cn=Monitor objectClass: monitorConnection cn: Connection 0 # Connection 0, Connections, Monitor dn: cn=Connection 0,cn=Connections,cn=Monitor objectClass: monitorConnection cn: Connection 0 # search result search: 2 result: 0 Success # numResponses: 3 # numEntries: 2 Was this issue fixed already? -- Peter Varkoly Sr. Developer SUSE Linux Enterprise Applications SUSE LINUX Products GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany

1 0

Re: (ITS#8030) How to find the LDAP User last logon time
by michael＠stroeder.com 21 Jan '15

21 Jan '15

Please note that the issue tracker is not for support requests. Usage questions should be sent to the openldap-technical mailing list where more people are subscribed who can answer and learn as well. raj.chinnu81(a)gmail.com wrote: > I am searching the option to find out the last logon time for LDAP user with > much no positive findings, > > I found that using 'lastbind overly' we can find the user last login time, if > yes > Please help me with the procedure to get this work. Yes, slapo-lastbind is the way to go. Please re-post on openldap-technical mailing list. Ciao, Michael.

1 0

(ITS#8030) How to find the LDAP User last logon time
by raj.chinnu81＠gmail.com 20 Jan '15

20 Jan '15

Full_Name: rajagopal Version: -2.4.39-3.el7 OS: RHEL7 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (119.225.42.74) Hi, I am searching the option to find out the last logon time for LDAP user with much no positive findings, I found that using 'lastbind overly' we can find the user last login time, if yes Please help me with the procedure to get this work. Thanks & Regards Raj

1 0

(ITS#8029) How to find the LDAP User last logon time
by raj.chinnu81＠gmail.com 20 Jan '15

20 Jan '15

Full_Name: rajagopal Version: -2.4.39-3.el7 OS: RHEL7 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (119.225.42.74) Hi, I am searching the option to find out the last logon time for LDAP user with much no positive findings, I found that using 'lastbind overly' we can find the user last login time, if yes Please help me with the procedure to get this work. Thanks & Regards Raj

1 0

Re: (ITS#8026) net-nds/openldap - powerpc-apple-darwin9-gcc: Undefined symbols for architecture ppc: "_slapi_pblock_set"
by hyc＠symas.com 20 Jan '15

20 Jan '15

daniel.ibnzayd(a)inquisitor.com wrote: > Full_Name: Daniel Ibn Zayd > Version: 2.4.38-r2 > OS: PPC > URL: https://bugs.gentoo.org/show_bug.cgi?id=536306 > Submission from: (NULL) (166.84.1.2) > > > Both of the latest builds marked stable for ppc (openldap-2.4.35-r1 and > openldap-2.4.38-r2) error out when they get to the libaddrdnvalues-plugin: > > * Building contrib-module: addrdnvalues plugin > Undefined symbols for architecture ppc: > "_slapi_pblock_set", referenced from: > _addrdnvalues_preop_init in cclXT5Cy.o > "_slapi_log_error", referenced from: > _addrdnvalues_preop_init in cclXT5Cy.o > _addrdnvalues_preop_add in cclXT5Cy.o > "_slapi_pblock_get", referenced from: > _addrdnvalues_preop_add in cclXT5Cy.o > "_slapi_entry_add_rdn_values", referenced from: > _addrdnvalues_preop_add in cclXT5Cy.o > "_slapi_send_ldap_result", referencefrfrom: > _addrdnvalues_preop_add in cclXT5Cy.o > "_ldap_err2string", referenced from: > _addrdnvalues_preop_add in cclXT5Cy.o > ld: symbol(s) not found for architecture ppc > > I've researched the plug-in and its components searching for possible fixes but > have not turned up anything. Any clues here are greatly appreciated. Appears to be a bug in gentoo's build rules for this plugin. The OpenLDAP Project doesn't supply build rules for this plugin, so there's nothing for us to fix here. Closing this ITS. Note that the function of this particular plugin was superseded long ago, there's no reason to even build it. > PS: This is a Gentoo Prefix set up I am working with. > > https://bugs.gentoo.org/show_bug.cgi?id=536306 > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8028) NULL pointer dereference in ldap_new_connection()
by hyc＠symas.com 20 Jan '15

20 Jan '15

rohanskurane(a)gmail.com wrote: > Full_Name: Rohan Kurane > Version: 2.4.40 > OS: BSD 7.2 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (64.80.217.3) > > > In ldap_new_connection() in request.c, while setting up a connection to the LDAP > server, there is a possibility of dereferencing a NULL pointer in > lc->locnn_server Fixed in git master. Please don't send HTML emails, they're particularly unreadable with embedded code like this. > > if ( connect ) { > LDAPURLDesc **srvp, *srv = NULL; > > async % LDADAP_BOOL_GET( &ld->ld_options, LDAP_BOOL_CONNECT_ASYNC ); > > for ( srvp = srvlist; *srvp != NULL; srvp = &(*srvp)->lud_next ) { > int rc; > > rc = ldap_int_open_connection( ld, lc, *srvp, async ); > if ( rc != -1 ) { > srv = *srvp; > > 9%9 if ( ld->ld_urllist_proc && ( !async || rc != -2 ) ) { > ld->ld_urllist_proc( ld, srvlist, srvp, ld->ld_urllist_params ); > } > > break; > } > } > > if ( srv == NULL ) { > if ( !use_ldsb ) { > ber_sockbuf_free( lc->lconn_sb ); > %%D > LDAP_FREE( (char *)lc ); > ld->ld_errno = LDAP_SERVER_DOWN; > return( NULL ); > } > > lc->lconn_server = ldap_url_dup( srv ); > } > > ldap_url_dup() does a bunch of malloc's to set up lc->lconn_server. If any of > those malloc's fail, it returns NULL. The code does not check for a NULL > lconn_server pointer and tries to reference lud_exts. That can cause a > segmentation fault. > > if ( connect ) { > #ifdef HAVE_TLS > if ( lc->lconn_server->lud_exts ) { > int rc, ext = find_tls_ext( lc->lconn_server ); > if ( ext ) { > LDAPConn *savedefconn; > > Even thou this should not happen, is this a known issue and are there any plans > to fix the openldap library ? > > Thank you > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8028) NULL pointer dereference in ldap_new_connection()
by rohanskurane＠gmail.com 20 Jan '15

20 Jan '15

Full_Name: Rohan Kurane Version: 2.4.40 OS: BSD 7.2 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (64.80.217.3) In ldap_new_connection() in request.c, while setting up a connection to the LDAP server, there is a possibility of dereferencing a NULL pointer in lc->locnn_server if ( connect ) { LDAPURLDesc **srvp, *srv = NULL; async % LDADAP_BOOL_GET( &ld->ld_options, LDAP_BOOL_CONNECT_ASYNC ); for ( srvp = srvlist; *srvp != NULL; srvp = &(*srvp)->lud_next ) { int rc; rc = ldap_int_open_connection( ld, lc, *srvp, async ); if ( rc != -1 ) { srv = *srvp; 9%9 if ( ld->ld_urllist_proc && ( !async || rc != -2 ) ) { ld->ld_urllist_proc( ld, srvlist, srvp, ld->ld_urllist_params ); } break; } } if ( srv == NULL ) { if ( !use_ldsb ) { ber_sockbuf_free( lc->lconn_sb ); %%D LDAP_FREE( (char *)lc ); ld->ld_errno = LDAP_SERVER_DOWN; return( NULL ); } lc->lconn_server = ldap_url_dup( srv ); } ldap_url_dup() does a bunch of malloc's to set up lc->lconn_server. If any of those malloc's fail, it returns NULL. The code does not check for a NULL lconn_server pointer and tries to reference lud_exts. That can cause a segmentation fault. if ( connect ) { #ifdef HAVE_TLS if ( lc->lconn_server->lud_exts ) { int rc, ext = find_tls_ext( lc->lconn_server ); if ( ext ) { LDAPConn *savedefconn; Even thou this should not happen, is this a known issue and are there any plans to fix the openldap library ? Thank you

1 0

Re: (ITS#8027) ldapsearch -E deref=member: crashes slapd
by hyc＠symas.com 19 Jan '15

19 Jan '15

ryan(a)nardis.ca wrote: > Full_Name: Ryan Tandy > Version: master (7df548d), RE24 (2b14bbc) > OS: Debian unstable > URL: > Submission from: (NULL) (142.32.208.227) > > > If you use the deref control but leave the list of requested attributes empty, > slapd crashes. > > ldapsearch [...] -E deref=member: > The ldapsearch manpage implies this probably isn't valid, but it still accepted > it. (FWIW, I tried it just to see whether it would return all attributes or > none.) I couldn't tell from draft-ldap-deref-00 whether an empty attr list is > considered a valid request. > Patched in master to reject a request with an empty attr list. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8027) ldapsearch -E deref=member: crashes slapd
by michael＠stroeder.com 19 Jan '15

19 Jan '15

hyc(a)symas.com wrote: > As I read the grammar in the draft section 2.2 the attributeList is not > OPTIONAL so this is definitely not a valid request. But this invalid request must not crash slapd with slapo-deref installed. Ciao, Michael.

1 0

Re: (ITS#8027) ldapsearch -E deref=member: crashes slapd
by hyc＠symas.com 19 Jan '15

19 Jan '15

ryan(a)nardis.ca wrote: > Full_Name: Ryan Tandy > Version: master (7df548d), RE24 (2b14bbc) > OS: Debian unstable > URL: > Submission from: (NULL) (142.32.208.227) > > > If you use the deref control but leave the list of requested attributes empty, > slapd crashes. > > ldapsearch [...] -E deref=member: > > #0 0x0000000000516ef0 in deref_parseCtrl (op=0x7fffec000940, rs=0x7ffff57eeac0, > ctrl=0x7fffec001238) at deref.c:225 > #1 0x000000000046a84d in slap_parse_ctrl (op=0x7fffec000940, rs=0x7ffff57eeac0, > control=0x7fffec001238, text=0x7ffff57eeae0) > at controls.c:693 > #2 0x000000000046b0f5 in get_ctrls2 (op=0x7fffec000940, rs=0x7ffff57eeac0, > sendres=1, ctag=160) at controls.c:886 > #3 0x000000000046a8ff in get_ctrls (op=0x7fffec000940, rs=0x7ffff57eeac0, > sendres=1) at controls.c:723 > #4 0x000000000042e94e in do_search (op=0x7fffec000940, rs=0x7ffff57eeac0) at > search.c:195 > #5 0x000000000042bdf3 in connection_operation (ctx=0x7ffff57eebf0, > arg_v=0x7fffec000940) at connection.c:1134 > #6 0x000000000042c3a3 in connection_read_thread (ctx=0x7ffff57eebf0, argv=0xb) > at connection.c:1280 > #7 0x0000000000538938 in ldap_int_thread_pool_wrapper (xpool=0x892bc0) at > tpool.c:958 > #8 0x00007ffff79b00a4 in start_thread (arg=0x7ffff57ef700) at > pthread_create.c:309 > #9 0x00007ffff76e4ccd in clone () at > ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 > > (line numbers are from master) > > The ldapsearch manpage implies this probably isn't valid, but it still accepted > it. (FWIW, I tried it just to see whether it would return all attributes or > none.) I couldn't tell from draft-ldap-deref-00 whether an empty attr list is > considered a valid request. As I read the grammar in the draft section 2.2 the attributeList is not OPTIONAL so this is definitely not a valid request. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8027) ldapsearch -E deref=member: crashes slapd
by ryan＠nardis.ca 19 Jan '15

19 Jan '15

Full_Name: Ryan Tandy Version: master (7df548d), RE24 (2b14bbc) OS: Debian unstable URL: Submission from: (NULL) (142.32.208.227) If you use the deref control but leave the list of requested attributes empty, slapd crashes. ldapsearch [...] -E deref=member: #0 0x0000000000516ef0 in deref_parseCtrl (op=0x7fffec000940, rs=0x7ffff57eeac0, ctrl=0x7fffec001238) at deref.c:225 #1 0x000000000046a84d in slap_parse_ctrl (op=0x7fffec000940, rs=0x7ffff57eeac0, control=0x7fffec001238, text=0x7ffff57eeae0) at controls.c:693 #2 0x000000000046b0f5 in get_ctrls2 (op=0x7fffec000940, rs=0x7ffff57eeac0, sendres=1, ctag=160) at controls.c:886 #3 0x000000000046a8ff in get_ctrls (op=0x7fffec000940, rs=0x7ffff57eeac0, sendres=1) at controls.c:723 #4 0x000000000042e94e in do_search (op=0x7fffec000940, rs=0x7ffff57eeac0) at search.c:195 #5 0x000000000042bdf3 in connection_operation (ctx=0x7ffff57eebf0, arg_v=0x7fffec000940) at connection.c:1134 #6 0x000000000042c3a3 in connection_read_thread (ctx=0x7ffff57eebf0, argv=0xb) at connection.c:1280 #7 0x0000000000538938 in ldap_int_thread_pool_wrapper (xpool=0x892bc0) at tpool.c:958 #8 0x00007ffff79b00a4 in start_thread (arg=0x7ffff57ef700) at pthread_create.c:309 #9 0x00007ffff76e4ccd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 (line numbers are from master) The ldapsearch manpage implies this probably isn't valid, but it still accepted it. (FWIW, I tried it just to see whether it would return all attributes or none.) I couldn't tell from draft-ldap-deref-00 whether an empty attr list is considered a valid request.

1 0

(ITS#8026) net-nds/openldap - powerpc-apple-darwin9-gcc: Undefined symbols for architecture ppc: "_slapi_pblock_set"
by daniel.ibnzayd＠inquisitor.com 19 Jan '15

19 Jan '15

Full_Name: Daniel Ibn Zayd Version: 2.4.38-r2 OS: PPC URL: https://bugs.gentoo.org/show_bug.cgi?id=536306 Submission from: (NULL) (166.84.1.2) Both of the latest builds marked stable for ppc (openldap-2.4.35-r1 and openldap-2.4.38-r2) error out when they get to the libaddrdnvalues-plugin: * Building contrib-module: addrdnvalues plugin Undefined symbols for architecture ppc: "_slapi_pblock_set", referenced from: _addrdnvalues_preop_init in cclXT5Cy.o "_slapi_log_error", referenced from: _addrdnvalues_preop_init in cclXT5Cy.o _addrdnvalues_preop_add in cclXT5Cy.o "_slapi_pblock_get", referenced from: _addrdnvalues_preop_add in cclXT5Cy.o "_slapi_entry_add_rdn_values", referenced from: _addrdnvalues_preop_add in cclXT5Cy.o "_slapi_send_ldap_result", referencefrfrom: _addrdnvalues_preop_add in cclXT5Cy.o "_ldap_err2string", referenced from: _addrdnvalues_preop_add in cclXT5Cy.o ld: symbol(s) not found for architecture ppc I've researched the plug-in and its components searching for possible fixes but have not turned up anything. Any clues here are greatly appreciated. PS: This is a Gentoo Prefix set up I am working with. https://bugs.gentoo.org/show_bug.cgi?id=536306

1 0

Re: (ITS#7971) LMDB: Uncarefully appointment when beginning a readonly txn.
by h.b.furuseth＠usit.uio.no 16 Jan '15

16 Jan '15

On 18/10/14 00:46, leo(a)yuriev.ru wrote: > (...) > - ti->mti_readers[i].mr_pid = pid; > - ti->mti_readers[i].mr_tid = tid; > + r = &ti->mti_readers[i]; > + r->mr_txnid = (txnid_t)-1; > + r->mr_tid = tid; > + r->mr_pid = pid; /* should be written last, see ITS#. */ > if (i == nr) > ti->mti_numreaders = ++nr; > /* Save numreaders for un-mutexed mdb_env_close() */ > env->me_numreaders = nr; Actually that too is too early to set mr_pid: We must be sure env_close() will reset it. But it is also too late, we must get rid of any garbage value in md_pid at once to protect it from other processes' env_close(). Fixing. Also, I'll rename the confusingly named me_numreaders to me_close_readers and drop some confused code using it.

1 0

Re: (ITS#8025) OpenLDAP authorisation on a client causes policykit GUI features to fail
by hyc＠symas.com 16 Jan '15

16 Jan '15

hyc(a)symas.com wrote: > There's nothing here that indicates any connection to OpenLDAP either. > Bug in pam_ldap, closing this ITS. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7969) LMDB: Globally shared fields of meta-data are not 'volatile'.
by hyc＠openldap.org 16 Jan '15

16 Jan '15

This discussion should be moved to the openldap-devel list as there's more being discussed now than the patch itself. Леонид Юрьев wrote: > 2015-01-14 8:49 GMT+03:00 Hallvard Breien Furuseth <h.b.furuseth(a)usit.uio.no>: >> On 13/01/15 19:23, Hallvard Breien Furuseth wrote: >>> >>> Yes, that didn't come out right. I don't mind inserting the >>> volatile, but I don't know that it helps either. As far as I >>> understand if it was broken without volatile, then it's still >>> broken with it - just hopefully less likely to break. And LMDB >>> couldn't be releaseed with your original unportable sync code >>> along with the volatile. >> >> >> Sorry, nevermind. Of course when the writer does sync well enough, >> volatile + the txn_renew loop will have to do for a sync primitive in >> the reader. > >> I suppose this requires that sync in the writer thread >> will shake other threads as well, it won't be private to the writer. > > In general this is wrong, more precisely depend on 'volatile' on > shared variables and usage of barriers/fences by the readers. Actually 'volatile' is meaningless at the hardware level. It only serves to prevent the compiler from reordering or eliminating accesses. In the case of LMDB the compiler cannot eliminate accesses because they are to global memory, and the compiler cannot determine anything about the liveness or side-effects of global memory accesses. > Sync-ops due lock/release a mutex by writer issue a memory-barrier for > its own thread. > With this compiler must update all modified variables, which shaded in > the CPU registers. > Next a hardware write-barrier (aka release) in the mutex-release code > enforce all changes to be visible for other threads (e.g. flush the > cache). > But 'be visible' here mean 'publish' and other threads can access > these changes, but if want this. > In general, to see changes made by the writer, all other threads > should issue a read-barrier (aka acquire). > On most arches such barrier just inform compiler that memory was > changed and variables which cached in the registers must be reloaded. > But in some cases (like Itanium) this barrier will be taken in account > for instruction scheduling. > For 'volatile' compiler should generate barriers each time on read or > write such variables. No. volatile doesn't generate hardware barriers. > More general, memory-barriers are very important to HPC, distributed > computing and for super-computers. > For example read-barriers may pull changes from internode-bus or other > nodes, and write-barriers - publish the local changes. > So, the one way to avoid a race bugs - thinking in terms of > publish/pull changes. "race bug" by definition occurs when multiple writers may modify a particular memory object, causing its value to be indeterminate to an observer. By definition, no such bugs can occur in LMDB because only single writers can ever modify any memory object. In the case of arbitrary readers viewing writes, these are the possible cases: 1) reader is on same CPU as writer, writes cached there is no issue, the reader sees what the writer wrote. 2) reader is on same CPU as writer, writes not cached there is no issue, the reader must fetch data from global memory 3) reader is on different CPU, writes not cached there is no issue, the reader's CPU must fetch the data from global memory - same as (2) 4) reader is on different CPU, writes are cached the reader may see the cached/stale data, or the CPU may fetch the new data from global memory Only case (4) has any ambiguity, and LMDB's reader table is specifically designed not to care about the ambiguity. I.e., whether fresh or stale data is seen is irrelevant, LMDB will operate correctly because it does not fresh data. Correct processing of the reader table only depends on the oldest data in the table, so staleness is an asset here. Correct processing of changes to the meta page requires exclusion from other writers, so the write mutex is used. This also guarantees that all changes to the meta page are flushed to global memory before the next writer begins. In ITS#7970 you discuss a "heisenbug" which can theoretically occur if multiple write txns complete in the span of 2 memory accesses by a reader. In practice such a bug cannot occur because the act of completing a write txn involves multiple blocking calls (I/O system calls, mutex acquisition/release, etc.) which will force writers to pause relative to any readers in question. In contrast the reader performs no blocking calls at all, and in the window of vulnerability it is performing only 2 single-word memory accesses. Demonstrating the bug by manually inserting yield()s only proves the point - without those manually inserted yields you cannot actually trigger any situation where the reader thread will be descheduled between those two instructions. You discuss the ramifications of such a bug as the writer potentially overwriting pages that the reader needs. None of this can actually occur in current LMDB code, again by design. The fact that you encountered these issues while debugging your LIFO patch only reflects on the problems I already pointed out with the LIFO approach. Hallvard and I had this exact same discussion a few years ago; his example used the debugger to pause the reader at that point. Certainly, if you go out of your way to manually halt the reader at a specific instruction you can break the reader. Without outside intervention it won't happen. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8025) OpenLDAP authorisation on a client causes policykit GUI features to fail
by hyc＠symas.com 16 Jan '15

16 Jan '15

mez(a)debian.org wrote: > Full_Name: Martin Meredith > Version: 2.4.31 > OS: Xubuntu 14.04 > URL: > Submission from: (NULL) (81.174.171.8) > > > when combining libmap-ldap and policykit within ubuntu 14.04, this causes the > GUI portions of PolicyKit to fail. As a side effect, this causes the lockscreen > within Xubuntu 14.04 to be bypassable by a VT switch. > > I'm unsure as to what information you need to help diagnose this issue, but I > can provide any info as required. > > PolicyKit bug:- https://bugs.freedesktop.org/show_bug.cgi?id=88492 Sounds like your issue is with pam_ldap, which is not OpenLDAP software. As such, there is no code we own here that we can investigate for you. I note that PADL, the original authors of pam_ldap, stopped supporting it many many years ago. For several years they were referring support customers to Symas, but we at Symas also deprecated pam_ldap many years ago in favor of nssov. > Ubuntu Bug: https://bugs.launchpad.net/policykit-1/+bug/1410195 There's nothing here that indicates any connection to OpenLDAP either. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8025) OpenLDAP authorisation on a client causes policykit GUI features to fail
by mez＠debian.org 16 Jan '15

16 Jan '15

Full_Name: Martin Meredith Version: 2.4.31 OS: Xubuntu 14.04 URL: Submission from: (NULL) (81.174.171.8) when combining libmap-ldap and policykit within ubuntu 14.04, this causes the GUI portions of PolicyKit to fail. As a side effect, this causes the lockscreen within Xubuntu 14.04 to be bypassable by a VT switch. I'm unsure as to what information you need to help diagnose this issue, but I can provide any info as required. PolicyKit bug:- https://bugs.freedesktop.org/show_bug.cgi?id=88492 Ubuntu Bug: https://bugs.launchpad.net/policykit-1/+bug/1410195

1 0

Re: (ITS#8024) Password Policy Logging Enhancement
by michael＠stroeder.com 14 Jan '15

14 Jan '15

mwarren(a)symas.com wrote: > We would like make a new feature request for enhanced logging within the > Password Policy Module. A customer has a need for logging of automated password > lockouts which occur after a certain number of failed binds within a given time > window. Pertinent info would include the DN of the locked out user as well as > the source IP of the failed attempt(s). When running a consumer with slapo-accesslog (yes, not for delta-syncrepl) slapo-ppolicy's modifications are written to the accesslog-DB. I use it in a highly secure environment for seeing logins (slapo-lastbind) and login failures (but no failure lockout). Having just a syslog entry in this case would probably better regarding performance though. Maybe even a info message along with the BIND RESULT message would do. Ciao, Michael.

1 0

(ITS#8024) Password Policy Logging Enhancement
by mwarren＠symas.com 14 Jan '15

14 Jan '15

Full_Name: Mark Warren Version: 2.4.x OS: Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (72.35.133.119) Greetings, We would like make a new feature request for enhanced logging within the Password Policy Module. A customer has a need for logging of automated password lockouts which occur after a certain number of failed binds within a given time window. Pertinent info would include the DN of the locked out user as well as the source IP of the failed attempt(s). Best Regards, Mark

1 0

Re: (ITS#7969) LMDB: Globally shared fields of meta-data are not 'volatile'.
by leo＠yuriev.ru 14 Jan '15

14 Jan '15

2015-01-14 8:49 GMT+03:00 Hallvard Breien Furuseth <h.b.furuseth(a)usit.uio.no>: > On 13/01/15 19:23, Hallvard Breien Furuseth wrote: >> >> Yes, that didn't come out right. I don't mind inserting the >> volatile, but I don't know that it helps either. As far as I >> understand if it was broken without volatile, then it's still >> broken with it - just hopefully less likely to break. And LMDB >> couldn't be releaseed with your original unportable sync code >> along with the volatile. > > > Sorry, nevermind. Of course when the writer does sync well enough, > volatile + the txn_renew loop will have to do for a sync primitive in > the reader. > I suppose this requires that sync in the writer thread > will shake other threads as well, it won't be private to the writer. In general this is wrong, more precisely depend on 'volatile' on shared variables and usage of barriers/fences by the readers. Sync-ops due lock/release a mutex by writer issue a memory-barrier for its own thread. With this compiler must update all modified variables, which shaded in the CPU registers. Next a hardware write-barrier (aka release) in the mutex-release code enforce all changes to be visible for other threads (e.g. flush the cache). But 'be visible' here mean 'publish' and other threads can access these changes, but if want this. In general, to see changes made by the writer, all other threads should issue a read-barrier (aka acquire). On most arches such barrier just inform compiler that memory was changed and variables which cached in the registers must be reloaded. But in some cases (like Itanium) this barrier will be taken in account for instruction scheduling. For 'volatile' compiler should generate barriers each time on read or write such variables. More general, memory-barriers are very important to HPC, distributed computing and for super-computers. For example read-barriers may pull changes from internode-bus or other nodes, and write-barriers - publish the local changes. So, the one way to avoid a race bugs - thinking in terms of publish/pull changes.

1 0

Re: (ITS#8023) slappasswd with sha2 overlay can generate hashes but not salted hashes
by quanah＠zimbra.com 14 Jan '15

14 Jan '15

--On Wednesday, January 14, 2015 11:00 AM +0000 freebsd(a)jonathanprice.org wrote: > Hi, > > I tried 2.4.39 under FreeBSD and still had the same issue. > > I have also tried the packages for both CentOS 7 and Debian Wheezy, but > unfortunately neither of them include the SHA2 overlay by default. > > Finally, I tried installing zimbra-core and zimbra-ldap under CentOS. > When I used this installation, it worked successfully. > > I ran slapd -V on the zimbra installation, and it's 2.4.39. However, > based on it still not working on 2.4.39 on FreeBSD it appears to have > narrowed it down to two reasons: - An issue with the packaging under > FreeBSD > - The functionality is specific to Zimbra > > The next step in the process to narrow this down is to do a manual > compilation on CentOS, including the SHA2 overlay. If this works, then it > would confirm it to be a FreeBSD issue, and if it doesn't work that would > strongly suggest that Zimbra has modified something. You could simply grab the LTB project builds. I'm pretty sure they build out the contrib modules. In any case, I already noted that Zimbra doesn't patch anything in OpenLDAP that would affect this area. --Quanah -- Quanah Gibson-Mount Platform Architect Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#8023) slappasswd with sha2 overlay can generate hashes but not salted hashes
by freebsd＠jonathanprice.org 14 Jan '15

14 Jan '15

Hi,=0A=0AI tried 2.4.39 under FreeBSD and still had the same issue.=0A=0A= I have also tried the packages for both CentOS 7 and Debian Wheezy, but u= nfortunately neither of them include the SHA2 overlay by default.=0A=0AFi= nally, I tried installing zimbra-core and zimbra-ldap under CentOS. When = I used this installation, it worked successfully.=0A=0AI ran slapd -V on = the zimbra installation, and it's 2.4.39. However, based on it still not = working on 2.4.39 on FreeBSD it appears to have narrowed it down to two r= easons:=0A- An issue with the packaging under FreeBSD=0A- The functionali= ty is specific to Zimbra=0A=0AThe next step in the process to narrow this= down is to do a manual compilation on CentOS, including the SHA2 overlay= . If this works, then it would confirm it to be a FreeBSD issue, and if i= t doesn't work that would strongly suggest that Zimbra has modified somet= hing.=0A=0AThanks for the assistance so far,=0A=0A-Jonathan=0A=0AJanuary = 13 2015 8:00 PM, "Quanah Gibson-Mount" <quanah(a)zimbra.com> wrote: =0A> --= On Tuesday, January 13, 2015 7:24 PM +0000 Jonathan Price=0A> <freebsd@jo= nathanprice.org> wrote:=0A> =0A>> I do apologise for the confusion, I'll = try to clarify below:=0A>> =0A>> Here is the command you ran successfully= :=0A>> /opt/zimbra/openldap/sbin/slappasswd -h=0A>> '{SSHA512}' -o module= -path=3D/opt/zimbra/openldap/sbin/openldap -o=0A>> module-load=3Dpw-sha2 = -s test=0A>> {SSHA512}TSwAWmK3sv42RbAasugMPR8d7GLozXtKU00v5Jdd4ebmXBsOpt5= We5HNkXxFfy5=0A>> Ptaoa/KUsmTV5484NA3UmrHrOpyUVnEh9=0A>> =0A>> Here is an= example of me running just a plain SHA512=0A>> slappasswd -h '{SHA512}' = -o module-path=3D/usr/local/libexec/openldap -o=0A>> module-load=3Dpw-sha= 2=0A>> {SHA512}7iaw3Ur350mqGo7jwQrpkj9hiYB3Lkc/iBml1JQODbJ6wYX4oOHV+E+IvI= h/1nsUN=0A>> zLDBMxfqa2Ob1f1ACio/w=3D=3D=0A>> =0A>> And here is an exampl= e of me running a salted SHA512 (SSHA512)=0A>> slappasswd -h '{SSHA512}' = -o module-path=3D/usr/local/libexec/openldap -o=0A>> module-load=3Dpw-sha= 2 -s test=0A>> Password verification failed.=0A>> =0A>> I hope this helps= to clarify.=0A> =0A> Yes, thank you. So I'm using 2.4.39. There were som= e minor changes to=0A> slapd-sha2 in 2.4.40. I will see if I can reproduc= e the issue with current=0A> RE24.=0A> =0A> --Quanah=0A> =0A> --=0A> =0A>= Quanah Gibson-Mount=0A> Platform Architect=0A> Zimbra, Inc. =0A> _______= ________________________=0A> =0A> Zimbra :: the leader in open source mes= saging and collaboration

1 0

Re: (ITS#7969) LMDB: Globally shared fields of meta-data are not 'volatile'.
by h.b.furuseth＠usit.uio.no 13 Jan '15

13 Jan '15

On 13/01/15 19:23, Hallvard Breien Furuseth wrote: > Yes, that didn't come out right. I don't mind inserting the > volatile, but I don't know that it helps either. As far as I > understand if it was broken without volatile, then it's still > broken with it - just hopefully less likely to break. And LMDB > couldn't be releaseed with your original unportable sync code > along with the volatile. Sorry, nevermind. Of course when the writer does sync well enough, volatile + the txn_renew loop will have to do for a sync primitive in the reader. I suppose this requires that sync in the writer thread will shake other threads as well, it won't be private to the writer. -- Hallvard

1 0

Re: (ITS#8023) slappasswd with sha2 overlay can generate hashes but not salted hashes
by quanah＠zimbra.com 13 Jan '15

13 Jan '15

--On Tuesday, January 13, 2015 7:24 PM +0000 Jonathan Price <freebsd(a)jonathanprice.org> wrote: > I do apologise for the confusion, I'll try to clarify below: > > Here is the command you ran successfully: > /opt/zimbra/openldap/sbin/slappasswd -h > '{SSHA512}' -o module-path=/opt/zimbra/openldap/sbin/openldap -o > module-load=pw-sha2 -s test > {SSHA512}TSwAWmK3sv42RbAasugMPR8d7GLozXtKU00v5Jdd4ebmXBsOpt5We5HNkXxFfy5 > Ptaoa/KUsmTV5484NA3UmrHrOpyUVnEh9 > > Here is an example of me running just a plain SHA512 > slappasswd -h '{SHA512}' -o module-path=/usr/local/libexec/openldap -o > module-load=pw-sha2 > {SHA512}7iaw3Ur350mqGo7jwQrpkj9hiYB3Lkc/iBml1JQODbJ6wYX4oOHV+E+IvIh/1nsUN > zLDBMxfqa2Ob1f1ACio/w== > > And here is an example of me running a salted SHA512 (SSHA512) > slappasswd -h '{SSHA512}' -o module-path=/usr/local/libexec/openldap -o > module-load=pw-sha2 -s test > Password verification failed. > > I hope this helps to clarify. Yes, thank you. So I'm using 2.4.39. There were some minor changes to slapd-sha2 in 2.4.40. I will see if I can reproduce the issue with current RE24. --Quanah -- Quanah Gibson-Mount Platform Architect Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#8023) slappasswd with sha2 overlay can generate hashes but not salted hashes
by freebsd＠jonathanprice.org 13 Jan '15

13 Jan '15

I do apologise for the confusion, I'll try to clarify below: Here is the command you ran successfully: /opt/zimbra/openldap/sbin/slappasswd -h '{SSHA512}' -o module-path=/opt/zimbra/openldap/sbin/openldap -o module-load=pw-sha2 -s test {SSHA512}TSwAWmK3sv42RbAasugMPR8d7GLozXtKU00v5Jdd4ebmXBsOpt5We5HNkXxFfy5 Ptaoa/KUsmTV5484NA3UmrHrOpyUVnEh9 Here is an example of me running just a plain SHA512 slappasswd -h '{SHA512}' -o module-path=/usr/local/libexec/openldap -o module-load=pw-sha2 {SHA512}7iaw3Ur350mqGo7jwQrpkj9hiYB3Lkc/iBml1JQODbJ6wYX4oOHV+E+IvIh/1nsUNzLDBMxfqa2Ob1f1ACio/w== And here is an example of me running a salted SHA512 (SSHA512) slappasswd -h '{SSHA512}' -o module-path=/usr/local/libexec/openldap -o module-load=pw-sha2 -s test Password verification failed. I hope this helps to clarify. On 2015-01-13 19:14, Quanah Gibson-Mount wrote: > --On Tuesday, January 13, 2015 7:11 PM +0000 Jonathan Price > <freebsd(a)jonathanprice.org> wrote: > >> Hi, >> >> From the original email: >> However, if I replace {SHA512} with {SSHA512} it produces the following >> output: >> Password verification failed. > > You also were not clear *where* you did this replacement. It is > certainly not valid to do this replacement on the generated hash, as the > generated has was non-salted, and just adding another S in there will > not magically make it salted. It is valid to do this replacement in the > slappasswd line when generating a hash, as per my example, so that a > salted hash is generated. > > --Quanah > > >> It's interesting to see that it does work under certain conditions then. >> It appears that your OpenLDAP installation is part of a Zimbra >> installation. Does Zimbra make any modifications to OpenLDAP, or is it >> just built on top of it? >> >> Either way, I think I'm going to try it on Debian, just to rule out it >> being a FreeBSD issue, which it quite well could be at this point. >> >> On 2015-01-13 19:01, Quanah Gibson-Mount wrote: >>> --On Tuesday, January 13, 2015 6:52 PM +0000 freebsd(a)jonathanprice.org >>> wrote: >>> >>>> Full_Name: Jonathan Price >>>> Version: 2.4.40 >>>> OS: FreeBSD 10.1 >>>> URL: ftp://ftp.openldap.org/incoming/ >>>> Submission from: (NULL) (80.47.105.54) >>>> >>>> >>>> I have compiled version 2.4.40 with the SHA2 module enabled. >>>> >>>> I then run the slappasswd with the following arguments: >>>> slappasswd -h '{SHA512}' -o module-path=/usr/local/libexec/openldap -o >>>> module-load=pw-sha2 >>> >>> You requested a non salted hash -> SHA512 >>> >>> Did you try requesting a salted hash? -> SSHA512 >>> >>> Works fine for me, and I've been using it in production for quite some >>> time. >>> >>> [zimbra@zre-ldap003 ~]$ /opt/zimbra/openldap/sbin/slappasswd -h >>> '{SSHA512}' -o module-path=/opt/zimbra/openldap/sbin/openldap -o >>> module-load=pw-sha2 -s test >>> {SSHA512}TSwAWmK3sv42RbAasugMPR8d7GLozXtKU00v5Jdd4ebmXBsOpt5We5HNkXxFfy5 >>> Ptaoa/KUsmTV5484NA3UmrHrOpyUVnEh9 >>> >>> >>> >>> --Quanah >>> >>> -- >>> >>> Quanah Gibson-Mount >>> Platform Architect >>> Zimbra, Inc. >>> -------------------- >>> Zimbra :: the leader in open source messaging and collaboration > > > > -- > > Quanah Gibson-Mount > Platform Architect > Zimbra, Inc. > -------------------- > Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#8023) slappasswd with sha2 overlay can generate hashes but not salted hashes
by quanah＠zimbra.com 13 Jan '15

13 Jan '15

--On Tuesday, January 13, 2015 7:11 PM +0000 Jonathan Price <freebsd(a)jonathanprice.org> wrote: > Hi, > > From the original email: > However, if I replace {SHA512} with {SSHA512} it produces the following > output: > Password verification failed. You also were not clear *where* you did this replacement. It is certainly not valid to do this replacement on the generated hash, as the generated has was non-salted, and just adding another S in there will not magically make it salted. It is valid to do this replacement in the slappasswd line when generating a hash, as per my example, so that a salted hash is generated. --Quanah > It's interesting to see that it does work under certain conditions then. > It appears that your OpenLDAP installation is part of a Zimbra > installation. Does Zimbra make any modifications to OpenLDAP, or is it > just built on top of it? > > Either way, I think I'm going to try it on Debian, just to rule out it > being a FreeBSD issue, which it quite well could be at this point. > > On 2015-01-13 19:01, Quanah Gibson-Mount wrote: >> --On Tuesday, January 13, 2015 6:52 PM +0000 freebsd(a)jonathanprice.org >> wrote: >> >>> Full_Name: Jonathan Price >>> Version: 2.4.40 >>> OS: FreeBSD 10.1 >>> URL: ftp://ftp.openldap.org/incoming/ >>> Submission from: (NULL) (80.47.105.54) >>> >>> >>> I have compiled version 2.4.40 with the SHA2 module enabled. >>> >>> I then run the slappasswd with the following arguments: >>> slappasswd -h '{SHA512}' -o module-path=/usr/local/libexec/openldap -o >>> module-load=pw-sha2 >> >> You requested a non salted hash -> SHA512 >> >> Did you try requesting a salted hash? -> SSHA512 >> >> Works fine for me, and I've been using it in production for quite some >> time. >> >> [zimbra@zre-ldap003 ~]$ /opt/zimbra/openldap/sbin/slappasswd -h >> '{SSHA512}' -o module-path=/opt/zimbra/openldap/sbin/openldap -o >> module-load=pw-sha2 -s test >> {SSHA512}TSwAWmK3sv42RbAasugMPR8d7GLozXtKU00v5Jdd4ebmXBsOpt5We5HNkXxFfy5 >> Ptaoa/KUsmTV5484NA3UmrHrOpyUVnEh9 >> >> >> >> --Quanah >> >> -- >> >> Quanah Gibson-Mount >> Platform Architect >> Zimbra, Inc. >> -------------------- >> Zimbra :: the leader in open source messaging and collaboration -- Quanah Gibson-Mount Platform Architect Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#8023) slappasswd with sha2 overlay can generate hashes but not salted hashes
by quanah＠zimbra.com 13 Jan '15

13 Jan '15

--On Tuesday, January 13, 2015 7:11 PM +0000 Jonathan Price <freebsd(a)jonathanprice.org> wrote: > Hi, > > From the original email: > However, if I replace {SHA512} with {SSHA512} it produces the following > output: > Password verification failed. > > It's interesting to see that it does work under certain conditions then. > It appears that your OpenLDAP installation is part of a Zimbra > installation. Does Zimbra make any modifications to OpenLDAP, or is it > just built on top of it? > > Either way, I think I'm going to try it on Debian, just to rule out it > being a FreeBSD issue, which it quite well could be at this point. We make a few modifications to OpenLDAP, but nothing affecting the pw-sha2 module, which we use as-is. --Quanah -- Quanah Gibson-Mount Platform Architect Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#8023) slappasswd with sha2 overlay can generate hashes but not salted hashes
by freebsd＠jonathanprice.org 13 Jan '15

13 Jan '15

Hi, From the original email: However, if I replace {SHA512} with {SSHA512} it produces the following output: Password verification failed. It's interesting to see that it does work under certain conditions then. It appears that your OpenLDAP installation is part of a Zimbra installation. Does Zimbra make any modifications to OpenLDAP, or is it just built on top of it? Either way, I think I'm going to try it on Debian, just to rule out it being a FreeBSD issue, which it quite well could be at this point. On 2015-01-13 19:01, Quanah Gibson-Mount wrote: > --On Tuesday, January 13, 2015 6:52 PM +0000 freebsd(a)jonathanprice.org > wrote: > >> Full_Name: Jonathan Price >> Version: 2.4.40 >> OS: FreeBSD 10.1 >> URL: ftp://ftp.openldap.org/incoming/ >> Submission from: (NULL) (80.47.105.54) >> >> >> I have compiled version 2.4.40 with the SHA2 module enabled. >> >> I then run the slappasswd with the following arguments: >> slappasswd -h '{SHA512}' -o module-path=/usr/local/libexec/openldap -o >> module-load=pw-sha2 > > You requested a non salted hash -> SHA512 > > Did you try requesting a salted hash? -> SSHA512 > > Works fine for me, and I've been using it in production for quite some > time. > > [zimbra@zre-ldap003 ~]$ /opt/zimbra/openldap/sbin/slappasswd -h > '{SSHA512}' -o module-path=/opt/zimbra/openldap/sbin/openldap -o > module-load=pw-sha2 -s test > {SSHA512}TSwAWmK3sv42RbAasugMPR8d7GLozXtKU00v5Jdd4ebmXBsOpt5We5HNkXxFfy5Ptaoa/KUsmTV5484NA3UmrHrOpyUVnEh9 > > > > --Quanah > > -- > > Quanah Gibson-Mount > Platform Architect > Zimbra, Inc. > -------------------- > Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#8023) slappasswd with sha2 overlay can generate hashes but not salted hashes
by quanah＠zimbra.com 13 Jan '15

13 Jan '15

--On Tuesday, January 13, 2015 6:52 PM +0000 freebsd(a)jonathanprice.org wrote: > Full_Name: Jonathan Price > Version: 2.4.40 > OS: FreeBSD 10.1 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (80.47.105.54) > > > I have compiled version 2.4.40 with the SHA2 module enabled. > > I then run the slappasswd with the following arguments: > slappasswd -h '{SHA512}' -o module-path=/usr/local/libexec/openldap -o > module-load=pw-sha2 You requested a non salted hash -> SHA512 Did you try requesting a salted hash? -> SSHA512 Works fine for me, and I've been using it in production for quite some time. [zimbra@zre-ldap003 ~]$ /opt/zimbra/openldap/sbin/slappasswd -h '{SSHA512}' -o module-path=/opt/zimbra/openldap/sbin/openldap -o module-load=pw-sha2 -s test {SSHA512}TSwAWmK3sv42RbAasugMPR8d7GLozXtKU00v5Jdd4ebmXBsOpt5We5HNkXxFfy5Ptaoa/KUsmTV5484NA3UmrHrOpyUVnEh9 --Quanah -- Quanah Gibson-Mount Platform Architect Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

(ITS#8023) slappasswd with sha2 overlay can generate hashes but not salted hashes
by freebsd＠jonathanprice.org 13 Jan '15

13 Jan '15

Full_Name: Jonathan Price Version: 2.4.40 OS: FreeBSD 10.1 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (80.47.105.54) I have compiled version 2.4.40 with the SHA2 module enabled. I then run the slappasswd with the following arguments: slappasswd -h '{SHA512}' -o module-path=/usr/local/libexec/openldap -o module-load=pw-sha2 This works successfully, and in this example I used the word "test" and it produced the following output: {SHA512}7iaw3Ur350mqGo7jwQrpkj9hiYB3Lkc/iBml1JQODbJ6wYX4oOHV+E+IvIh/1nsUNzLDBMxfqa2Ob1f1ACio/w== However, if I replace {SHA512} with {SSHA512} it produces the following output: Password verification failed. I have tested SHA256 SHA384 and SHA512. All three of these work fine. All three of SSHA256, SSHA384 and SSHA512 do not work however. It appears that there is an issue with slappasswd and salted SHA2 hashes. I have checked that 2.4.40 is new enough to have a version of the SHA2 overlay, and also checked the source to make sure it was definitely a new enough version, and can confirm that it is. Unfortunately, beyond this basic level of checking, I'm not a C programmer so I can't investigate the issue further myself.

1 0

Re: (ITS#7969) LMDB: Globally shared fields of meta-data are not 'volatile'.
by h.b.furuseth＠usit.uio.no 13 Jan '15

13 Jan '15

On 13/01/15 17:54, leo(a)yuriev.ru wrote: > 2015-01-13 13:50 GMT+03:00 Hallvard Breien Furuseth <h.b.furuseth(a)usit.uio.no>: >> comp.programming.threads's advise is that volatile is useless for thread >> sync: You need sync primitives, and then you don't need volatile since >> the compiler/hardware may not move the memory access past the primitive. > > Excuse me, but seems you are misunderstand comp.programming.threads in > case of blocking/locking multithreading and non-blocking > multithreading (aka lock-free/wait-free). Yes, that didn't come out right. I don't mind inserting the volatile, but I don't know that it helps either. As far as I understand if it was broken without volatile, then it's still broken with it - just hopefully less likely to break. And LMDB couldn't be releaseed with your original unportable sync code along with the volatile. > (...) >> Unless the primitive's spec says so, I guess. E.g. with an "asm" >> statement where the compiler does not know what the assembly code does >> and must be told not to mess with ordering. > > Please study GCC's manual about __asm __volalile (:::"memory"), this > is just a "full compiler fence". Thanks, looks good. Will read the rest later. -- Hallvard

1 0

Re: (ITS#7969) LMDB: Globally shared fields of meta-data are not 'volatile'.
by leo＠yuriev.ru 13 Jan '15

13 Jan '15

2015-01-13 13:50 GMT+03:00 Hallvard Breien Furuseth <h.b.furuseth(a)usit.uio.no>: > On 01/12/2015 07:15 PM, leo(a)yuriev.ru wrote: >> >> Unfortunately IMDB programmed without accounting possibility of memory >> reordering and cache incoherency (as a proof - the 'volatile' was >> missed before ITS#7969). >> So, currently seems LMDB is durable only on x86 and may be on MIPS. > > > I gather "volatile" in thread code can be read as "warning: dubious > code" or at best "please access this sooner rather than later". > > comp.programming.threads's advise is that volatile is useless for thread > sync: You need sync primitives, and then you don't need volatile since > the compiler/hardware may not move the memory access past the primitive. Excuse me, but seems you are misunderstand comp.programming.threads in case of blocking/locking multithreading and non-blocking multithreading (aka lock-free/wait-free). Yes, `volatile` don't needed, when a locking (e.g. mutex) is used for access to any variables that shared between threads. But LMDB don't uses locking primitives for interaction between writers and readers. LMDB properties declared as: - readers don't block writers and writers don't block readers; - read transactions are extremely cheap, and can be performed using no mallocs or any other _blocking_ calls; THEREFORE 'volatile' and memory-ordering ISSUES MUST BE HANDLED VERY CAREFULLY AND ANXIOUSLY. > Unless the primitive's spec says so, I guess. E.g. with an "asm" > statement where the compiler does not know what the assembly code does > and must be told not to mess with ordering. Please study GCC's manual about __asm __volalile (:::"memory"), this is just a "full compiler fence". > The best fix is likely a format change which reduces the need for sync > primitives, for LDMBv2. Checksum the MDB_meta, and/or replace the > variable part with a single word which refers to the meta. > Don't know if we can get rid of unportable sync code though. Yes it is possible, but requires C++, see http://en.cppreference.com/w/cpp/atomic/memory_order By using just C, include C99 - not possible, see http://stackoverflow.com/questions/27301371/memory-order-consistency-model-… > Leaving the rest to Howard, I'm too busy just now to do more than blow > hot air:-) Ok ;) I created gist for public review, please share it. https://gist.github.com/leo-yuriev/ba186a6bf5cf3a27bae7 Leonid.

1 0

Re: (ITS#8018) a lot of warnings building with -Wall
by h.b.furuseth＠usit.uio.no 13 Jan '15

13 Jan '15

On 01/12/2015 05:22 PM, leo(a)yuriev.ru wrote: > 2015-01-12 17:19 GMT+03:00 Hallvard Breien Furuseth < > h.b.furuseth(a)usit.uio.no>: >> The Debug() change is going to give merge conflicts all over >> the place when your <https://github.com/ReOpen/ReOpenLDAP> and >> OpenLDAP pull updates from each other. >> >> https://github.com/ReOpen/ReOpenLDAP should not be used for merge, but > only for cherry-picking. Oops, right. "cherry-picking conflicts":-) > It has stripped history and merged with OpenLDAP/mdb.master, because > mdb.RE/0.9 don't includes a few critical fixes (ITS 7969,7970,7971) for our > case. > I think that is the reason for patch conflicts. > Patchset got RE25 should remain among my sent emails. Presumably I can send > it for you again, if you are interested. No, I was talking about maintaining the two projects in the future. Presumably you'll want to copy fixes from OpenLDAP, and we'll want fixes from you. Any such change next to a Debug() statement will not apply cleanly, and must be hand-edited. >> How about defining Debug1(), Debug2(), etc for 1,2... arguments? >> That's still annoying, but it keeps the code C90-compatible and >> keeps -Wformat quiet. There's already similar code for Log<N>() >> in include/ldap_log.h, but it was never used much or even merged >> into RE24. (Because it got scheduled for RE25 which was going to >> start Real Soon Now, IIRC:-) > > IMHO the Log<N> recipe is onerously for support. > On other hand, nowadays all non-obsolete compilers support most of C99 > features. > Baseline features, such as "variadic macro", supported long ago (Microsoft > was the most conservative to 2012). > Maybe it is time to enable C99 in RE25? Dunno. People do use old compilers. OpenLDAP is conservative about portability. It requires C90 to build, but exports K&R1-compatible headers. Well, it tries. K&R1 compat was broken by OpenLDAP 2.0 (ITS#6171) and nobody who cares about K&R1 has bothered to fix it. When we do move towards C99, maybe we'll first test the waters by making Debug() use __VA_ARGS__ but not change the Debug() calls yet, so we'll only have a few lines to revert it it proves premature. We'll hear what Kurt says. -- Hallvard

1 0

Re: (ITS#8022) Backend Meta does not work with ssl-backends
by dirk.kastens＠uni-osnabrueck.de 13 Jan '15

13 Jan '15

Hi, > I've pushed fixes for both of these problems to git master and back-meta > works on ldaps:// for me now. Since I didn't reproduce your symptoms, I > have no idea if this will improve things for you or not. Please test and > follow up, thanks. Great! This perfectly works for me. I just compiled the latest git version. Now the meta directory works with ldaps. Thanks for your support. -- Dirk Kastens Universitaet Osnabrueck, Rechenzentrum (Computer Center) Albrechtstr. 28, 49069 Osnabrueck, Germany Tel.: +49-541-969-2347, FAX: -2470

1 0

Re: (ITS#7969) LMDB: Globally shared fields of meta-data are not 'volatile'.
by h.b.furuseth＠usit.uio.no 13 Jan '15

13 Jan '15

On 01/12/2015 07:15 PM, leo(a)yuriev.ru wrote: > Unfortunately IMDB programmed without accounting possibility of memory > reordering and cache incoherency (as a proof - the 'volatile' was > missed before ITS#7969). > So, currently seems LMDB is durable only on x86 and may be on MIPS. I gather "volatile" in thread code can be read as "warning: dubious code" or at best "please access this sooner rather than later". comp.programming.threads's advise is that volatile is useless for thread sync: You need sync primitives, and then you don't need volatile since the compiler/hardware may not move the memory access past the primitive. Unless the primitive's spec says so, I guess. E.g. with an "asm" statement where the compiler does not know what the assembly code does and must be told not to mess with ordering. The best fix is likely a format change which reduces the need for sync primitives, for LDMBv2. Checksum the MDB_meta, and/or replace the variable part with a single word which refers to the meta. Don't know if we can get rid of unportable sync code though. Leaving the rest to Howard, I'm too busy just now to do more than blow hot air:-) -- Hallvard

1 0

Re: (ITS#7969) LMDB: Globally shared fields of meta-data are not 'volatile'.
by leo＠yuriev.ru 13 Jan '15

13 Jan '15

2014-11-16 0:04 GMT+03:00 Hallvard Breien Furuseth <h.b.furuseth(a)usit.uio.no>: > __sync_synchronize() needs an #ifdef around it. > #ifdef __GNUC__, or something more general? I have implemented 'portable' memory barriers for a set of compilers. https://github.com/ReOpen/ReOpenLDAP/commit/a1ec3a4a8290c84846b4e98ff3abf38… Feedback welcomed. Leonid.

1 0

Re: (ITS#8022) Backend Meta does not work with ssl-backends
by hyc＠symas.com 12 Jan '15

12 Jan '15

Dirk Kastens wrote: > Hi, > >> How about running ldapsearch with -d7, and also the corresponding search >> against back-meta using slapd -d7. Upload the output somewhere that we >> can review. > > See ftp.openldap.org/incoming/... > > You will find the following files: > > ldapsearch_direct: directly searching the source server with ssl > > ldapsearch_meta_starttls: searching the meta directory where the source > server is connected via starttls (this works) > > ldapsearch_meta_ssl: searching the meta directory where the source > server is connected via ssl (this doesn't work) > > slapd_meta_starttls: meta directory where the source server is connected > via starttls (this works) > > slapd_meta_ssl: meta directory where the source server is connected via > ssl (this doesn't work). slapd hangs after searching and has to be > killed with "kill -4". > > Hope this helps. > I wasn't able to exactly reproduce the problem shown in your logs, but I did find a couple of problems here. For ldaps:// URLs, the back-meta config parser was ignoring the TLS settings. As such, it wasn't able to establish any ldaps session at all, unless there were usable settings in ldap.conf/ldaprc/etc... Also, back-meta was specifying that libldap should open its outbound connections asynchronously, and this wasn't working properly if the async connection succeeded immediately. (I.e., when told to open async, libldap would expect the connection to take a long time to complete, and defer the rest of initialization to a following library call. If it completed immediately, libldap would mis-handle the rest of initialization.) I've pushed fixes for both of these problems to git master and back-meta works on ldaps:// for me now. Since I didn't reproduce your symptoms, I have no idea if this will improve things for you or not. Please test and follow up, thanks. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7969) LMDB: Globally shared fields of meta-data are not 'volatile'.
by leo＠yuriev.ru 12 Jan '15

12 Jan '15

2014-11-16 0:04 GMT+03:00 Hallvard Breien Furuseth <h.b.furuseth(a)usit.uio.no>: > __sync_synchronize() needs an #ifdef around it. > #ifdef __GNUC__, or something more general? > > Not sure what to do for the #else. We could mostly > avoid using mm_txnid when the lockfile is initialized > and use mti_txnid instead (e.g. in pick_meta). And > penalize MDB_NOLOCK with some sync call in the #else. > > -- > Hallvard > Unfortunately IMDB programmed without accounting possibility of memory reordering and cache incoherency (as a proof - the 'volatile' was missed before ITS#7969). So, currently seems LMDB is durable only on x86 and may be on MIPS. I think memory-ordering (e.g. barriers) functions should be defined in separate h-file, which will select implementation depends on cpu-arch & compiler. see http://en.wikipedia.org/wiki/Memory_ordering The minimum seems to be required a memory_barrier_coherence() - to make cache coherence and suppress dangerously load/store reordering, e.g. flush/invalidate cache on some cpu-arch, but do nothing on x86 and so on. Then __sync_synchronize() and MIPS's cache invalidate should be replaced with memory_barrier_coherence(). But instead the original version recreate cleanly on the basis of 1Hippeus engine (manager of shared memory and queues for zero-copy IPC, google it). Leonid

1 0

Re: (ITS#7958) LMDB: LIFO-reclaiming, write-performance improvement & bugfixes
by leo＠yuriev.ru 12 Jan '15

12 Jan '15

For informaion only - Nowadays 'lifo' and 'coalesce' features implemented in ReOpenLDAP fork. 1) lmdb: MDB_LIFORECLAIM & MDB_COALESCE modes. https://github.com/ReOpen/ReOpenLDAP/commit/829c2063b602238b5c93ea36a981de3= d0d7994bc 2) lmdb-backend: support config for 'lifo' and 'coalesce' envflags. https://github.com/ReOpen/ReOpenLDAP/commit/08b4a41b5b837548444ef0fef761494= 0c41d882a With the couple of issues: 1) lmdb in 'writemap' mode may inconsistent even with checkpoints https://github.com/ReOpen/ReOpenLDAP/issues/1 2) lifo feature should be synchonized with checkpoints https://github.com/ReOpen/ReOpenLDAP/issues/2 However currently it gives a reasonable boost (5-10 times) of write-performance in our use case. Leonid. 2014-10-20 0:27 GMT+03:00 =D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4 =D0=AE=D1=80= =D1=8C=D0=B5=D0=B2 <leo(a)yuriev.ru>: >>>> We are using the combination: >>>> envflags writemap nosync lifo >>>> checkpoint 0 1 >>>> >>>> If the checkpoint is set in seconds, it gives us the assurance >>>> consistent state database on disk. >>>> However, without this patch meta-pages can be written by the kernel >>>> before the data. >>>> >>>> In fact, for a full guarantee in case of death slapd process, >>>> meta-page should be written explicitly. >> >> >> No, the DB can never go inconsistent due to a process crashing - the pag= es >> in OS cache are always correct. It can only go inconsistent if the OS >> crashes and a proper sync has not occurred. > > Yes, Howard, you are right. > But apparently I need to be more precise. > Talking about "death" of slapd, I meant all the reasons, including power = off. > > For example, a power-off case: > - The main power is turned off and the system switches to the UPS. > - Given the notice, OS starts an emergency stop processes. > - For some reason (does not have enough time to stop) slapd receives SIGK= ILL. > - OS tries to write mmap-region of the DB-file and begins with the > lower address. > - let the meta-pages has written completely, but for the rest of the > data is not enough battery power. > - Now DB is completely destroyed on the disk. > > To avoid this, the meta-pages should not be included in the rw-mapped > region, and should be written explicity after a data pages. > > >>>>>> commit 8ddd63161aeb2689822d1a8d27385d62e4e341ae >>>>>> Author: Leo Yuriev <leo(a)yuriev.ru> >>>>>> Date: 2014-09-19 22:47:19 +0400 >>>>>> >>>>>> BUGFIX - lmdb: properly sync meta-pages in mdb_sync_env(). >>>>>> >>>>>> Meta-pages may be updated during data-syncing in mdb_sync_env= (), >>>>>> in this case database would be inconsistent. >>>>>> >>>>>> Check-and-retry if lead txn-id changed during flushing data i= n >>>>>> mdb_sync_env(). >> >> Fundamentally, you are trying to make an inherently unsafe configuration >> "safer", but it's impossible. Assume you have mlock'd the meta pages int= o >> memory, so the OS never flushes them itself any more, and you're running >> with NOSYNC. That means, within 3 transactions, the data pages on disk w= ill >> be out of sync with the meta pages on disk. If the OS crashes at that po= int, >> the entire DB will be lost. > > Not a problem. > I had explained above - we should write meta-pages explicitly after > the data sync. > But also we should not perform reclaiming ahead of the last checkpoint. > >> The only way to make this mode of operation somewhat safe is to defer >> reclaiming pages for even longer. E.g., instead of halting at current_tx= nid >> - 3, halt at current_txnid - 22, in which case the data pointed to by th= e >> on-disk meta pages cannot get obsolete until 20 transactions have occurr= ed. >> >> Note that in combination with your LIFO patch, it's pretty much guarante= ed >> that the on-disk meta pages will be useless after only 2 un-sync'd >> transactions. > > Yes, Howard, you are right. > But I think there is confusion in the discussion because of mixing of > LIFO-feature and changes for checkpoints consistency in a NOSYNC and > WRITEMAP+NOSYNC modes. > For a "NOSYNC + checkpoints" topic I will submit a separate ITS (like > a 'volaile' related 7969,7970,7971). > My opinion - it is a flaw, and no reason to don't fix it. > > Continuing the conversation about checkpoints in a LIFO context. > I saw the problem, that you specified, and thinking over its solution, > but have not yet found "golden ratio". > And since we are having a serious problem with syncrepl, then I put > off this task with an excuse "LIFO-patch not does worse than it was." > > In general, we should do not reclaim anything ahead of the txn, that > is synced to the disk (let this be named a R-rule). > To do so we need a second field like mti_txnid, but which will be > update only at the end of mdb_env_write_meta(). > Finally we should start search in mdb_find_oldest() from value of this > new field instead of the current txn number. > This seems to will be work fine. > > However, I stopped on a reasoning - about the purpose of the > checkpoints, about design LMDB as a product, about the expectations of > the user and the necessary configuration parameters: > - checkpoints are needed ONLY in nosync modes; > - if the user does NOT activate the checkpoints, he do not care about > consistency; > - but if it is turned on, we MUST provide consistency on the checkpoints; > - otherwise a checkpoints feature is thoughtless and should be REMOVED. > Therefore implementation of checkpoints & reclaiming should be updated > to conform to the "R-rule", that noted above. > > From this point of view a LIFO-feature also should be refined, but > nevertheless can be very useful. > - SYNC mode =3D takes a benefit from storage with write-back cache > (assume powered by battery). > - ASYNC/NOSYNC without checkpoint =3D significant reduction of > committed/dirty pages and thereby much less write-iops. > - ASYNC/NOSYNC with checkpoint =3D seems to same as a SYNC case. > > Total of all the above - I think first we need to fix a reclaiming or > delete the checkpoints, and then I will complete LIFO. > >> -- Howard Chu >> CTO, Symas Corp. http://www.symas.com >> Director, Highland Sun http://highlandsun.com/hyc/ >> Chief Architect, OpenLDAP http://www.openldap.org/project/ > > Thank for conversation. > Leonid.

1 0

Re: (ITS#8022) Backend Meta does not work with ssl-backends
by hyc＠symas.com 12 Jan '15

12 Jan '15

Dirk Kastens wrote: > Hi, > >> Then you should have simply followed up to #7983. > > The case has been closed. > >> Does ldapsearch work against the target servers using the certificate >> configurations you listed? > Yes. I can directly query the target servers from the redhat server > using ldapsearch and ssl. How about running ldapsearch with -d7, and also the corresponding search against back-meta using slapd -d7. Upload the output somewhere that we can review. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8018) a lot of warnings building with -Wall
by leo＠yuriev.ru 12 Jan '15

12 Jan '15

--f46d04426a60f30297050c76e815 Content-Type: text/plain; charset=UTF-8 2015-01-12 17:19 GMT+03:00 Hallvard Breien Furuseth < h.b.furuseth(a)usit.uio.no>: > On 01/06/2015 04:31 PM, hyc(a)symas.com wrote: > >> Leonid Yuriev wrote: >> >>> Yes of course, all warnings could be hushed by compiler options, but >>> this also hides the real mistakes. >>> So, I prefer fix the warnings, instead of disable gcc/clang features. >>> >>> Please review attached patchset and merge. >>> >> >> Your patch assumes CPP features that are not guaranteed to be present. >> Rejecting this patchset. >> > > The Debug() change is going to give merge conflicts all over > the place when your <https://github.com/ReOpen/ReOpenLDAP> and > OpenLDAP pull updates from each other. > > https://github.com/ReOpen/ReOpenLDAP should not be used for merge, but only for cherry-picking. It has stripped history and merged with OpenLDAP/mdb.master, because mdb.RE/0.9 don't includes a few critical fixes (ITS 7969,7970,7971) for our case. I think that is the reason for patch conflicts. Patchset got RE25 should remain among my sent emails. Presumably I can send it for you again, if you are interested. > How about defining Debug1(), Debug2(), etc for 1,2... arguments? > That's still annoying, but it keeps the code C90-compatible and > keeps -Wformat quiet. There's already similar code for Log<N>() > in include/ldap_log.h, but it was never used much or even merged > into RE24. (Because it got scheduled for RE25 which was going to > start Real Soon Now, IIRC:-) > > -- > Hallvard > IMHO the Log<N> recipe is onerously for support. On other hand, nowadays all non-obsolete compilers support most of C99 features. Baseline features, such as "variadic macro", supported long ago (Microsoft was the most conservative to 2012). Maybe it is time to enable C99 in RE25? Leonid. --f46d04426a60f30297050c76e815 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div><div><br></div></div><div class=3D"gmail_extra"><div = class=3D"gmail_quote">2015-01-12 17:19 GMT+03:00 Hallvard Breien Furuseth <= span dir=3D"ltr"><<a href=3D"mailto:h.b.furuseth@usit.uio.no" target=3D"= _blank">h.b.furuseth(a)usit.uio.no</a>></span>:<br><blockquote class=3D"gm= ail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;border-l= eft-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><span = class=3D"">On 01/06/2015 04:31 PM, <a href=3D"mailto:hyc@symas.com" target= =3D"_blank">hyc(a)symas.com</a> wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;p= adding-left:1ex"> Leonid Yuriev wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;p= adding-left:1ex"> Yes of course, all warnings could be hushed by compiler options, but<br> this also hides the real mistakes.<br> So, I prefer fix the warnings, instead of disable gcc/clang features.<br> <br> Please review attached patchset and merge.<br> </blockquote> <br> Your patch assumes CPP features that are not guaranteed to be present.<br> Rejecting this patchset.<br> </blockquote> <br></span> The Debug() change is going to give merge conflicts all over<br> the place when your <<a href=3D"https://github.com/ReOpen/ReOpenLDAP" ta= rget=3D"_blank">https://github.com/ReOpen/<u></u>ReOpenLDAP</a>> and<br> OpenLDAP pull updates from each other.<br> <br></blockquote><div><a href=3D"https://github.com/ReOpen/ReOpenLDAP">http= s://github.com/ReOpen/ReOpenLDAP</a> should not be used for merge, but only= for cherry-picking.</div><div>It has stripped history and merged with Open= LDAP/mdb.master, because mdb.RE/0.9 don't includes a few critical fixes= (ITS 7969,7970,7971) for our case.</div><div>I think that is the reason fo= r patch conflicts.</div><div>Patchset got RE25 should remain among my sent = emails. Presumably I can send it for you again, if you are interested.</div= ><div>=C2=A0<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px= 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);bor= der-left-style:solid;padding-left:1ex"> How about defining Debug1(), Debug2(), etc for 1,2... arguments?<br> That's still annoying, but it keeps the code C90-compatible and<br> keeps -Wformat quiet.=C2=A0 There's already similar code for Log<N&g= t;()<br> in include/ldap_log.h, but it was never used much or even merged<br> into RE24.=C2=A0 (Because it got scheduled for RE25 which was going to<br> start Real Soon Now, IIRC:-)<span class=3D""><font color=3D"#888888"><br> <br> -- <br> Hallvard<br> </font></span></blockquote></div><br></div><div class=3D"gmail_extra"><div = class=3D"gmail_extra"><div class=3D"gmail_extra">IMHO the Log<N> reci= pe is onerously for support.</div><div class=3D"gmail_extra">On other hand,= nowadays all non-obsolete compilers support most of C99 features.</div><di= v class=3D"gmail_extra">Baseline features, such as "variadic macro&quo= t;, supported long ago (Microsoft was the most conservative to 2012).</div>= <div class=3D"gmail_extra">Maybe it is time to enable C99 in RE25?<br></div= ><div class=3D"gmail_extra"><br></div><div class=3D"gmail_extra">Leonid.</d= iv></div></div></div> --f46d04426a60f30297050c76e815--

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs