openldap-bugs December 2018

openldap-bugs@openldap.org

10 participants
25 discussions

Re: (ITS#8890) Bug fix patches unbreaking the build on x32 systems
by hyc＠symas.com 19 Dec '18

19 Dec '18

Thorsten Glaser wrote: > Howard Chu dixit: >=20 >> The remaining format patches for debug statements will have no impact >> on the correctness of execution or output. This and all of the >> following patches are being rejected. >=20 > There are multiple systems on which sizeof(time_t) > sizeof(long), > with more expected as 2038 dawns. >=20 > Are you saying you want to prohibit users on these platforms from > debugging OpenLDAP? >=20 >>> --- a/contrib/slapd-modules/smbk5pwd/smbk5pwd.c >>> +++ b/contrib/slapd-modules/smbk5pwd/smbk5pwd.c >>> @@ -605,7 +605,7 @@ static int smbk5pwd_exop_passwd( >>> keys[0].bv_val =3D ch_malloc( LDAP_PVT_INTTYPE_CHARS(long) ); >>> keys[0].bv_len =3D snprintf(keys[0].bv_val, >>> LDAP_PVT_INTTYPE_CHARS(long), >>> - "%ld", slap_get_time()); >>> + "%lld", (long long)slap_get_time()); >>> BER_BVZERO( &keys[1] ); >=20 > This doesn=E2=80=99t look like debugging output to me. I already addressed that patch explicitly. "The rest" covers all the othe= rs. You might have correctly identified a Y2038 issue. You certainly have not written a correct fix for it. --=20 -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8890) Bug fix patches unbreaking the build on x32 systems
by tg＠debian.org 19 Dec '18

19 Dec '18

Howard Chu dixit: >The remaining format patches for debug statements will have no impact >on the correctness of execution or output. This and all of the >following patches are being rejected. There are multiple systems on which sizeof(time_t) > sizeof(long), with more expected as 2038 dawns. Are you saying you want to prohibit users on these platforms from debugging OpenLDAP? >> --- a/contrib/slapd-modules/smbk5pwd/smbk5pwd.c >> +++ b/contrib/slapd-modules/smbk5pwd/smbk5pwd.c >> @@ -605,7 +605,7 @@ static int smbk5pwd_exop_passwd( >> =09=09keys[0].bv_val =3D ch_malloc( LDAP_PVT_INTTYPE_CHARS(long) ); >> =09=09keys[0].bv_len =3D snprintf(keys[0].bv_val, >> =09=09=09LDAP_PVT_INTTYPE_CHARS(long), >> -=09=09=09"%ld", slap_get_time()); >> +=09=09=09"%lld", (long long)slap_get_time()); >> =09=09BER_BVZERO( &keys[1] ); This doesn=E2=80=99t look like debugging output to me. bye, //mirabilos --=20 Yes, I hate users and I want them to suffer. =09-- Marco d'Itri on gmane.linux.debian.devel.general

1 0

Re: (ITS#8890) Bug fix patches unbreaking the build on x32 systems
by hyc＠symas.com 19 Dec '18

19 Dec '18

tg(a)debian.org wrote: > Full_Name: mirabilos > Version: 2.4.46 > OS: Debian > URL:=20 > Submission from: (NULL) (2a01:4f8:150:946c::42:3) >=20 >=20 > I=C3=A2=E2=82=AC=E2=84=A2ve just managed to unbreak the build of openld= ap on x32 > (basically amd64ilp32: a 64-bit x86 architecture using 32-bit > =C3=A2=E2=82=AC=C5=93long=C3=A2=E2=82=AC=C2=9D and pointers) by fixing = a couple of simple GCC warnings. >=20 > Original bugreport: https://bugs.debian.org/905237 >=20 > Please find the attached patches (pretty machinal, so not > copyright-relevant) and include them in your next release. > Thanks! >=20 > --- a/clients/tools/common.c > +++ b/clients/tools/common.c > @@ -2326,7 +2326,7 @@ void tool_print_ctrls( > /* known controls */ > for ( j =3D 0; tool_ctrl_response[j].oid !=3D NULL; j++ ) { > if ( strcmp( tool_ctrl_response[j].oid, ctrls[i]->ldctl_oid ) =3D=3D= 0 ) { > - if ( !tool_ctrl_response[j].mask & tool_type ) { > + if ( !(tool_ctrl_response[j].mask & tool_type) ) { > /* this control should not appear > * with this tool; warning? */ > } This is patching an if statement with an empty body - it is a pure no-op = that the compiler will optimize away. There's no point in changing this other = than to shut up stupid static analyzer tools. But since you're not the first p= erson to run a stupid static analyzer tool and point this out, I've merged this= "fix." The following 3 patches are valid and have been merged. > --- a/servers/slapd/backend.c > +++ b/servers/slapd/backend.c > @@ -1500,7 +1500,7 @@ fe_acl_group( > * or if filter parsing fails. > * In the latter case, > * we should give up. */ > - if ( ludp->lud_filter !=3D NULL && ludp->lud_filter !=3D '\0') { > + if ( ludp->lud_filter !=3D NULL && ludp->lud_filter[0] !=3D '\0')= { > filter =3D str2filter_x( op, ludp->lud_filter ); > if ( filter =3D=3D NULL ) { > /* give up... */ > --- a/servers/slapd/overlays/constraint.c > +++ b/servers/slapd/overlays/constraint.c > @@ -446,7 +446,7 @@ constraint_cf_gen( ConfigArgs *c ) > } > =20 > if ( ap.restrict_lud->lud_attrs !=3D NULL ) { > - if ( ap.restrict_lud->lud_attrs[0] !=3D '\0' ) { > + if ( ap.restrict_lud->lud_attrs[0] !=3D NULL ) { > snprintf( c->cr_msg, sizeof( c->cr_msg ), > "%s %s: attrs not allowed in restrict URI %s\n", > c->argv[0], c->argv[1], arg); > --- a/servers/slapd/syntax.c > +++ b/servers/slapd/syntax.c > @@ -219,8 +219,8 @@ syn_add( > } > =20 > assert( (*lsei)->lsei_values !=3D NULL ); > - if ( (*lsei)->lsei_values[0] =3D=3D '\0' > - || (*lsei)->lsei_values[1] !=3D '\0' ) > + if ( (*lsei)->lsei_values[0] =3D=3D NULL > + || (*lsei)->lsei_values[1] !=3D NULL ) > { > Debug( LDAP_DEBUG_ANY, "syn_add(%s): exactly one substitute syntax= must be > present\n", > ssyn->ssyn_syn.syn_oid, 0, 0 ); This chunk no longer exists in the repo, so it's been omitted. > --- a/tests/progs/slapd-addel.c > +++ b/tests/progs/slapd-addel.c > @@ -173,7 +173,7 @@ main( int argc, char **argv ) > =20 > } > =20 > - if (( attrs =3D=3D NULL ) || ( *attrs =3D=3D '\0' )) { > + if (( attrs =3D=3D NULL ) || ( *attrs =3D=3D NULL )) { > =20 > fprintf( stderr, "%s: invalid attrs in file \"%s\".\n", > argv[0], filename ); This patch is trying to cast to a long long and print the result into a b= uffer that is explicitly sized for a long. That's clearly a bug. The remaining = format patches for debug statements will have no impact on the correctness of execution or output. This and all of the following patches are being reje= cted. > --- a/contrib/slapd-modules/smbk5pwd/smbk5pwd.c > +++ b/contrib/slapd-modules/smbk5pwd/smbk5pwd.c > @@ -605,7 +605,7 @@ static int smbk5pwd_exop_passwd( > keys[0].bv_val =3D ch_malloc( LDAP_PVT_INTTYPE_CHARS(long) ); > keys[0].bv_len =3D snprintf(keys[0].bv_val, > LDAP_PVT_INTTYPE_CHARS(long), > - "%ld", slap_get_time()); > + "%lld", (long long)slap_get_time()); > BER_BVZERO( &keys[1] ); > =09 > ml->sml_desc =3D ad_sambaPwdLastSet; > @@ -627,7 +627,7 @@ static int smbk5pwd_exop_passwd( > keys[0].bv_val =3D ch_malloc( LDAP_PVT_INTTYPE_CHARS(long) ); > keys[0].bv_len =3D snprintf(keys[0].bv_val, > LDAP_PVT_INTTYPE_CHARS(long), > - "%ld", slap_get_time() + pi->smb_must_change); > + "%lld", (long long)slap_get_time() + (long long)pi->smb_must_chan= ge); > BER_BVZERO( &keys[1] ); > =20 > ml->sml_desc =3D ad_sambaPwdMustChange; > @@ -650,7 +650,7 @@ static int smbk5pwd_exop_passwd( > keys[0].bv_val =3D ch_malloc( LDAP_PVT_INTTYPE_CHARS(long) ); > keys[0].bv_len =3D snprintf(keys[0].bv_val, > LDAP_PVT_INTTYPE_CHARS(long), > - "%ld", slap_get_time() + pi->smb_can_change); > + "%lld", (long long)slap_get_time() + (long long)pi->smb_can_chang= e); > BER_BVZERO( &keys[1] ); > =20 > ml->sml_desc =3D ad_sambaPwdCanChange; > --- a/libraries/libldap/os-ip.c > +++ b/libraries/libldap/os-ip.c > @@ -282,8 +282,8 @@ ldap_int_poll( > int rc; > =09 > =20 > - osip_debug(ld, "ldap_int_poll: fd: %d tm: %ld\n", > - s, tvp ? tvp->tv_sec : -1L, 0); > + osip_debug(ld, "ldap_int_poll: fd: %d tm: %lld\n", > + s, tvp ? (long long)tvp->tv_sec : -1LL, 0); > =20 > #ifdef HAVE_POLL > { > @@ -432,8 +432,8 @@ ldap_pvt_connect(LDAP *ld, ber_socket_t > opt_tv =3D &tv; > } > =20 > - osip_debug(ld, "ldap_pvt_connect: fd: %d tm: %ld async: %d\n", > - s, opt_tv ? tv.tv_sec : -1L, async); > + osip_debug(ld, "ldap_pvt_connect: fd: %d tm: %lld async: %d\n", > + s, opt_tv ? (long long)tv.tv_sec : -1LL, async); > =20 > if ( opt_tv && ldap_pvt_ndelay_on(ld, s) =3D=3D -1 ) > return ( -1 ); > --- a/libraries/libldap/os-local.c > +++ b/libraries/libldap/os-local.c > @@ -176,8 +176,8 @@ ldap_pvt_connect(LDAP *ld, ber_socket_t > opt_tv =3D &tv; > } > =20 > - oslocal_debug(ld, "ldap_connect_timeout: fd: %d tm: %ld async: %d\n", > - s, opt_tv ? tv.tv_sec : -1L, async); > + oslocal_debug(ld, "ldap_connect_timeout: fd: %d tm: %lld async: %d\n"= , > + s, opt_tv ? (long long)tv.tv_sec : -1LL, async); > =20 > if ( ldap_pvt_ndelay_on(ld, s) =3D=3D -1 ) return -1; > =20 > --- a/libraries/libldap/result.c > +++ b/libraries/libldap/result.c > @@ -264,8 +264,8 @@ wait4msg( > Debug( LDAP_DEBUG_TRACE, "wait4msg ld %p msgid %d (infinite timeout)= \n", > (void *)ld, msgid, 0 ); > } else { > - Debug( LDAP_DEBUG_TRACE, "wait4msg ld %p msgid %d (timeout %ld usec)= \n", > - (void *)ld, msgid, (long)timeout->tv_sec * 1000000 + timeout->tv_us= ec ); > + Debug( LDAP_DEBUG_TRACE, "wait4msg ld %p msgid %d (timeout %lld usec= )\n", > + (void *)ld, msgid, (long long)timeout->tv_sec * 1000000LL + timeout= ->tv_usec > ); > } > #endif /* LDAP_DEBUG */ > =20 > --- a/servers/slapd/back-ldap/bind.c > +++ b/servers/slapd/back-ldap/bind.c > @@ -3008,14 +3008,14 @@ ldap_back_conn2str( const ldapconn_base_ > } > =20 > if ( lc->lcb_create_time !=3D 0 ) { > - len =3D snprintf( tbuf, sizeof(tbuf), "%ld", lc->lcb_create_time ); > + len =3D snprintf( tbuf, sizeof(tbuf), "%lld", (long long)lc->lcb_cre= ate_time > ); > if ( ptr + sizeof(" created=3D") + len >=3D end ) return -1; > ptr =3D lutil_strcopy( ptr, " created=3D" ); > ptr =3D lutil_strcopy( ptr, tbuf ); > } > =20 > if ( lc->lcb_time !=3D 0 ) { > - len =3D snprintf( tbuf, sizeof(tbuf), "%ld", lc->lcb_time ); > + len =3D snprintf( tbuf, sizeof(tbuf), "%lld", (long long)lc->lcb_tim= e ); > if ( ptr + sizeof(" modified=3D") + len >=3D end ) return -1; > ptr =3D lutil_strcopy( ptr, " modified=3D" ); > ptr =3D lutil_strcopy( ptr, tbuf ); > --- a/servers/slapd/back-meta/config.c > +++ b/servers/slapd/back-meta/config.c > @@ -1277,8 +1277,8 @@ meta_back_cf_gen( ConfigArgs *c ) > if ( mc->mc_network_timeout =3D=3D 0 ) { > return 1; > } > - bv.bv_len =3D snprintf( c->cr_msg, sizeof(c->cr_msg), "%ld", > - mc->mc_network_timeout ); > + bv.bv_len =3D snprintf( c->cr_msg, sizeof(c->cr_msg), "%lld", > + (long long)mc->mc_network_timeout ); > bv.bv_val =3D c->cr_msg; > value_add_one( &c->rvalue_vals, &bv ); > break; > --- a/servers/slapd/overlays/dds.c > +++ b/servers/slapd/overlays/dds.c > @@ -417,7 +417,7 @@ dds_op_add( Operation *op, SlapReply *rs > assert( ttl <=3D DDS_RF2589_MAX_TTL ); > =20 > bv.bv_val =3D ttlbuf; > - bv.bv_len =3D snprintf( ttlbuf, sizeof( ttlbuf ), "%ld", ttl ); > + bv.bv_len =3D snprintf( ttlbuf, sizeof( ttlbuf ), "%lld", (long long= )ttl ); > assert( bv.bv_len < sizeof( ttlbuf ) ); > =20 > /* FIXME: apparently, values in op->ora_e are malloc'ed > @@ -695,7 +695,7 @@ dds_op_modify( Operation *op, SlapReply > goto done; > } > =20 > - bv_entryTtl.bv_len =3D snprintf( textbuf, sizeof( textbuf ), "%ld"= , entryTtl > ); > + bv_entryTtl.bv_len =3D snprintf( textbuf, sizeof( textbuf ), "%lld= ", (long > long)entryTtl ); > break; > =20 > default: > @@ -917,7 +917,7 @@ dds_response( Operation *op, SlapReply * > ttl =3D (ttl < 0) ? 0 : ttl; > assert( ttl <=3D DDS_RF2589_MAX_TTL ); > =20 > - len =3D snprintf( ttlbuf, sizeof(ttlbuf), "%ld", ttl ); > + len =3D snprintf( ttlbuf, sizeof(ttlbuf), "%lld", (long long)ttl ); > if ( len < 0 ) > { > goto done; > @@ -1177,7 +1177,7 @@ dds_op_extended( Operation *op, SlapRepl > ttlmod.sml_values =3D ttlvalues; > ttlmod.sml_numvals =3D 1; > ttlvalues[ 0 ].bv_val =3D ttlbuf; > - ttlvalues[ 0 ].bv_len =3D snprintf( ttlbuf, sizeof( ttlbuf ), "%ld",= ttl ); > + ttlvalues[ 0 ].bv_len =3D snprintf( ttlbuf, sizeof( ttlbuf ), "%lld"= , (long > long)ttl ); > BER_BVZERO( &ttlvalues[ 1 ] ); > =20 > /* the entryExpireTimestamp is added by modify */ > @@ -1205,8 +1205,8 @@ dds_op_extended( Operation *op, SlapRepl > rs->sr_rspoid =3D ch_strdup( slap_EXOP_REFRESH.bv_val ); > =20 > Log3( LDAP_DEBUG_TRACE, LDAP_LEVEL_INFO, > - "%s REFRESH dn=3D\"%s\" TTL=3D%ld\n", > - op->o_log_prefix, op->o_req_ndn.bv_val, ttl ); > + "%s REFRESH dn=3D\"%s\" TTL=3D%lld\n", > + op->o_log_prefix, op->o_req_ndn.bv_val, (long long)ttl ); > } > =20 > ber_free_buf( ber ); > --- a/servers/slapd/overlays/pcache.c > +++ b/servers/slapd/overlays/pcache.c > @@ -2728,8 +2728,8 @@ pc_bind_search( Operation *op, SlapReply > pbi->bi_flags |=3D BI_HASHED; > } else { > Debug( pcache_debug, "pc_bind_search: cache is stale, " > - "reftime: %ld, current time: %ld\n", > - pbi->bi_cq->bindref_time, op->o_time, 0 ); > + "reftime: %lld, current time: %lld\n", > + (long long)pbi->bi_cq->bindref_time, (long long)op->o_time, 0 ); > } > } else if ( pbi->bi_si ) { > /* This search result is going into the cache */ > @@ -3831,9 +3831,9 @@ pc_cf_gen( ConfigArgs *c ) > struct berval bv; > switch( c->type ) { > case PC_MAIN: > - bv.bv_len =3D snprintf( c->cr_msg, sizeof( c->cr_msg ), "%s %d %d %= d %ld", > + bv.bv_len =3D snprintf( c->cr_msg, sizeof( c->cr_msg ), "%s %d %d %= d %lld", > cm->db.bd_info->bi_type, cm->max_entries, cm->numattrsets, > - cm->num_entries_limit, cm->cc_period ); > + cm->num_entries_limit, (long long)cm->cc_period ); > bv.bv_val =3D c->cr_msg; > value_add_one( &c->rvalue_vals, &bv ); > break; > @@ -3875,12 +3875,12 @@ pc_cf_gen( ConfigArgs *c ) > /* HEADS-UP: always print all; > * if optional =3D=3D 0, ignore */ > bv.bv_len =3D snprintf( c->cr_msg, sizeof( c->cr_msg ), > - " %d %ld %ld %ld %ld", > + " %d %lld %lld %lld %lld", > temp->attr_set_index, > - temp->ttl, > - temp->negttl, > - temp->limitttl, > - temp->ttr ); > + (long long)temp->ttl, > + (long long)temp->negttl, > + (long long)temp->limitttl, > + (long long)temp->ttr ); > bv.bv_len +=3D temp->querystr.bv_len + 2; > bv.bv_val =3D ch_malloc( bv.bv_len+1 ); > ptr =3D bv.bv_val; > @@ -3897,9 +3897,9 @@ pc_cf_gen( ConfigArgs *c ) > for (temp=3Dqm->templates; temp; temp=3Dtemp->qmnext) { > if ( !temp->bindttr ) continue; > bv.bv_len =3D snprintf( c->cr_msg, sizeof( c->cr_msg ), > - " %d %ld %s ", > + " %d %lld %s ", > temp->attr_set_index, > - temp->bindttr, > + (long long)temp->bindttr, > ldap_pvt_scope2str( temp->bindscope )); > bv.bv_len +=3D temp->bindbase.bv_len + temp->bindftemp.bv_len + 4; > bv.bv_val =3D ch_malloc( bv.bv_len + 1 ); > --- a/servers/slapd/syncrepl.c > +++ b/servers/slapd/syncrepl.c > @@ -2395,8 +2395,8 @@ syncrepl_message_to_op( > op->o_callback =3D &cb; > slap_op_time( &op->o_time, &op->o_tincr ); > =20 > - Debug( LDAP_DEBUG_SYNC, "syncrepl_message_to_op: %s tid %x\n", > - si->si_ridtxt, op->o_tid, 0 ); > + Debug( LDAP_DEBUG_SYNC, "syncrepl_message_to_op: %s tid %lx\n", > + si->si_ridtxt, (unsigned long)op->o_tid, 0 ); > =20 > switch( op->o_tag ) { > case LDAP_REQ_ADD: > @@ -2905,8 +2905,8 @@ syncrepl_entry( > int freecsn =3D 1; > =20 > Debug( LDAP_DEBUG_SYNC, > - "syncrepl_entry: %s LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_%s) tid %x\n", > - si->si_ridtxt, syncrepl_state2str( syncstate ), op->o_tid ); > + "syncrepl_entry: %s LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_%s) tid %lx\n", > + si->si_ridtxt, syncrepl_state2str( syncstate ), (unsigned long)op->o= _tid ); > =20 > if (( syncstate =3D=3D LDAP_SYNC_PRESENT || syncstate =3D=3D LDAP_SYN= C_ADD ) ) { > if ( !si->si_refreshPresent && !si->si_refreshDone ) { >=20 >=20 --=20 -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8944) Support Microsoft AD DirSync as a consumer
by hyc＠openldap.org 18 Dec '18

18 Dec '18

Full_Name: Howard Chu Version: 2.5 OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (80.233.39.182) Submitted by: hyc Code in master is now available for operating slapd as a replication consumer against Microsoft ActiveDirectory using their DirSync protocol. Note that no attempt has been made to work with anything other than user and group entries. A sample config, based on the test017 syncrepl consumer config: #### include ./schema/core.schema include ./schema/cosine.schema include ./schema/inetorgperson.schema include ./schema/nis.schema include ./schema/msuser.schema attributeoptions range= database mdb suffix "dc=ldapsync,dc=local" rootdn "cn=Replica,dc=ldapsync,dc=local" rootpw secret directory ./testrun/db.2.a index objectClass eq index cn,sn,uid pres,eq,sub index entryUUID,entryCSN eq syncrepl rid=1 provider=ldap://ldapsync/ binddn="cn=Administrator,cn=users,dc=ldapsync,dc=local" bindmethod=simple credentials=MSAD-secret searchbase="dc=ldapsync,dc=local" filter="(|(objectClass=user)(objectclass=group))" schemachecking=off scope=sub type=dirSync interval=00:00:00:03 updateref ldap://ldapsync/ database monitor #### Note that DirSync doesn't have a persist mode, it only works by refreshOnly-style polling. DirSync support for Modifications is quite braindead; the protocol has no way to indicate deletion of single-valued attributes. This has been left as-is for now, possibly some workarounds can be added for that later.

1 0

Re: (ITS#8943) speed up parallel add/delete
by derek＠shannon-data.com 13 Dec '18

13 Dec '18

On Thursday, December 13, 2018 07:54:12 AM Derek Zhou wrote: > > On December 13, 2018 1:47:44 AM GMT+08:00, Quanah Gibson-Mount <quanah(a)symas.com> wrote: > >Hi, > > > >Thanks for the report! I have some questions: > > > >a) Did you test with writemap enabled for mdb? > No, but I did test the unpatched slapd with writemap. it was not much faster. I can test the patch with writemap to see what happens. I tested my patch with writemap, it works with same marginal performance gain like without the patch. -- Derek Zhou Shannon Systems http://www.shannon-sys.com

1 0

Re: (ITS#8943) speed up parallel add/delete
by derek＠shannon-data.com 13 Dec '18

13 Dec '18

On December 13, 2018 1:47:44 AM GMT+08:00, Quanah Gibson-Mount <quanah@sym= as=2Ecom> wrote: >Hi, > >Thanks for the report! I have some questions: > >a) Did you test with writemap enabled for mdb? No, but I did test the unpatched slapd with writemap=2E it was not much fa= ster=2E I can test the patch with writemap to see what happens=2E > >b) Did you test the effect on other operating systems? (I=2Ee=2E, Windows= ,=20 >Solaris, FreeBSD, etc)? No , I dont have any=2E The only thing I have is a x86-64 pc with linux=2E= I did try to avoid touching any os specific codes > >c) Did you test multiple filesystems under Linux(I=2Ee=2E, ext2, ext3, >ext4,=20 >xfs, etc)? > no=2E I only tested with ext4=2E >d) What filesystem did you use when testing? Was journaling enabled or > >disabled? What was the default flush behavior of the filesystem you >used? I used nobarrier, data=3Dwriteback mount options, which are faster than th= e default, but not significantly=2E I am using a very fast SSD though=2E >I=2Ee=2E, if you set dbnosync at the mdb level, the underlying filesystem >flush=20 >behavior will then be in effect (with ext4, for example, this defaults >to 5=20 >seconds) something like that=2E it also depends on vm setting, like dirty ratio=2E > >e) Did you test when the database is also writing to an underlying=20 >accesslog database that is also backed by mdb? > no, I dont konw how=2E --=20 Sent from my Android device with K-9 Mail=2E Please excuse my brevity=2E

1 0

Re: (ITS#8943) speed up parallel add/delete
by quanah＠symas.com 12 Dec '18

12 Dec '18

--On Wednesday, December 12, 2018 3:54 AM +0000 derek(a)shannon-data.com wrote: > Full_Name: Derek Zhou > Version: 2.4 git HEAD > OS: linux > URL: > Submission from: (NULL) (116.236.169.22) > > > the mdb backend does not really support parallelism in rw transactions, so > add/delete are serialized by a mutex inside the mdb backend as small > transactions. In my patch, all add/delete are put in a queue and executed > in a dedicated worker thread. It may seem pointless at first, but by > doing them all in the same thread, now I can merge concurrent ops from > different clients into larger transactions, and reduce the number of > expensive txn_commit calls (fsyncs). Hi, Thanks for the report! I have some questions: a) Did you test with writemap enabled for mdb? b) Did you test the effect on other operating systems? (I.e., Windows, Solaris, FreeBSD, etc)? c) Did you test multiple filesystems under Linux(I.e., ext2, ext3, ext4, xfs, etc)? d) What filesystem did you use when testing? Was journaling enabled or disabled? What was the default flush behavior of the filesystem you used? I.e., if you set dbnosync at the mdb level, the underlying filesystem flush behavior will then be in effect (with ext4, for example, this defaults to 5 seconds) e) Did you test when the database is also writing to an underlying accesslog database that is also backed by mdb? Thanks, Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

(ITS#8943) speed up parallel add/delete
by derek＠shannon-data.com 12 Dec '18

12 Dec '18

Full_Name: Derek Zhou Version: 2.4 git HEAD OS: linux URL: Submission from: (NULL) (116.236.169.22) the mdb backend does not really support parallelism in rw transactions, so add/delete are serialized by a mutex inside the mdb backend as small transactions. In my patch, all add/delete are put in a queue and executed in a dedicated worker thread. It may seem pointless at first, but by doing them all in the same thread, now I can merge concurrent ops from different clients into larger transactions, and reduce the number of expensive txn_commit calls (fsyncs). http://proxy.shannon-data.com/~derek/patches/openldap/0001-add-rw_worker-in… http://proxy.shannon-data.com/~derek/patches/openldap/0002-add-config-varia… Legal notice ============ The attached patch file is derived from OpenLDAP Software. All of the modifications to OpenLDAP Software represented in the following patch(es) were developed by Derek Zhou <derek(a)shannon-data.com>. I have not assigned rights and/or interest in this work to any party. I, Derek Zhou, hereby place the following modifications to OpenLDAP Software (and only these modifications) into the public domain. Hence, these modifications may be freely used and/or redistributed for any purpose with or without attribution and/or other notice.

1 0

Re: (ITS#8847) New LDAP URL syntax to support binding to specific IP address at client side
by hyc＠symas.com 11 Dec '18

11 Dec '18

Sharma, Ramakant 2. (Nokia - IN/Bangalore) wrote: >>> The work of parse_and_validate is wasted since you call inet_pton bot= h for >>>validating and for using the result later. You should simply save the = IPv4 and IPv6 binary >>>addresses when the validation succeeds. There should be no get_IPvX_fr= om_lst() functions >>>at all. >=20 > I am bit confused regarding this point. >=20 > The intent of parse_and_validate_sourceip() is to set only valid intern= et address=A0 to=A0 "ldo_local_IP_addresses" so that we don=92t overwrite > =93ldo_local_IP_addresses=94 in case user provides some invalid address= es either from ldap.conf or from application. Sure, the validation step is required. The result of validation is either= a struct in_addr or a struct in6_addr. You should simply store them directly in the ldapoptions struct. Maybe ad= d a flag to tell that each one has been set. > Now during connect, the need is to get either IPv4 or IPv6 internet add= ress based on target LDAP server's address family type. So, once it is kn= own that target > LDAP server is using IPv4 or IPv6. Code will fetch the related configur= ed bind IP from "ldo_local_IP_addresses". Yes. All that should be required here is to fetch the struct in_addr or i= n6_addr value accordingly. >=20 > Since at this step we are not sure at which index the related bind IP i= s present so, I used inet_pton call to get the matching internet bind IP = (internet > address) from the list (intent was not to use binary address) This is stupid and wasted work. > If the concern is to avoid inet_pton call at this stage then, I can use= "." to search IPv4 and ":" to search IPv6 address in respective getters = (because we > already know that at this stage the fetched IP will always be valid) This is stupid and wasted work. > Kindly let me know your opinion. >=20 > =A0 >=20 > BR, >=20 > Ramakant Sharma >=20 > Senior R&D Engineer >=20 > Nokia Networks, Bangalore >=20 > =A0 >=20 --=20 -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8847) New LDAP URL syntax to support binding to specific IP address at client side
by hyc＠symas.com 06 Dec '18

06 Dec '18

Thanks for the updated patch. diff --git a/include/ldap.h b/include/ldap.h index c58c576d3..78e07d49f 100644 --- a/include/ldap.h +++ b/include/ldap.h @@ -138,6 +138,7 @@ LDAP_BEGIN_DECL #define LDAP_OPT_CONNECT_ASYNC 0x5010 /* create connections asynchronously */ #define LDAP_OPT_CONNECT_CB 0x5011 /* connection callbacks */ #define LDAP_OPT_SESSION_REFCNT 0x5012 /* session reference count */ +#define LDAP_OPT_SOCKET_BIND_ADDRESSES 0x5013 /* user configured bind IPs */ /* OpenLDAP TLS options */ #define LDAP_OPT_X_TLS 0x6000 You've got strange whitespace here. Please re-read the CONTRIBUTING docs; we use 4-column tab stops. diff --git a/libraries/libldap/bind_ip.c b/libraries/libldap/bind_ip.c new file mode 100644 index 000000000..56eddda43 --- /dev/null +++ b/libraries/libldap/bind_ip.c @@ -0,0 +1,102 @@ +/* LIBLDAP bind_ip.c*/ +/* $OpenLDAP$ */ +/* This work is part of OpenLDAP Software <http://www.openldap.org/>. + * I see no reason why these functions are exposed in their own source file or exported from libldap. parse_and_validate_sourceip() could just be a static function inside options.c get_IPvX_from_lst() can just be statics in os-ip.c. In particular, you've defined these as global but they don't have an ldap_ prefix, which is just wrong. The work of parse_and_validate is wasted since you call inet_pton both for validating and for using the result later. You should simply save the IPv4 and IPv6 binary addresses when the validation succeeds. There should be no get_IPvX_from_lst() functions at all. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs December 2018