Re: (ITS#8890) Bug fix patches unbreaking the build on x32 systems
by hyc＠symas.com
Thorsten Glaser wrote:
> Howard Chu dixit:
>=20
>> The remaining format patches for debug statements will have no impact
>> on the correctness of execution or output. This and all of the
>> following patches are being rejected.
>=20
> There are multiple systems on which sizeof(time_t) > sizeof(long),
> with more expected as 2038 dawns.
>=20
> Are you saying you want to prohibit users on these platforms from
> debugging OpenLDAP?
>=20
>>> --- a/contrib/slapd-modules/smbk5pwd/smbk5pwd.c
>>> +++ b/contrib/slapd-modules/smbk5pwd/smbk5pwd.c
>>> @@ -605,7 +605,7 @@ static int smbk5pwd_exop_passwd(
>>> keys[0].bv_val =3D ch_malloc( LDAP_PVT_INTTYPE_CHARS(long) );
>>> keys[0].bv_len =3D snprintf(keys[0].bv_val,
>>> LDAP_PVT_INTTYPE_CHARS(long),
>>> - "%ld", slap_get_time());
>>> + "%lld", (long long)slap_get_time());
>>> BER_BVZERO( &keys[1] );
>=20
> This doesn=E2=80=99t look like debugging output to me.
I already addressed that patch explicitly. "The rest" covers all the othe=
rs.
You might have correctly identified a Y2038 issue. You certainly have not
written a correct fix for it.
--=20
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
3 years, 7 months
Re: (ITS#8890) Bug fix patches unbreaking the build on x32 systems
by tg＠debian.org
Howard Chu dixit:
>The remaining format patches for debug statements will have no impact
>on the correctness of execution or output. This and all of the
>following patches are being rejected.
There are multiple systems on which sizeof(time_t) > sizeof(long),
with more expected as 2038 dawns.
Are you saying you want to prohibit users on these platforms from
debugging OpenLDAP?
>> --- a/contrib/slapd-modules/smbk5pwd/smbk5pwd.c
>> +++ b/contrib/slapd-modules/smbk5pwd/smbk5pwd.c
>> @@ -605,7 +605,7 @@ static int smbk5pwd_exop_passwd(
>> =09=09keys[0].bv_val =3D ch_malloc( LDAP_PVT_INTTYPE_CHARS(long) );
>> =09=09keys[0].bv_len =3D snprintf(keys[0].bv_val,
>> =09=09=09LDAP_PVT_INTTYPE_CHARS(long),
>> -=09=09=09"%ld", slap_get_time());
>> +=09=09=09"%lld", (long long)slap_get_time());
>> =09=09BER_BVZERO( &keys[1] );
This doesn=E2=80=99t look like debugging output to me.
bye,
//mirabilos
--=20
Yes, I hate users and I want them to suffer.
=09-- Marco d'Itri on gmane.linux.debian.devel.general
3 years, 7 months
Re: (ITS#8890) Bug fix patches unbreaking the build on x32 systems
by hyc＠symas.com
tg(a)debian.org wrote:
> Full_Name: mirabilos
> Version: 2.4.46
> OS: Debian
> URL:=20
> Submission from: (NULL) (2a01:4f8:150:946c::42:3)
>=20
>=20
> I=C3=A2=E2=82=AC=E2=84=A2ve just managed to unbreak the build of openld=
ap on x32
> (basically amd64ilp32: a 64-bit x86 architecture using 32-bit
> =C3=A2=E2=82=AC=C5=93long=C3=A2=E2=82=AC=C2=9D and pointers) by fixing =
a couple of simple GCC warnings.
>=20
> Original bugreport: https://bugs.debian.org/905237
>=20
> Please find the attached patches (pretty machinal, so not
> copyright-relevant) and include them in your next release.
> Thanks!
>=20
> --- a/clients/tools/common.c
> +++ b/clients/tools/common.c
> @@ -2326,7 +2326,7 @@ void tool_print_ctrls(
> /* known controls */
> for ( j =3D 0; tool_ctrl_response[j].oid !=3D NULL; j++ ) {
> if ( strcmp( tool_ctrl_response[j].oid, ctrls[i]->ldctl_oid ) =3D=3D=
0 ) {
> - if ( !tool_ctrl_response[j].mask & tool_type ) {
> + if ( !(tool_ctrl_response[j].mask & tool_type) ) {
> /* this control should not appear
> * with this tool; warning? */
> }
This is patching an if statement with an empty body - it is a pure no-op =
that
the compiler will optimize away. There's no point in changing this other =
than
to shut up stupid static analyzer tools. But since you're not the first p=
erson
to run a stupid static analyzer tool and point this out, I've merged this=
"fix."
The following 3 patches are valid and have been merged.
> --- a/servers/slapd/backend.c
> +++ b/servers/slapd/backend.c
> @@ -1500,7 +1500,7 @@ fe_acl_group(
> * or if filter parsing fails.
> * In the latter case,
> * we should give up. */
> - if ( ludp->lud_filter !=3D NULL && ludp->lud_filter !=3D '\0') {
> + if ( ludp->lud_filter !=3D NULL && ludp->lud_filter[0] !=3D '\0')=
{
> filter =3D str2filter_x( op, ludp->lud_filter );
> if ( filter =3D=3D NULL ) {
> /* give up... */
> --- a/servers/slapd/overlays/constraint.c
> +++ b/servers/slapd/overlays/constraint.c
> @@ -446,7 +446,7 @@ constraint_cf_gen( ConfigArgs *c )
> }
> =20
> if ( ap.restrict_lud->lud_attrs !=3D NULL ) {
> - if ( ap.restrict_lud->lud_attrs[0] !=3D '\0' ) {
> + if ( ap.restrict_lud->lud_attrs[0] !=3D NULL ) {
> snprintf( c->cr_msg, sizeof( c->cr_msg ),
> "%s %s: attrs not allowed in restrict URI %s\n",
> c->argv[0], c->argv[1], arg);
> --- a/servers/slapd/syntax.c
> +++ b/servers/slapd/syntax.c
> @@ -219,8 +219,8 @@ syn_add(
> }
> =20
> assert( (*lsei)->lsei_values !=3D NULL );
> - if ( (*lsei)->lsei_values[0] =3D=3D '\0'
> - || (*lsei)->lsei_values[1] !=3D '\0' )
> + if ( (*lsei)->lsei_values[0] =3D=3D NULL
> + || (*lsei)->lsei_values[1] !=3D NULL )
> {
> Debug( LDAP_DEBUG_ANY, "syn_add(%s): exactly one substitute syntax=
must be
> present\n",
> ssyn->ssyn_syn.syn_oid, 0, 0 );
This chunk no longer exists in the repo, so it's been omitted.
> --- a/tests/progs/slapd-addel.c
> +++ b/tests/progs/slapd-addel.c
> @@ -173,7 +173,7 @@ main( int argc, char **argv )
> =20
> }
> =20
> - if (( attrs =3D=3D NULL ) || ( *attrs =3D=3D '\0' )) {
> + if (( attrs =3D=3D NULL ) || ( *attrs =3D=3D NULL )) {
> =20
> fprintf( stderr, "%s: invalid attrs in file \"%s\".\n",
> argv[0], filename );
This patch is trying to cast to a long long and print the result into a b=
uffer
that is explicitly sized for a long. That's clearly a bug. The remaining =
format
patches for debug statements will have no impact on the correctness of
execution or output. This and all of the following patches are being reje=
cted.
> --- a/contrib/slapd-modules/smbk5pwd/smbk5pwd.c
> +++ b/contrib/slapd-modules/smbk5pwd/smbk5pwd.c
> @@ -605,7 +605,7 @@ static int smbk5pwd_exop_passwd(
> keys[0].bv_val =3D ch_malloc( LDAP_PVT_INTTYPE_CHARS(long) );
> keys[0].bv_len =3D snprintf(keys[0].bv_val,
> LDAP_PVT_INTTYPE_CHARS(long),
> - "%ld", slap_get_time());
> + "%lld", (long long)slap_get_time());
> BER_BVZERO( &keys[1] );
> =09
> ml->sml_desc =3D ad_sambaPwdLastSet;
> @@ -627,7 +627,7 @@ static int smbk5pwd_exop_passwd(
> keys[0].bv_val =3D ch_malloc( LDAP_PVT_INTTYPE_CHARS(long) );
> keys[0].bv_len =3D snprintf(keys[0].bv_val,
> LDAP_PVT_INTTYPE_CHARS(long),
> - "%ld", slap_get_time() + pi->smb_must_change);
> + "%lld", (long long)slap_get_time() + (long long)pi->smb_must_chan=
ge);
> BER_BVZERO( &keys[1] );
> =20
> ml->sml_desc =3D ad_sambaPwdMustChange;
> @@ -650,7 +650,7 @@ static int smbk5pwd_exop_passwd(
> keys[0].bv_val =3D ch_malloc( LDAP_PVT_INTTYPE_CHARS(long) );
> keys[0].bv_len =3D snprintf(keys[0].bv_val,
> LDAP_PVT_INTTYPE_CHARS(long),
> - "%ld", slap_get_time() + pi->smb_can_change);
> + "%lld", (long long)slap_get_time() + (long long)pi->smb_can_chang=
e);
> BER_BVZERO( &keys[1] );
> =20
> ml->sml_desc =3D ad_sambaPwdCanChange;
> --- a/libraries/libldap/os-ip.c
> +++ b/libraries/libldap/os-ip.c
> @@ -282,8 +282,8 @@ ldap_int_poll(
> int rc;
> =09
> =20
> - osip_debug(ld, "ldap_int_poll: fd: %d tm: %ld\n",
> - s, tvp ? tvp->tv_sec : -1L, 0);
> + osip_debug(ld, "ldap_int_poll: fd: %d tm: %lld\n",
> + s, tvp ? (long long)tvp->tv_sec : -1LL, 0);
> =20
> #ifdef HAVE_POLL
> {
> @@ -432,8 +432,8 @@ ldap_pvt_connect(LDAP *ld, ber_socket_t
> opt_tv =3D &tv;
> }
> =20
> - osip_debug(ld, "ldap_pvt_connect: fd: %d tm: %ld async: %d\n",
> - s, opt_tv ? tv.tv_sec : -1L, async);
> + osip_debug(ld, "ldap_pvt_connect: fd: %d tm: %lld async: %d\n",
> + s, opt_tv ? (long long)tv.tv_sec : -1LL, async);
> =20
> if ( opt_tv && ldap_pvt_ndelay_on(ld, s) =3D=3D -1 )
> return ( -1 );
> --- a/libraries/libldap/os-local.c
> +++ b/libraries/libldap/os-local.c
> @@ -176,8 +176,8 @@ ldap_pvt_connect(LDAP *ld, ber_socket_t
> opt_tv =3D &tv;
> }
> =20
> - oslocal_debug(ld, "ldap_connect_timeout: fd: %d tm: %ld async: %d\n",
> - s, opt_tv ? tv.tv_sec : -1L, async);
> + oslocal_debug(ld, "ldap_connect_timeout: fd: %d tm: %lld async: %d\n"=
,
> + s, opt_tv ? (long long)tv.tv_sec : -1LL, async);
> =20
> if ( ldap_pvt_ndelay_on(ld, s) =3D=3D -1 ) return -1;
> =20
> --- a/libraries/libldap/result.c
> +++ b/libraries/libldap/result.c
> @@ -264,8 +264,8 @@ wait4msg(
> Debug( LDAP_DEBUG_TRACE, "wait4msg ld %p msgid %d (infinite timeout)=
\n",
> (void *)ld, msgid, 0 );
> } else {
> - Debug( LDAP_DEBUG_TRACE, "wait4msg ld %p msgid %d (timeout %ld usec)=
\n",
> - (void *)ld, msgid, (long)timeout->tv_sec * 1000000 + timeout->tv_us=
ec );
> + Debug( LDAP_DEBUG_TRACE, "wait4msg ld %p msgid %d (timeout %lld usec=
)\n",
> + (void *)ld, msgid, (long long)timeout->tv_sec * 1000000LL + timeout=
->tv_usec
> );
> }
> #endif /* LDAP_DEBUG */
> =20
> --- a/servers/slapd/back-ldap/bind.c
> +++ b/servers/slapd/back-ldap/bind.c
> @@ -3008,14 +3008,14 @@ ldap_back_conn2str( const ldapconn_base_
> }
> =20
> if ( lc->lcb_create_time !=3D 0 ) {
> - len =3D snprintf( tbuf, sizeof(tbuf), "%ld", lc->lcb_create_time );
> + len =3D snprintf( tbuf, sizeof(tbuf), "%lld", (long long)lc->lcb_cre=
ate_time
> );
> if ( ptr + sizeof(" created=3D") + len >=3D end ) return -1;
> ptr =3D lutil_strcopy( ptr, " created=3D" );
> ptr =3D lutil_strcopy( ptr, tbuf );
> }
> =20
> if ( lc->lcb_time !=3D 0 ) {
> - len =3D snprintf( tbuf, sizeof(tbuf), "%ld", lc->lcb_time );
> + len =3D snprintf( tbuf, sizeof(tbuf), "%lld", (long long)lc->lcb_tim=
e );
> if ( ptr + sizeof(" modified=3D") + len >=3D end ) return -1;
> ptr =3D lutil_strcopy( ptr, " modified=3D" );
> ptr =3D lutil_strcopy( ptr, tbuf );
> --- a/servers/slapd/back-meta/config.c
> +++ b/servers/slapd/back-meta/config.c
> @@ -1277,8 +1277,8 @@ meta_back_cf_gen( ConfigArgs *c )
> if ( mc->mc_network_timeout =3D=3D 0 ) {
> return 1;
> }
> - bv.bv_len =3D snprintf( c->cr_msg, sizeof(c->cr_msg), "%ld",
> - mc->mc_network_timeout );
> + bv.bv_len =3D snprintf( c->cr_msg, sizeof(c->cr_msg), "%lld",
> + (long long)mc->mc_network_timeout );
> bv.bv_val =3D c->cr_msg;
> value_add_one( &c->rvalue_vals, &bv );
> break;
> --- a/servers/slapd/overlays/dds.c
> +++ b/servers/slapd/overlays/dds.c
> @@ -417,7 +417,7 @@ dds_op_add( Operation *op, SlapReply *rs
> assert( ttl <=3D DDS_RF2589_MAX_TTL );
> =20
> bv.bv_val =3D ttlbuf;
> - bv.bv_len =3D snprintf( ttlbuf, sizeof( ttlbuf ), "%ld", ttl );
> + bv.bv_len =3D snprintf( ttlbuf, sizeof( ttlbuf ), "%lld", (long long=
)ttl );
> assert( bv.bv_len < sizeof( ttlbuf ) );
> =20
> /* FIXME: apparently, values in op->ora_e are malloc'ed
> @@ -695,7 +695,7 @@ dds_op_modify( Operation *op, SlapReply
> goto done;
> }
> =20
> - bv_entryTtl.bv_len =3D snprintf( textbuf, sizeof( textbuf ), "%ld"=
, entryTtl
> );
> + bv_entryTtl.bv_len =3D snprintf( textbuf, sizeof( textbuf ), "%lld=
", (long
> long)entryTtl );
> break;
> =20
> default:
> @@ -917,7 +917,7 @@ dds_response( Operation *op, SlapReply *
> ttl =3D (ttl < 0) ? 0 : ttl;
> assert( ttl <=3D DDS_RF2589_MAX_TTL );
> =20
> - len =3D snprintf( ttlbuf, sizeof(ttlbuf), "%ld", ttl );
> + len =3D snprintf( ttlbuf, sizeof(ttlbuf), "%lld", (long long)ttl );
> if ( len < 0 )
> {
> goto done;
> @@ -1177,7 +1177,7 @@ dds_op_extended( Operation *op, SlapRepl
> ttlmod.sml_values =3D ttlvalues;
> ttlmod.sml_numvals =3D 1;
> ttlvalues[ 0 ].bv_val =3D ttlbuf;
> - ttlvalues[ 0 ].bv_len =3D snprintf( ttlbuf, sizeof( ttlbuf ), "%ld",=
ttl );
> + ttlvalues[ 0 ].bv_len =3D snprintf( ttlbuf, sizeof( ttlbuf ), "%lld"=
, (long
> long)ttl );
> BER_BVZERO( &ttlvalues[ 1 ] );
> =20
> /* the entryExpireTimestamp is added by modify */
> @@ -1205,8 +1205,8 @@ dds_op_extended( Operation *op, SlapRepl
> rs->sr_rspoid =3D ch_strdup( slap_EXOP_REFRESH.bv_val );
> =20
> Log3( LDAP_DEBUG_TRACE, LDAP_LEVEL_INFO,
> - "%s REFRESH dn=3D\"%s\" TTL=3D%ld\n",
> - op->o_log_prefix, op->o_req_ndn.bv_val, ttl );
> + "%s REFRESH dn=3D\"%s\" TTL=3D%lld\n",
> + op->o_log_prefix, op->o_req_ndn.bv_val, (long long)ttl );
> }
> =20
> ber_free_buf( ber );
> --- a/servers/slapd/overlays/pcache.c
> +++ b/servers/slapd/overlays/pcache.c
> @@ -2728,8 +2728,8 @@ pc_bind_search( Operation *op, SlapReply
> pbi->bi_flags |=3D BI_HASHED;
> } else {
> Debug( pcache_debug, "pc_bind_search: cache is stale, "
> - "reftime: %ld, current time: %ld\n",
> - pbi->bi_cq->bindref_time, op->o_time, 0 );
> + "reftime: %lld, current time: %lld\n",
> + (long long)pbi->bi_cq->bindref_time, (long long)op->o_time, 0 );
> }
> } else if ( pbi->bi_si ) {
> /* This search result is going into the cache */
> @@ -3831,9 +3831,9 @@ pc_cf_gen( ConfigArgs *c )
> struct berval bv;
> switch( c->type ) {
> case PC_MAIN:
> - bv.bv_len =3D snprintf( c->cr_msg, sizeof( c->cr_msg ), "%s %d %d %=
d %ld",
> + bv.bv_len =3D snprintf( c->cr_msg, sizeof( c->cr_msg ), "%s %d %d %=
d %lld",
> cm->db.bd_info->bi_type, cm->max_entries, cm->numattrsets,
> - cm->num_entries_limit, cm->cc_period );
> + cm->num_entries_limit, (long long)cm->cc_period );
> bv.bv_val =3D c->cr_msg;
> value_add_one( &c->rvalue_vals, &bv );
> break;
> @@ -3875,12 +3875,12 @@ pc_cf_gen( ConfigArgs *c )
> /* HEADS-UP: always print all;
> * if optional =3D=3D 0, ignore */
> bv.bv_len =3D snprintf( c->cr_msg, sizeof( c->cr_msg ),
> - " %d %ld %ld %ld %ld",
> + " %d %lld %lld %lld %lld",
> temp->attr_set_index,
> - temp->ttl,
> - temp->negttl,
> - temp->limitttl,
> - temp->ttr );
> + (long long)temp->ttl,
> + (long long)temp->negttl,
> + (long long)temp->limitttl,
> + (long long)temp->ttr );
> bv.bv_len +=3D temp->querystr.bv_len + 2;
> bv.bv_val =3D ch_malloc( bv.bv_len+1 );
> ptr =3D bv.bv_val;
> @@ -3897,9 +3897,9 @@ pc_cf_gen( ConfigArgs *c )
> for (temp=3Dqm->templates; temp; temp=3Dtemp->qmnext) {
> if ( !temp->bindttr ) continue;
> bv.bv_len =3D snprintf( c->cr_msg, sizeof( c->cr_msg ),
> - " %d %ld %s ",
> + " %d %lld %s ",
> temp->attr_set_index,
> - temp->bindttr,
> + (long long)temp->bindttr,
> ldap_pvt_scope2str( temp->bindscope ));
> bv.bv_len +=3D temp->bindbase.bv_len + temp->bindftemp.bv_len + 4;
> bv.bv_val =3D ch_malloc( bv.bv_len + 1 );
> --- a/servers/slapd/syncrepl.c
> +++ b/servers/slapd/syncrepl.c
> @@ -2395,8 +2395,8 @@ syncrepl_message_to_op(
> op->o_callback =3D &cb;
> slap_op_time( &op->o_time, &op->o_tincr );
> =20
> - Debug( LDAP_DEBUG_SYNC, "syncrepl_message_to_op: %s tid %x\n",
> - si->si_ridtxt, op->o_tid, 0 );
> + Debug( LDAP_DEBUG_SYNC, "syncrepl_message_to_op: %s tid %lx\n",
> + si->si_ridtxt, (unsigned long)op->o_tid, 0 );
> =20
> switch( op->o_tag ) {
> case LDAP_REQ_ADD:
> @@ -2905,8 +2905,8 @@ syncrepl_entry(
> int freecsn =3D 1;
> =20
> Debug( LDAP_DEBUG_SYNC,
> - "syncrepl_entry: %s LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_%s) tid %x\n",
> - si->si_ridtxt, syncrepl_state2str( syncstate ), op->o_tid );
> + "syncrepl_entry: %s LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_%s) tid %lx\n",
> + si->si_ridtxt, syncrepl_state2str( syncstate ), (unsigned long)op->o=
_tid );
> =20
> if (( syncstate =3D=3D LDAP_SYNC_PRESENT || syncstate =3D=3D LDAP_SYN=
C_ADD ) ) {
> if ( !si->si_refreshPresent && !si->si_refreshDone ) {
>=20
>=20
--=20
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
3 years, 7 months
(ITS#8944) Support Microsoft AD DirSync as a consumer
by hyc＠openldap.org
Full_Name: Howard Chu
Version: 2.5
OS:
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (80.233.39.182)
Submitted by: hyc
Code in master is now available for operating slapd as a replication consumer
against Microsoft ActiveDirectory using their DirSync protocol. Note that no
attempt has been
made to work with anything other than user and group entries.
A sample config, based on the test017 syncrepl consumer config:
####
include ./schema/core.schema
include ./schema/cosine.schema
include ./schema/inetorgperson.schema
include ./schema/nis.schema
include ./schema/msuser.schema
attributeoptions range=
database mdb
suffix "dc=ldapsync,dc=local"
rootdn "cn=Replica,dc=ldapsync,dc=local"
rootpw secret
directory ./testrun/db.2.a
index objectClass eq
index cn,sn,uid pres,eq,sub
index entryUUID,entryCSN eq
syncrepl rid=1
provider=ldap://ldapsync/
binddn="cn=Administrator,cn=users,dc=ldapsync,dc=local"
bindmethod=simple
credentials=MSAD-secret
searchbase="dc=ldapsync,dc=local"
filter="(|(objectClass=user)(objectclass=group))"
schemachecking=off
scope=sub
type=dirSync
interval=00:00:00:03
updateref ldap://ldapsync/
database monitor
####
Note that DirSync doesn't have a persist mode, it only works by
refreshOnly-style polling. DirSync support for Modifications is quite braindead;
the protocol has no way to indicate deletion of single-valued attributes. This
has been left as-is for now, possibly some workarounds can be added for that
later.
3 years, 7 months
Re: (ITS#8943) speed up parallel add/delete
by derek＠shannon-data.com
On Thursday, December 13, 2018 07:54:12 AM Derek Zhou wrote:
>
> On December 13, 2018 1:47:44 AM GMT+08:00, Quanah Gibson-Mount <quanah(a)symas.com> wrote:
> >Hi,
> >
> >Thanks for the report! I have some questions:
> >
> >a) Did you test with writemap enabled for mdb?
> No, but I did test the unpatched slapd with writemap. it was not much faster. I can test the patch with writemap to see what happens.
I tested my patch with writemap, it works with same marginal performance gain like without the patch.
--
Derek Zhou
Shannon Systems
http://www.shannon-sys.com
3 years, 8 months
Re: (ITS#8943) speed up parallel add/delete
by derek＠shannon-data.com
On December 13, 2018 1:47:44 AM GMT+08:00, Quanah Gibson-Mount <quanah@sym=
as=2Ecom> wrote:
>Hi,
>
>Thanks for the report! I have some questions:
>
>a) Did you test with writemap enabled for mdb?
No, but I did test the unpatched slapd with writemap=2E it was not much fa=
ster=2E I can test the patch with writemap to see what happens=2E
>
>b) Did you test the effect on other operating systems? (I=2Ee=2E, Windows=
,=20
>Solaris, FreeBSD, etc)?
No , I dont have any=2E The only thing I have is a x86-64 pc with linux=2E=
I did try to avoid touching any os specific codes
>
>c) Did you test multiple filesystems under Linux(I=2Ee=2E, ext2, ext3,
>ext4,=20
>xfs, etc)?
>
no=2E I only tested with ext4=2E
>d) What filesystem did you use when testing? Was journaling enabled or
>
>disabled? What was the default flush behavior of the filesystem you
>used?
I used nobarrier, data=3Dwriteback mount options, which are faster than th=
e default, but not significantly=2E I am using a very fast SSD though=2E
>I=2Ee=2E, if you set dbnosync at the mdb level, the underlying filesystem
>flush=20
>behavior will then be in effect (with ext4, for example, this defaults
>to 5=20
>seconds)
something like that=2E it also depends on vm setting, like dirty ratio=2E
>
>e) Did you test when the database is also writing to an underlying=20
>accesslog database that is also backed by mdb?
>
no, I dont konw how=2E
--=20
Sent from my Android device with K-9 Mail=2E Please excuse my brevity=2E
3 years, 8 months
Re: (ITS#8943) speed up parallel add/delete
by quanah＠symas.com
--On Wednesday, December 12, 2018 3:54 AM +0000 derek(a)shannon-data.com
wrote:
> Full_Name: Derek Zhou
> Version: 2.4 git HEAD
> OS: linux
> URL:
> Submission from: (NULL) (116.236.169.22)
>
>
> the mdb backend does not really support parallelism in rw transactions, so
> add/delete are serialized by a mutex inside the mdb backend as small
> transactions. In my patch, all add/delete are put in a queue and executed
> in a dedicated worker thread. It may seem pointless at first, but by
> doing them all in the same thread, now I can merge concurrent ops from
> different clients into larger transactions, and reduce the number of
> expensive txn_commit calls (fsyncs).
Hi,
Thanks for the report! I have some questions:
a) Did you test with writemap enabled for mdb?
b) Did you test the effect on other operating systems? (I.e., Windows,
Solaris, FreeBSD, etc)?
c) Did you test multiple filesystems under Linux(I.e., ext2, ext3, ext4,
xfs, etc)?
d) What filesystem did you use when testing? Was journaling enabled or
disabled? What was the default flush behavior of the filesystem you used?
I.e., if you set dbnosync at the mdb level, the underlying filesystem flush
behavior will then be in effect (with ext4, for example, this defaults to 5
seconds)
e) Did you test when the database is also writing to an underlying
accesslog database that is also backed by mdb?
Thanks,
Quanah
--
Quanah Gibson-Mount
Product Architect
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:
<http://www.symas.com>
3 years, 8 months
(ITS#8943) speed up parallel add/delete
by derek＠shannon-data.com
Full_Name: Derek Zhou
Version: 2.4 git HEAD
OS: linux
URL:
Submission from: (NULL) (116.236.169.22)
the mdb backend does not really support parallelism in rw transactions, so
add/delete are serialized by a mutex inside the mdb backend as small
transactions. In my patch, all add/delete are put in a queue and executed in a
dedicated worker thread. It may seem pointless at first, but by doing them all
in the same thread, now I can merge concurrent ops from different clients into
larger transactions, and reduce the number of expensive txn_commit calls
(fsyncs).
http://proxy.shannon-data.com/~derek/patches/openldap/0001-add-rw_worker-...
http://proxy.shannon-data.com/~derek/patches/openldap/0002-add-config-var...
Legal notice
============
The attached patch file is derived from OpenLDAP Software. All of the
modifications to OpenLDAP Software represented in the following patch(es) were
developed by Derek Zhou <derek(a)shannon-data.com>. I have not assigned rights
and/or interest in this work to any party.
I, Derek Zhou, hereby place the following modifications to OpenLDAP Software
(and only these modifications) into the public domain. Hence, these
modifications may be freely used and/or redistributed for any purpose with or
without attribution and/or other notice.
3 years, 8 months
Re: (ITS#8847) New LDAP URL syntax to support binding to specific IP address at client side
by hyc＠symas.com
Sharma, Ramakant 2. (Nokia - IN/Bangalore) wrote:
>>> The work of parse_and_validate is wasted since you call inet_pton bot=
h for
>>>validating and for using the result later. You should simply save the =
IPv4 and IPv6 binary
>>>addresses when the validation succeeds. There should be no get_IPvX_fr=
om_lst() functions
>>>at all.
>=20
> I am bit confused regarding this point.
>=20
> The intent of parse_and_validate_sourceip() is to set only valid intern=
et address=A0 to=A0 "ldo_local_IP_addresses" so that we don=92t overwrite
> =93ldo_local_IP_addresses=94 in case user provides some invalid address=
es either from ldap.conf or from application.
Sure, the validation step is required. The result of validation is either=
a struct in_addr or a struct in6_addr.
You should simply store them directly in the ldapoptions struct. Maybe ad=
d a flag to tell that each one has been set.
> Now during connect, the need is to get either IPv4 or IPv6 internet add=
ress based on target LDAP server's address family type. So, once it is kn=
own that target
> LDAP server is using IPv4 or IPv6. Code will fetch the related configur=
ed bind IP from "ldo_local_IP_addresses".
Yes. All that should be required here is to fetch the struct in_addr or i=
n6_addr value accordingly.
>=20
> Since at this step we are not sure at which index the related bind IP i=
s present so, I used inet_pton call to get the matching internet bind IP =
(internet
> address) from the list (intent was not to use binary address)
This is stupid and wasted work.
> If the concern is to avoid inet_pton call at this stage then, I can use=
"." to search IPv4 and ":" to search IPv6 address in respective getters =
(because we
> already know that at this stage the fetched IP will always be valid)
This is stupid and wasted work.
> Kindly let me know your opinion.
>=20
> =A0
>=20
> BR,
>=20
> Ramakant Sharma
>=20
> Senior R&D Engineer
>=20
> Nokia Networks, Bangalore
>=20
> =A0
>=20
--=20
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
3 years, 8 months
Re: (ITS#8847) New LDAP URL syntax to support binding to specific IP address at client side
by hyc＠symas.com
Thanks for the updated patch.
diff --git a/include/ldap.h b/include/ldap.h
index c58c576d3..78e07d49f 100644
--- a/include/ldap.h
+++ b/include/ldap.h
@@ -138,6 +138,7 @@ LDAP_BEGIN_DECL
#define LDAP_OPT_CONNECT_ASYNC 0x5010 /* create connections asynchronously */
#define LDAP_OPT_CONNECT_CB 0x5011 /* connection callbacks */
#define LDAP_OPT_SESSION_REFCNT 0x5012 /* session reference count */
+#define LDAP_OPT_SOCKET_BIND_ADDRESSES 0x5013 /* user configured bind IPs */
/* OpenLDAP TLS options */
#define LDAP_OPT_X_TLS 0x6000
You've got strange whitespace here. Please re-read the CONTRIBUTING docs; we use 4-column tab stops.
diff --git a/libraries/libldap/bind_ip.c b/libraries/libldap/bind_ip.c
new file mode 100644
index 000000000..56eddda43
--- /dev/null
+++ b/libraries/libldap/bind_ip.c
@@ -0,0 +1,102 @@
+/* LIBLDAP bind_ip.c*/
+/* $OpenLDAP$ */
+/* This work is part of OpenLDAP Software <http://www.openldap.org/>.
+ *
I see no reason why these functions are exposed in their own source file or exported from libldap.
parse_and_validate_sourceip() could just be a static function inside options.c
get_IPvX_from_lst() can just be statics in os-ip.c. In particular, you've defined these
as global but they don't have an ldap_ prefix, which is just wrong.
The work of parse_and_validate is wasted since you call inet_pton both for validating
and for using the result later. You should simply save the IPv4 and IPv6 binary addresses
when the validation succeeds. There should be no get_IPvX_from_lst() functions at all.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
3 years, 8 months