Hi Howard, Thanks for your response. I may be missing something, but let me share my thoughts based on code.

ldap_set_option called from many threads, not only that even ldap_int_tls_connect, ldap_pvt_tls_init_def_ctx as multiple thread can connect to LDAP server. Based on code lookup, i feel not every members of option structure is read only.

Let's consider ldapoptions->ldo_tls_ctx which is global and can be used by many threads. Suppose there is two threads and thread A may be at point 1 and thread B may be at point 2, based on scheduling chances of getting it messed is more, which problem i am facing now. Have a look and please share in case my assumptions are wrong or i am missing something.

tls2.c ---------------------------

1. Let's see where ldo_tls_ctx get allocated. ldap_pvt_tls_init_def_ctx( int is_server ) { *struct ldapoptions *lo = LDAP_INT_GLOBAL_OPT(); // Accessed global option structure* int rc; LDAP_MUTEX_LOCK( &tls_def_ctx_mutex ); *rc = ldap_int_tls_init_ctx( lo, is_server ); // Inside this function lo->ldo_tls_ctx is allocated.* LDAP_MUTEX_UNLOCK( &tls_def_ctx_mutex ); return rc; }

static int *ldap_int_tls_init_ctx( struct ldapoptions *lo, int is_server )* { ... * lo->ldo_tls_ctx = ti->ti_ctx_new( lo ); //The same global variable is allocated with memory* if ( lo->ldo_tls_ctx == NULL ) { Debug( LDAP_DEBUG_ANY, "TLS: could not allocate default ctx.\n", 0,0,0); rc = -1; goto error_exit; } ... }

2. Let's see where it's getting freed. ldo_tls_ctx the same from global option context.

*ldap_pvt_tls_destroy( void )* { *struct ldapoptions *lo = LDAP_INT_GLOBAL_OPT(); //Accessed the global structure *

* ldap_int_tls_destroy( lo );*

... }

void *ldap_int_tls_destroy( struct ldapoptions *lo )* { if ( lo->ldo_tls_ctx ) { * ldap_pvt_tls_ctx_free( lo->ldo_tls_ctx ); //Freeing the global tls context* lo->ldo_tls_ctx = NULL; } }

So if thread 1 where allocation done, and scheduling thread 2 at destroy part, thread 1 again got chance to access and fill some data , it's already freed by thread 2. If the tls context is part of global option structure and all threads will be viewing the same.

I can see a local ldo_tls_ctx exists per ld, but it's again being assigned with same global option ldo_tls_ctx.

static int ldap_int_tls_connect( LDAP *ld, LDAPConn *conn, const char *host ) { ... Sockbuf *sb = conn->lconn_sb; int err; tls_session *ssl = NULL;

if ( HAS_TLS( sb )) { ber_sockbuf_ctrl( sb, LBER_SB_OPT_GET_SSL, (void *)&ssl ); } else { struct ldapoptions *lo; tls_ctx *ctx;

*ctx = ld->ld_options.ldo_tls_ctx; //Accessing the local ldo_tls_ctx.* ssl = alloc_handle( ctx, 0 );

if ( ssl == NULL ) return -1;

#ifdef LDAP_DEBUG ber_sockbuf_add_io( sb, &ber_sockbuf_io_debug, LBER_SBIOD_LEVEL_TRANSPORT, (void *)"tls_" ); #endif ber_sockbuf_add_io( sb, tls_imp->ti_sbio, LBER_SBIOD_LEVEL_TRANSPORT, (void *)ssl );

lo = LDAP_INT_GLOBAL_OPT(); if( ctx == NULL ) { *ctx = lo->ldo_tls_ctx; //Getting the ldo_tls_ctx from context from global context which was allocated inside alloc_handle further functions*. * ld->ld_options.ldo_tls_ctx = ctx;* *// Assigned the same global context to local variable.* tls_ctx_ref( ctx ); } ...

Thanks Sachidananda

On Tue, Aug 6, 2019 at 9:44 PM Howard Chu hyc@symas.com wrote:

sachidananda sahu wrote:

...

Hi All,

Any one can give a thought on this ?

Your problem description makes no sense. Unless you're explicitly calling ldap_set_option in multiple threads, the option structure is read-only.

The default libldap is not threadsafe, nor is it meant to be. You should probably be using libldap_r.

...

On Thu, Aug 1, 2019 at 7:55 PM sachidananda sahu <sachi059@gmail.com

mailto:sachi059@gmail.com> wrote:

...

Hi All,

I recently upgraded to openldap 2.4.47, it's working with single

threaded connection but with multi threaded getting problem due to global structure of

...

ldapoptions in init.c

-------------------------
init.c
-------------------------

*struct* ldapoptions ldap_int_global_options  =
        { LDAP_UNINITIALIZED , LDAP_DEBUG_NONE,
                LDAP_LDO_NULLARG ,
                LDAP_LDO_CONNECTIONLESS_NULLARG,
                LDAP_LDO_TLS_NULLARG,
                LDAP_LDO_SASL_NULLARG ,
                LDAP_LDO_GSSAPI_NULLARG,
                LDAP_LDO_MUTEX_NULLARG  };,


This global structure is accessed at multiple places (such as

ldap_pvt_tls_init_def_ctx , alloc_handle, ldap_int_tls_connect , *ldap_pvt_tls_destroy , ldap_ld_free*)

...

in tls2.c using the macro lo = LDAP_INT_GLOBAL_OPT
();

So in case of multi threaded application multiple ldap connection

will be using this global structure, for example ldo_tls_ctx of lapoptions will be used.

...

In one thread it can be creating a tls connection and in one it can

be destroying the connection. As it's global so it is getting corrupted.

...

Is openldap library thread safe completely ? Because this variable

seems to be not for this tls context variable, is there any other way of using this

...

context . As i can see a local variable ldo_tls_ctx exist in dap

ld->ldc->ldap_options->ldo_tls_ctx structure, but it's just got assigned with the same

...

address of global structure in  ldap_int_tls_connect.

So can someone share some thoughts on it ?

--
Regards,
Sachidananda Sahu

-- Regards, Sachidananda Sahu +91-9035265767

-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

many thanks for the information. Now have couple of queries.

Then which call i should use to create the local context and destroy the tls context.

Currently i was calling ldap_start_tls_s->ldap_int_tls_start->ldap_int_tls_connect : here it was doing ctx http://opengrok-prd.eng.netapp.com/source/s?defs=ctx&project=dev = ld http://opengrok-prd.eng.netapp.com/source/s?defs=ld&project=dev-> ld_options http://opengrok-prd.eng.netapp.com/source/s?defs=ld_options&project=dev. ldo_tls_ctx http://opengrok-prd.eng.netapp.com/source/s?defs=ldo_tls_ctx&project=dev;, So as this connection is newly started and to add tls context over LDAP we call ldap_start_tls_s .....So when we should allocate the assign memory for this local context.

Currently It;s going for alloc_handle and creating a default context using global and assigning it to local context.

Similarly many places it access the global option->ldo_tls_ctx many places, which function we should use for the destory of the context, considering multiple threads are there and creating and destroying connection.

If any reference implementation link also there, you can share.

Many thanks Howard for your insights.

On Tue, Aug 6, 2019 at 10:42 PM Howard Chu hyc@symas.com wrote:

sachidananda sahu wrote:

...

Hi Howard, Thanks for your response. I may be missing something, but let me share

my thoughts based on code.

...

ldap_set_option called from many threads, not only that even

ldap_int_tls_connect, ldap_pvt_tls_init_def_ctx as multiple thread can connect to LDAP server.

...

Based on code lookup, i feel not every members of option structure is

read only.

...

Let's consider ldapoptions->ldo_tls_ctx which is global and can be used

by many threads. Suppose there is two threads and thread A may be at point 1 and thread

...

B may be at point 2, based on scheduling chances of getting it messed is

more, which problem i am facing now. Have a look and please share in case my

...

assumptions are wrong or i am missing something.

init_def_ctx only gets called from alloc_handle() if there was no existing ctx.

...

tls2.c

1. Let's see where ldo_tls_ctx get allocated.

ldap_pvt_tls_init_def_ctx( int is_server )

...

2. Let's see where it's getting freed. ldo_tls_ctx the same from global

option context.

...

*ldap_pvt_tls_destroy( void )* {

This is an OpenLDAP-specific private API. If you're calling this function, you're misusing the API.

...

On Tue, Aug 6, 2019 at 9:44 PM Howard Chu <hyc@symas.com <mailto:

hyc@symas.com>> wrote:

...

sachidananda sahu wrote:
> Hi All,
>
> Any one can give a thought on this ?

Your problem description makes no sense. Unless you're explicitly

calling ldap_set_option in multiple threads,

...

the option structure is read-only.

The default libldap is not threadsafe, nor is it meant to be. You

should probably be using libldap_r.

...

>
>
>
> On Thu, Aug 1, 2019 at 7:55 PM sachidananda sahu <

sachi059@gmail.com mailto:sachi059@gmail.com <mailto:sachi059@gmail.com mailto:sachi059@gmail.com>>

...

wrote:
>
>
>     Hi All,
>
>     I recently upgraded to openldap 2.4.47, it's working with

single threaded connection but with multi threaded getting problem due to global structure of

...

>     ldapoptions in init.c
>
>     -------------------------
>     init.c
>     -------------------------
>
>     *struct* ldapoptions ldap_int_global_options  =
>             { LDAP_UNINITIALIZED , LDAP_DEBUG_NONE,
>                     LDAP_LDO_NULLARG ,
>                     LDAP_LDO_CONNECTIONLESS_NULLARG,
>                     LDAP_LDO_TLS_NULLARG,
>                     LDAP_LDO_SASL_NULLARG ,
>                     LDAP_LDO_GSSAPI_NULLARG,
>                     LDAP_LDO_MUTEX_NULLARG  };,
>
>
>     This global structure is accessed at multiple places (such as

ldap_pvt_tls_init_def_ctx , alloc_handle, ldap_int_tls_connect , *ldap_pvt_tls_destroy ,

...

ldap_ld_free*)
>
>     in tls2.c using the macro lo = LDAP_INT_GLOBAL_OPT
>     ();
>
>     So in case of multi threaded application multiple ldap

connection will be using this global structure, for example ldo_tls_ctx of lapoptions will be used.

...

>     In one thread it can be creating a tls connection and in one

it can be destroying the connection. As it's global so it is getting corrupted.

...

>
>     Is openldap library thread safe completely ? Because this

variable seems to be not for this tls context variable, is there any other way of using this

...

>     context . As i can see a local variable ldo_tls_ctx exist in

dap ld->ldc->ldap_options->ldo_tls_ctx structure, but it's just got assigned with the same

...

>     address of global structure in  ldap_int_tls_connect.
>
>     So can someone share some thoughts on it ?
>

-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

Hi Howard,

Some clarification with respect to this statement "The context is per-process state, not per-thread state. No threads should be trying to destroy the context."

Consider in my process i have two threads, each want to connect to their respective LDAP server. As two LDAP server so two different certificate and two different tls context. In that case that process specific global variable will be overwritten, considering concurrent case (One thread is in ldap_int_tls_start as part of allocation and further use and other thread in ldap_unbind_ext--->ldap_ld_free-->ldap_int_tls_destroy(&ld->ld_options );).

So the concern is two treads connecting to two LDAP server independently (starting ldo_tls_ctx from same global variable or ldap_unbind_ext where they free the ldo_tls_ctx) . The same global variable getting used through two different LDAP server connection, two different ld context. So clarity needed with this concurrent scenario,

1. Here we are storing the local context ldo_tls_ctx with after extracting from global option structure ldo_tls_ctx.

ldap_int_tls_connect( LDAP *ld, LDAPConn *conn, const char *host ) { ... Sockbuf *sb = conn->lconn_sb; int err; tls_session *ssl = NULL;

if ( HAS_TLS( sb )) { ber_sockbuf_ctrl( sb, LBER_SB_OPT_GET_SSL, (void *)&ssl ); } else { struct ldapoptions *lo; tls_ctx *ctx;

* ctx = ld->ld_options.ldo_tls_ctx;* ------------------------- ssl = alloc_handle( ctx, 0 );

if ( ssl == NULL ) return -1;

#ifdef LDAP_DEBUG ber_sockbuf_add_io( sb, &ber_sockbuf_io_debug, LBER_SBIOD_LEVEL_TRANSPORT, (void *)"tls_" ); #endif ber_sockbuf_add_io( sb, tls_imp->ti_sbio, LBER_SBIOD_LEVEL_TRANSPORT, (void *)ssl );

lo = LDAP_INT_GLOBAL_OPT(); if( ctx == NULL ) { *ctx = lo->ldo_tls_ctx; -------------------- Storing the global context in to ld specific local context. * ld->ld_options.ldo_tls_ctx = ctx; tls_ctx_ref( ctx ); } ... }

2. ldap_ld_free is deleting that ld specific context but inside that the context is same as global option structure ldo_tls_ctx.

int ldap_ld_free( LDAP *ld, int close, LDAPControl **sctrls, LDAPControl **cctrls ) { ... #ifdef HAVE_TLS * ldap_int_tls_destroy( &ld->ld_options ); ----------------- Destroying the tls_ctx inside this function, though it is extracting from ld, but it carries the address of global option ldo_tls_ctx as per point 1.* #endif ..... }

void *ldap_int_tls_destroy*( struct ldapoptions *lo ) { if ( lo->ldo_tls_ctx ) {

* ldap_pvt_tls_ctx_free( lo->ldo_tls_ctx ); lo->ldo_tls_ctx = NULL;* } ... }

On Wed, Aug 7, 2019 at 12:05 AM Howard Chu hyc@symas.com wrote:

sachidananda sahu wrote:

...

many thanks for the information. Now have couple of queries.

Then which call i should use to create the local context and destroy the

tls context.

None. The context is created automatically and used as needed. The context is per-process state, not per-thread state. No threads should be trying to destroy the context.

The reference implementations are in clients/tools in the source tree.

...

Currently i was

calling ldap_start_tls_s->ldap_int_tls_start->ldap_int_tls_connect : here it was doing ctx

...

http://opengrok-prd.eng.netapp.com/source/s?defs=ctx&project=dev= ld <

http://opengrok-prd.eng.netapp.com/source/s?defs=ld&project=dev

...

->ld_options <http://opengrok-prd.eng.netapp.com/source/s?defs=ld_options&project=dev .ldo_tls_ctx <

http://opengrok-prd.eng.netapp.com/source/s?defs=ldo_tls_ctx&project=dev...;, So as this connection is newly started and to add tls context over LDAP we call

...

ldap_start_tls_s .....So when we should allocate the assign memory for

this local context.

...

Currently It;s going for alloc_handle and creating a default context

using global and assigning it to local context.

...

Similarly many places it access the global option->ldo_tls_ctx many

places, which function we should use for the destory of the context, considering multiple

...

threads are there and creating and destroying connection.

If any reference implementation link also there, you can share.

Many thanks Howard for your insights.

On Tue, Aug 6, 2019 at 10:42 PM Howard Chu <hyc@symas.com <mailto:

hyc@symas.com>> wrote:

...

sachidananda sahu wrote:
> Hi Howard,
> Thanks for your response. I may be missing something, but let me

share my thoughts based on code.

...

>
> ldap_set_option called from many threads, not only that even

ldap_int_tls_connect, ldap_pvt_tls_init_def_ctx as multiple thread can connect to LDAP server.

...

> Based on code lookup, i feel not every members of option structure

is read only.

...

>
> Let's consider ldapoptions->ldo_tls_ctx which is global and can be

used by many threads. Suppose there is two threads and thread A may be at point 1 and

...

thread
> B may be at point 2, based on scheduling chances of getting it

messed is more, which problem i am facing now. Have a look and please share in case my

...

> assumptions are wrong or i am missing something.

init_def_ctx only gets called from alloc_handle() if there was no

existing ctx.

...

>
> tls2.c
> ---------------------------
>
> 1. Let's see where ldo_tls_ctx get allocated.
> ldap_pvt_tls_init_def_ctx( int is_server )

> 2. Let's see where it's getting freed. ldo_tls_ctx the same from

global option context.

...

>
> *ldap_pvt_tls_destroy( void )*
> {
This is an OpenLDAP-specific private API. If you're calling this

function, you're misusing the API.

...

> On Tue, Aug 6, 2019 at 9:44 PM Howard Chu <hyc@symas.com <mailto:

hyc@symas.com> <mailto:hyc@symas.com mailto:hyc@symas.com>> wrote:

...

>
>     sachidananda sahu wrote:
>     > Hi All,
>     >
>     > Any one can give a thought on this ?
>
>     Your problem description makes no sense. Unless you're

explicitly calling ldap_set_option in multiple threads,

...

>     the option structure is read-only.
>
>     The default libldap is not threadsafe, nor is it meant to be.

You should probably be using libldap_r.

...

>     >
>     >
>     >
>     > On Thu, Aug 1, 2019 at 7:55 PM sachidananda sahu <

sachi059@gmail.com mailto:sachi059@gmail.com <mailto:sachi059@gmail.com

...

<mailto:sachi059@gmail.com>> <mailto:sachi059@gmail.com <mailto:

sachi059@gmail.com> <mailto:sachi059@gmail.com <mailto:sachi059@gmail.com

...

...

...

...

>     wrote:
>     >
>     >
>     >     Hi All,
>     >
>     >     I recently upgraded to openldap 2.4.47, it's working

with single threaded connection but with multi threaded getting problem due to global

...

structure of
>     >     ldapoptions in init.c
>     >
>     >     -------------------------
>     >     init.c
>     >     -------------------------
>     >
>     >     *struct* ldapoptions ldap_int_global_options  =
>     >             { LDAP_UNINITIALIZED , LDAP_DEBUG_NONE,
>     >                     LDAP_LDO_NULLARG ,
>     >                     LDAP_LDO_CONNECTIONLESS_NULLARG,
>     >                     LDAP_LDO_TLS_NULLARG,
>     >                     LDAP_LDO_SASL_NULLARG ,
>     >                     LDAP_LDO_GSSAPI_NULLARG,
>     >                     LDAP_LDO_MUTEX_NULLARG  };,
>     >
>     >
>     >     This global structure is accessed at multiple places

(such as ldap_pvt_tls_init_def_ctx , alloc_handle, ldap_int_tls_connect ,

...

*ldap_pvt_tls_destroy ,
>     ldap_ld_free*)
>     >
>     >     in tls2.c using the macro lo = LDAP_INT_GLOBAL_OPT
>     >     ();
>     >
>     >     So in case of multi threaded application multiple ldap

connection will be using this global structure, for example ldo_tls_ctx of lapoptions

...

will be used.
>     >     In one thread it can be creating a tls connection and in

one it can be destroying the connection. As it's global so it is getting corrupted.

...

>     >
>     >     Is openldap library thread safe completely ? Because

this variable seems to be not for this tls context variable, is there any other way of

...

using this
>     >     context . As i can see a local variable ldo_tls_ctx

exist in dap ld->ldc->ldap_options->ldo_tls_ctx structure, but it's just got assigned with

...

the same
>     >     address of global structure in  ldap_int_tls_connect.
>     >
>     >     So can someone share some thoughts on it ?

-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/