openldap-bugs

openldap-bugs@openldap.org

1 participants
20959 discussions

Re: (ITS#4943) tpool.c pause vs. finish
by h.b.furuseth＠usit.uio.no 08 Jun '07

08 Jun '07

OK, here's where we stand now, with my TODOs, questions and some new issues. 3+ pages. I thought this would be a brief message:-( Hopefully I'll get to some of it this weekend, but no promises. TODOs (unless someone protests) and questions: * When a thread is finishing, make it go active for context_reset(). Should make slapi module destructors safer, and I don't see that it can hurt. * I'd like to rename thread contexts to tasks(?) and user contexts back to contexts (as in RE23). The current terminology is confusing. * Save and restore the old data for key slap_sasl_bind, and just in case (I can't tell if it is needed) smbk5pwd. Might as well do it everywhere we have setkey - do something - delete key in one function call. Since setkey(non-null) isn't supposed to overwrite existing keys, the change should not be able to hurt. It could protect something I didn't think of or a future slapd change. API: the smallest change would be to call pool_getkey() first, but I prefer a pool_swapkey(ctx, key, &data, &kfree) function. Then in RE24, maybe remove the &kfree parameter to pool_getkey(). Could omit the &kfree parameter to swapkey() if new data was guaranteed non-NULL, but that'd require a malloc in bdb (silly says Howard) or passing data in a &union{integer, ptr} (cumbersome say I). * Axe semaphore code in tpool.c, the ldap_lazy_sem_init() call in slapd/daemon.c, and with them LDAP_PVT_THREAD_POOL_SEM_LOAD_CONTROL in ldap_pvt_thread.h and SLAP_SEM_LOAD_CONTROL in slap.h. * Is the ldap_pvt_thread_pool_context_reset() call in slapd/init.c indeed safe - ie. its slap_sl_mem_destroy() call? That's not during startup, but in slap_destroy(). Slapd does various stuff after that. * API changes: Do we include the assert(no multiple pools) in the import to RE23? Remove keys from ltu_key[] _before_ calling ltk_free() with them? That prevents functions called from ltk_free from seeing freed thread data, but it can also break ltk_free functions that call such functions _before_ freeing it. Probably not for RE23. * Scheduling: If several threads call pool_pause(), then once there is a pause tpool does not schedule them all. They could get handled then, or another thread could undo the pause so tpool would wait to pause again. Is that deliberate? Dubious code, new issues, old ones I'm not sure we finished with: * thread_keys[] still has problems. - pool_context() breaks if several ldap_pvt_thread_t bit patterns can represent the same thread ID: TID_HASH() would give different hashes for the same thread, and pool_context() stops searching when it hits a slot with ctx == NULL. (My previous bug report was a bit wrong.) The best fix would be to use use real thread-specific data instead. Just one key with the ctx for now, that minimizes the changes. OTOH it also means we'll do thread-specific key lookup twice - first in pthread to get the ctx, and then in ltu_key[] to find our data. Anyway, I said I'd do that later but might as well get on with it, at least for pthreads. Except I don't know if that's OS-dependent enough that it should wait for RE24? When that's not implemented, I'd prefer to cast the thread ID to an integer when that is safe, and hash that instead of its bytes. Won't work for structs/unions, and is not formally guaranteed for pointers or in case of pthreads even integers. BTW, when it doesn't work, we've got a problem in ITS#4983 (tls.c id_callback) as well. When that's not possible, we could search the entire table before giving up. Could make thread_keys[] smaller and reduce the maximum allowed #threads on such hosts. (What should the limit be?) - thread_keys is a poor hash table implementation with a poor hash function. For the hash function, I'll insert a variant of Paul Hsieh's SuperFastHash() instead, or something simpler if we hash an integer. For the hash table, the real fix would be to grab a proper hash table implementation and use that. We might find use for it elsewhere too. Though simple fixes will handle all but some rare cases, and I don't know how much we care: The table is never resized and will slow down lookup when it gets full. A quick fix would be to allow max (70% table size) threads or something. And double the size of the table, if we still want to allow 1024 threads. However if the admin experiments with ltp_max_count for a while so many threads stop/start/stop/start, the table slowly fills up with DELETED_THREAD_CTX entries. Longer search chains, longer thread ID lookup time. Should happen rarely, but we don't advise against it in the docs either. Do we care, beyond documenting it? If yes, we need a real hash table implementation, or at least to re-hash the table when there are too many DELETED_THREAD_CTX entries. Or use a chained hash table. No malloc needed, use the thread's stack space. For open addressing, I was thinking quadric probing would be better than linear, but maybe not with a proper hash function. I don't know what that does to cache issues. Oops - for caching, maybe I shouldn't have removed the thread id from thread_keys[]. Or maybe the hash value should be stored there instead, so we can compare with == instead of ldap_pvt_thread_equal. * I wonder if there are long-lived ldap_pvt_thread_cond_wait(), [r]mutex_lock() or rdwr_[rw]lock() calls which should make the thread go inactive first (after waiting for any pause to finish), so that (a) a pool_pause() will not block waiting for them, and (b) all threads in the pool can't be blocked in long waits without tpool knowing about it. * back-bdb/tools.c uses pool_submit() to start long-lived threads: bdb_tool_trickle_task() and bdb_tool_index_task(). They don't exit until slapd is stopping, so they'll block slapd if a thread requests a pause. Also the pool thinks its has more available threads that it does, so scheduled tasks might never be executed. Possibly that's not a problem with slaptools. As complex as slapd is now with overlays and plugins and whatnot, I have no idea. If it's a problem, it is not enough for them to regularly reschedule themselves and exit: They have long-lived waits. See previous issue. * slapd/bconfig.c maintains its own idea of the max number of threads in connection_pool_max and slap_tool_thread_max, which does not match tpool's idea if the limit is <=0 or larger than LDAP_MAXTHR. Since slapd acts on its own count, I imagine that can cause breakage. Simplest fix might be to set max #threads and then read it back, I haven't checked. Would need to return LDAP_MAXTHR instead of 0 for "as many as allowed". Note however also the issue above: tpool's idea of actually available threads can be too high too. * I said my new ltp_pause waits might be too aggressive. Maybe I should clarify: I don't see any instances where they can be dropped, but it's possible that slapd expects pauses to be less aggressive somewhere. About pool_submit() and the ltp_mutex unlock-relock: Some history and notes, just to finish up previous discussion. Some possible TODOs, but they are not needed since it's not broken: Originally, pool_submit() unlocked ltp_mutex before thread_create() and re-locked it afterwards for further maintenance. That was not because of the semaphore code. It looks like it could work even in non-threaded mode where thread_create(,,fn,arg) just called fn(arg), though tpool.c was always wrapped in an #ifndef <--without-threads>. I don't think non-threaded would have worked anyway since rev 1.24 (apr 2003), which introduced thread_keys[]. Caller and callee would have gotten the same thread ID, so tpool would have confused their contexts. Previously the new context was pushed onto ltp_active_list, and would have temporarily hid the old context. That's also when thread_create() was moved after the ltp_mutex re-lock, which would mean the mutex was locked while pool_wrapper() was running and tried to re-lock the mutex. And now, my rev.171 has removed the unlock-relock. With that I can also remove the search for ctx in ltp_pending_list when backing out of pool_submit, it will be at the head of that list. With tpool only supporting threaded mode, another cleanup could be for pool_submit to start a thread before inserting ctx in ltp_pending_list instead of after. That done, it can fail if ltp_open_count is still 0, and we won't need the code for backing out of pool_submit(). Oh, the comment /* no open threads at all?!? */: That's not strange. Happens in the first pool_submit() call(s) if thread_create() fails. Though in that case there was no danger of removing the wrong ctx from ltp_pending_list after all (fixed in rev 1.63). -- Regards, Hallvard

1 0

Re: (ITS#5005) test017 fails
by quanah＠zimbra.com 08 Jun '07

08 Jun '07

--On June 8, 2007 6:43:50 PM +0000 dieter(a)dkluenter.de wrote: > Full_Name: Dieter Kluenter > Version: HEAD > OS: Linux > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (84.142.196.121) > > > here the output of test017 logfiles > > ---slapd.1.log---------------- > ber_scanf fmt (}) ber: > <= get_ctrls: n=1 rc=2 err="Sync control : cookie parsing error" > send_ldap_result: conn=7 op=1 p=3 > send_ldap_result: err=2 matched="" text="Sync control : cookie parsing > error" send_ldap_response: msgid=2 tag=101 err=2 Hm, is this possibly related to the fix for ITS#4977? If you revert that checkin for syncprov.c today, does the test pass? <http://www.openldap.org/devel/cvsweb.cgi/servers/slapd/overlays/syncprov.c.…> --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

(ITS#5005) test017 fails
by dieter＠dkluenter.de 08 Jun '07

08 Jun '07

Full_Name: Dieter Kluenter Version: HEAD OS: Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (84.142.196.121) here the output of test017 logfiles ---slapd.1.log---------------- >>> slap_listener(ldap://localhost:9011/) conn=7 fd=13 ACCEPT from IP=127.0.0.1:13352 (IP=127.0.0.1:9011) connection_get(13) connection_get(13): got connid=7 connection_read(13): checking for input on id=7 ber_get_next ber_get_next: tag 0x30 len 46 contents: ber_get_next do_bind ber_scanf fmt ({imt) ber: ber_scanf fmt (m}) ber: >>> dnPrettyNormal: <cn=manager,dc=example,dc=com> => ldap_bv2dn(cn=manager,dc=example,dc=com,0) <= ldap_bv2dn(cn=manager,dc=example,dc=com)=0 => ldap_dn2bv(272) <= ldap_dn2bv(cn=manager,dc=example,dc=com)=0 => ldap_dn2bv(272) <= ldap_dn2bv(cn=manager,dc=example,dc=com)=0 <<< dnPrettyNormal: <cn=manager,dc=example,dc=com>, <cn=manager,dc=example,dc=com> do_bind: version=3 dn="cn=manager,dc=example,dc=com" method=128 conn=7 op=0 BIND dn="cn=manager,dc=example,dc=com" method=128 ==> bdb_bind: dn: cn=manager,dc=example,dc=com conn=7 op=0 BIND dn="cn=Manager,dc=example,dc=com" mech=SIMPLE ssf=0 do_bind: v3 bind: "cn=manager,dc=example,dc=com" to "cn=Manager,dc=example,dc=com" send_ldap_result: conn=7 op=0 p=3 send_ldap_result: err=0 matched="" text="" send_ldap_response: msgid=1 tag=97 err=0 ber_flush2: 14 bytes to sd 13 conn=7 op=0 RESULT tag=97 err=0 text= connection_get(13) connection_get(13): got connid=7 connection_read(13): checking for input on id=7 ber_get_next ber_get_next: tag 0x30 len 109 contents: ber_get_next do_search ber_scanf fmt ({miiiib) ber: >>> dnPrettyNormal: <dc=example,dc=com> => ldap_bv2dn(dc=example,dc=com,0) <= ldap_bv2dn(dc=example,dc=com)=0 => ldap_dn2bv(272) <= ldap_dn2bv(dc=example,dc=com)=0 => ldap_dn2bv(272) <= ldap_dn2bv(dc=example,dc=com)=0 <<< dnPrettyNormal: <dc=example,dc=com>, <dc=example,dc=com> SRCH "dc=example,dc=com" 2 0 0 0 0 ber_scanf fmt (m) ber: filter: (objectClass=*) ber_scanf fmt ({M}}) ber: => get_ctrls ber_scanf fmt ({m) ber: ber_scanf fmt (m) ber: => get_ctrls: oid="1.3.6.1.4.1.4203.1.9.1.1" (noncritical) ber_scanf fmt ({i) ber: ber_scanf fmt (m) ber: ber_scanf fmt (b) ber: ber_scanf fmt (}) ber: <= get_ctrls: n=1 rc=2 err="Sync control : cookie parsing error" send_ldap_result: conn=7 op=1 p=3 send_ldap_result: err=2 matched="" text="Sync control : cookie parsing error" send_ldap_response: msgid=2 tag=101 err=2 ber_flush2: 49 bytes to sd 13 conn=7 op=1 SEARCH RESULT tag=101 err=2 nentries=0 text=Sync control : cookie parsing error do_search: get_ctrls failed connection_get(13) connection_get(13): got connid=7 connection_read(13): checking for input on id=7 ber_get_next ber_get_next: tag 0x30 len 5 contents: ber_get_next ber_get_next on fd 13 failed errno=0 (Success) connection_closing: readying conn=7 sd=13 for close connection_close: deferring conn=7 sd=13 do_unbind conn=7 op=2 UNBIND connection_resched: attempting closing conn=7 sd=13 connection_close: conn=7 sd=13 conn=7 fd=13 closed daemon: shutdown requested and initiated. --------------------------------------- ---- slapd.2.log ---------------------- >>> slap_listener(ldap://localhost:9012/) conn=1 fd=13 ACCEPT from IP=127.0.0.1:3877 (IP=127.0.0.1:9012) connection_get(13) connection_get(13): got connid=1 connection_read(13): checking for input on id=1 ber_get_next ber_get_next: tag 0x30 len 46 contents: ber_get_next do_bind ber_scanf fmt ({imt) ber: ber_scanf fmt (m}) ber: >>> dnPrettyNormal: <cn=Manager,dc=example,dc=com> => ldap_bv2dn(cn=Manager,dc=example,dc=com,0) <= ldap_bv2dn(cn=Manager,dc=example,dc=com)=0 => ldap_dn2bv(272) <= ldap_dn2bv(cn=Manager,dc=example,dc=com)=0 => ldap_dn2bv(272) <= ldap_dn2bv(cn=manager,dc=example,dc=com)=0 <<< dnPrettyNormal: <cn=Manager,dc=example,dc=com>, <cn=manager,dc=example,dc=com> do_bind: version=3 dn="cn=Manager,dc=example,dc=com" method=128 conn=1 op=0 BIND dn="cn=Manager,dc=example,dc=com" method=128 ==> bdb_bind: dn: cn=Manager,dc=example,dc=com bdb_dn2entry("cn=manager,dc=example,dc=com") => bdb_dn2id("dc=example,dc=com") <= bdb_dn2id: get failed: DB_NOTFOUND: No matching key/data pair found (-30989) send_ldap_result: conn=1 op=0 p=3 send_ldap_result: err=49 matched="" text="" send_ldap_response: msgid=1 tag=97 err=49 ber_flush2: 14 bytes to sd 13 conn=1 op=0 RESULT tag=97 err=49 text= connection_get(13) connection_get(13): got connid=1 connection_read(13): checking for input on id=1 ber_get_next ber_get_next on fd 13 failed errno=0 (Success) connection_closing: readying conn=1 sd=13 for close connection_close: conn=1 sd=13 conn=1 fd=13 closed (connection lost) daemon: shutdown requested and initiated. slapd shutdown: waiting for 0 threads to terminate lt-slapd shutdown: initiated ====> bdb_cache_release_all lt-slapd destroy: freeing system resources. slapd stopped. -------------------------------------------------------- ---test.out--------------- ldap_initialize( ldap://localhost:9012 ) ldap_bind: Invalid credentials (49) ------------------------------ -Dieter

1 0

Re: (ITS#4983) libldap_r/tls.c compile failure
by h.b.furuseth＠usit.uio.no 08 Jun '07

08 Jun '07

h.b.furuseth(a)usit.uio.no writes: >kurt(a)OpenLDAP.org writes: >> tls.c: In function `tls_thread_self': >> tls.c:412: error: invalid operands to binary / >> tls.c:415: warning: comparison between pointer and integer > > Well, that was quick... my tls.c rev 1.154 patch, tls_thread_self(), > deliberately causes that for non-integer thread types. What to do? I found this in ITS#4723, from Howard Chu at 29 Nov 2006: > In the current OpenSSL, the address of errno is tested as well. Since > this is always unique per thread, there's really no need to set the id > callback any more. The problem with just using CRYPTO_set_id_callback > is that it doesn't work on platforms where a thread ID is not an > integer (e.g. OS/390). I don't think CRYPTO_set_idptr_callback was > available in earlier OpenSSL releases. Yet Howard added CRYPTO_set_id_callback to tls.c (rev.144) the next day. Why? Should we wrap CRYPTO_set_id_callback() in #if (OpenSSL is old)? Well, I can think of a way the &errno hack can break: The OS could create a small virtual address space which maps to different physical memory for different threads, and put things like errno there. OpenSSL 0.9.9 will apparently have CRYPTO_set_idptr_callback, so we can use an #ifdef for that. For whatever isn't handled above, my suggestion for now is to keep the failure if ldap_pvt_thread_t is a struct/union or wider than long, and see if anyone still complains. -- Regards, Hallvard

1 0

Re: (ITS#4965) slapd stops if access to cn=monitor is restricted
by ando＠sys-net.it 08 Jun '07

08 Jun '07

ali.pouya(a)free.fr wrote: > OK Pierangelo; > Now I have an explanation : > If you declare the monitor database AFTER bdb slapd starts well. > But if you decalre monitor BEFORE bdb slapd cannot start. > You can test it. OK, I see. The difference is that if the monitor database comes first, it's already open when the bdb tries to register its custom monitor stuff. So it can barf out when the addition fails because of no enough permissions. If it comes later, back-bdb's custom monitor stuff ends up in the limbo, which is then flushed when the monitor database starts up. At this point, a failure in flushing the limbo is not considered critical. I see two/three solutions: 1) allow databases to fail monitor info registration without aborting 2) make startup fail also when limbo flushing fails 3) let the monitor database have a default rootdn. I'm not oriented towards any of the above, yet. p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati(a)sys-net.it ---------------------------------------

1 0

Re: (ITS#4965) slapd stops if access to cn=monitor is restricted
by ali.pouya＠free.fr 08 Jun '07

08 Jun '07

OK Pierangelo; Now I have an explanation : If you declare the monitor database AFTER bdb slapd starts well. But if you decalre monitor BEFORE bdb slapd cannot start. You can test it. Thanks for your help Best regards Ali

1 0

(ITS#5003) issue about upper_needs_cast
by caojunliang＠huawei.com 08 Jun '07

08 Jun '07

Full_Name: Cao Junliang Version: openldap-2.3.35 OS: windows xp URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (210.21.230.126) Hello, when I set the option upper_needs_cast as Yes.It only takes effect on id_query when adding an entry,but does not take effect on child id query when deleting an entry. I found that in the src file back-sql/init.c line 374 --379 there are not operation about upper cast. I want to know if what I think is right. Cao Junliang

1 0

Re: (ITS#5002) Windows Service: logoff causes shutdown
by hyc＠symas.com 08 Jun '07

08 Jun '07

This bug was fixed several years ago. You should update to a current release. This ITS will be closed. mark.lakes(a)alcatel-lucent.com wrote: > Full_Name: Mark Lakes > Version: 2.1.22 > OS: Windows > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (198.206.187.100) > > > Summary: LDAP service stops when user logs off. > > The LDAP server is registered as a Windows Service and works correctly with > SCM. > It is registered as an automatic service so should be running all the time. > When a user logs off from the box, the logoff triggers the service to stop > running. > This happens on WindowsXP and Windows 2003 Server that are configured on a > Domain. > > I have a solution. > The problem is that despite the fact it is a service, it receives an event > called CTRL_LOGOFF_EVENT. Since it is not handling this event, the default > behavior by Windows is to shutdown the service. > Add the following code to the source at an appropriate place(probably somewhere > in ntservice.c) for when it runs as a service: > > /** > This is used to handle CTRL_LOGOFF_EVENT from shutting us down when we are > running as a service. > Return TRUE works. We dont really care what the crtlType is. > */ > BOOL WINAPI > WinSigHANDLER_ROUTINE(DWORD ctrlType) > { > /* > char logstr[256]; > sprintf(logstr, "NTservice: caught signal(%d)\n", ctrlType); > writeStatusLog(logstr); > switch (ctrlType) > { > case CTRL_LOGOFF_EVENT: > ... > default: > > } */ > return TRUE; > } > > Then, register the above function in the appropriate place as follows: > SetConsoleCtrlHandler(WinSigHANDLER_ROUTINE, TRUE); > > > > -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#5002) Windows Service: logoff causes shutdown
by mark.lakes＠alcatel-lucent.com 08 Jun '07

08 Jun '07

Full_Name: Mark Lakes Version: 2.1.22 OS: Windows URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (198.206.187.100) Summary: LDAP service stops when user logs off. The LDAP server is registered as a Windows Service and works correctly with SCM. It is registered as an automatic service so should be running all the time. When a user logs off from the box, the logoff triggers the service to stop running. This happens on WindowsXP and Windows 2003 Server that are configured on a Domain. I have a solution. The problem is that despite the fact it is a service, it receives an event called CTRL_LOGOFF_EVENT. Since it is not handling this event, the default behavior by Windows is to shutdown the service. Add the following code to the source at an appropriate place(probably somewhere in ntservice.c) for when it runs as a service: /** This is used to handle CTRL_LOGOFF_EVENT from shutting us down when we are running as a service. Return TRUE works. We dont really care what the crtlType is. */ BOOL WINAPI WinSigHANDLER_ROUTINE(DWORD ctrlType) { /* char logstr[256]; sprintf(logstr, "NTservice: caught signal(%d)\n", ctrlType); writeStatusLog(logstr); switch (ctrlType) { case CTRL_LOGOFF_EVENT: ... default: } */ return TRUE; } Then, register the above function in the appropriate place as follows: SetConsoleCtrlHandler(WinSigHANDLER_ROUTINE, TRUE);

1 0

(ITS#5001) Incorrect SSF display
by quanah＠OpenLDAP.org 07 Jun '07

07 Jun '07

Full_Name: Quanah Gibson-Mount Version: 2.3.35 OS: NA URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (71.202.148.128) I was playing with SASL connections on an TLS encrypted connection, and noticed the SSF displayed is not correct in the SASL part. Jun 7 12:51:27 ldap-dev1 slapd[11858]: conn=4494 fd=29 TLS established tls_ssf=256 ssf=256 Jun 7 12:51:27 ldap-dev1 slapd[11858]: conn=4494 op=4 BIND dn="uid=quanah,cn=accounts,dc=stanford,dc=edu" mech=GSSAPI ssf=56 I believe that the second line above should really display "sasl_ssf=56", or probably even better, "sasl_ssf=56 ssf=256", similar to how the first line has "tls_ssf=256 ssf=256", so that it is clear that there are different security factors in play here. --Quanah

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs