openldap-bugs

openldap-bugs@openldap.org

1 participants
20959 discussions

Re: (ITS#4996) Use SRV records to locate local server for command-line clients
by ando＠sys-net.it 09 Jun '07

09 Jun '07

Committed to HEAD. Command-line clients now use DNS SRV to look up the host names related to a DN and build a list of URIs for subsequent use. To trigger, use -H "<proto>:///DN". The <host> portion must be empty, and a DN must be present. The DN must be in the "dc=<component>" form; it's turned into a domain and the host list is looked up using DNS SRV. After that, a list of URIs is built using the returned host/ports, and using <proto> as the LDAP scheme. Please test. p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati(a)sys-net.it ---------------------------------------

1 0

Re: (ITS#4936) default modulepath
by ando＠sys-net.it 09 Jun '07

09 Jun '07

It would be pretty straightforward to call lt_dlsetsearchpath() (actually, the module_path() wrapper) from module_init() with a default value. This could be, for example, LDAP_LIBEXECDIR, where slapd itself gets installed by default. Comments? p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati(a)sys-net.it ---------------------------------------

1 0

Re: (ITS#4943) tpool.c pause vs. finish
by h.b.furuseth＠usit.uio.no 09 Jun '07

09 Jun '07

I wrote: > I'll just make [the hash table] chained. (...) > Solves the multi-pool problem too. No it doesn't. pool_pause() would stilll only pause one pool, while purgekey() needs all pools to be paused. I guess I'll chain it anyway. I forgot one bug in my summary: Slapd does not always check if pool_submit() and pool_setkey() succeed, even though failure is not a bug. (They can run into some limits.) pool_submit(): daemon.c:slapd_daemon_task() with rtask->routine connection.c:connections_shutdown() with c_clientfunc connection.c:connection_read() with c_clientfunc back-bdb/tools.c:bdb_tool_entry_open() with trickle and index tasks pool_setkey(): connection.c:connection_fake_init2() for slapi object extensions backend.c:be_isroot_pw() for SLAPD_SPASSWD passwd.c:slap_passwd_check() for SLAPD_SPASSWD sl_malloc.c:slap_sl_mem_create() back-bdb/search.c:search_stack() back-meta/conn.c:meta_back_candidates_get() slapd-modules/smbk5pwd/smbk5pwd.c:smbk5pwd_op_bind() -- Regards, Hallvard

1 0

Re: (ITS#4986) manageDSAit might control cause problems in database selection
by ando＠sys-net.it 09 Jun '07

09 Jun '07

A tentative fix is in HEAD (servers/slapd/backend.c 1.377 -> 1.378). Please test. p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati(a)sys-net.it ---------------------------------------

1 0

Re: (ITS#4943) tpool.c pause vs. finish
by h.b.furuseth＠usit.uio.no 09 Jun '07

09 Jun '07

hyc(a)symas.com writes: > h.b.furuseth(a)usit.uio.no wrote: >> * When a thread is finishing, make it go active for context_reset(). >> Should make slapi module destructors safer, and I don't see that it >> can hurt. > > I don't understand this. Let me dig a bit back in the ITS... "context_reset() can call connection_fake_destroy(), which via slapi_int_free_object_extensions() calls slapi module destructors, which can do who-knows-what." >> * I'd like to rename thread contexts to tasks(?) and user contexts back >> to contexts (as in RE23). The current terminology is confusing. > > I don't care about this either way. Then I will. I still confuse myself sometimes, and I certainly would if I came back to this code next year. >> (...) > Never use malloc unless absolutely necessary. We have enough problems > with heap fragmentation already. Aha. OK. >> * Scheduling: If several threads call pool_pause(), then once there is a >> pause tpool does not schedule them all. They could get handled then, >> or another thread could undo the pause so tpool would wait to pause >> again. Is that deliberate? > > I don't understand the question. The point of the pause is to prevent > any other thread (in the pool) from running. Why should tpool schedule > any other threads at this point? Two threads call pool_pause(). Eventually the pool gets paused, and one pool_pause() call returns. When that thread calls pool_resume(), the other thread waiting in pool_pause() may or may not get scheduled. >> - pool_context() breaks if several ldap_pvt_thread_t bit patterns can >> represent the same thread ID: TID_HASH() would give different hashes >> for the same thread, and pool_context() stops searching when it hits >> a slot with ctx == NULL. (My previous bug report was a bit wrong.) > > How can a single thread ID be represented by multiple bit patterns? A struct/union with padding bytes seems the least unlikely possibility. A pointer in hardware where several address representations map to the same physical address, and the compiler handles that by normalizing pointers when they are compared/subtracted. Like DOS "huge" pointers would have been if the compiler normalized them when comparing them instead of when incrementing/decrementing them. 20-bit address bus, 32-bit pointers, physical address = 16 * <segment half of pointer> + <offset half of pointer>. Or, not that I take this too seriously: A pthread implementation which uses e.g. a 32-bit integer type for thread IDs but just ignores the top 22 bits. The Posix spec seems to allow that, and thread IDs are to be compared with pthread_equal(). >> The best fix would be to use use real thread-specific data instead. >> Just one key with the ctx for now, that minimizes the changes. OTOH >> it also means we'll do thread-specific key lookup twice - first in >> pthread to get the ctx, and then in ltu_key[] to find our data. >> Anyway, I said I'd do that later but might as well get on with it, >> at least for pthreads. Except I don't know if that's OS-dependent >> enough that it should wait for RE24? > > The best fix may be to just use the address of the thread's stack as the > thread ID. That's actually perfect for our purpose. How does a function called from somewhere inside the thread know that address? That's almost what the user context is, and thus what thread_keys[] maps the thread ID to. >> - thread_keys is a poor hash table implementation with a poor hash >> function. > > Ideally it would not be a hash table at all, but a direct lookup based > on the thread ID. If we can. See above. Anyway, I don't know why I keep kvetching about the hash table implementation. I'll just make it chained. As I finally figured out, that can be done without mallocs. Solves the multi-pool problem too. >> * I wonder if there are long-lived ldap_pvt_thread_cond_wait(), >> [r]mutex_lock() or rdwr_[rw]lock() calls which should make the thread >> go inactive first (after waiting for any pause to finish), so that >> (a) a pool_pause() will not block waiting for them, and >> (b) all threads in the pool can't be blocked in long waits without >> tpool knowing about it. > > Long-lived lock calls would be a quite noticeable bug. This seems to be > a non-issue. Well, it'd have to be with code which is very rarely executed, or unusual configurations. But yes, I feel relaxed enough about it. >> * back-bdb/tools.c uses pool_submit() to start long-lived threads: >> (...) > Tool threads are completely separate from slapd threads. This is not an > issue. Fine. I don't understand that in this context, but I'll take your word for it:-) >> * slapd/bconfig.c maintains its own idea of the max number of threads >> in connection_pool_max and slap_tool_thread_max, which does not match >> tpool's idea if the limit is <=0 or larger than LDAP_MAXTHR. >> >> Since slapd acts on its own count, I imagine that can cause breakage. >> Simplest fix might be to set max #threads and then read it back, >> I haven't checked. Would need to return LDAP_MAXTHR instead of 0 >> for "as many as allowed". Note however also the issue above: >> tpool's idea of actually available threads can be too high too. > > The practical limits are actually pretty low. Even if you're on a 1024 > processor machine with many terabytes of RAM, it's not a good idea to > have many thousands of threads... Scheduler overhead would overshadow > any useful work. OK. I forgot how much RAM a slapd thread requires. But then it won't hurt to give slapd a max limit lower than LDAP_MAXTHR either. >> * I said my new ltp_pause waits might be too aggressive. Maybe I should >> clarify: I don't see any instances where they can be dropped, but it's >> possible that slapd expects pauses to be less aggressive somewhere. > > I don't understand this comment. Clarifying the clarification: I don't know slapd well enough to know if the extra pauses do something bad to it. >> Oh, the comment /* no open threads at all?!? */: That's not strange. >> Happens in the first pool_submit() call(s) if thread_create() fails. >> Though in that case there was no danger of removing the wrong ctx >> from ltp_pending_list after all (fixed in rev 1.63). > > (Sure, but thread_create() should never fail if you have a sane > configuration.) And if the rest of the machine doesn't run wild. I imagine thread_create() can fail when fork() can fail on some OSes. -- Regards, Hallvard

1 0

Re: (ITS#5003) issue about upper_needs_cast
by ando＠sys-net.it 09 Jun '07

09 Jun '07

caojunliang(a)huawei.com wrote: > when I set the option upper_needs_cast as Yes.It only takes effect on id_query > when adding an entry,but does not take effect on child id query when deleting an > entry. > > I found that in the src file back-sql/init.c > > line 374 --379 > > there are not operation about upper cast. > > I want to know if what I think is right. Not clear what's your point. "upper_needs_cast" is a hack to work around the fact that some RDBMSes need an explicit cast before passing a literal into whatever "uppercase" function they provide. Can you point out what's the problem and what's the expected behavior? I don't see any literal passed to the query that looks up the parent's and that needs any special treatment. A log of the offending operation, and an example of what you'd consider correct, would help. p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati(a)sys-net.it ---------------------------------------

1 0

Re: (ITS#4965) slapd stops if access to cn=monitor is restricted
by ando＠sys-net.it 09 Jun '07

09 Jun '07

This is now made consistent in HEAD: the inability to register custom monitor stuff always results in a failure. Note that the software provides different means to prevent this feature: a) allow anonymous read access to the monitor database b) define a rootdn (even without any possibility to auth) for the monitor database c) disable monitoring for all bdb/hdb databases p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati(a)sys-net.it ---------------------------------------

1 0

Re: (ITS#5005) test017 fails
by ando＠sys-net.it 09 Jun '07

09 Jun '07

dieter(a)dkluenter.de wrote: The problem is in slap_parse_sync_cookie(); when no cookie is passed, a string containing "rid=001" is parsed. The parser expects it to end with a comma. If passing "rid=001" is correct, the fix is trivial: instead of checking for (*next != ','), check for (*next && *next != ','). Index: servers/slapd/ldapsync.c =================================================================== RCS file: /repo/OpenLDAP/pkg/ldap/servers/slapd/ldapsync.c,v retrieving revision 1.42 diff -u -r1.42 ldapsync.c --- servers/slapd/ldapsync.c 18 May 2007 12:46:52 -0000 1.42 +++ servers/slapd/ldapsync.c 9 Jun 2007 09:42:24 -0000 @@ -180,7 +180,10 @@ if ( !strncmp( next, "rid=", STRLENOF("rid=") )) { rid_ptr = next; cookie->rid = strtoul( &rid_ptr[ STRLENOF( "rid=" ) ], &next, 10 ); - if ( next == rid_ptr || next > end || *next != ',' ) { + if ( next == rid_ptr + || next > end + || ( *next && *next != ',' ) ) + { return -1; } if ( *next == ',' ) { @@ -194,7 +197,10 @@ if ( !strncmp( next, "sid=", STRLENOF("sid=") )) { rid_ptr = next; cookie->sid = strtoul( &rid_ptr[ STRLENOF( "sid=" ) ], &next, 16 ); - if ( next == rid_ptr || next > end || *next != ',' ) { + if ( next == rid_ptr + || next > end + || ( *next && *next != ',' ) ) + { return -1; } if ( *next == ',' ) { I'm not committing this fix because I'm not sure it doesn't break anything else. p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati(a)sys-net.it ---------------------------------------

1 0

Re: (ITS#4965) slapd stops if access to cn=monitor is restricted
by ando＠sys-net.it 09 Jun '07

09 Jun '07

ando(a)sys-net.it wrote: > The difference is that if the monitor database comes first, it's already > open when the bdb tries to register its custom monitor stuff. So it can > barf out when the addition fails because of no enough permissions. > > If it comes later, back-bdb's custom monitor stuff ends up in the limbo, > which is then flushed when the monitor database starts up. At this > point, a failure in flushing the limbo is not considered critical. > > I see two/three solutions: > > 1) allow databases to fail monitor info registration without aborting > > 2) make startup fail also when limbo flushing fails > > 3) let the monitor database have a default rootdn. > > I'm not oriented towards any of the above, yet. I think the point is: if one explicitly requests monitoring support (by default on) in back-bdb, then it must either succeed or slapd should not start. If one does not explicitly request it, giving up monitoring with a warning should be fine. This is complicated by the need to support run-time configuration: in fact, if monitoring is explicitly requested but it cannot be registered, the creation of that database should fail, but the daemon should not stop. p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati(a)sys-net.it ---------------------------------------

1 0

Re: (ITS#4943) tpool.c pause vs. finish
by hyc＠symas.com 09 Jun '07

09 Jun '07

h.b.furuseth(a)usit.uio.no wrote: > OK, here's where we stand now, with my TODOs, questions and some > new issues. 3+ pages. I thought this would be a brief message:-( > Hopefully I'll get to some of it this weekend, but no promises. > > > TODOs (unless someone protests) and questions: > > * When a thread is finishing, make it go active for context_reset(). > Should make slapi module destructors safer, and I don't see that it > can hurt. I don't understand this. > * I'd like to rename thread contexts to tasks(?) and user contexts back > to contexts (as in RE23). The current terminology is confusing. I don't care about this either way. > > * Save and restore the old data for key slap_sasl_bind, and just in case > (I can't tell if it is needed) smbk5pwd. > > Might as well do it everywhere we have setkey - do something - delete > key in one function call. Since setkey(non-null) isn't supposed to > overwrite existing keys, the change should not be able to hurt. It > could protect something I didn't think of or a future slapd change. > > API: the smallest change would be to call pool_getkey() first, > but I prefer a pool_swapkey(ctx, key, &data, &kfree) function. > Then in RE24, maybe remove the &kfree parameter to pool_getkey(). > > Could omit the &kfree parameter to swapkey() if new data was > guaranteed non-NULL, but that'd require a malloc in bdb (silly says > Howard) or passing data in a &union{integer, ptr} (cumbersome say I). Never use malloc unless absolutely necessary. We have enough problems with heap fragmentation already. > * Axe semaphore code in tpool.c, the ldap_lazy_sem_init() call in > slapd/daemon.c, and with them LDAP_PVT_THREAD_POOL_SEM_LOAD_CONTROL > in ldap_pvt_thread.h and SLAP_SEM_LOAD_CONTROL in slap.h. OK. > * Is the ldap_pvt_thread_pool_context_reset() call in slapd/init.c > indeed safe - ie. its slap_sl_mem_destroy() call? That's not during > startup, but in slap_destroy(). Slapd does various stuff after that. Yes. > * API changes: > > Do we include the assert(no multiple pools) in the import to RE23? > Remove keys from ltu_key[] _before_ calling ltk_free() with them? > That prevents functions called from ltk_free from seeing freed thread > data, but it can also break ltk_free functions that call such > functions _before_ freeing it. Probably not for RE23. > > * Scheduling: If several threads call pool_pause(), then once there is a > pause tpool does not schedule them all. They could get handled then, > or another thread could undo the pause so tpool would wait to pause > again. Is that deliberate? I don't understand the question. The point of the pause is to prevent any other thread (in the pool) from running. Why should tpool schedule any other threads at this point? > Dubious code, new issues, old ones I'm not sure we finished with: > > * thread_keys[] still has problems. > > - pool_context() breaks if several ldap_pvt_thread_t bit patterns can > represent the same thread ID: TID_HASH() would give different hashes > for the same thread, and pool_context() stops searching when it hits > a slot with ctx == NULL. (My previous bug report was a bit wrong.) How can a single thread ID be represented by multiple bit patterns? > The best fix would be to use use real thread-specific data instead. > Just one key with the ctx for now, that minimizes the changes. OTOH > it also means we'll do thread-specific key lookup twice - first in > pthread to get the ctx, and then in ltu_key[] to find our data. > Anyway, I said I'd do that later but might as well get on with it, > at least for pthreads. Except I don't know if that's OS-dependent > enough that it should wait for RE24? The best fix may be to just use the address of the thread's stack as the thread ID. That's actually perfect for our purpose. > When that's not implemented, I'd prefer to cast the thread ID to an > integer when that is safe, and hash that instead of its bytes. > Won't work for structs/unions, and is not formally guaranteed for > pointers or in case of pthreads even integers. BTW, when it doesn't > work, we've got a problem in ITS#4983 (tls.c id_callback) as well. > > When that's not possible, we could search the entire table before > giving up. Could make thread_keys[] smaller and reduce the maximum > allowed #threads on such hosts. (What should the limit be?) > > - thread_keys is a poor hash table implementation with a poor hash > function. Ideally it would not be a hash table at all, but a direct lookup based on the thread ID. > Oops - for caching, maybe I shouldn't have removed the thread id > from thread_keys[]. Or maybe the hash value should be stored there > instead, so we can compare with == instead of ldap_pvt_thread_equal. > > * I wonder if there are long-lived ldap_pvt_thread_cond_wait(), > [r]mutex_lock() or rdwr_[rw]lock() calls which should make the thread > go inactive first (after waiting for any pause to finish), so that > (a) a pool_pause() will not block waiting for them, and > (b) all threads in the pool can't be blocked in long waits without > tpool knowing about it. Long-lived lock calls would be a quite noticeable bug. This seems to be a non-issue. > * back-bdb/tools.c uses pool_submit() to start long-lived threads: > bdb_tool_trickle_task() and bdb_tool_index_task(). They don't exit > until slapd is stopping, so they'll block slapd if a thread requests > a pause. Also the pool thinks its has more available threads that it > does, so scheduled tasks might never be executed. > > Possibly that's not a problem with slaptools. As complex as slapd is > now with overlays and plugins and whatnot, I have no idea. Tool threads are completely separate from slapd threads. This is not an issue. > * slapd/bconfig.c maintains its own idea of the max number of threads > in connection_pool_max and slap_tool_thread_max, which does not match > tpool's idea if the limit is <=0 or larger than LDAP_MAXTHR. > > Since slapd acts on its own count, I imagine that can cause breakage. > Simplest fix might be to set max #threads and then read it back, > I haven't checked. Would need to return LDAP_MAXTHR instead of 0 > for "as many as allowed". Note however also the issue above: > tpool's idea of actually available threads can be too high too. The practical limits are actually pretty low. Even if you're on a 1024 processor machine with many terabytes of RAM, it's not a good idea to have many thousands of threads... Scheduler overhead would overshadow any useful work. > * I said my new ltp_pause waits might be too aggressive. Maybe I should > clarify: I don't see any instances where they can be dropped, but it's > possible that slapd expects pauses to be less aggressive somewhere. I don't understand this comment. > > About pool_submit() and the ltp_mutex unlock-relock: > > Some history and notes, just to finish up previous discussion. > Some possible TODOs, but they are not needed since it's not broken: > > Originally, pool_submit() unlocked ltp_mutex before thread_create() > and re-locked it afterwards for further maintenance. That was not > because of the semaphore code. It looks like it could work even in > non-threaded mode where thread_create(,,fn,arg) just called fn(arg), > though tpool.c was always wrapped in an #ifndef <--without-threads>. > > I don't think non-threaded would have worked anyway since rev 1.24 > (apr 2003), which introduced thread_keys[]. Caller and callee would > have gotten the same thread ID, so tpool would have confused their > contexts. Previously the new context was pushed onto ltp_active_list, > and would have temporarily hid the old context. > > That's also when thread_create() was moved after the ltp_mutex > re-lock, which would mean the mutex was locked while pool_wrapper() > was running and tried to re-lock the mutex. And now, my rev.171 has > removed the unlock-relock. > > With that I can also remove the search for ctx in ltp_pending_list > when backing out of pool_submit, it will be at the head of that list. > > With tpool only supporting threaded mode, another cleanup could be for > pool_submit to start a thread before inserting ctx in ltp_pending_list > instead of after. That done, it can fail if ltp_open_count is still > 0, and we won't need the code for backing out of pool_submit(). OK. > Oh, the comment /* no open threads at all?!? */: That's not strange. > Happens in the first pool_submit() call(s) if thread_create() fails. > Though in that case there was no danger of removing the wrong ctx > from ltp_pending_list after all (fixed in rev 1.63). (Sure, but thread_create() should never fail if you have a sane configuration.) -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs