openldap-devel

openldap-devel@openldap.org

1443 discussions

Language bindings
by Lorenzo Pastrana 09 Jan '09

09 Jan '09

Hi, I'm planning to make a v8(http://code.google.com/p/v8/) - ldap binding/extension Trying to do it well, I'd be glad to hear any recommendation ( musts / pitfalls / post mortem / etc. ), as well as any useful documentation / reference / previous works etc. Thanks. Lo. Lorenzo Pastrana - Happy End Vision -------------------------- Web Shop Multimédia Design Visual Communication & Publishing -------------------------- Tél.: +33 1 42 47 83 09 Fax.: +33 1 47 70 70 19 E-mail : lorenzo.pastrana(a)happyend.fr

1 0

Re: commit: ldap/servers/slapd syncrepl.c
by Rein Tollevik 29 Dec '08

29 Dec '08

hyc(a)OpenLDAP.org wrote: > Update of /repo/OpenLDAP/pkg/ldap/servers/slapd > > Modified Files: > syncrepl.c 1.434 -> 1.435 > > Log Message: > ITS#5866 skip CSN check for glue entries > > > CVS Web URLs: > http://www.openldap.org/devel/cvsweb.cgi/servers/slapd/ > http://www.openldap.org/devel/cvsweb.cgi/servers/slapd/syncrepl.c This patch adds test for "!is_entry_glue( rs->sr_entry )" twice, both in the outer and the inner if condition. A suspect it should only have been in the inner condition? Rein

2 1

ITS#5875: build fix for sparc64/gcc-3.3
by manu＠netbsd.org 28 Dec '08

28 Dec '08

Hello If it does not raise any concern, I will commit this patch, which fixes the build on sparc64/gcc-3.3. It has been tested in NetBSD package system for 5 years, so that looks quite safe... -- Emmanuel Dreyfus http://hcpnet.free.fr/pubz manu(a)netbsd.org

2 1

hide attribute
by manu＠netbsd.org 25 Dec '08

25 Dec '08

Hello Many badly designed software fetch all attribute when looking up an user in the directory, instead of just fetching the one they are interested in. My user objects have jpegPhoto attribute, which get fetched with the whole user object. jpegPhoto are big, so this cause unnescesary load on the network and LDAP servers and it slows down login process on the bad software. Setting up ACL to deny read access to jpegPhoto is not always feasible, nor it is easily maintainable. A nicer approach would probably to have a hidden jpegPhoto: it would not be sent to a client requesting all attributes, but a client explicitely requesting a set of attribute including jpegPhoto would get it. AFAIK, there is no way to do that for now. Am I right? I suspect an overlay would be the right way of implementing it (slapo-cloak?). Would it be of enough interest to go into server/slapd/overlays ? If it does, I will contribute it. -- Emmanuel Dreyfus http://hcpnet.free.fr/pubz manu(a)netbsd.org

3 10

Syntax/matching rule implementation - floating point numbers
by Hallvard B Furuseth 17 Dec '08

17 Dec '08

Here are some notes about how slapd implements a syntax/matching rule. I'm moving this to openldap-devel, it seems to belong there by now. A slapd search first uses any indexes to reduce the number of entries to examine, and then it checks these entries against the filter. An index entry contains the indexed value translated to a string which can be compared with simple memcmp(), and a list of entry IDs for the entries that contain that value. So e.g. caseIgnoreMatch translates upper- and lowercase characters to the same character in the index. If (x < y) implies (memcmp(indexed(x), indexed(y)) < 0), then ORDERING match can use indexing - the 'eq' index. Examples include the 'integer' indexing format since OpenLDAP 2.4.7, and 'generalizedTime'. Index keys should not be large, so string indexing stores a hash of the indexed value while integer and generalizedTime use binary formats. The integer index format is quite different from normal binary integer formats, to make ORDERING (i.e. memcmp()) work. You may want to normalize or prettify values received over the protocol before storing them, e.g. store incoming "nan" as "NaN". For syntax/matching rule examples see schema_init.c - for syntax 'foo': syntax_defs[] DESC 'Foo', mrule_defs[] NAME 'fooMatch','fooOrderingMatch'. fooValidate() checks the syntax of an attribute value. fooMatch() implements the EQUALITY and ORDERING rules. fooIndexer(), fooFilter() make the syntax indexable. fooNormalize() normalizes values, so they can be compared just with memcmp(). fooPretty() translates to a nicer to deal with but not normalized value, e.g. with DN syntax CN=foo\,bar ==> cn=foo\2Cbar. For a non-normalized syntax, look at 'foo'='integer'. For a normalized index, look at 'foo'='generalizedTime'. Note that having normalized the value, the 'generalizedTimeMatch' rule in mrule_defs[] simply uses octetStringMatch to compare instead of a more complex rule. 'generalizedTimeOrderingMatch' cannot do that though. If you go for a binary attribute syntax like some IEEE format, ask someone else for advise:-) There is also a distinction between whether such a syntax is transferred with the ";binary" option. (I think certificates must and are transferred as ASN.1 BER values. OTOH Octet String can contain any set of bytes, but is not ;binary.) Regarding a syntax for 'Real': Look at ITS#745 <http://www.openldap.org/its/?findid=745>. Nothing came of it, but maybe the author is around. NaN is a problem. NaN in math does not compare equal to itself, but such behavior would be messy for LDAP values. I haven't tested, but I think you could not have a multi-valued real attribute which contains NaN and some other number: When storing a value, LDAP uses the equality matching rule to check if you are trying to store a duplicate of an existing value. OTOH I expect realOrderingMatch(NaN, anything)=Undefined is OK, but I haven't tested that either. (Remember that LDAP filters have three-valued logic: A compare can return True, False, Undefined. Or an error, which isn't quite the same.) If you make it defined, and if current math practice doesn't say anything, I guess NaN should be larger than other values (except Inf?) to match Server Side Sorting (RFC 2891)'s treatment of absent values. OpenLDAP doesn't implement server side sorting, but people keep asking so maybe it will someday. -- Hallvard

2 5

Re: commit: ldap/servers/slapd syncrepl.c
by hyc＠symas.com 02 Dec '08

02 Dec '08

If anyone has some free time it would be a good idea to extend tests 17, 18, and 19 for these cases. Modify an entry (some non-RDN attribute) and then rename the entry. We should test Refresh when both ops have already occurred to the entry, as well as Refresh when each op occurs in a separate refresh pass. And as noted in -bugs, we should test with an entry using a single-valued RDN attribute. On Wed, Dec 03, 2008 at 03:49:54AM +0000, hyc(a)OpenLDAP.org wrote: > Update of /repo/OpenLDAP/pkg/ldap/servers/slapd > > Modified Files: > syncrepl.c 1.431 -> 1.432 > > Log Message: > More for prev commit - if doing both modrdn and modify at once, only > set the CSN on the modify op. > > > CVS Web URLs: > http://www.openldap.org/devel/cvsweb.cgi/servers/slapd/ > http://www.openldap.org/devel/cvsweb.cgi/servers/slapd/syncrepl.c > > Changes are generally available on cvs.openldap.org (and CVSweb) > within 30 minutes of being committed.

2 1

OpenLDAP stuck deep in BDB?
by Quanah Gibson-Mount 02 Dec '08

02 Dec '08

I've seen this now with two different masters, using BDB 4.2.52 + patches, which we long considered stable: Thread 3 (Thread 1115703648 (LWP 31507)): #0 0x0000003fca90bd9c in __pread_nocancel () from /lib64/tls/libpthread.so.0 #1 0x0000002a97348edc in __os_io (dbenv=0xdc3100, op=1, fhp=0xf7d2d0, pgno=71531, pagesize=4096, buf=0x2cb08f8cf0 "_3", niop=0x42681b70) at ../dist/../os/os_rw.c:55 #2 0x0000002a973405ea in __memp_pgread (dbmfp=0xeb5780, mutexp=0x2cbba53150, bhp=0x2cb08f8c58, can_create=0) at ../dist/../mp/mp_bh.c:219 #3 0x0000002a973413f7 in __memp_fget (dbmfp=0xeb5780, pgnoaddr=0x42681cc4, flags=0, addrp=0x42681cc8) at ../dist/../mp/mp_fget.c:580 #4 0x0000002a972d18b3 in __bam_search (dbc=0xdb1dc40, root_pgno=Variable "root_pgno" is not available. ) at ../dist/../btree/bt_search.c:307 #5 0x0000002a972c6e27 in __bam_c_search (dbc=0xdb1dc40, root_pgno=0, key=0x42681f40, flags=28, exactp=0x42681df4) at ../dist/../btree/bt_cursor.c:2547 #6 0x0000002a972c7b7e in __bam_c_get (dbc=0xdb1dc40, key=0x42681f40, data=0x42681f20, flags=28, pgnop=0x42681e64) at ../dist/../btree/bt_cursor.c:962 #7 0x0000002a9730f51b in __db_c_get (dbc_arg=0x28ea700, key=0x42681f40, data=0x42681f20, flags=28) at ../dist/../db/db_cam.c:643 #8 0x0000002a973177d7 in __db_c_get_pp (dbc=0x28ea700, key=0x42681f40, data=0x42681f20, flags=28) at ../dist/../db/db_iface.c:1836 #9 0x0000002a97183dc2 in bdb_dn2id (op=0x428025e0, dn=0x42682028, ei=0x42682010, locker=44, lock=0x42681fd0) at dn2id.c:315 #10 0x0000002a971896fd in bdb_cache_find_ndn (op=0x428025e0, locker=44, ndn=0x291c370, res=0x42682310) at cache.c:341 #11 0x0000002a97189e39 in bdb_cache_find_id (op=0x428025e0, tid=0x0, id=25145, eip=0x42682310, islocked=0, locker=44, lock=0x42682250) at cache.c:716 #12 0x0000002a9717b1c7 in bdb_search (op=0x428025e0, rs=0x42802570) at search.c:696 #13 0x0000002a975921eb in syncprov_findcsn (op=0x5ccd900, mode=FIND_PRESENT) at syncprov.c:681 #14 0x0000002a9759690a in syncprov_op_search (op=0x5ccd900, rs=0x42803d60) at syncprov.c:2074 #15 0x000000000049b935 in overlay_op_walk (op=0x5ccd900, rs=0x42803d60, which=op_search, oi=0xded8c0, on=0xded700) at backover.c:640 #16 0x000000000049bb91 in over_op_func (op=0x5ccd900, rs=0x42803d60, which=op_search) at backover.c:702 #17 0x000000000049bc27 in over_op_search (op=0x5ccd900, rs=0x42803d60) at backover.c:724 #18 0x000000000042f1d8 in fe_op_search (op=0x5ccd900, rs=0x42803d60) at search.c:355 #19 0x000000000042ecac in do_search (op=0x5ccd900, rs=0x42803d60) at search.c:217 #20 0x000000000042bd9c in connection_operation (ctx=0x42803e90, arg_v=0x5ccd900) at connection.c:1133 #21 0x000000000042c28c in connection_read_thread (ctx=0x42803e90, argv=0x1e) at connection.c:1261 ---Type <return> to continue, or q <return> to quit--- #22 0x0000002a956c7c77 in ldap_int_thread_pool_wrapper (xpool=0x8a1f00) at tpool.c:478 #23 0x0000003fca90610a in start_thread () from /lib64/tls/libpthread.so.0 #24 0x0000003fca0c68c3 in clone () from /lib64/tls/libc.so.6 #25 0x0000000000000000 in ?? () Has anyone else seen anything like this? It seems there's possibly a bug inside BDB 4.2.52 that gets hit occasionally. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

ITS#5835 insights?
by Quanah Gibson-Mount 02 Dec '08

02 Dec '08

Does anyone have any thoughts on ITS#5835? The server keeps hitting the assert in connection.c: static void connection_close( Connection *c ) { ber_socket_t sd = AC_SOCKET_INVALID; assert( connections != NULL ); assert( c != NULL ); /* ITS#4667 we may have gotten here twice */ if ( c->c_conn_state == SLAP_C_INVALID ) return; assert( c->c_struct_state == SLAP_C_USED ); --> assert( c->c_conn_state == SLAP_C_CLOSING ); Which was called by connections_timeout_idle in connection.c: if( difftime( c->c_activitytime+global_idletimeout, now) < 0 ) { /* close it */ connection_closing( c, "idletimeout" ); --> connection_close( c ); i++; With the modified debug logging Howard checked in, I see the following steps taking place: Dec 2 03:06:05 ldap slapd[8292]: connection_closing: readying conn=3049 sd=28 for close so we know it made it into connection_closing. Then we see: Dec 2 03:06:05 ldap slapd[8292]: connection_resched: attempting closing conn=3049 sd=28 which means it made it to connection_resched, which calls connection_close: Debug( LDAP_DEBUG_CONNS, "connection_resched: " "attempting closing conn=%lu sd=%d\n", conn->c_connid, sd, 0 ); connection_close( conn ); Doesn't this directly conflict with line 242 above, which also calls connection_close on the connection? I also see this comment in the connection_closing code: /* ITS#4667 this may allow another thread to drop into * connection_resched / connection_close before we * finish, but that's OK. */ I'm honestly not sure this is really "OK". The only place I see connection_resched called is from connection_operation, and I don't see exactly how I'ved ended up in there in this code path. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

sd= vs fd=?
by Quanah Gibson-Mount 01 Dec '08

01 Dec '08

I was looking at connection.c, and we intermix sd= and fd= in our connection debug logs, even though they are being used for the same thing as best I can tell. Is there any reason not to just standardize on fd? --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

slapadd performance degradation from 2.3.43 to 2.4.12
by Ralf Haferkamp 26 Nov '08

26 Nov '08

Hi, While doing a few slapadd testruns comparing the RE23 with the RE24 version I ran into a strange issue. I ran test with different LDIFs (100k, 500k and 1000k Entries) and especially with the 500k and 1000k LDIF, slapadd from 2.3.43 was significantly faster than the 2.4.12 version. 2.3.43 loaded the 500k database in 13m54s while it took 33m49s with 2.4.12. The 1000k testcase to 29m41s on 2.3 (still faster than the 500k on 2.4). I didn't finish the 2.4 run with the 1000k database I stopped it after about an hour. I used exactly the same configuration on exactly the same hardware/os for the tests (a HP Proliant DL 580 G3 with 4 3.33GHz Xeons, 8GB of RAM, SLES10-SP2, ext3 filesystem). BerkelyDB Version was 4.5.20 with the following DB_CONFIG: set_cachesize 0 4294967295 1 set_lg_regionmax 262144 set_lg_bsize 2097152 cachesize in slapd.conf was large enough to hold the entire database and tool- threads was set to 4. I did some profiling (with valgrinds callgrind tool) to find out where all the time is spend and it revealed that 2.4 spend a significantly larger amount of systime in the pwrite() function than 2.3. Most of that seemed to come from the bdb_tool_trickle_task() that calls libdb's memp_trickle() function. Just to verify this I ran a testbuild with the trickle_task disabled(). And slapadd's performance was back to a normal level, comparable to the 2.3.43 release. AFAIK the trickle_task() was introduced into 2.4 to increase slapadd throughput but has exactly the opposite effect on my test system. Did anybody else make similar experiences? Or do you see anything thats obviously wrong with my testcases? -- Ralf

4 17

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-devel