August 2008 - openldap-bugs

Re: (ITS#5662) Comments in schema declarations separated by semicolon

by h.b.furuseth＠usit.uio.no

michael(a)stroeder.com writes: >hyc(a)symas.com wrote: >>Michael Ströder wrote: >>> The comments wouldn't have to persist in cn=config. >> >> Perhaps not, but that makes the data in cn=config less comprehensible >> than it would be in slapd.conf. Well, that's no different from #comments at the beginning of lines. Or the default values which back-config inserts. One can't tell whether a value is present because the admin knew and thought about it or because cn=config defaulted it. However: >> Who benefits from this feature? > > An admin copying&pasting a schema from an standard document which uses > this format. I'm currently looking at such a document with ~500 > occurences of OIDs used in declarations instead of NAMEs. Which one? It's not RFC 4512 format. RFC 4512 uses ';' for comments _about_ the syntax of schema elements, not _in_ their syntax. -- Hallvard

14 years, 9 months

1
0
0 / 0

Re: (ITS#5664) Deadlocks when writing in parallell (two processes)

by stelios.xx.grigoriadis＠ericsson.com

tom.bjorkholm(a)aastra.com wrote: > Full_Name: Stelios Grigoriadis & Tom Björkholm > Version: 2.3.39 > OS: Novell SLES 10 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (194.237.142.7) > > > We get a lot of DB_LOCK_DEADLOCK when using client programs that for a period of > time continuously writes to OpenLDAP. > Version is 2.3.39. > > The information added is of the form: > ebcmdCustomer=0+ebcmdDir=220xx,ou=AuthCodes,ebcmdVersion=0,ebcmdProduct=ebcmd,dc=example,dc=com > where xx varies. > > Snippet of the output: > Mar 27 13:03:21 ldapt1 slapd[7589]: => bdb_dn2id_add: subtree > (ebcmdCustomer=0+ebcmdDir=22037,ou=authcodes,ebcmdVersion=0,ebcmdProduct=ebcmd,dc=example,dc=com) > put failed: -30995 > Mar 27 13:03:26 ldapt1 slapd[7589]: => bdb_idl_insert_key: c_put id failed: > DB_LOCK_DEADLOCK: Locker killed to resolve a deadlock (-30995) > Mar 27 13:03:26 ldapt1 slapd[7589]: => bdb_dn2id_add: parent > (ou=authcodes,ebcmdVersion=0,ebcmdProduct=ebcmd,dc=example,dc=com) insert > failed: -30995 > Mar 27 13:03:28 ldapt1 slapd[7589]: => bdb_idl_insert_key: c_put id failed: > DB_LOCK_DEADLOCK: Locker killed to resolve a deadlock (-30995) > Mar 27 13:03:28 ldapt1 slapd[7589]: => bdb_dn2id_add: parent > (ou=authcodes,ebcmdVersion=0,ebcmdProduct=ebcmd,dc=example,dc=com) insert > failed: -30995 > Mar 27 13:03:36 ldapt1 slapd[7589]: => bdb_idl_insert_key: c_put id failed: > DB_LOCK_DEADLOCK: Locker killed to resolve a deadlock (-30995) > Mar 27 13:03:36 ldapt1 slapd[7589]: => bdb_dn2id_add: parent > (ou=authcodes,ebcmdVersion=0,ebcmdProduct=ebcmd,dc=example,dc=com) insert > failed: -30995 > Mar 27 13:03:38 ldapt1 slapd[7589]: => bdb_idl_insert_key: c_put id failed: > DB_LOCK_DEADLOCK: Locker killed to resolve a deadlock (-30995) > > > We've temporarily fixed the problem by introducing a static mutex before any add/update operation. By doing so, we have effectively serialized the add/update operations from within slapd. This is just intended as a temporary solution as we hope the issue will be resolved in future releases. The patch: # This patch file is derived from OpenLDAP Software. All of the modifications to OpenLDAP Software # represented in the following patch(es) were developed by Stelios Grigoriadis stelios.xx.grigoriadis(a)ericsson.com. # These modifications are not subject to any license of Ericsson AB. # I, Stelios Grigoriadis, hereby place the following modifications to OpenLDAP Software (and only these modifications) # into the public domain. Hence, these modifications may be freely used and/or redistributed for any purpose with or # without attribution and/or other notice. # Bug Fix - This patch fixes the bug ITS#5133. # The fix works as follows. A periodic check in the runqueue (called do_mastercheck). The intervall is determined by # a slapd.conf parameter (mastercheckint) in the syncrepl section and is optional. If it's not specified, it's not # inserted in the runqueue. --- servers/slapd/connection.c 2007-06-15 01:49:38.000000000 +0200 +++ connection.c 2008-06-10 16:30:08.000000000 +0200 @@ -1052,19 +1052,24 @@ /* FIXME: returns 0 in case of failure */ ldap_pvt_mp_add_ulong(slap_counters.sc_ops_initiated, 1); ldap_pvt_thread_mutex_unlock( &slap_counters.sc_ops_mutex ); + static pthread_mutex_t op_upd_mutex = PTHREAD_MUTEX_INITIALIZER; + int upd_tag=0; op->o_threadctx = ctx; #ifdef LDAP_DEVEL op->o_tid = ldap_pvt_thread_pool_tid( ctx ); #endif /* LDAP_DEVEL */ switch ( tag ) { - case LDAP_REQ_BIND: - case LDAP_REQ_UNBIND: case LDAP_REQ_ADD: case LDAP_REQ_DELETE: case LDAP_REQ_MODDN: case LDAP_REQ_MODIFY: + ldap_pvt_thread_mutex_lock( &op_upd_mutex ); + upd_tag=1; + break; + case LDAP_REQ_BIND: + case LDAP_REQ_UNBIND: case LDAP_REQ_COMPARE: case LDAP_REQ_SEARCH: case LDAP_REQ_ABANDON: @@ -1178,6 +1183,10 @@ connection_resched( conn ); ldap_pvt_thread_mutex_unlock( &conn->c_mutex ); + if (upd_tag) { + ldap_pvt_thread_mutex_unlock( &op_upd_mutex ); + } + return NULL; }

14 years, 9 months

1
0
0 / 0

Re: (ITS#5662) Comments in schema declarations separated by semicolon

by michael＠stroeder.com

hyc(a)symas.com wrote: > Michael Ströder wrote: >> Howard Chu wrote: >>> As for truncating the comments on input - kind of pointless. In >>> particular, that means those comments won't be persisted in cn=config. >> The comments wouldn't have to persist in cn=config. > > Perhaps not, but that makes the data in cn=config less comprehensible than it > would be in slapd.conf. On the other hand, OID macros *are* preserved in > cn=config, which is yet another reason to use them. > > Who benefits from this feature? An admin copying&pasting a schema from an standard document which uses this format. I'm currently looking at such a document with ~500 occurences of OIDs used in declarations instead of NAMEs. > The comment is there for the benefit of a > human admin who has to read this definition. If the comment is stripped, then > the benefit is lost; anyone inspecting the schema remotely will just be faced > with the unwieldy numeric OIDs. Well, in web2ldap's schema viewer I'm always trying to display the NAME instead of the OID if possible. slapd could also normalize the schema declarations to list the NAME instead of OID if available (but that would be another ITS). > It still wouldn't persist because e.g. at_unparse() (which provides the value > that actually gets stored) works by constructing the string out of the LDAP > Attribute structure. So again, all of the comments would be gone, and there > would be no embedded line endings. My request was simply to truncate and ignore the comments. Ciao, Michael.

14 years, 9 months

1
0
0 / 0

Re: (ITS#5662) Comments in schema declarations separated by semicolon

by hyc＠symas.com

Michael Ströder wrote: > Howard Chu wrote: >> As for truncating the comments on input - kind of pointless. In >> particular, that means those comments won't be persisted in cn=config. > > The comments wouldn't have to persist in cn=config. Perhaps not, but that makes the data in cn=config less comprehensible than it would be in slapd.conf. On the other hand, OID macros *are* preserved in cn=config, which is yet another reason to use them. Who benefits from this feature? The comment is there for the benefit of a human admin who has to read this definition. If the comment is stripped, then the benefit is lost; anyone inspecting the schema remotely will just be faced with the unwieldy numeric OIDs. Remember also, slapd.conf is going to be phased out down the road. It's better not to add features to it that people are going to rely on, if the analogous feature won't be exposed in cn=config. Instead, we should be making use of whatever features already exist that are supported in both. Hallvard wrote: > back-config could accept > attributeTypes:: base64("description with line endings and comments") > if it doesn't already. It still wouldn't persist because e.g. at_unparse() (which provides the value that actually gets stored) works by constructing the string out of the LDAP Attribute structure. So again, all of the comments would be gone, and there would be no embedded line endings. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

14 years, 9 months

1
0
0 / 0

Re: (ITS#5662) Comments in schema declarations separated by semicolon

by michael＠stroeder.com

Howard Chu wrote: > As for truncating the comments on input - kind of pointless. In > particular, that means those comments won't be persisted in cn=config. The comments wouldn't have to persist in cn=config. Ciao, Michael.

14 years, 9 months

1
0
0 / 0

Re: (ITS#5662) Comments in schema declarations separated by semicolon

by h.b.furuseth＠usit.uio.no

Michael Ströder writes: >> but someone who schema via some LDAP-friendly tool might. > > ??? > > My schema viewer always trys to display NAME(s) of any schema element > linked. Oops, some words disappeard. I meant someone who reads/writes schema using a LDAP-friendly (and LDIF-friendly) tool, so they'd see the base64-decoded version of the attribute. -- Hallvard

14 years, 9 months

1
0
0 / 0

Re: (ITS#5662) Comments in schema declarations separated by semicolon

by michael＠stroeder.com

h.b.furuseth(a)usit.uio.no wrote: > Personally I wouldn't write such a file anytime > soon, Me neither. But if you just copy&paste schema declarations from a RFC or I-D it would be handy to accept this. > but someone who schema via some LDAP-friendly tool might. ??? My schema viewer always trys to display NAME(s) of any schema element linked. Ciao, Michael.

14 years, 9 months

1
0
0 / 0

Re: (ITS#5662) Comments in schema declarations separated by semicolon

by hyc＠symas.com

Michael Ströder wrote: > hyc(a)symas.com wrote: >> michael(a)stroeder.com wrote: >>> It might be helpful if the schema config file parser would accept lines with >>> comments at the end separated by semicolon. >> The parser doesn't know anything about line endings. Remember, lines with >> leading white space are continuations of the previous line. The schema parser >> is just fed a single string as one long line, the line endings were removed >> long before it ever got there. > > Maybe lines with ; could be truncated before passing it to the schema > parser? I don't know if that has any impact on processing slapd.conf in > general. > The slapd.conf parser has to already know in advance that this > part is a schema declaration (started by key-word objectclass etc.) and > could change pre-processing? Actually no, the parser doesn't know there's anything special about schema declarations. It doesn't parse the lines any differently than any other directives. Nor should it. As for truncating the comments on input - kind of pointless. In particular, that means those comments won't be persisted in cn=config. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

14 years, 9 months

1
0
0 / 0

Re: (ITS#5662) Comments in schema declarations separated by semicolon

by h.b.furuseth＠usit.uio.no

hyc(a)symas.com writes: > It would be smarter to just standardize OID macros for each of these > OIDs, the way X.500 did, and not require the humans reading these > specs to memorize these numeric OIDs... Yup. slapd/schema/syntaxes.schema? Syntax names like olsDirectoryString or just DirectoryString? (I don't know if OID macro names are a separate namespace or not.) > I don't see any straightforward way to modify our parser as you're > suggesting. Unless someone else feels like jumping in, I think we > should reject this ITS. back-config could accept attributeTypes:: base64("description with line endings and comments") if it doesn't already. Personally I wouldn't write such a file anytime soon, but someone who schema via some LDAP-friendly tool might. -- Hallvard

14 years, 9 months

1
0
0 / 0

Re: (ITS#5662) Comments in schema declarations separated by semicolon

by michael＠stroeder.com

hyc(a)symas.com wrote: > michael(a)stroeder.com wrote: >> It might be helpful if the schema config file parser would accept lines with >> comments at the end separated by semicolon. > > The parser doesn't know anything about line endings. Remember, lines with > leading white space are continuations of the previous line. The schema parser > is just fed a single string as one long line, the line endings were removed > long before it ever got there. Maybe lines with ; could be truncated before passing it to the schema parser? I don't know if that has any impact on processing slapd.conf in general. The slapd.conf parser has to already know in advance that this part is a schema declaration (started by key-word objectclass etc.) and could change pre-processing? Ciao, Michael.

14 years, 9 months

1
0
0 / 0

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs August 2008