Re: (ITS#4817) modify "replace" command firing conflict error where it shouldn't be
by hyc＠symas.com
Are you by any chance using slapadd -q? As noted in the documentation, that
turns off a lot of consistency checks. One of the things it turns off is the
input sorting, because you're only supposed to feed valid LDIF to slapadd -q.
daniel(a)ncsu.edu wrote:
>> daniel(a)ncsu.edu wrote:
>>> Hi Howard! As much as I hate to say this, it did not solve the problem
>>> I
>>> was running into. =( I rebuilt openldap with the entry.c patch and
>>> rebuilt the LDAP database from scratch (pulling from sources and using
>>> slapadd). Then I ran the following statement via ldapmodify (as always,
>>> XXX'd out sensitive info):
>> The slapcat output you provide shows that the attribute was already
>> slapadded
>> incorrectly. Please provide the "pulling from sources" input you used with
>> slapadd for this entry.
--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc
OpenLDAP Core Team http://www.openldap.org/project/
16 years, 4 months
Re: (ITS#4817) modify "replace" command firing conflict error where it shouldn't be
by daniel＠ncsu.edu
> daniel(a)ncsu.edu wrote:
>> Note that I have not changed anything to add the ou's as a group and the
>> ncsuCurriculumCode's as a group, but from what I understood, this should
>> work fine for openldap, even if it may or may not be truly legit based
>> off
>> the RFCs. And again, I have not changed anything in this code and have
>> been adding things like this for the past 2 years with OpenLDAP 2.2 and
>> earlier. I mean sure I can go edit my code to add those ou's and
>> ncsucurriculumcodes in the correct order (ie, bunched together), but if
>> you are saying openldap should be able to handle this, I'd like to stick
>> with it and see it resolved instead of letting it be a dangling
>> difference
>> behavior since 2.2. If you want to make an official-like statement that
>> what I'm doing there is not proper and not supported, then so be it,
>> I'll
>> take that as golden and adjust my code. =) (in other words, would you
>> rather it be that this is flat out not supported? .. of course if it's
>> not
>> supported it would be nicer if openldap yelled about it instead of
>> accepting it and just being mad afterwards)
>
> It is supposed to work. On my tests it does work. I have no idea what's
> happening on your side.
>
> Here's a snippet from running slapadd with -d -1 for the entry I tested
> with.
> Since I didn't have your NCSU schema I substituted some other
> attributetypes
> instead, but it should be equivalent. Note that I used "ou:" and "Ou:"
> which
> would have triggered the error before. The fact that "ou" only turns up
> once
> in the oc_check_allowed messages indicates that it correctly parsed them
> into
> a single attribute.
I'm going to send you this information "out of band". There's so much of
it that printed and I'm not 100% sure what is any is not sensitive in the
output. =(
Daniel
>
> Please send your corresponding debug output for this operation.
>
> slapadd startup: initiated.
> backend_startup_one: starting "dc=example,dc=com"
> bdb_db_open: dc=example,dc=com
> bdb_db_open: Warning - No DB_CONFIG file found in directory
> ./testrun/db.1.a: (2)
> Expect poor performance for suffix dc=example,dc=com.
> bdb_db_open: dbenv_open(./testrun/db.1.a)
> => str2entry: "dn: uid=XXX,ou=people,dc=example,dc=com
> objectClass: person
> objectClass: inetOrgPerson
> objectClass: openldapperson
> uid: XXX
> cn: XXX
> sn: XXX
> title: Senior
> description: XXX
> organizationalStatus: registered
> o: NC State University
> gn: XXX
> initials: XXX
> displayName: XXX
> personalTitle: XXX
> departmentNumber: XXX
> destinationIndicator: SR
> Ou: Physics
> businessCategory: PY
> ou: B S - Philosophy
> businessCategory: LSL
> mail: XXX(a)unity.ncsu.edu
> labeleduri: XXX(a)unity.ncsu.edu
> registeredAddress: XXX
> postalAddress: XXX
> telephoneNumber: XXX
> l: Raleigh
> st: NC
> postalCode: 27603
> employeeType: staff
> "
> >>> dnPrettyNormal: <uid=XXX,ou=people,dc=example,dc=com>
> <<< dnPrettyNormal: <uid=XXX,ou=people,dc=example,dc=com>,
> <uid=xxx,ou=people,dc=example,dc=com>
> <= str2entry(uid=XXX,ou=people,dc=example,dc=com) -> 0x7734f0
> oc_check_required entry (uid=XXX,ou=people,dc=example,dc=com), objectClass
> "OpenLDAPperson"
> oc_check_allowed type "objectClass"
> oc_check_allowed type "uid"
> oc_check_allowed type "cn"
> oc_check_allowed type "sn"
> oc_check_allowed type "title"
> oc_check_allowed type "description"
> oc_check_allowed type "organizationalStatus"
> oc_check_allowed type "o"
> oc_check_allowed type "givenName"
> oc_check_allowed type "initials"
> oc_check_allowed type "displayName"
> oc_check_allowed type "personalTitle"
> oc_check_allowed type "departmentNumber"
> oc_check_allowed type "destinationIndicator"
> oc_check_allowed type "ou"
> oc_check_allowed type "businessCategory"
> oc_check_allowed type "mail"
> oc_check_allowed type "labeledURI"
> oc_check_allowed type "registeredAddress"
> oc_check_allowed type "postalAddress"
> oc_check_allowed type "telephoneNumber"
> oc_check_allowed type "l"
> oc_check_allowed type "st"
> oc_check_allowed type "postalCode"
> oc_check_allowed type "employeeType"
> oc_check_allowed type "structuralObjectClass"
> => bdb_tool_entry_put( -1, "uid=XXX,ou=people,dc=example,dc=com" )
>
>
> --
> -- Howard Chu
> Chief Architect, Symas Corp. http://www.symas.com
> Director, Highland Sun http://highlandsun.com/hyc
> OpenLDAP Core Team http://www.openldap.org/project/
>
16 years, 4 months
Re: (ITS#4817) modify "replace" command firing conflict error where it shouldn't be
by hyc＠symas.com
daniel(a)ncsu.edu wrote:
> Note that I have not changed anything to add the ou's as a group and the
> ncsuCurriculumCode's as a group, but from what I understood, this should
> work fine for openldap, even if it may or may not be truly legit based off
> the RFCs. And again, I have not changed anything in this code and have
> been adding things like this for the past 2 years with OpenLDAP 2.2 and
> earlier. I mean sure I can go edit my code to add those ou's and
> ncsucurriculumcodes in the correct order (ie, bunched together), but if
> you are saying openldap should be able to handle this, I'd like to stick
> with it and see it resolved instead of letting it be a dangling difference
> behavior since 2.2. If you want to make an official-like statement that
> what I'm doing there is not proper and not supported, then so be it, I'll
> take that as golden and adjust my code. =) (in other words, would you
> rather it be that this is flat out not supported? .. of course if it's not
> supported it would be nicer if openldap yelled about it instead of
> accepting it and just being mad afterwards)
It is supposed to work. On my tests it does work. I have no idea what's
happening on your side.
Here's a snippet from running slapadd with -d -1 for the entry I tested with.
Since I didn't have your NCSU schema I substituted some other attributetypes
instead, but it should be equivalent. Note that I used "ou:" and "Ou:" which
would have triggered the error before. The fact that "ou" only turns up once
in the oc_check_allowed messages indicates that it correctly parsed them into
a single attribute.
Please send your corresponding debug output for this operation.
slapadd startup: initiated.
backend_startup_one: starting "dc=example,dc=com"
bdb_db_open: dc=example,dc=com
bdb_db_open: Warning - No DB_CONFIG file found in directory ./testrun/db.1.a: (2)
Expect poor performance for suffix dc=example,dc=com.
bdb_db_open: dbenv_open(./testrun/db.1.a)
=> str2entry: "dn: uid=XXX,ou=people,dc=example,dc=com
objectClass: person
objectClass: inetOrgPerson
objectClass: openldapperson
uid: XXX
cn: XXX
sn: XXX
title: Senior
description: XXX
organizationalStatus: registered
o: NC State University
gn: XXX
initials: XXX
displayName: XXX
personalTitle: XXX
departmentNumber: XXX
destinationIndicator: SR
Ou: Physics
businessCategory: PY
ou: B S - Philosophy
businessCategory: LSL
mail: XXX(a)unity.ncsu.edu
labeleduri: XXX(a)unity.ncsu.edu
registeredAddress: XXX
postalAddress: XXX
telephoneNumber: XXX
l: Raleigh
st: NC
postalCode: 27603
employeeType: staff
"
>>> dnPrettyNormal: <uid=XXX,ou=people,dc=example,dc=com>
<<< dnPrettyNormal: <uid=XXX,ou=people,dc=example,dc=com>,
<uid=xxx,ou=people,dc=example,dc=com>
<= str2entry(uid=XXX,ou=people,dc=example,dc=com) -> 0x7734f0
oc_check_required entry (uid=XXX,ou=people,dc=example,dc=com), objectClass
"OpenLDAPperson"
oc_check_allowed type "objectClass"
oc_check_allowed type "uid"
oc_check_allowed type "cn"
oc_check_allowed type "sn"
oc_check_allowed type "title"
oc_check_allowed type "description"
oc_check_allowed type "organizationalStatus"
oc_check_allowed type "o"
oc_check_allowed type "givenName"
oc_check_allowed type "initials"
oc_check_allowed type "displayName"
oc_check_allowed type "personalTitle"
oc_check_allowed type "departmentNumber"
oc_check_allowed type "destinationIndicator"
oc_check_allowed type "ou"
oc_check_allowed type "businessCategory"
oc_check_allowed type "mail"
oc_check_allowed type "labeledURI"
oc_check_allowed type "registeredAddress"
oc_check_allowed type "postalAddress"
oc_check_allowed type "telephoneNumber"
oc_check_allowed type "l"
oc_check_allowed type "st"
oc_check_allowed type "postalCode"
oc_check_allowed type "employeeType"
oc_check_allowed type "structuralObjectClass"
=> bdb_tool_entry_put( -1, "uid=XXX,ou=people,dc=example,dc=com" )
--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc
OpenLDAP Core Team http://www.openldap.org/project/
16 years, 4 months
Re: (ITS#4817) modify "replace" command firing conflict error where it shouldn't be
by daniel＠ncsu.edu
> daniel(a)ncsu.edu wrote:
>> Hi Howard! As much as I hate to say this, it did not solve the problem
>> I
>> was running into. =( I rebuilt openldap with the entry.c patch and
>> rebuilt the LDAP database from scratch (pulling from sources and using
>> slapadd). Then I ran the following statement via ldapmodify (as always,
>> XXX'd out sensitive info):
>
> The slapcat output you provide shows that the attribute was already
> slapadded
> incorrectly. Please provide the "pulling from sources" input you used with
> slapadd for this entry.
dn: uid=XXX,ou=students,ou=people,dc=ncsu,dc=edu
objectClass: person
objectClass: inetOrgPerson
objectClass: ncsuPerson
uid: XXX
cn: XXX
sn: XXX
title: Sophomore
ncsuTwoPartName: XXX
organizationalStatus: registered
o: NC State University
gn: XXX
ncsuMiddleName: XXX
initials: XXX
displayName: XXX
ncsuAltDisplayName: XXX
ncsuCampusID: XXX
ncsuClassCode: SO
ou: Political Science - Int Politics Conc
ncsuCurriculumCode: LIP
ou: B A - History
ncsuCurriculumCode: LAH
mail: XXX
ncsuPrimaryEMail: XXX
registeredAddress: XXX
postalAddress: XXX
telephoneNumber: XXX
l: Graham
st: NC
postalCode: XXX
ncsuPrimaryRole: student
Note that I have not changed anything to add the ou's as a group and the
ncsuCurriculumCode's as a group, but from what I understood, this should
work fine for openldap, even if it may or may not be truly legit based off
the RFCs. And again, I have not changed anything in this code and have
been adding things like this for the past 2 years with OpenLDAP 2.2 and
earlier. I mean sure I can go edit my code to add those ou's and
ncsucurriculumcodes in the correct order (ie, bunched together), but if
you are saying openldap should be able to handle this, I'd like to stick
with it and see it resolved instead of letting it be a dangling difference
behavior since 2.2. If you want to make an official-like statement that
what I'm doing there is not proper and not supported, then so be it, I'll
take that as golden and adjust my code. =) (in other words, would you
rather it be that this is flat out not supported? .. of course if it's not
supported it would be nicer if openldap yelled about it instead of
accepting it and just being mad afterwards)
I moved away from 2.2 because it looks like you all don't support it
anymore and I started running into stability problems. (which we have
not narrowed down to be OpenLDAP's 'fault'.. just upgrading OpenLDAP was
a good first and easy step 1, and actually seems to have helped)
Daniel
> --
> -- Howard Chu
> Chief Architect, Symas Corp. http://www.symas.com
> Director, Highland Sun http://highlandsun.com/hyc
> OpenLDAP Core Team http://www.openldap.org/project/
>
16 years, 4 months
Re: (ITS#4822) After test049 is over, restarting the producer dumps core
by hyc＠symas.com
ando(a)sys-net.it wrote:
> Full_Name: Pierangelo Masarati
> Version: HEAD
> OS: Linux 2.6 (CentOS 4.4)
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (81.72.89.40)
> Submitted by: ando
>
>
> After test049 is over, restarting the producer dumps core. This happens in the
> consumer's code (!) which, as far as I can understand, is there in cn=config
> only for the purpose of replicating to the consumer and set it to be, well, a
> consumer.
>
> Same happens when re-starting slapd with the consumer's database.
>
> The problem is related to the pending CSN being a NULL berval; the assertion
> occurs while trying to copy it, since the source of a ber copy cannot be NULL.
> If it is correct that pending CSN be NULL, the test must be performed before
> calling ber_bvreplace(); otherwise, this needs to be fixed.
>
> At this point I suggest that test049 includes stopping and restarting both (when
> this bug is fixed), re-checking that contents match.
>
> <producer>
> ...
> Core was generated by `../../../servers/slapd/slapd -F slapd.d/ -h ldap://:9011
> -s0 -dstats'.
Recall from http://www.openldap.org/lists/openldap-devel/200701/msg00077.html
the trick to keep the consumer from starting only works if the providerURI
matches a listenerURI. The providerURI is ldap://localhost:9011 which does
not match what you used here.
The other half of this ITS remains to be investigated...
> Program terminated with signal 6, Aborted.
> ...
> (gdb) bt
> #0 0x003957a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2
> #1 0x00fbf7a5 in raise () from /lib/tls/libc.so.6
> #2 0x00fc1209 in abort () from /lib/tls/libc.so.6
> #3 0x00fb8d91 in __assert_fail () from /lib/tls/libc.so.6
> #4 0x08218a30 in ber_bvreplace_x (dst=0xb6d15ff4, src=0x97ae258, ctx=0x0)
> at memory.c:703
> #5 0x080f34d1 in slap_queue_csn (op=0xb6d15f30, csn=0xb6d15bf0)
> at ctxcsn.c:167
> #6 0x080e91bc in syncrepl_del_nonpresent (op=0xb6d15f30, si=0x977ea68,
> uuids=0x0, cookiecsn=0xb6d15d9c) at syncrepl.c:2097
> #7 0x080e49a0 in do_syncrep2 (op=0xb6d15f30, si=0x977ea68) at syncrepl.c:728
> #8 0x080e58e1 in do_syncrepl (ctx=0xb6d162a4, arg=0x977a008)
> at syncrepl.c:1010
> #9 0x081e5948 in ldap_int_thread_pool_wrapper (xpool=0x9701bc8) at tpool.c:704
> #10 0x00692371 in start_thread () from /lib/tls/libpthread.so.0
> #11 0x0105fffe in clone () from /lib/tls/libc.so.6
> </producer>
>
> <consumer>
> ...
> Core was generated by `../../../servers/slapd/slapd -F slapd.d/ -h ldap://:9011
> -s0 -dstats'.
> Program terminated with signal 6, Aborted.
> ...
> (gdb) bt
> #0 0x003957a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2
> #1 0x00f057a5 in raise () from /lib/tls/libc.so.6
> #2 0x00f07209 in abort () from /lib/tls/libc.so.6
> #3 0x00efed91 in __assert_fail () from /lib/tls/libc.so.6
> #4 0x08218a30 in ber_bvreplace_x (dst=0xb6cd7ff4, src=0x8dec878, ctx=0x0)
> at memory.c:703
> #5 0x080f34d1 in slap_queue_csn (op=0xb6cd7f30, csn=0xb6cd7bf0)
> at ctxcsn.c:167
> #6 0x080e91bc in syncrepl_del_nonpresent (op=0xb6cd7f30, si=0x8db80b0,
> uuids=0x0, cookiecsn=0xb6cd7d9c) at syncrepl.c:2097
> #7 0x080e49a0 in do_syncrep2 (op=0xb6cd7f30, si=0x8db80b0) at syncrepl.c:728
> #8 0x080e58e1 in do_syncrepl (ctx=0xb6cd82a4, arg=0x8da5408)
> at syncrepl.c:1010
> #9 0x081e5948 in ldap_int_thread_pool_wrapper (xpool=0x8d3fbc8) at tpool.c:704
> #10 0x003cb371 in start_thread () from /lib/tls/libpthread.so.0
> #11 0x00fa5ffe in clone () from /lib/tls/libc.so.6
> </consumer>
>
>
>
--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc
OpenLDAP Core Team http://www.openldap.org/project/
16 years, 4 months
(ITS#4822) After test049 is over, restarting the producer dumps core
by ando＠sys-net.it
Full_Name: Pierangelo Masarati
Version: HEAD
OS: Linux 2.6 (CentOS 4.4)
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (81.72.89.40)
Submitted by: ando
After test049 is over, restarting the producer dumps core. This happens in the
consumer's code (!) which, as far as I can understand, is there in cn=config
only for the purpose of replicating to the consumer and set it to be, well, a
consumer.
Same happens when re-starting slapd with the consumer's database.
The problem is related to the pending CSN being a NULL berval; the assertion
occurs while trying to copy it, since the source of a ber copy cannot be NULL.
If it is correct that pending CSN be NULL, the test must be performed before
calling ber_bvreplace(); otherwise, this needs to be fixed.
At this point I suggest that test049 includes stopping and restarting both (when
this bug is fixed), re-checking that contents match.
<producer>
...
Core was generated by `../../../servers/slapd/slapd -F slapd.d/ -h ldap://:9011
-s0 -dstats'.
Program terminated with signal 6, Aborted.
...
(gdb) bt
#0 0x003957a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2
#1 0x00fbf7a5 in raise () from /lib/tls/libc.so.6
#2 0x00fc1209 in abort () from /lib/tls/libc.so.6
#3 0x00fb8d91 in __assert_fail () from /lib/tls/libc.so.6
#4 0x08218a30 in ber_bvreplace_x (dst=0xb6d15ff4, src=0x97ae258, ctx=0x0)
at memory.c:703
#5 0x080f34d1 in slap_queue_csn (op=0xb6d15f30, csn=0xb6d15bf0)
at ctxcsn.c:167
#6 0x080e91bc in syncrepl_del_nonpresent (op=0xb6d15f30, si=0x977ea68,
uuids=0x0, cookiecsn=0xb6d15d9c) at syncrepl.c:2097
#7 0x080e49a0 in do_syncrep2 (op=0xb6d15f30, si=0x977ea68) at syncrepl.c:728
#8 0x080e58e1 in do_syncrepl (ctx=0xb6d162a4, arg=0x977a008)
at syncrepl.c:1010
#9 0x081e5948 in ldap_int_thread_pool_wrapper (xpool=0x9701bc8) at tpool.c:704
#10 0x00692371 in start_thread () from /lib/tls/libpthread.so.0
#11 0x0105fffe in clone () from /lib/tls/libc.so.6
</producer>
<consumer>
...
Core was generated by `../../../servers/slapd/slapd -F slapd.d/ -h ldap://:9011
-s0 -dstats'.
Program terminated with signal 6, Aborted.
...
(gdb) bt
#0 0x003957a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2
#1 0x00f057a5 in raise () from /lib/tls/libc.so.6
#2 0x00f07209 in abort () from /lib/tls/libc.so.6
#3 0x00efed91 in __assert_fail () from /lib/tls/libc.so.6
#4 0x08218a30 in ber_bvreplace_x (dst=0xb6cd7ff4, src=0x8dec878, ctx=0x0)
at memory.c:703
#5 0x080f34d1 in slap_queue_csn (op=0xb6cd7f30, csn=0xb6cd7bf0)
at ctxcsn.c:167
#6 0x080e91bc in syncrepl_del_nonpresent (op=0xb6cd7f30, si=0x8db80b0,
uuids=0x0, cookiecsn=0xb6cd7d9c) at syncrepl.c:2097
#7 0x080e49a0 in do_syncrep2 (op=0xb6cd7f30, si=0x8db80b0) at syncrepl.c:728
#8 0x080e58e1 in do_syncrepl (ctx=0xb6cd82a4, arg=0x8da5408)
at syncrepl.c:1010
#9 0x081e5948 in ldap_int_thread_pool_wrapper (xpool=0x8d3fbc8) at tpool.c:704
#10 0x003cb371 in start_thread () from /lib/tls/libpthread.so.0
#11 0x00fa5ffe in clone () from /lib/tls/libc.so.6
</consumer>
16 years, 4 months
Re: (ITS#4821) test043 cores; apparently incorrect resource usage
by ando＠sys-net.it
hyc(a)symas.com wrote:
> ando(a)sys-net.it wrote:
>> hyc(a)symas.com wrote:
>>
>>> Working now with patched libldap_r/tpool.c
>> But now it leaks a lot at shutdown...
>
> Sigh, previous patch was missing a line. OK now.
Comfirmed. I'll close this ITS.
Thanks, p.
16 years, 4 months