- openldap-bugs - openldap.org

Re: (ITS#6165) Support offline modify (slapmodify) for cn=config
by masarati＠aero.polimi.it 18 Apr '10

18 Apr '10

> Full_Name: Quanah Gibson-Mount > Version: 2.4.16 > OS: NA > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (75.111.29.239) > > > The slapmodify function already exists, but we've never built a tool > around it. > It would be useful to be able to have a utility that validates commands > that > users can use to modify cn=config while slapd is offline, rather than > resorting > to text editors that don't do validation of the changes being made. What about this? <ftp://ftp.openldap.org/incoming/slapmodify> ;) p.

1 0

Re: (ITS#6482) slapcat doesn't continue with -c after the first corrupted database entry
by masarati＠aero.polimi.it 18 Apr '10

18 Apr '10

> The main point wasn't to tell you how important the data was, but that > tools like ldapsearch or other tools connecting to slapd using port 389 > still saw the good data. Because it was cached? Was this true after restarting the server? > And slapd never complained about a corrupted > database. As a result of that the data corruption was hidden (all other > programs worked, only three non-important entries were not accessible) > until I tried to use slapcat to export the data for an upgrade. > > Regarding "If the underlying database fails, there's little slapcat can > do.": As long as slapd sees the data, because it sends it in responses > to ldap-queries, the database hasn't failed so much, that slapcat > couldn't retrieve those data. Well, as I said in my previous message, slapcat is asking the underlying database to provide entries. If the underlying database fails, there's little slapcat can do. When the underlying database is back-bdb/hdb, slapd accesses data using indexes (dn/filter indexes mapped to IDs; the entry is then accessed using BDB in DB_SET mode), while slapcat accesses it using BDB cursors in DB_NEXT mode. Your approach of guessing IDs in the hope it works sounds a bit naive; I think in your case it worked by chance. Perhaps, a more "robust" approach would be to have back-bdb/hdb bi_tool_entry_next() to try to use itself the DB_SET approach instead of the DB_NEXT when the latter fails, with slapcat only knowing about the failure (so it can abort if -c is not passed). > Regarding "any garbage coming out of a corrupted database": slapd never > complained about the database being corrupted, it started without any > problems (except that three entries were unreadable), the entries after > those damaged entries made sense and weren't garbage. Berkeley's > db_tools claimed that there were some errors in the db, but slapd worked > without problems, only slapcat (and ldapdelete when trying to delete > those entries) complained that it couldn't decode them (which seems to > be rather a problem of the application layer than of the database layer, > but might have been caused by some little db corruption). When things go wrong, and specifically when this happens outside the code of our pertinence (in this case BDB), usually the best we can do is to rely on what any diagnostics/recovery capability they provide. Again, what worked for you may not work for another kind of corruption. Actually, going on might provide inconsistent data, something as bad as data loss. > I tried tools like "db_recover", but they didn't help, What about catastrophic recovery, using archived log files? > so I wanted to > extract the good ldap entries (using slapcat ...) to drop the database > and recreate it with the exported ldif, which failed because slapcat > stopped after the first failure (even with -c). Using the submitted > patch I was able to export all readable data into a ldif-file and to > recreate the database. > > summary: > * slapd never complained, ldapsearch listed the data, only few entries > were missing (because for them following was true: > > <= entry_decode: slap_str2undef_ad(object�..!p): > AttributeDescription > contains inappropriate characters) > > --> it seems that some attribute-values were damaged, but not the dn? > > * if ldapsearch is able to retrieve the data, slapcat should also do In general, I concur. However, slapd and tools access data in a slightly different manner, so this may not be trivial, and might require a redesign. > * I don't know how you want to recover an corrupted database. I tried to > recover all good entries and then to recreate it. And for that I wanted > to use slapcat (or is there any other tool for exporting data from the > ldap database than slapcat with option -c)? None I know about. > * maybe the way -c works shouldn't be changed, but there should be an > option for trying data recovery harder without having to patch openldap > (which is complicated for users of other distros not working directly > with the sources, but with precompiled packages) I'd take this as a feature request for redesigning the tools iteration procedure in order to try to overcome minor db corruptions. p.

1 0

Re: (ITS#6528) test020 with hdb: Query 15: ldapsearch failed (4)
by michael＠stroeder.com 18 Apr '10

18 Apr '10

Seems to work now with recent checkout. Ciao, Michael.

1 0

Re: (ITS#6225) GSSAPI support for client library (fixes)
by mimir＠samba.org 18 Apr '10

18 Apr '10

--LZvS9be/3tNcYl/X Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Howard, On Fri, Jul 24, 2009 at 09:20:46PM -0700, Howard Chu wrote: > mimir(a)samba.org wrote: > >Full_Name: Rafal Szczesniak > >Version: Stable (2.4.16) > >OS: GNU/Linux > >URL: http://www.samba.org/~mimir/gssapi-bugfix.diff > >Submission from: (NULL) (67.51.54.234) > > > > > >This is a bugfix patch for OpenLDAP's gssapi support. > > > One of these patches appears to duplicate ITS#6092. As the original > author of this code, could you also take a look at #6091 and #6093? I'm really sorry but after I looked at current code in HEAD and I thought I= liked it (the actual issue was fixed), following with this thread completely went of= f my radar. I've found it while looking through my mailbox. The last chunk of this its' patch still applies though. It's a minor thing = but fixes potential memory leak. I can open a separate ITS for it if you like. As for the other patches - #6091 is (was) convenient but not absolutely nec= essary and #6093 is interesting but I'm not sure if it isn's a bit too much. If yo= u're connecting invalid (from kerberos point of view) hostname then, well, find out what's = the correct=20 hostname first. This its # can be closed of course. cheers, --=20 Rafal Szczesniak Samba Team member http://www.samba.org Likewise Software http://www.likewise.com --LZvS9be/3tNcYl/X Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) iD8DBQFLy1e3Hvdfyv3qiKkRAk7ZAJ0VPkzc89VAxXh/AF4EZVCJTvS/fgCfeZK7 TULoTjp8aXQkK0oGZNShxsY= =zucU -----END PGP SIGNATURE----- --LZvS9be/3tNcYl/X--

1 0

Re: (ITS#6482) slapcat doesn't continue with -c after the first corrupted database entry
by sysadmin＠htl-leonding.ac.at 18 Apr '10

18 Apr '10

This is a multi-part message in MIME format. --------------020204070007000004050507 Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: 7bit The main point wasn't to tell you how important the data was, but that tools like ldapsearch or other tools connecting to slapd using port 389 still saw the good data. And slapd never complained about a corrupted database. As a result of that the data corruption was hidden (all other programs worked, only three non-important entries were not accessible) until I tried to use slapcat to export the data for an upgrade. Regarding "If the underlying database fails, there's little slapcat can do.": As long as slapd sees the data, because it sends it in responses to ldap-queries, the database hasn't failed so much, that slapcat couldn't retrieve those data. Regarding "any garbage coming out of a corrupted database": slapd never complained about the database being corrupted, it started without any problems (except that three entries were unreadable), the entries after those damaged entries made sense and weren't garbage. Berkeley's db_tools claimed that there were some errors in the db, but slapd worked without problems, only slapcat (and ldapdelete when trying to delete those entries) complained that it couldn't decode them (which seems to be rather a problem of the application layer than of the database layer, but might have been caused by some little db corruption). I tried tools like "db_recover", but they didn't help, so I wanted to extract the good ldap entries (using slapcat ...) to drop the database and recreate it with the exported ldif, which failed because slapcat stopped after the first failure (even with -c). Using the submitted patch I was able to export all readable data into a ldif-file and to recreate the database. summary: * slapd never complained, ldapsearch listed the data, only few entries were missing (because for them following was true: <= entry_decode: slap_str2undef_ad(object�..!p): AttributeDescription contains inappropriate characters) --> it seems that some attribute-values were damaged, but not the dn? * if ldapsearch is able to retrieve the data, slapcat should also do * I don't know how you want to recover an corrupted database. I tried to recover all good entries and then to recreate it. And for that I wanted to use slapcat (or is there any other tool for exporting data from the ldap database than slapcat with option -c)? * maybe the way -c works shouldn't be changed, but there should be an option for trying data recovery harder without having to patch openldap (which is complicated for users of other distros not working directly with the sources, but with precompiled packages) --------------020204070007000004050507 Content-Type: text/html; charset=ISO-8859-15 Content-Transfer-Encoding: 7bit <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html> <head> <meta http-equiv="content-type" content="text/html; charset=ISO-8859-15"> </head> <body bgcolor="#ffffff" text="#000000"> The main point wasn't to tell you how important the data was, but that tools like ldapsearch or other tools connecting to slapd using port 389 still saw the good data. And slapd never complained about a corrupted database. As a result of that the data corruption was hidden (all other programs worked, only three non-important entries were not accessible) until I tried to use slapcat to export the data for an upgrade. Regarding "If the underlying database fails, there's little slapcat can do.": As long as slapd sees the data, because it sends it in responses to ldap-queries, the database hasn't failed so much, that slapcat couldn't retrieve those data. Regarding "any garbage coming out of a corrupted database": slapd never complained about the database being corrupted, it started without any problems (except that three entries were unreadable), the entries after those damaged entries made sense and weren't garbage. Berkeley's db_tools claimed that there were some errors in the db, but slapd worked without problems, only slapcat (and ldapdelete when trying to delete those entries) complained that it couldn't decode them (which seems to be rather a problem of the application layer than of the database layer, but might have been caused by some little db corruption). I tried tools like "db_recover", but they didn't help, so I wanted to extract the good ldap entries (using slapcat ...) to drop the database and recreate it with the exported ldif, which failed because slapcat stopped after the first failure (even with -c). Using the submitted patch I was able to export all readable data into a ldif-file and to recreate the database. summary: * slapd never complained, ldapsearch listed the data, only few entries were missing (because for them following was true: <pre><= entry_decode: slap_str2undef_ad(object&#65533;..!p): AttributeDescription contains inappropriate characters) --> it seems that some attribute-values were damaged, but not the dn? </pre> * if ldapsearch is able to retrieve the data, slapcat should also do * I don't know how you want to recover an corrupted database. I tried to recover all good entries and then to recreate it. And for that I wanted to use slapcat (or is there any other tool for exporting data from the ldap database than slapcat with option -c)? * maybe the way -c works shouldn't be changed, but there should be an option for trying data recovery harder without having to patch openldap (which is complicated for users of other distros not working directly with the sources, but with precompiled packages) </body> </html> --------------020204070007000004050507--

1 0

Re: (ITS#6482) slapcat doesn't continue with -c after the first corrupted database entry
by masarati＠aero.polimi.it 18 Apr '10

18 Apr '10

> Maybe I haven't described the situation correctly: > The database was used for managing user accounts (via nss_ldap). The > database seems to had three corrupted entries in the middle (good > entries were following after that). The command "getent passwd" i.e. > still listed all users except those three with corrupted entries, but > export via slapcat and reimport of the data would have led to loss of > many more accounts. Lossing half your data is not the expected behaviour > for a tool, which should list all the existing (or at least somehow > readable) database entries. I'm not questioning the importance of the data slapcat couldn't export. > The important point is that slapcat cancelled the export at the first > corrupted database entry and didn't even try to get to the non-corrupted > database entries (which were seen by the other LDAP-tools). This is not correct. Slapcat tries. It asks the underlying backend to provide the subsequent entries. If the underlying database fails, there's little slapcat can do. > It didn't > even try in "continuation" mode, so there was no way to rescue all the > readable data via slapcat. I've tested slapcat in continuation mode by forcibly having be_entry_get() return NULL, and it works as expected: it continues to the next one, as soon as be_entry_next() is able to return a valid ID. if it can't, then anything you'd get from be_entry_next() with an arbitrary ID is unreliable. > For slapcat all entries occuring after the > first corrupt entry were lost. As the remaining good entries were still > accessible and visible via the other tools communicating with slapd via > the port 389, the expected behaviour for slapcat would have been to try > harder to ignore the damaged entries and to recover the remaining > readable entries after them. Maybe slapcat really failed to extract the > next entries id, but in "continuation"-mode it should try hard to > workaround any errors caused by non-readable entries. In case of an > non-readable entry my patch continues stupidly incrementing the id by 1 > and trying to read that entry. As soon as it can extract an entry, it > continues processing. If by "try hard" you mean that any garbage coming out of a corrupted database would be fine for you, then you can fix your patch by having it stop when id == NOID. To me, it sounds a bit too hard. What about recovering the database first, did you try? p.

1 0

Re: (ITS#6528) test020 with hdb: Query 15: ldapsearch failed (4)
by masarati＠aero.polimi.it 18 Apr '10

18 Apr '10

Thanks, I'm working at this. p. > Full_Name: Michael Ströder > Version: HEAD > OS: > URL: > Submission from: (NULL) (84.163.121.239) > > > Note: It seems to work with back-bdb > > $ ./run -b hdb test020 > Cleaning up test run directory leftover from previous run. > Running ./scripts/test020-proxycache for hdb... > Starting master slapd on TCP/IP port 9011... > Using ldapsearch to check that master slapd is running... > Using ldapadd to populate the master directory... > Starting proxy cache on TCP/IP port 9012... > Using ldapsearch to check that proxy slapd is running... > Making queries on the proxy cache... > Query 1: filter:(sn=Jon) attrs:all (expect nothing) > Query 2: filter:(|(cn=*Jon*)(sn=Jon*)) attrs:cn sn title uid > Query 3: filter:(sn=Smith*) attrs:cn sn uid > Query 4: filter:(sn=Doe*) attrs:cn sn title uid > Query 5: filter:(uid=johnd) attrs:mail postaladdress telephonenumber cn > uid > Query 6: filter:(mail=*@mail.alumni.example.com) attrs:cn sn title uid > Query 7: filter:(mail=*) attrs:cn sn title uid > Query 8: filter:(mail=*example.com) attrs:cn sn title uid > ldapsearch failed (4) > Query 9: filter:(uid=b*) attrs:mail > ldapsearch failed (4) > Query 1 not cacheable > Query 2 cacheable > Query 3 cacheable > Query 4 cacheable > Query 5 cacheable > Query 6 cacheable > Query 7 not cacheable > Query 8 cacheable > Query 9 cacheable > Successfully verified cacheability > Query 10: filter:(|(cn=*Jones)(sn=Jones)) attrs:cn sn title uid > Query 11: filter:(sn=Smith) attrs:cn sn title uid > Query 12: filter:(uid=bjorn) attrs:mail postaladdress telephonenumber cn > uid > Query 13: filter:(mail=jaj@mail.alumni.example.com) attrs:cn sn title uid > Query 14: filter:(mail=*example.com) attrs:cn sn title uid > ldapsearch failed (4) > Query 15: filter:(uid=b*) attrs:mail > ldapsearch failed (4) > Query 10 answerable > Query 11 answerable > Query 12 answerable > Query 13 not answerable > Query 14 not answerable > Query 15 answerable > Successfully verified answerability > Filtering ldapsearch results... > Filtering original ldif... > Comparing filter output... > Comparison failed > > >

1 0

Re: (ITS#6527) Build errors in function � slap_cf_aux_table_unparse�
by masarati＠aero.polimi.it 18 Apr '10

18 Apr '10

Thanks, Quanah is working at this. p. > Full_Name: Michael Ströder > Version: RE24 > OS: openSUSE 11.2 Linux > URL: > Submission from: (NULL) (84.163.121.239) > > > HEAD builds and make test works. > > RE24 fails to build since recent check-ins (anon CVS) yesterday: > > make[3]: Leaving directory > `/usr/src/michael/openldap/OPENLDAP_REL_ENG_2_4/openldap/servers/slapd/overlays' > ../../build/mkversion -v "2.4.X" -s -n Versionstr slapd > version.c > cc -g -O2 -DSLAP_SCHEMA_EXPOSE -I../../include -I. -I./slapi -I. > -I../../include > -c -o main.o main.c > cc -g -O2 -DSLAP_SCHEMA_EXPOSE -I../../include -I. -I./slapi -I. > -I../../include > -c -o globals.o globals.c > cc -g -O2 -DSLAP_SCHEMA_EXPOSE -I../../include -I. -I./slapi -I. > -I../../include > -c -o bconfig.o bconfig.c > cc -g -O2 -DSLAP_SCHEMA_EXPOSE -I../../include -I. -I./slapi -I. > -I../../include > -c -o config.o config.c > config.c: In function �slap_cf_aux_table_unparse�: > config.c:1575: error: duplicate case value > config.c:1547: error: previously used here > make[2]: *** [config.o] Error 1 > make[2]: Leaving directory > `/usr/src/michael/openldap/OPENLDAP_REL_ENG_2_4/openldap/servers/slapd' > make[1]: *** [all-common] Error 1 > make[1]: Leaving directory > `/usr/src/michael/openldap/OPENLDAP_REL_ENG_2_4/openldap/servers' > make: *** [all-common] Error 1 > > >

1 0

(ITS#6528) test020 with hdb: Query 15: ldapsearch failed (4)
by michael＠stroeder.com 18 Apr '10

18 Apr '10

Full_Name: Michael Ströder Version: HEAD OS: URL: Submission from: (NULL) (84.163.121.239) Note: It seems to work with back-bdb $ ./run -b hdb test020 Cleaning up test run directory leftover from previous run. Running ./scripts/test020-proxycache for hdb... Starting master slapd on TCP/IP port 9011... Using ldapsearch to check that master slapd is running... Using ldapadd to populate the master directory... Starting proxy cache on TCP/IP port 9012... Using ldapsearch to check that proxy slapd is running... Making queries on the proxy cache... Query 1: filter:(sn=Jon) attrs:all (expect nothing) Query 2: filter:(|(cn=*Jon*)(sn=Jon*)) attrs:cn sn title uid Query 3: filter:(sn=Smith*) attrs:cn sn uid Query 4: filter:(sn=Doe*) attrs:cn sn title uid Query 5: filter:(uid=johnd) attrs:mail postaladdress telephonenumber cn uid Query 6: filter:(mail=*@mail.alumni.example.com) attrs:cn sn title uid Query 7: filter:(mail=*) attrs:cn sn title uid Query 8: filter:(mail=*example.com) attrs:cn sn title uid ldapsearch failed (4) Query 9: filter:(uid=b*) attrs:mail ldapsearch failed (4) Query 1 not cacheable Query 2 cacheable Query 3 cacheable Query 4 cacheable Query 5 cacheable Query 6 cacheable Query 7 not cacheable Query 8 cacheable Query 9 cacheable Successfully verified cacheability Query 10: filter:(|(cn=*Jones)(sn=Jones)) attrs:cn sn title uid Query 11: filter:(sn=Smith) attrs:cn sn title uid Query 12: filter:(uid=bjorn) attrs:mail postaladdress telephonenumber cn uid Query 13: filter:(mail=jaj@mail.alumni.example.com) attrs:cn sn title uid Query 14: filter:(mail=*example.com) attrs:cn sn title uid ldapsearch failed (4) Query 15: filter:(uid=b*) attrs:mail ldapsearch failed (4) Query 10 answerable Query 11 answerable Query 12 answerable Query 13 not answerable Query 14 not answerable Query 15 answerable Successfully verified answerability Filtering ldapsearch results... Filtering original ldif... Comparing filter output... Comparison failed

1 0

Re: (ITS#6482) slapcat doesn't continue with -c after the first corrupted database entry
by sysadmin＠htl-leonding.ac.at 18 Apr '10

18 Apr '10

Maybe I haven't described the situation correctly: The database was used for managing user accounts (via nss_ldap). The database seems to had three corrupted entries in the middle (good entries were following after that). The command "getent passwd" i.e. still listed all users except those three with corrupted entries, but export via slapcat and reimport of the data would have led to loss of many more accounts. Lossing half your data is not the expected behaviour for a tool, which should list all the existing (or at least somehow readable) database entries. The important point is that slapcat cancelled the export at the first corrupted database entry and didn't even try to get to the non-corrupted database entries (which were seen by the other LDAP-tools). It didn't even try in "continuation" mode, so there was no way to rescue all the readable data via slapcat. For slapcat all entries occuring after the first corrupt entry were lost. As the remaining good entries were still accessible and visible via the other tools communicating with slapd via the port 389, the expected behaviour for slapcat would have been to try harder to ignore the damaged entries and to recover the remaining readable entries after them. Maybe slapcat really failed to extract the next entries id, but in "continuation"-mode it should try hard to workaround any errors caused by non-readable entries. In case of an non-readable entry my patch continues stupidly incrementing the id by 1 and trying to read that entry. As soon as it can extract an entry, it continues processing.

1 0

(ITS#6527) Build errors in function �slap_cf_aux_table_unparse�
by michael＠stroeder.com 18 Apr '10

18 Apr '10

Full_Name: Michael Ströder Version: RE24 OS: openSUSE 11.2 Linux URL: Submission from: (NULL) (84.163.121.239) HEAD builds and make test works. RE24 fails to build since recent check-ins (anon CVS) yesterday: make[3]: Leaving directory `/usr/src/michael/openldap/OPENLDAP_REL_ENG_2_4/openldap/servers/slapd/overlays' ../../build/mkversion -v "2.4.X" -s -n Versionstr slapd > version.c cc -g -O2 -DSLAP_SCHEMA_EXPOSE -I../../include -I. -I./slapi -I. -I../../include -c -o main.o main.c cc -g -O2 -DSLAP_SCHEMA_EXPOSE -I../../include -I. -I./slapi -I. -I../../include -c -o globals.o globals.c cc -g -O2 -DSLAP_SCHEMA_EXPOSE -I../../include -I. -I./slapi -I. -I../../include -c -o bconfig.o bconfig.c cc -g -O2 -DSLAP_SCHEMA_EXPOSE -I../../include -I. -I./slapi -I. -I../../include -c -o config.o config.c config.c: In function slap_cf_aux_table_unparse: config.c:1575: error: duplicate case value config.c:1547: error: previously used here make[2]: *** [config.o] Error 1 make[2]: Leaving directory `/usr/src/michael/openldap/OPENLDAP_REL_ENG_2_4/openldap/servers/slapd' make[1]: *** [all-common] Error 1 make[1]: Leaving directory `/usr/src/michael/openldap/OPENLDAP_REL_ENG_2_4/openldap/servers' make: *** [all-common] Error 1

1 0

Re: (ITS#6458) syncrepl fails if attribute has no equalty maching rule
by masarati＠aero.polimi.it 17 Apr '10

17 Apr '10

Should now be fixed in HEAD; please test. Thanks, p.

1 0

Re: (ITS#6458) syncrepl fails if attribute has no equalty maching rule
by masarati＠aero.polimi.it 17 Apr '10

17 Apr '10

> I see the issue, and I've isolated it to the case where you're removing a > value. In that case, in attr_cmp(), nn == 0 and no < o; the original test > for modifying an attribute without matching rule fails. I've fixed it > according to this patch, not sure whether it's appropriate for other cases > > <ftp://ftp.openldap.org/incoming/pierangelo-masarati-2010-04-17-syncrepl-no-…> > > Please test and review. Thanks, p. Please disregard, this patch is breaking other things (basically, producer's and consumer's entries are equivalent, but attrs are listed in a different order; this conforms to the LDAP model but breaks the original, preferable behavior). Working at a better fix. p.

1 0

Re: (ITS#6458) syncrepl fails if attribute has no equalty maching rule
by masarati＠aero.polimi.it 17 Apr '10

17 Apr '10

> Full_Name: Patrick Cernko > Version: 2.4.21 > OS: Linux > URL: > http://www.mpi-inf.mpg.de/~pcernko/syncrepl_fails_if_attribute_has_no_equal… > Submission from: (NULL) (139.19.3.67) > > > Hi OpenLDAP-Programmers, > > although some related bug reports were posted in the past years, this > issue > still seems not to be fixed: > > OpenLDAP's syncrepl replication fails if it comes to removing one of a > multi-value attribes of an object if this attribute does not have a > equality > matching rule defined in the schema. Modification works on the master > server but > fails to replicate to the slave(s) leaving them with the dit version just > before > and not replicating any other changes any more. > > I compiled a small example how to reproduce the problem with the (maybe > "well-known") nisNetgroupTriple attribute from nis.schema as shipped with > openldap-2.4.21: > > - extract the files from > http://www.mpi-inf.mpg.de/~pcernko/syncrepl_fails_if_attribute_has_no_equal… > into an empty working directory > - download the tarball to that working dir and configure similar to the > just > extracted configure.sh > - build and install it to get an install subdir in the working dir > - start the master using sh master.sh in the working dir > This will create the "master" database in your working dir, slapadd the > dit.ldif and start a master slapd with a unix socket /tmp/master. > - start the slave using sh slave.sh > This will create "slave" database in your working dir and start the > slave > slapd with a unix socket /tmp/slave which will immediately start syncing > from > the running master > - if you want you can search the 2 servers with > install/bin/ldapsearch -H ldapi://%2ftmp%2fmaster/ -b ou=openldap -LLL > resp. > install/bin/ldapsearch -H ldapi://%2ftmp%2fslave/ -b ou=openldap -LLL > - use "python mod.py" to remove one of the nisNetgroupTriple attributes > from the > netgroup object > - you will see a debug message from the slave server like this: > > bdb_modify_internal: delete nisNetgroupTriple > bdb_modify_internal: 18 modify/delete: nisNetgroupTriple: no equality > matching > rule > send_ldap_result: err=18 matched="" text="modify/delete: > nisNetgroupTriple: no > equality matching rule" > null_callback : error code 0x12 > slap_graduate_commit_csn: removing 0x29931c0 > 20100120124712.028626Z#000000#000#000000 > syncrepl_entry: rid=001 be_modify cn=netgroup,ou=openldap (18) > syncrepl_entry: rid=001 be_modify failed (18) > > if not, use "python mod.py add" to re-add the removed nisNetgroupTriple > and > try again. Sometimes, it seems to work (race condition?). > - you can also verify the difference with: > install/bin/ldapsearch -H ldapi://%2ftmp%2fmaster/ -b ou=openldap -LLL > cn=netgroup > install/bin/ldapsearch -H ldapi://%2ftmp%2fslave/ -b ou=openldap -LLL > cn=netgroup > - from now on, the slave will fail to sync any succeeding changes, as it > continuously tries to sync the modification above first. :-( > > Now, obviously having an attribute without equality matching rule is not a > good > idea in the first place. However, the (official?) nis.schema comes with > such an > attribute which we must use like this (e.g. we would have to patch the > Solaris > kernel to change it for the Solaris NFS server). > > It is possible to modify a nisNetgroup object like this: according to the > debugging output, I guess the master removes simply ALL nisNetgroupTriple > attributes and adds only the attributes that should remain. When the > corresponding syncrepl_entry is handled by the slave, it seems the it in > contrast tries to really only remove the 2-be-removed attribute which does > not > work as it cannot identify the right one due to a missing eq-match > operator. > > I suggest the best way to solve this issue is to try to get syncrepl do > exactly > the same actions (not resulting changes) as the server does. I see the issue, and I've isolated it to the case where you're removing a value. In that case, in attr_cmp(), nn == 0 and no < o; the original test for modifying an attribute without matching rule fails. I've fixed it according to this patch, not sure whether it's appropriate for other cases <ftp://ftp.openldap.org/incoming/pierangelo-masarati-2010-04-17-syncrepl-no-…> Please test and review. Thanks, p.

1 0

Re: (ITS#6463) dnssrv_back_referrals: ldaps not handled, search base gone
by masarati＠aero.polimi.it 17 Apr '10

17 Apr '10

> Full_Name: Jochen Keutel > Version: 2.4.21 > OS: Solaris 10 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (87.159.206.14) > > > dnssrv_back_referrals() always returns ldap URLs - even when the original > request was a ldaps query. This is a (logical) duplicate of ITS#6462; see related answers. > Additionally only protocol and hostname are put into the URL - the search > base > doesn't appear any more. > > Problematic code: > > url.bv_len = STRLENOF( "ldap://" ) + strlen( hosts[i] ); > url.bv_val = ch_malloc( url.bv_len + 1 ); > > strcpy( url.bv_val, "ldap://" ); > strcpy( &url.bv_val[STRLENOF( "ldap://" )], hosts[i] ); > > I'm not sure whether you want me to deliver a complete patch or not ... This behavior conforms to RFC4511, section 4.1.10: - If the <dn> part is present, the client uses this name in its next request to progress the operation, and if it is not present the client uses the same name as in the original request. I note that few lines above the same RFC states: - It is RECOMMENDED that the <dn> part be present to avoid ambiguity. However, slapd-dnssrv(5) computes the referral from DNS SRV records, rather than returning data it knows about. So returning the <dn> part here would be incorrect, in my opinion, as the client could legitimately rely on the fact that the returned DN is appropriate, while it may not. p.

1 0

Re: (ITS#6462) DNS SRV records: ldaps ???
by masarati＠aero.polimi.it 17 Apr '10

17 Apr '10

> --On Wednesday, January 27, 2010 1:28 AM +0000 jochen(a)keutel.de wrote: > >> Full_Name: Jochen Keutel >> Version: 2.4.21 >> OS: Solaris 10 >> URL: ftp://ftp.openldap.org/incoming/ > >> protocol is either "ldap" oder "ldaps". > > LDAPS is not a defined protocol. I would suggest using LDAP with > startTLS, > which is RFC Defined. One could argue that OpenLDAP supports ldaps in many places. I agree in general with your reply, but I'd treat this ITS as a feature request, although I'm not going to give it any special priority. p.

1 0

Re: (ITS#6491) LDAP Error code 80 - Dn index delete failed
by masarati＠aero.polimi.it 17 Apr '10

17 Apr '10

> Full_Name: Robert Hanson > Version: 2.4.21 > OS: RedHat 4.7 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (74.203.208.250) > > > 1) start with an empty database; use slapadd to add a large (3 MB) > database. 2) > Use ldapbrowser to delete nearly all the nodes. After this process, I get > to a > point where a subnode can't be deleted; I see the message "12:00:27 PM: > Failed > to delete entry ou=Agents, lcc=Call Center 1, ou=Company, o=Spanlink > Communications > Reason: [LDAP: error code 80 - DN index delete failed] > 12:15:19 PM: Failed to delete entry ou=Agents, lcc=Call Center 1, > ou=Company, > o=Spanlink Communications > Reason: [LDAP: error code 80 - DN index delete failed]" in the ldapbrowser > window. > > Steps to reproduce: (on redhat linux; I think any linux will do) > 1) build bdb 4.8.26 from the distro; build openldap 2.4.21 from the > distro. > 2) copy built slapd to an appropriate place. link it to slapadd. > 3) FTP: ftp.calabrio.com, user openldap, pass *a68pcJH (Account is > scheduled to > close 3/23/2010; let me know if you need the file uploaded again). unzip > the > file, it contains the slapcat.out; a blank database (in a subfolder), and > a > slapd.conf file. > 4) Edit the slapd.conf file as needed to point to the directory where you > put > the database. > > That was the setup, now here are the steps to reproduce the problem: > > 5) use slapadd -c -f slapd.conf -l slapcat.out to create the database. > 6) start slapd (using slapd -d 1 -f slapd.conf) > 7) use ldapbrowser to delete the "lcc=call center 1" node. > > It takes a long time, but eventually you'll see it fail while trying to > delete > the "agents" node. Can you reproduce with -dtrace? It'll generate a large log file, please only post the part related to the failure. p.

1 0

Re: (ITS#6472) Syncrepl : loop problem with moddn on a new node
by masarati＠aero.polimi.it 17 Apr '10

17 Apr '10

A fix is in HEAD, please test slapd/syncrepl.c 1.502 -> 1.503 Thanks for reporting, p.

1 0

Re: (ITS#6472) Syncrepl : loop problem with moddn on a new node
by hyc＠symas.com 17 Apr '10

17 Apr '10

masarati(a)aero.polimi.it wrote: > >> It's not clear to me where the issue is. What is the "right" sequence the >> add of the new superior and the mordrdn should be transmitted? Should the >> provider operate differently, or should the consumer check all syncrepl >> messages and try to rebuild the final state, instead of giving up when the >> internal lookup for the newsuperior fails? Probably, a workaround could >> be to perform the modrdn by crating the new superior as a glue object, >> which eventually will be replaced by the actual add. > > I've quickly hacked things this way, and it seems to work fine. > > <ftp://ftp.openldap.org/incoming/pierangelo-masarati-2010-04-17-sync-rename.…> > > Please let me know if this approach is sound enough, I might have > overlooked some implications. Patch looks good, solution makes sense. This is one of the reasons we would expect glue entries to be used. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#6524) gnutls cipher spec is unclear
by matthijs＠cacholong.nl 17 Apr '10

17 Apr '10

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Kurt Zeilenga wrote: > On Apr 17, 2010, at 3:15 AM, matthijs(a)cacholong.nl wrote: > >> On Apr 16, 2010, at 6:01 PM, Howard Chu wrote: >>> If Peter wants his patch considered for inclusion in OpenLDAP he = >> should write to the ITS himself, we cannot accept 3rd party = >> contributions. >> >> What's the reasoning behind this? > > 3rd parties (middle men) not only hinder good communication between the authors(s) of the contribution and the Project/Foundation, they introduce various risks to both the author and the Project/Foundation. > > Regard, Kurt (Executive Director, The OpenLDAP Foundation) > Hi, Thanks for the explanation, I'll keep it in mind when forwarding patches and / or bug reports from the Debian bugtracker. Regards, Matthijs Mohlmann -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkvKDnUACgkQ2n1ROIkXqbDFtwCgjOrjYmDPXtqYpQrQTNzq7gqL RcsAmgKRRJB28cke29WMHhKIIcB/+q7L =QTpi -----END PGP SIGNATURE-----

1 0

Re: (ITS#6472) Syncrepl : loop problem with moddn on a new node
by masarati＠aero.polimi.it 17 Apr '10

17 Apr '10

> It's not clear to me where the issue is. What is the "right" sequence the > add of the new superior and the mordrdn should be transmitted? Should the > provider operate differently, or should the consumer check all syncrepl > messages and try to rebuild the final state, instead of giving up when the > internal lookup for the newsuperior fails? Probably, a workaround could > be to perform the modrdn by crating the new superior as a glue object, > which eventually will be replaced by the actual add. I've quickly hacked things this way, and it seems to work fine. <ftp://ftp.openldap.org/incoming/pierangelo-masarati-2010-04-17-sync-rename.…> Please let me know if this approach is sound enough, I might have overlooked some implications. p.

1 0

Re: (ITS#6472) Syncrepl : loop problem with moddn on a new node
by masarati＠aero.polimi.it 17 Apr '10

17 Apr '10

>> Full_Name: Julien COMBES >> Version: 2.4.21 >> OS: Debian 5.0.4 >> URL: ftp://ftp.openldap.org/incoming/its-syncrepl-loop-moddn.tar.bz2 >> Submission from: (NULL) (212.23.175.185) >> >> >> Hello, >> >> I think I have found a loop problem with syncrepl replication with >> openldap >> 2.4.21, BDB 4.7.25 with all patches and hdb database. The problem >> appears >> sometimes when an entry is moved with "modrdbn -s" in a node which has >> just been >> created. I have reproduced the problem with the creation of a node and a >> moddn >> while the consumer was stopped and then restarted after. >> >> The problem follows these steps : >> - When it starts, the consumer does a request objectClass=* on the >> provider : >> Feb 12 09:09:19 ldapma24-ida01 slapd[30445]: conn=1007 op=1 SRCH >> base="dc=my,dc=domain" scope=2 deref=0 filter="(objectClass=*)" >> >> - The consumer finds the modrdn and tries to do this : >> Feb 12 09:09:19 ldapra24-ida01 slapd[12156]: >> ==>hdb_modrdn(cn=user1,ou=A,dc=my,dc=domain,cn=user1,ou=X,dc=my,dc=domain) >> >> - The consumer fails with these errors : >> Feb 12 09:09:19 ldapra24-ida01 slapd[12156]: => >> hdb_dn2id("ou=x,dc=my,dc=domain") >> Feb 12 09:09:19 ldapra24-ida01 slapd[12156]: <= hdb_dn2id: get failed: >> DB_NOTFOUND: No matching key/data pair found (-30988) >> Feb 12 09:09:19 ldapra24-ida01 slapd[12156]: hdb_modrdn: >> newSup(ndn=ou=x,dc=my,dc=domain) not here! >> Feb 12 09:09:19 ldapra24-ida01 slapd[12156]: send_ldap_result: conn=-1 >> op=0 p=0 >> Feb 12 09:09:19 ldapra24-ida01 slapd[12156]: send_ldap_result: err=32 >> matched="" >> text="new superior not found" >> >> - The consumer retries the request objectClass=* on the provider and >> loops on >> the problem. The replication doesn't work anymore. >> >> To reproduce the problem, I have used these steps : >> - start an empty provider >> - ldapadd the entries in mydomain.ldif >> ldapadd -x -h 127.0.0.1 -D "dc=my,dc=domain" -W -f mydomain.ldif >> - start the consumer. >> - stop the consumer when replication is finished >> - ldapadd the new node >> ldapadd -x -h 127.0.0.1 -D "dc=my,dc=domain" -W -f add.ldif >> - modrdn -s >> ldapmodrdn -x -h 127.0.0.1 -D "dc=my,dc=domain" -W -r -s >> "ou=X,dc=my,dc=domain" >> "cn=user1,ou=A,dc=my,dc=domain" "cn=user1" >> - start the consumer >> >> I join in its-syncrepl-loop-moddn.tar.bz2 : >> - slapd.conf of provider and consummer >> - log files of provider and consummer >> - mydomain.ldif and add.ldif > > Thanks for the detailed report. The bug is confirmed, and it's not > related to back-hdb, but seems to be syncrepl-related in general. It's not clear to me where the issue is. What is the "right" sequence the add of the new superior and the mordrdn should be transmitted? Should the provider operate differently, or should the consumer check all syncrepl messages and try to rebuild the final state, instead of giving up when the internal lookup for the newsuperior fails? Probably, a workaround could be to perform the modrdn by crating the new superior as a glue object, which eventually will be replaced by the actual add. p.

1 0

Re: (ITS#6472) Syncrepl : loop problem with moddn on a new node
by masarati＠aero.polimi.it 17 Apr '10

17 Apr '10

> Full_Name: Julien COMBES > Version: 2.4.21 > OS: Debian 5.0.4 > URL: ftp://ftp.openldap.org/incoming/its-syncrepl-loop-moddn.tar.bz2 > Submission from: (NULL) (212.23.175.185) > > > Hello, > > I think I have found a loop problem with syncrepl replication with > openldap > 2.4.21, BDB 4.7.25 with all patches and hdb database. The problem appears > sometimes when an entry is moved with "modrdbn -s" in a node which has > just been > created. I have reproduced the problem with the creation of a node and a > moddn > while the consumer was stopped and then restarted after. > > The problem follows these steps : > - When it starts, the consumer does a request objectClass=* on the > provider : > Feb 12 09:09:19 ldapma24-ida01 slapd[30445]: conn=1007 op=1 SRCH > base="dc=my,dc=domain" scope=2 deref=0 filter="(objectClass=*)" > > - The consumer finds the modrdn and tries to do this : > Feb 12 09:09:19 ldapra24-ida01 slapd[12156]: > ==>hdb_modrdn(cn=user1,ou=A,dc=my,dc=domain,cn=user1,ou=X,dc=my,dc=domain) > > - The consumer fails with these errors : > Feb 12 09:09:19 ldapra24-ida01 slapd[12156]: => > hdb_dn2id("ou=x,dc=my,dc=domain") > Feb 12 09:09:19 ldapra24-ida01 slapd[12156]: <= hdb_dn2id: get failed: > DB_NOTFOUND: No matching key/data pair found (-30988) > Feb 12 09:09:19 ldapra24-ida01 slapd[12156]: hdb_modrdn: > newSup(ndn=ou=x,dc=my,dc=domain) not here! > Feb 12 09:09:19 ldapra24-ida01 slapd[12156]: send_ldap_result: conn=-1 > op=0 p=0 > Feb 12 09:09:19 ldapra24-ida01 slapd[12156]: send_ldap_result: err=32 > matched="" > text="new superior not found" > > - The consumer retries the request objectClass=* on the provider and > loops on > the problem. The replication doesn't work anymore. > > To reproduce the problem, I have used these steps : > - start an empty provider > - ldapadd the entries in mydomain.ldif > ldapadd -x -h 127.0.0.1 -D "dc=my,dc=domain" -W -f mydomain.ldif > - start the consumer. > - stop the consumer when replication is finished > - ldapadd the new node > ldapadd -x -h 127.0.0.1 -D "dc=my,dc=domain" -W -f add.ldif > - modrdn -s > ldapmodrdn -x -h 127.0.0.1 -D "dc=my,dc=domain" -W -r -s > "ou=X,dc=my,dc=domain" > "cn=user1,ou=A,dc=my,dc=domain" "cn=user1" > - start the consumer > > I join in its-syncrepl-loop-moddn.tar.bz2 : > - slapd.conf of provider and consummer > - log files of provider and consummer > - mydomain.ldif and add.ldif Thanks for the detailed report. The bug is confirmed, and it's not related to back-hdb, but seems to be syncrepl-related in general. p.

1 0

Re: (ITS#6482) slapcat doesn't continue with -c after the first corrupted database entry
by masarati＠aero.polimi.it 17 Apr '10

17 Apr '10

> Full_Name: Himmelbauer > Version: 2.4.19 > OS: Gentoo Linux > URL: ftp://ftp.openldap.org/incoming/himmelbauer-100301.patch > Submission from: (NULL) (92.248.100.200) > > > Was not able to retrieve database entries from a corrupted LDAP database > using > slapcat (openldap 2.4.19) with option -c. It aborts after the first > corrupted > entry with > > <= entry_decode: slap_str2undef_ad(object�!p): > AttributeDescription > contains inappropriate characters > # no data for entry id=xxxxxxxx > > and doesn't continue with the next valid entry as I would have expected. > > A damaged database is not good, but as slapcat is described as the tool to > recover the non-corrupted data in various forums, it would be good when it > really continues on errors. Other ldap-related tools like ldapsearch > ignore the > damaged data, so it is really annoying, when you want to recover your data > and > don't even get all the readable entries. > > The patch corrects the handling of damaged entries by slapcat when -c is > used, > so that the output is continued with the next non-damaged entry. When -c > is not > used slapcat stops after the first corrupted entry (as expected). > > The patch works for me, but I know that it is not really correct, because > when > there would be no non-damaged entry left it would either cause a endless > loop or > some crashes. > > So a modification would be needed to stop the loop when the last database > entry > was processed, but I didn't know how to access that id. I do not quite understand this report and your fix. The code seems to work as intended. "continue" here means keep processing the database if an entry can't be extracted from the database. This does by no means imply that processing can always continue. If the database is corrupted, it is likely that from the first appearance of the corruption, not even the next entry id can be retrieved at all. Otherwise, if the only error consists in not being able to extract an entry, continuation occurs as expected in the current code. What likely happened in your case was that as soon as the database corruption was encountered, be_entry_next() was unable to compute the id of the next entry. This is why you don't even get a log of the ids of the entries that couldn't be extracted. I'd close this ITS. p.

1 0

Re: (ITS#6524) gnutls cipher spec is unclear
by Kurt＠OpenLDAP.org 17 Apr '10

17 Apr '10

On Apr 17, 2010, at 3:15 AM, matthijs(a)cacholong.nl wrote: > On Apr 16, 2010, at 6:01 PM, Howard Chu wrote: >> If Peter wants his patch considered for inclusion in OpenLDAP he =3D > should write to the ITS himself, we cannot accept 3rd party =3D > contributions. >=20 > What's the reasoning behind this? 3rd parties (middle men) not only hinder good communication between the = authors(s) of the contribution and the Project/Foundation, they = introduce various risks to both the author and the Project/Foundation. Regard, Kurt (Executive Director, The OpenLDAP Foundation)

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs