openldap-bugs

openldap-bugs@openldap.org

1 participants
20965 discussions

Re: (ITS#6165) Support offline modify (slapmodify) for cn=config
by masarati＠aero.polimi.it 18 Apr '10

18 Apr '10

> Full_Name: Quanah Gibson-Mount > Version: 2.4.16 > OS: NA > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (75.111.29.239) > > > The slapmodify function already exists, but we've never built a tool > around it. > It would be useful to be able to have a utility that validates commands > that > users can use to modify cn=config while slapd is offline, rather than > resorting > to text editors that don't do validation of the changes being made. What about this? <ftp://ftp.openldap.org/incoming/slapmodify> ;) p.

1 0

Re: (ITS#6482) slapcat doesn't continue with -c after the first corrupted database entry
by masarati＠aero.polimi.it 18 Apr '10

18 Apr '10

> The main point wasn't to tell you how important the data was, but that > tools like ldapsearch or other tools connecting to slapd using port 389 > still saw the good data. Because it was cached? Was this true after restarting the server? > And slapd never complained about a corrupted > database. As a result of that the data corruption was hidden (all other > programs worked, only three non-important entries were not accessible) > until I tried to use slapcat to export the data for an upgrade. > > Regarding "If the underlying database fails, there's little slapcat can > do.": As long as slapd sees the data, because it sends it in responses > to ldap-queries, the database hasn't failed so much, that slapcat > couldn't retrieve those data. Well, as I said in my previous message, slapcat is asking the underlying database to provide entries. If the underlying database fails, there's little slapcat can do. When the underlying database is back-bdb/hdb, slapd accesses data using indexes (dn/filter indexes mapped to IDs; the entry is then accessed using BDB in DB_SET mode), while slapcat accesses it using BDB cursors in DB_NEXT mode. Your approach of guessing IDs in the hope it works sounds a bit naive; I think in your case it worked by chance. Perhaps, a more "robust" approach would be to have back-bdb/hdb bi_tool_entry_next() to try to use itself the DB_SET approach instead of the DB_NEXT when the latter fails, with slapcat only knowing about the failure (so it can abort if -c is not passed). > Regarding "any garbage coming out of a corrupted database": slapd never > complained about the database being corrupted, it started without any > problems (except that three entries were unreadable), the entries after > those damaged entries made sense and weren't garbage. Berkeley's > db_tools claimed that there were some errors in the db, but slapd worked > without problems, only slapcat (and ldapdelete when trying to delete > those entries) complained that it couldn't decode them (which seems to > be rather a problem of the application layer than of the database layer, > but might have been caused by some little db corruption). When things go wrong, and specifically when this happens outside the code of our pertinence (in this case BDB), usually the best we can do is to rely on what any diagnostics/recovery capability they provide. Again, what worked for you may not work for another kind of corruption. Actually, going on might provide inconsistent data, something as bad as data loss. > I tried tools like "db_recover", but they didn't help, What about catastrophic recovery, using archived log files? > so I wanted to > extract the good ldap entries (using slapcat ...) to drop the database > and recreate it with the exported ldif, which failed because slapcat > stopped after the first failure (even with -c). Using the submitted > patch I was able to export all readable data into a ldif-file and to > recreate the database. > > summary: > * slapd never complained, ldapsearch listed the data, only few entries > were missing (because for them following was true: > > <= entry_decode: slap_str2undef_ad(object�..!p): > AttributeDescription > contains inappropriate characters) > > --> it seems that some attribute-values were damaged, but not the dn? > > * if ldapsearch is able to retrieve the data, slapcat should also do In general, I concur. However, slapd and tools access data in a slightly different manner, so this may not be trivial, and might require a redesign. > * I don't know how you want to recover an corrupted database. I tried to > recover all good entries and then to recreate it. And for that I wanted > to use slapcat (or is there any other tool for exporting data from the > ldap database than slapcat with option -c)? None I know about. > * maybe the way -c works shouldn't be changed, but there should be an > option for trying data recovery harder without having to patch openldap > (which is complicated for users of other distros not working directly > with the sources, but with precompiled packages) I'd take this as a feature request for redesigning the tools iteration procedure in order to try to overcome minor db corruptions. p.

1 0

Re: (ITS#6528) test020 with hdb: Query 15: ldapsearch failed (4)
by michael＠stroeder.com 18 Apr '10

18 Apr '10

Seems to work now with recent checkout. Ciao, Michael.

1 0

Re: (ITS#6225) GSSAPI support for client library (fixes)
by mimir＠samba.org 18 Apr '10

18 Apr '10

--LZvS9be/3tNcYl/X Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Howard, On Fri, Jul 24, 2009 at 09:20:46PM -0700, Howard Chu wrote: > mimir(a)samba.org wrote: > >Full_Name: Rafal Szczesniak > >Version: Stable (2.4.16) > >OS: GNU/Linux > >URL: http://www.samba.org/~mimir/gssapi-bugfix.diff > >Submission from: (NULL) (67.51.54.234) > > > > > >This is a bugfix patch for OpenLDAP's gssapi support. > > > One of these patches appears to duplicate ITS#6092. As the original > author of this code, could you also take a look at #6091 and #6093? I'm really sorry but after I looked at current code in HEAD and I thought I= liked it (the actual issue was fixed), following with this thread completely went of= f my radar. I've found it while looking through my mailbox. The last chunk of this its' patch still applies though. It's a minor thing = but fixes potential memory leak. I can open a separate ITS for it if you like. As for the other patches - #6091 is (was) convenient but not absolutely nec= essary and #6093 is interesting but I'm not sure if it isn's a bit too much. If yo= u're connecting invalid (from kerberos point of view) hostname then, well, find out what's = the correct=20 hostname first. This its # can be closed of course. cheers, --=20 Rafal Szczesniak Samba Team member http://www.samba.org Likewise Software http://www.likewise.com --LZvS9be/3tNcYl/X Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) iD8DBQFLy1e3Hvdfyv3qiKkRAk7ZAJ0VPkzc89VAxXh/AF4EZVCJTvS/fgCfeZK7 TULoTjp8aXQkK0oGZNShxsY= =zucU -----END PGP SIGNATURE----- --LZvS9be/3tNcYl/X--

1 0

Re: (ITS#6482) slapcat doesn't continue with -c after the first corrupted database entry
by sysadmin＠htl-leonding.ac.at 18 Apr '10

18 Apr '10

This is a multi-part message in MIME format. --------------020204070007000004050507 Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: 7bit The main point wasn't to tell you how important the data was, but that tools like ldapsearch or other tools connecting to slapd using port 389 still saw the good data. And slapd never complained about a corrupted database. As a result of that the data corruption was hidden (all other programs worked, only three non-important entries were not accessible) until I tried to use slapcat to export the data for an upgrade. Regarding "If the underlying database fails, there's little slapcat can do.": As long as slapd sees the data, because it sends it in responses to ldap-queries, the database hasn't failed so much, that slapcat couldn't retrieve those data. Regarding "any garbage coming out of a corrupted database": slapd never complained about the database being corrupted, it started without any problems (except that three entries were unreadable), the entries after those damaged entries made sense and weren't garbage. Berkeley's db_tools claimed that there were some errors in the db, but slapd worked without problems, only slapcat (and ldapdelete when trying to delete those entries) complained that it couldn't decode them (which seems to be rather a problem of the application layer than of the database layer, but might have been caused by some little db corruption). I tried tools like "db_recover", but they didn't help, so I wanted to extract the good ldap entries (using slapcat ...) to drop the database and recreate it with the exported ldif, which failed because slapcat stopped after the first failure (even with -c). Using the submitted patch I was able to export all readable data into a ldif-file and to recreate the database. summary: * slapd never complained, ldapsearch listed the data, only few entries were missing (because for them following was true: <= entry_decode: slap_str2undef_ad(object�..!p): AttributeDescription contains inappropriate characters) --> it seems that some attribute-values were damaged, but not the dn? * if ldapsearch is able to retrieve the data, slapcat should also do * I don't know how you want to recover an corrupted database. I tried to recover all good entries and then to recreate it. And for that I wanted to use slapcat (or is there any other tool for exporting data from the ldap database than slapcat with option -c)? * maybe the way -c works shouldn't be changed, but there should be an option for trying data recovery harder without having to patch openldap (which is complicated for users of other distros not working directly with the sources, but with precompiled packages) --------------020204070007000004050507 Content-Type: text/html; charset=ISO-8859-15 Content-Transfer-Encoding: 7bit <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html> <head> <meta http-equiv="content-type" content="text/html; charset=ISO-8859-15"> </head> <body bgcolor="#ffffff" text="#000000"> The main point wasn't to tell you how important the data was, but that tools like ldapsearch or other tools connecting to slapd using port 389 still saw the good data. And slapd never complained about a corrupted database. As a result of that the data corruption was hidden (all other programs worked, only three non-important entries were not accessible) until I tried to use slapcat to export the data for an upgrade. Regarding "If the underlying database fails, there's little slapcat can do.": As long as slapd sees the data, because it sends it in responses to ldap-queries, the database hasn't failed so much, that slapcat couldn't retrieve those data. Regarding "any garbage coming out of a corrupted database": slapd never complained about the database being corrupted, it started without any problems (except that three entries were unreadable), the entries after those damaged entries made sense and weren't garbage. Berkeley's db_tools claimed that there were some errors in the db, but slapd worked without problems, only slapcat (and ldapdelete when trying to delete those entries) complained that it couldn't decode them (which seems to be rather a problem of the application layer than of the database layer, but might have been caused by some little db corruption). I tried tools like "db_recover", but they didn't help, so I wanted to extract the good ldap entries (using slapcat ...) to drop the database and recreate it with the exported ldif, which failed because slapcat stopped after the first failure (even with -c). Using the submitted patch I was able to export all readable data into a ldif-file and to recreate the database. summary: * slapd never complained, ldapsearch listed the data, only few entries were missing (because for them following was true: <pre><= entry_decode: slap_str2undef_ad(object&#65533;..!p): AttributeDescription contains inappropriate characters) --> it seems that some attribute-values were damaged, but not the dn? </pre> * if ldapsearch is able to retrieve the data, slapcat should also do * I don't know how you want to recover an corrupted database. I tried to recover all good entries and then to recreate it. And for that I wanted to use slapcat (or is there any other tool for exporting data from the ldap database than slapcat with option -c)? * maybe the way -c works shouldn't be changed, but there should be an option for trying data recovery harder without having to patch openldap (which is complicated for users of other distros not working directly with the sources, but with precompiled packages) </body> </html> --------------020204070007000004050507--

1 0

Re: (ITS#6482) slapcat doesn't continue with -c after the first corrupted database entry
by masarati＠aero.polimi.it 18 Apr '10

18 Apr '10

> Maybe I haven't described the situation correctly: > The database was used for managing user accounts (via nss_ldap). The > database seems to had three corrupted entries in the middle (good > entries were following after that). The command "getent passwd" i.e. > still listed all users except those three with corrupted entries, but > export via slapcat and reimport of the data would have led to loss of > many more accounts. Lossing half your data is not the expected behaviour > for a tool, which should list all the existing (or at least somehow > readable) database entries. I'm not questioning the importance of the data slapcat couldn't export. > The important point is that slapcat cancelled the export at the first > corrupted database entry and didn't even try to get to the non-corrupted > database entries (which were seen by the other LDAP-tools). This is not correct. Slapcat tries. It asks the underlying backend to provide the subsequent entries. If the underlying database fails, there's little slapcat can do. > It didn't > even try in "continuation" mode, so there was no way to rescue all the > readable data via slapcat. I've tested slapcat in continuation mode by forcibly having be_entry_get() return NULL, and it works as expected: it continues to the next one, as soon as be_entry_next() is able to return a valid ID. if it can't, then anything you'd get from be_entry_next() with an arbitrary ID is unreliable. > For slapcat all entries occuring after the > first corrupt entry were lost. As the remaining good entries were still > accessible and visible via the other tools communicating with slapd via > the port 389, the expected behaviour for slapcat would have been to try > harder to ignore the damaged entries and to recover the remaining > readable entries after them. Maybe slapcat really failed to extract the > next entries id, but in "continuation"-mode it should try hard to > workaround any errors caused by non-readable entries. In case of an > non-readable entry my patch continues stupidly incrementing the id by 1 > and trying to read that entry. As soon as it can extract an entry, it > continues processing. If by "try hard" you mean that any garbage coming out of a corrupted database would be fine for you, then you can fix your patch by having it stop when id == NOID. To me, it sounds a bit too hard. What about recovering the database first, did you try? p.

1 0

Re: (ITS#6528) test020 with hdb: Query 15: ldapsearch failed (4)
by masarati＠aero.polimi.it 18 Apr '10

18 Apr '10

Thanks, I'm working at this. p. > Full_Name: Michael Ströder > Version: HEAD > OS: > URL: > Submission from: (NULL) (84.163.121.239) > > > Note: It seems to work with back-bdb > > $ ./run -b hdb test020 > Cleaning up test run directory leftover from previous run. > Running ./scripts/test020-proxycache for hdb... > Starting master slapd on TCP/IP port 9011... > Using ldapsearch to check that master slapd is running... > Using ldapadd to populate the master directory... > Starting proxy cache on TCP/IP port 9012... > Using ldapsearch to check that proxy slapd is running... > Making queries on the proxy cache... > Query 1: filter:(sn=Jon) attrs:all (expect nothing) > Query 2: filter:(|(cn=*Jon*)(sn=Jon*)) attrs:cn sn title uid > Query 3: filter:(sn=Smith*) attrs:cn sn uid > Query 4: filter:(sn=Doe*) attrs:cn sn title uid > Query 5: filter:(uid=johnd) attrs:mail postaladdress telephonenumber cn > uid > Query 6: filter:(mail=*@mail.alumni.example.com) attrs:cn sn title uid > Query 7: filter:(mail=*) attrs:cn sn title uid > Query 8: filter:(mail=*example.com) attrs:cn sn title uid > ldapsearch failed (4) > Query 9: filter:(uid=b*) attrs:mail > ldapsearch failed (4) > Query 1 not cacheable > Query 2 cacheable > Query 3 cacheable > Query 4 cacheable > Query 5 cacheable > Query 6 cacheable > Query 7 not cacheable > Query 8 cacheable > Query 9 cacheable > Successfully verified cacheability > Query 10: filter:(|(cn=*Jones)(sn=Jones)) attrs:cn sn title uid > Query 11: filter:(sn=Smith) attrs:cn sn title uid > Query 12: filter:(uid=bjorn) attrs:mail postaladdress telephonenumber cn > uid > Query 13: filter:(mail=jaj@mail.alumni.example.com) attrs:cn sn title uid > Query 14: filter:(mail=*example.com) attrs:cn sn title uid > ldapsearch failed (4) > Query 15: filter:(uid=b*) attrs:mail > ldapsearch failed (4) > Query 10 answerable > Query 11 answerable > Query 12 answerable > Query 13 not answerable > Query 14 not answerable > Query 15 answerable > Successfully verified answerability > Filtering ldapsearch results... > Filtering original ldif... > Comparing filter output... > Comparison failed > > >

1 0

Re: (ITS#6527) Build errors in function � slap_cf_aux_table_unparse�
by masarati＠aero.polimi.it 18 Apr '10

18 Apr '10

Thanks, Quanah is working at this. p. > Full_Name: Michael Ströder > Version: RE24 > OS: openSUSE 11.2 Linux > URL: > Submission from: (NULL) (84.163.121.239) > > > HEAD builds and make test works. > > RE24 fails to build since recent check-ins (anon CVS) yesterday: > > make[3]: Leaving directory > `/usr/src/michael/openldap/OPENLDAP_REL_ENG_2_4/openldap/servers/slapd/overlays' > ../../build/mkversion -v "2.4.X" -s -n Versionstr slapd > version.c > cc -g -O2 -DSLAP_SCHEMA_EXPOSE -I../../include -I. -I./slapi -I. > -I../../include > -c -o main.o main.c > cc -g -O2 -DSLAP_SCHEMA_EXPOSE -I../../include -I. -I./slapi -I. > -I../../include > -c -o globals.o globals.c > cc -g -O2 -DSLAP_SCHEMA_EXPOSE -I../../include -I. -I./slapi -I. > -I../../include > -c -o bconfig.o bconfig.c > cc -g -O2 -DSLAP_SCHEMA_EXPOSE -I../../include -I. -I./slapi -I. > -I../../include > -c -o config.o config.c > config.c: In function �slap_cf_aux_table_unparse�: > config.c:1575: error: duplicate case value > config.c:1547: error: previously used here > make[2]: *** [config.o] Error 1 > make[2]: Leaving directory > `/usr/src/michael/openldap/OPENLDAP_REL_ENG_2_4/openldap/servers/slapd' > make[1]: *** [all-common] Error 1 > make[1]: Leaving directory > `/usr/src/michael/openldap/OPENLDAP_REL_ENG_2_4/openldap/servers' > make: *** [all-common] Error 1 > > >

1 0

(ITS#6528) test020 with hdb: Query 15: ldapsearch failed (4)
by michael＠stroeder.com 18 Apr '10

18 Apr '10

Full_Name: Michael Ströder Version: HEAD OS: URL: Submission from: (NULL) (84.163.121.239) Note: It seems to work with back-bdb $ ./run -b hdb test020 Cleaning up test run directory leftover from previous run. Running ./scripts/test020-proxycache for hdb... Starting master slapd on TCP/IP port 9011... Using ldapsearch to check that master slapd is running... Using ldapadd to populate the master directory... Starting proxy cache on TCP/IP port 9012... Using ldapsearch to check that proxy slapd is running... Making queries on the proxy cache... Query 1: filter:(sn=Jon) attrs:all (expect nothing) Query 2: filter:(|(cn=*Jon*)(sn=Jon*)) attrs:cn sn title uid Query 3: filter:(sn=Smith*) attrs:cn sn uid Query 4: filter:(sn=Doe*) attrs:cn sn title uid Query 5: filter:(uid=johnd) attrs:mail postaladdress telephonenumber cn uid Query 6: filter:(mail=*@mail.alumni.example.com) attrs:cn sn title uid Query 7: filter:(mail=*) attrs:cn sn title uid Query 8: filter:(mail=*example.com) attrs:cn sn title uid ldapsearch failed (4) Query 9: filter:(uid=b*) attrs:mail ldapsearch failed (4) Query 1 not cacheable Query 2 cacheable Query 3 cacheable Query 4 cacheable Query 5 cacheable Query 6 cacheable Query 7 not cacheable Query 8 cacheable Query 9 cacheable Successfully verified cacheability Query 10: filter:(|(cn=*Jones)(sn=Jones)) attrs:cn sn title uid Query 11: filter:(sn=Smith) attrs:cn sn title uid Query 12: filter:(uid=bjorn) attrs:mail postaladdress telephonenumber cn uid Query 13: filter:(mail=jaj@mail.alumni.example.com) attrs:cn sn title uid Query 14: filter:(mail=*example.com) attrs:cn sn title uid ldapsearch failed (4) Query 15: filter:(uid=b*) attrs:mail ldapsearch failed (4) Query 10 answerable Query 11 answerable Query 12 answerable Query 13 not answerable Query 14 not answerable Query 15 answerable Successfully verified answerability Filtering ldapsearch results... Filtering original ldif... Comparing filter output... Comparison failed

1 0

Re: (ITS#6482) slapcat doesn't continue with -c after the first corrupted database entry
by sysadmin＠htl-leonding.ac.at 18 Apr '10

18 Apr '10

Maybe I haven't described the situation correctly: The database was used for managing user accounts (via nss_ldap). The database seems to had three corrupted entries in the middle (good entries were following after that). The command "getent passwd" i.e. still listed all users except those three with corrupted entries, but export via slapcat and reimport of the data would have led to loss of many more accounts. Lossing half your data is not the expected behaviour for a tool, which should list all the existing (or at least somehow readable) database entries. The important point is that slapcat cancelled the export at the first corrupted database entry and didn't even try to get to the non-corrupted database entries (which were seen by the other LDAP-tools). It didn't even try in "continuation" mode, so there was no way to rescue all the readable data via slapcat. For slapcat all entries occuring after the first corrupt entry were lost. As the remaining good entries were still accessible and visible via the other tools communicating with slapd via the port 389, the expected behaviour for slapcat would have been to try harder to ignore the damaged entries and to recover the remaining readable entries after them. Maybe slapcat really failed to extract the next entries id, but in "continuation"-mode it should try hard to workaround any errors caused by non-readable entries. In case of an non-readable entry my patch continues stupidly incrementing the id by 1 and trying to read that entry. As soon as it can extract an entry, it continues processing.

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs