openldap-bugs

openldap-bugs@openldap.org

1 participants
20960 discussions

Re: (ITS#6482) slapcat doesn't continue with -c after the first corrupted database entry
by sysadmin＠htl-leonding.ac.at 19 Apr '10

19 Apr '10

I had restarted the server many times and it still worked, so I don't think the data was cached. Maybe the different method of access caused the difference, because the non-corrupted entries were still accessible via their id, so direct access worked. Could it be that slapcat doesn't correctly handle the case of invalid entries (entries with decoding errors)? Before this generic patch, I checked if the current entry had the id of the first invalid entry and in this case I incremented the ID by three, got the resulting entry and continued the export --> slapcat "skipped" the entries which it couldn't decode and then continued processing normally. Sorry that I can't test catastrophic recovery any more, because I have deleted the corrupted database, but I think that the main problem was that slapcat (and also other ldap-tools) wasn't able to decode the few damaged entries and didn't try to work around that problem.

1 0

ITS#6055
by masarati＠aero.polimi.it 19 Apr '10

19 Apr '10

Refactored rdnval.c available at <ftp://ftp.openldap.org/incoming/pierangelo-masarati-2010-04-18-rdnval.c> attempts to repair the database at startup; useful when the overlay is configured for an existing database. Please test. p.

1 0

Re: ITS#6017
by masarati＠aero.polimi.it 19 Apr '10

19 Apr '10

> > --=-Ga69RST1xu0P4I4ibWcD > Content-Type: text/plain; charset="UTF-8" > Content-Transfer-Encoding: quoted-printable > > On Sun, 2010-04-18 at 22:59 +0200, masarati(a)aero.polimi.it wrote: >> Still needed? Would the entryUUID of the parent stored in a >> "parentUUID" >> attribute fit? > > While we may change our minds again, Samba4 currently does not require > parentUUID to be stored. We now calculate it dynamically at runtime.=20 OK, thanks. In any case, while refactoring the rdnval.c (ITS#6055), I also implemented this <ftp://ftp.openldap.org/incoming/pierangelo-masarati-2010-04-18-pguid.c>. It keeps the parentUUID in sync, and repairs the database at startup (useful when the overlay is added on existing databases). p.

1 0

Re: ITS#6017
by abartlet＠samba.org 19 Apr '10

19 Apr '10

--=-Ga69RST1xu0P4I4ibWcD Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sun, 2010-04-18 at 22:59 +0200, masarati(a)aero.polimi.it wrote: > Still needed? Would the entryUUID of the parent stored in a "parentUUID" > attribute fit? While we may change our minds again, Samba4 currently does not require parentUUID to be stored. We now calculate it dynamically at runtime.=20 Andrew Bartlett --=20 Andrew Bartlett http://samba.org/~abartlet/ Authentication Developer, Samba Team http://samba.org Samba Developer, Cisco Inc. --=-Ga69RST1xu0P4I4ibWcD Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) iD8DBQBLy49wz4A8Wyi0NrsRAgxwAJ96NWo95lkdiVV2WSpz9F/EdKaDfwCeO6Y3 +zF9u9EWpDvziGGAsOAyB8Q= =a/rb -----END PGP SIGNATURE----- --=-Ga69RST1xu0P4I4ibWcD--

1 0

ITS#6017
by masarati＠aero.polimi.it 18 Apr '10

18 Apr '10

Still needed? Would the entryUUID of the parent stored in a "parentUUID" attribute fit? p.

1 0

Re: (ITS#6165) Support offline modify (slapmodify) for cn=config
by masarati＠aero.polimi.it 18 Apr '10

18 Apr '10

> Full_Name: Quanah Gibson-Mount > Version: 2.4.16 > OS: NA > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (75.111.29.239) > > > The slapmodify function already exists, but we've never built a tool > around it. > It would be useful to be able to have a utility that validates commands > that > users can use to modify cn=config while slapd is offline, rather than > resorting > to text editors that don't do validation of the changes being made. What about this? <ftp://ftp.openldap.org/incoming/slapmodify> ;) p.

1 0

Re: (ITS#6482) slapcat doesn't continue with -c after the first corrupted database entry
by masarati＠aero.polimi.it 18 Apr '10

18 Apr '10

> The main point wasn't to tell you how important the data was, but that > tools like ldapsearch or other tools connecting to slapd using port 389 > still saw the good data. Because it was cached? Was this true after restarting the server? > And slapd never complained about a corrupted > database. As a result of that the data corruption was hidden (all other > programs worked, only three non-important entries were not accessible) > until I tried to use slapcat to export the data for an upgrade. > > Regarding "If the underlying database fails, there's little slapcat can > do.": As long as slapd sees the data, because it sends it in responses > to ldap-queries, the database hasn't failed so much, that slapcat > couldn't retrieve those data. Well, as I said in my previous message, slapcat is asking the underlying database to provide entries. If the underlying database fails, there's little slapcat can do. When the underlying database is back-bdb/hdb, slapd accesses data using indexes (dn/filter indexes mapped to IDs; the entry is then accessed using BDB in DB_SET mode), while slapcat accesses it using BDB cursors in DB_NEXT mode. Your approach of guessing IDs in the hope it works sounds a bit naive; I think in your case it worked by chance. Perhaps, a more "robust" approach would be to have back-bdb/hdb bi_tool_entry_next() to try to use itself the DB_SET approach instead of the DB_NEXT when the latter fails, with slapcat only knowing about the failure (so it can abort if -c is not passed). > Regarding "any garbage coming out of a corrupted database": slapd never > complained about the database being corrupted, it started without any > problems (except that three entries were unreadable), the entries after > those damaged entries made sense and weren't garbage. Berkeley's > db_tools claimed that there were some errors in the db, but slapd worked > without problems, only slapcat (and ldapdelete when trying to delete > those entries) complained that it couldn't decode them (which seems to > be rather a problem of the application layer than of the database layer, > but might have been caused by some little db corruption). When things go wrong, and specifically when this happens outside the code of our pertinence (in this case BDB), usually the best we can do is to rely on what any diagnostics/recovery capability they provide. Again, what worked for you may not work for another kind of corruption. Actually, going on might provide inconsistent data, something as bad as data loss. > I tried tools like "db_recover", but they didn't help, What about catastrophic recovery, using archived log files? > so I wanted to > extract the good ldap entries (using slapcat ...) to drop the database > and recreate it with the exported ldif, which failed because slapcat > stopped after the first failure (even with -c). Using the submitted > patch I was able to export all readable data into a ldif-file and to > recreate the database. > > summary: > * slapd never complained, ldapsearch listed the data, only few entries > were missing (because for them following was true: > > <= entry_decode: slap_str2undef_ad(object�..!p): > AttributeDescription > contains inappropriate characters) > > --> it seems that some attribute-values were damaged, but not the dn? > > * if ldapsearch is able to retrieve the data, slapcat should also do In general, I concur. However, slapd and tools access data in a slightly different manner, so this may not be trivial, and might require a redesign. > * I don't know how you want to recover an corrupted database. I tried to > recover all good entries and then to recreate it. And for that I wanted > to use slapcat (or is there any other tool for exporting data from the > ldap database than slapcat with option -c)? None I know about. > * maybe the way -c works shouldn't be changed, but there should be an > option for trying data recovery harder without having to patch openldap > (which is complicated for users of other distros not working directly > with the sources, but with precompiled packages) I'd take this as a feature request for redesigning the tools iteration procedure in order to try to overcome minor db corruptions. p.

1 0

Re: (ITS#6528) test020 with hdb: Query 15: ldapsearch failed (4)
by michael＠stroeder.com 18 Apr '10

18 Apr '10

Seems to work now with recent checkout. Ciao, Michael.

1 0

Re: (ITS#6225) GSSAPI support for client library (fixes)
by mimir＠samba.org 18 Apr '10

18 Apr '10

--LZvS9be/3tNcYl/X Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Howard, On Fri, Jul 24, 2009 at 09:20:46PM -0700, Howard Chu wrote: > mimir(a)samba.org wrote: > >Full_Name: Rafal Szczesniak > >Version: Stable (2.4.16) > >OS: GNU/Linux > >URL: http://www.samba.org/~mimir/gssapi-bugfix.diff > >Submission from: (NULL) (67.51.54.234) > > > > > >This is a bugfix patch for OpenLDAP's gssapi support. > > > One of these patches appears to duplicate ITS#6092. As the original > author of this code, could you also take a look at #6091 and #6093? I'm really sorry but after I looked at current code in HEAD and I thought I= liked it (the actual issue was fixed), following with this thread completely went of= f my radar. I've found it while looking through my mailbox. The last chunk of this its' patch still applies though. It's a minor thing = but fixes potential memory leak. I can open a separate ITS for it if you like. As for the other patches - #6091 is (was) convenient but not absolutely nec= essary and #6093 is interesting but I'm not sure if it isn's a bit too much. If yo= u're connecting invalid (from kerberos point of view) hostname then, well, find out what's = the correct=20 hostname first. This its # can be closed of course. cheers, --=20 Rafal Szczesniak Samba Team member http://www.samba.org Likewise Software http://www.likewise.com --LZvS9be/3tNcYl/X Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) iD8DBQFLy1e3Hvdfyv3qiKkRAk7ZAJ0VPkzc89VAxXh/AF4EZVCJTvS/fgCfeZK7 TULoTjp8aXQkK0oGZNShxsY= =zucU -----END PGP SIGNATURE----- --LZvS9be/3tNcYl/X--

1 0

Re: (ITS#6482) slapcat doesn't continue with -c after the first corrupted database entry
by sysadmin＠htl-leonding.ac.at 18 Apr '10

18 Apr '10

This is a multi-part message in MIME format. --------------020204070007000004050507 Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: 7bit The main point wasn't to tell you how important the data was, but that tools like ldapsearch or other tools connecting to slapd using port 389 still saw the good data. And slapd never complained about a corrupted database. As a result of that the data corruption was hidden (all other programs worked, only three non-important entries were not accessible) until I tried to use slapcat to export the data for an upgrade. Regarding "If the underlying database fails, there's little slapcat can do.": As long as slapd sees the data, because it sends it in responses to ldap-queries, the database hasn't failed so much, that slapcat couldn't retrieve those data. Regarding "any garbage coming out of a corrupted database": slapd never complained about the database being corrupted, it started without any problems (except that three entries were unreadable), the entries after those damaged entries made sense and weren't garbage. Berkeley's db_tools claimed that there were some errors in the db, but slapd worked without problems, only slapcat (and ldapdelete when trying to delete those entries) complained that it couldn't decode them (which seems to be rather a problem of the application layer than of the database layer, but might have been caused by some little db corruption). I tried tools like "db_recover", but they didn't help, so I wanted to extract the good ldap entries (using slapcat ...) to drop the database and recreate it with the exported ldif, which failed because slapcat stopped after the first failure (even with -c). Using the submitted patch I was able to export all readable data into a ldif-file and to recreate the database. summary: * slapd never complained, ldapsearch listed the data, only few entries were missing (because for them following was true: <= entry_decode: slap_str2undef_ad(object�..!p): AttributeDescription contains inappropriate characters) --> it seems that some attribute-values were damaged, but not the dn? * if ldapsearch is able to retrieve the data, slapcat should also do * I don't know how you want to recover an corrupted database. I tried to recover all good entries and then to recreate it. And for that I wanted to use slapcat (or is there any other tool for exporting data from the ldap database than slapcat with option -c)? * maybe the way -c works shouldn't be changed, but there should be an option for trying data recovery harder without having to patch openldap (which is complicated for users of other distros not working directly with the sources, but with precompiled packages) --------------020204070007000004050507 Content-Type: text/html; charset=ISO-8859-15 Content-Transfer-Encoding: 7bit <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html> <head> <meta http-equiv="content-type" content="text/html; charset=ISO-8859-15"> </head> <body bgcolor="#ffffff" text="#000000"> The main point wasn't to tell you how important the data was, but that tools like ldapsearch or other tools connecting to slapd using port 389 still saw the good data. And slapd never complained about a corrupted database. As a result of that the data corruption was hidden (all other programs worked, only three non-important entries were not accessible) until I tried to use slapcat to export the data for an upgrade. Regarding "If the underlying database fails, there's little slapcat can do.": As long as slapd sees the data, because it sends it in responses to ldap-queries, the database hasn't failed so much, that slapcat couldn't retrieve those data. Regarding "any garbage coming out of a corrupted database": slapd never complained about the database being corrupted, it started without any problems (except that three entries were unreadable), the entries after those damaged entries made sense and weren't garbage. Berkeley's db_tools claimed that there were some errors in the db, but slapd worked without problems, only slapcat (and ldapdelete when trying to delete those entries) complained that it couldn't decode them (which seems to be rather a problem of the application layer than of the database layer, but might have been caused by some little db corruption). I tried tools like "db_recover", but they didn't help, so I wanted to extract the good ldap entries (using slapcat ...) to drop the database and recreate it with the exported ldif, which failed because slapcat stopped after the first failure (even with -c). Using the submitted patch I was able to export all readable data into a ldif-file and to recreate the database. summary: * slapd never complained, ldapsearch listed the data, only few entries were missing (because for them following was true: <pre><= entry_decode: slap_str2undef_ad(object&#65533;..!p): AttributeDescription contains inappropriate characters) --> it seems that some attribute-values were damaged, but not the dn? </pre> * if ldapsearch is able to retrieve the data, slapcat should also do * I don't know how you want to recover an corrupted database. I tried to recover all good entries and then to recreate it. And for that I wanted to use slapcat (or is there any other tool for exporting data from the ldap database than slapcat with option -c)? * maybe the way -c works shouldn't be changed, but there should be an option for trying data recovery harder without having to patch openldap (which is complicated for users of other distros not working directly with the sources, but with precompiled packages) </body> </html> --------------020204070007000004050507--

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs