I'd like a slap tool which verifies an LDIF before I try to ldapadd/slapadd it. "slapadd -u -o value-check=yes" is fairly close. What does it fail to catch? I can think of:
- Duplicate entries. - Missing entries (if the initial DB is expected to be empty). - Child entries before parents (OK for slapadd to at least back-<bdb,hdb,mdb>).
- Issues which the tool can only catch if it opens the database, like attempts to add already-existing entries. I probably don't want to do that.
- Issues which overlays like slapo-unique would reject. Can't do that, since the overlay won't have a non-empty DB to check against and slap tools do not use overlays anyway. Might special-case "unique" though, since the "duplicate entries" check will need uniqueness code anyway.
Hallvard Breien Furuseth wrote:
I'd like a slap tool which verifies an LDIF before I try to ldapadd/slapadd it. "slapadd -u -o value-check=yes" is fairly close. What does it fail to catch? I can think of:
- Duplicate entries.
Quite an unrealistic requirement. You need to store the set of entryDNs to achieve this, and for a large LDIF you may need an actual database to manage this. Might as well just do a normal slapadd.
Missing entries (if the initial DB is expected to be empty).
Child entries before parents (OK for slapadd to at least back-<bdb,hdb,mdb>).
Issues which the tool can only catch if it opens the database, like attempts to add already-existing entries. I probably don't want to do that.
Issues which overlays like slapo-unique would reject. Can't do that, since the overlay won't have a non-empty DB to check against and slap tools do not use overlays anyway. Might special-case "unique" though, since the "duplicate entries" check will need uniqueness code anyway.
These requirements are definitely too loosely described to be implemented. But maybe prerequisites could be added to the slapadd operation, mimicking dynamic dns (rfc2136)? That way all prerequisites would be sent by the client, and the server would "just" have too check them before applying the changes. I know it looks a like as a ldapsearch (to check these prerequisites) followed by a ldapadd, but if implemented server side it could be done in an atomic way?
On Tue, Mar 24, 2015 at 11:56 PM, Howard Chu hyc@symas.com wrote:
Hallvard Breien Furuseth wrote:
I'd like a slap tool which verifies an LDIF before I try to ldapadd/slapadd it. "slapadd -u -o value-check=yes" is fairly close. What does it fail to catch? I can think of:
- Duplicate entries.
Quite an unrealistic requirement. You need to store the set of entryDNs to achieve this, and for a large LDIF you may need an actual database to manage this. Might as well just do a normal slapadd.
- Missing entries (if the initial DB is expected to be empty).
- Child entries before parents (OK for slapadd to at least
back-<bdb,hdb,mdb>).
- Issues which the tool can only catch if it opens the database, like
attempts to add already-existing entries. I probably don't want to do that.
- Issues which overlays like slapo-unique would reject. Can't do that, since the overlay won't have a non-empty DB to check against and slap tools do not use overlays anyway. Might special-case "unique" though, since the "duplicate entries" check will need uniqueness code anyway.
-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Le 24/03/15 23:56, Howard Chu a écrit :
Hallvard Breien Furuseth wrote:
I'd like a slap tool which verifies an LDIF before I try to ldapadd/slapadd it. "slapadd -u -o value-check=yes" is fairly close. What does it fail to catch? I can think of:
- Duplicate entries.
Quite an unrealistic requirement. You need to store the set of entryDNs to achieve this, and for a large LDIF you may need an actual database to manage this. Might as well just do a normal slapadd.
Let me disagree. A simple merge sort will do the trick. No need of a database to deal with that, even for a large LDIF. We have done some tests with the tool we use to handle bulkloads at ApacheDS, and that works quite fine, and quite fast enough.
On 25. mars 2015 00:56, Emmanuel Lécharny wrote:
Le 24/03/15 23:56, Howard Chu a écrit :
Hallvard Breien Furuseth wrote:
I'd like a slap tool which verifies an LDIF before I try to ldapadd/slapadd it. "slapadd -u -o value-check=yes" is fairly close. What does it fail to catch? I can think of:
Another requirement:
- An attribute should not occur several times in an entry with other attributes in between. E.g. "cn", "title", then "cn". That gives broken entries at least with slapadd -q.
- Duplicate entries.
Quite an unrealistic requirement. You need to store the set of entryDNs to achieve this, and for a large LDIF you may need an actual database to manage this. Might as well just do a normal slapadd.
No, I want it to be faster than that. Also I might do someting else with the verified LDIF anyway. Generate a diff against the current DB and feed it to ldapmodify.
DN checking would need to be an option, which could just document "you asked for unbounded memory usage, you got it". Though I suppose it could slapadd to a DB with just the DNs, not any attributes.
Let me disagree. A simple merge sort will do the trick. No need of a database to deal with that, even for a large LDIF. We have done some tests with the tool we use to handle bulkloads at ApacheDS, and that works quite fine, and quite fast enough.
Nice idea.
A reliable sorter would still need to link slapd or something else knowing the schema though, so it'll normalize the DNs correctly. Not quite sure if I'll bother to worry about that. Don't need it with LDIFs we generate ourselves, but LDIFs from elsewhere are less fun.
-
Emmanuel Lécharny -
Hallvard Breien Furuseth -
Howard Chu -
Mathieu