Hi Guys,
Hope this is the right list for this, haven't been lurking here previously so I don't have a feel for things yet.
I'm upgrading our OpenLDAP servers to use directory based configuration under Ubuntu/Lucid and am having some problems including the provided Cosine and iNetOrgPerson schemas.
It appears that if I symlink the LDIF files from /etc/ldap/schema/ into /etc/ldap/slapd.d/cn=config/cn=schema/ slapd will not start.
Running slapd in debug mode gives me the following output:
ldif_read_file: read entry file: "/etc/ldap/slapd.d//cn=config/cn=schema/cosine.ldif" => str2entry: "# RFC1274: Cosine and Internet X.500 schema <snip contents of the LDIF file being read in> => ldap_bv2dn(cn=cosine,cn=schema,cn=config,0) <= ldap_bv2dn(cn=cosine,cn=schema,cn=config)=0 => ldap_dn2bv(272) <= ldap_dn2bv(cn=cosine,cn=schema,cn=config)=0 => ldap_dn2bv(272) <= ldap_dn2bv(cn=cosine,cn=schema,cn=config)=0 <<< dnPrettyNormal: <cn=cosine,cn=schema,cn=config>, <cn=cosine,cn=schema,cn=config> <= str2entry(cn=cosine,cn=schema,cn=config) -> 0xb9124344 => test_filter PRESENT => access_allowed: search access to "cn=cosine,cn=schema,cn=config,cn=schema,cn=config" "objectClass" requested <= root access granted => access_allowed: search access granted by manage(=mwrscxd) <= test_filter 6 : config_add_internal: DN="cn=cosine,cn=schema,cn=config,cn=schema,cn=config" not child of DN="cn=schema,cn=config" config error processing cn=cosine,cn=schema,cn=config,cn=schema,cn=config: send_ldap_result: conn=-1 op=0 p=0 send_ldap_result: err=32 matched="" text=""
The DN specified in the LDIF file is as follows:
dn: cn=cosine,cn=schema,cn=config
But it looks like when it's reading in the file, it's postpending cn=schema,cn=config (presumably from the configuration directory path), as opposed to using the fully qualified DN.
Is there a way to fix this? I'm using packages to deploy software configurations, and it doesn't make sense for us to inject this schema with ldapadd (what seems to be the prescribed way) of adding schema to get around this DN problem (and handing adding/removing it when upstream updates the defintions - however infrequent/unlikely this is).
We also roll out our own schema definitions, but these have been converted to LDIF and it's no big deal to have the DN: line set to whatever will make slapd happy.
I hope this makes sense and that someone is able to help me understand directory based configuration a little better.
Cheers :)
Will Dowling
T: +61 (08) 6364 4880 F: +61 (08) 6364 4881 E: will@autodeist.com
--On Wednesday, September 01, 2010 1:46 PM +0800 Will Dowling will+lists_openldap@autodeist.com wrote:
I hope this makes sense and that someone is able to help me understand directory based configuration a little better.
You can't just symlink them. You have to copy them over, and then edit the dn.
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration
Quanah Gibson-Mount wrote:
--On Wednesday, September 01, 2010 1:46 PM +0800 Will Dowling will+lists_openldap@autodeist.com wrote:
I hope this makes sense and that someone is able to help me understand directory based configuration a little better.
You can't just symlink them. You have to copy them over, and then edit the dn.
No, you are never supposed to muck with any of the files inside slapd.d. You slapadd the LDIF files, same way you would load any other LDIF file into slapd.
No, you are never supposed to muck with any of the files inside slapd.d. You slapadd the LDIF files, same way you would load any other LDIF file into slapd.
Wow, okay. The online documentation doesn't make that clear at all (especially when it talks about converting your old config).
http://www.openldap.org/doc/admin24/slapdconf2.html
I'm not about to start picking fights with the Chief Architect though.
Keeping that in mind - are you advocating this from a design point of view (it won't work properly), or a precautionary one (you shouldn't unless you know what you're doing)?
If it's the former (it won't work properly), can you make any recommendations for best-practice in terms of maintaining changes to third-party packaged configurations?
For example, if we roll out updated schmea, would it be best to drop and re-add the schema - or diff the structure and create an update LDIF?
Seems a bit clunky if thats the case, but I have had a few settings not stick already (olcDatabaseDirectory).
Anyway, would love your insight and thanks for your time :)
Will Dowling
T: +61 (08) 6364 4880 F: +61 (08) 6364 4881 E: will@autodeist.com
Will Dowling wrote:
No, you are never supposed to muck with any of the files inside slapd.d. You slapadd the LDIF files, same way you would load any other LDIF file into slapd.
Wow, okay. The online documentation doesn't make that clear at all (especially when it talks about converting your old config).
http://www.openldap.org/doc/admin24/slapdconf2.html
I'm not about to start picking fights with the Chief Architect though.
Keeping that in mind - are you advocating this from a design point of view (it won't work properly), or a precautionary one (you shouldn't unless you know what you're doing)?
If you know what you're doing, you can binary edit a BDB database file if you really want to. But most people don't want to, and certainly most people won't know what they're doing.
cn=config is a slapd database and should be treated as such. The contents are not vanilla LDIF files, and database internals are always subject to change. It was designed to be used like other LDAP databases - using ldap* tools when slapd is running, and using slap* tools when slapd is offline.
If it's the former (it won't work properly), can you make any recommendations for best-practice in terms of maintaining changes to third-party packaged configurations?
For example, if we roll out updated schmea, would it be best to drop and re-add the schema - or diff the structure and create an update LDIF?
Applying a diff via ldapmodify would be Best; that was the intended use case.
Seems a bit clunky if thats the case, but I have had a few settings not stick already (olcDatabaseDirectory).
Anyway, would love your insight and thanks for your time :)
Howard Chu wrote:
cn=config is a slapd database and should be treated as such. The contents are not vanilla LDIF files, and database internals are always subject to change. It was designed to be used like other LDAP databases
- using ldap* tools when slapd is running, and using slap* tools when
slapd is offline.
While this is true for normal conditions couldn't there be situations where cn=config was tweaked in such a way that the server won't come up anymore? In this case tweaking the LDIF directly is an appropriate solution.
Ciao, Michael.
On 2010.09.02 11.30, Michael Ströder wrote:
Howard Chu wrote:
cn=config is a slapd database and should be treated as such. The contents are not vanilla LDIF files, and database internals are always subject to change. It was designed to be used like other LDAP databases
- using ldap* tools when slapd is running, and using slap* tools when
slapd is offline.
While this is true for normal conditions couldn't there be situations where cn=config was tweaked in such a way that the server won't come up anymore? In this case tweaking the LDIF directly is an appropriate solution.
"and using slap* tools when slapd is offline."..., no?
btb@bitrate.net wrote:
On 2010.09.02 11.30, Michael Ströder wrote:
Howard Chu wrote:
cn=config is a slapd database and should be treated as such. The contents are not vanilla LDIF files, and database internals are always subject to change. It was designed to be used like other LDAP databases
- using ldap* tools when slapd is running, and using slap* tools when
slapd is offline.
While this is true for normal conditions couldn't there be situations where cn=config was tweaked in such a way that the server won't come up anymore? In this case tweaking the LDIF directly is an appropriate solution.
"and using slap* tools when slapd is offline."..., no?
Note that slap* actually use the config, here cn=config (see command-line option -F)...
Actually those situation are likely very rare if one is always tweaking cn=config via LDAP.
Ciao, Michael.
No, you are never supposed to muck with any of the files inside slapd.d. You slapadd the LDIF files, same way you would load any other LDIF file into slapd.
Hi Howard,
I'm still having problems understanding the basis for this design.
Can you please elaborate on why the files in slapd.d are not considered editable? What is stopping OpenLDAP supporting processing user-edited files in this location?
Assuming users aren't editing the configuration "database", what is the point of making it a dot-d style directory as opposed to a seperate backend database?
This seems at-odds with the concept of a dot-d style configuration directory: a reasonable person having experience with other software packages would assume that the contents of the slapd.d directories are editable and read into memory when slapd is started.
I'm all for the benefits that are introduced by making the configuration reflectable and allowing for runtime configuration - don't get me wrong.
But I think there is room for valuable other benefits to be realised also, and I'm trying to figure out whether or not there is opportunity for my organisation to contribute resources to help realise this.
Thank you and I look forward to your reply :)
Will Dowling
T: +61 (08) 6364 4880 F: +61 (08) 6364 4881 E: will@autodeist.com
Will Dowling wrote:
No, you are never supposed to muck with any of the files inside slapd.d. You slapadd the LDIF files, same way you would load any other LDIF file into slapd.
Hi Howard,
I'm still having problems understanding the basis for this design.
Can you please elaborate on why the files in slapd.d are not considered editable? What is stopping OpenLDAP supporting processing user-edited files in this location?
These files are database entries. Each entry contains operational attributes such as modifyTimestamp and such. If you edit by hand without updating the operational attributes correctly, Things Can Break.
Also, these are not RFC-compliant LDIF files. They are a slapd-specific format that is 98% LDIF-compliant. We will never document what the differences are from the standard format, because internal file formats are *always* open to change without notice.
Assuming users aren't editing the configuration "database", what is the point of making it a dot-d style directory as opposed to a seperate backend database?
It is of course possible for a careful, clueful admin to edit the files by hand without breaking anything. Just like it is possible for a careful, clueful admin to binary edit the *.bdb files in a back-bdb or back-hdb database. But let's face it, the majority of people out there, and particularly the people having problems that drive them to post on this mailing list, are neither careful enough nor clueful enough to qualify for these activities.
This seems at-odds with the concept of a dot-d style configuration directory: a reasonable person having experience with other software packages would assume that the contents of the slapd.d directories are editable and read into memory when slapd is started.
A reasonable person should understand (since the documentation states this clearly, multiple times) that this is an LDAP database and is meant to be manipulated using LDAP operations. Period, end of story.
Once you get used to manually editing the raw files, you run into the very real danger of editing while slapd is running, and then confusing the hell out of yourself when things behave in ways other than you expected. The only way to avoid creating such messes is to simply be disciplined enough in the first place to only manipulate things the way they were designed to be done.
If you go back into the openldap-devel discussions from 2003 or so when we first started working on cn=config you may see that there's of course a desire to keep slapd.d human-readable, to make things less painful in case of disaster recovery. So yes, while cn=config is an LDAP database and we could at any point in the future migrate it to a binary file format, it's not really likely to happen. But, in the interests of keeping our options open, we continue to drive the point - cn=config is an LDAP database. The underlying storage format is open to change at any time without warning. You should *never* rely on the internal storage format of any of these databases. You should *only* use the standard tools to manipulate them, that's why those tools exist in the first place.
I'm all for the benefits that are introduced by making the configuration reflectable and allowing for runtime configuration - don't get me wrong.
But I think there is room for valuable other benefits to be realised also, and I'm trying to figure out whether or not there is opportunity for my organisation to contribute resources to help realise this.
openldap-technical@openldap.org
-
btb＠bitrate.net -
Howard Chu -
Michael Ströder -
Quanah Gibson-Mount -
Will Dowling