Hi all,
Having started to look at the changes required to migrate from a slapd.conf setup to a cn=config setup, one of things I'm struggling with is how to load new LDAP schemas into cn=config.
I've seen the guides similar to this one here: http://blogger.ziesemer.com/2011/01/ldap-authentication-for-samba.html which suggest hacking together a temporary slapd.conf file containing just the include directives, run slaptest, and then hack the output so that it can be loaded into cn=config using ldapadd.
Given that this is a quite a common task, is there no way of generating the LDIF directly to be loaded into the directory, e.g.
slaptest -s /etc/ldap/schema/myschema.schema [ -n <schemanum> ] -l myschema.ldif
Or then again, is this functionality already there but I just haven't managed to find it yet? I'd be grateful if someone could point me in the right direction and/or give me some hints as to the best way to manage schemas in the new regime.
Many thanks,
Mark.
Mark Cave-Ayland wrote:
Hi all,
Having started to look at the changes required to migrate from a slapd.conf setup to a cn=config setup, one of things I'm struggling with is how to load new LDAP schemas into cn=config.
I've seen the guides similar to this one here: http://blogger.ziesemer.com/2011/01/ldap-authentication-for-samba.html which suggest hacking together a temporary slapd.conf file containing just the include directives, run slaptest, and then hack the output so that it can be loaded into cn=config using ldapadd.
His step 1 and 2 were fine. Everything after that is garbage.
1: schemaConvert.conf #### include /etc/ldap/schema/core.schema include /etc/ldap/schema/cosine.schema include /etc/ldap/schema/nis.schema include /etc/ldap/schema/inetorgperson.schema include samba.schema ####
2: slaptest mkdir config && slaptest -f schemaConvert.conf -F config
3: slapcat slapcat -F config -n0 -s cn=schema,cn=config
and all of your converted schema will pop out, ready to be slapadd'd or ldapadd'd anywhere else.
Given that this is a quite a common task, is there no way of generating the LDIF directly to be loaded into the directory, e.g.
slaptest -s /etc/ldap/schema/myschema.schema [ -n<schemanum> ] -l myschema.ldif
Or then again, is this functionality already there but I just haven't managed to find it yet? I'd be grateful if someone could point me in the right direction and/or give me some hints as to the best way to manage schemas in the new regime.
On 29/06/11 11:59, Howard Chu wrote:
Having started to look at the changes required to migrate from a slapd.conf setup to a cn=config setup, one of things I'm struggling with is how to load new LDAP schemas into cn=config.
I've seen the guides similar to this one here: http://blogger.ziesemer.com/2011/01/ldap-authentication-for-samba.html which suggest hacking together a temporary slapd.conf file containing just the include directives, run slaptest, and then hack the output so that it can be loaded into cn=config using ldapadd.
His step 1 and 2 were fine. Everything after that is garbage.
1: schemaConvert.conf #### include /etc/ldap/schema/core.schema include /etc/ldap/schema/cosine.schema include /etc/ldap/schema/nis.schema include /etc/ldap/schema/inetorgperson.schema include samba.schema ####
2: slaptest mkdir config && slaptest -f schemaConvert.conf -F config
3: slapcat
slapcat -F config -n0 -s cn=schema,cn=config
and all of your converted schema will pop out, ready to be slapadd'd or ldapadd'd anywhere else.
Hi Howard,
Thanks for the response - this makes a bit more sense now. Just to clarify another point: when you generate schemaConvert.conf, I guess that you need to include *all* schemas in your current cn=config matching the existing order, as well as the new one you are trying to add?
Also that begs another question: what happens if you want to modify an existing schema, e.g. if I need to hack a schema by hand and reload it into openldap so that it takes effect? Normally I would change the schema file on disk, restart slapd and it would just work.
With cn=config, I'm guessing I can't just delete the relevant schema entry from LDAP and re-add it since there would be a small window whereby the schema would be missing/invisible to slapd. This could cause all sorts of issues, such as pushing the incremental deletion out to any slaves, as well as breaking any concurrent lookups... :(
ATB,
Mark.
Mark Cave-Ayland wrote:
On 29/06/11 11:59, Howard Chu wrote:
Having started to look at the changes required to migrate from a slapd.conf setup to a cn=config setup, one of things I'm struggling with is how to load new LDAP schemas into cn=config.
I've seen the guides similar to this one here: http://blogger.ziesemer.com/2011/01/ldap-authentication-for-samba.html which suggest hacking together a temporary slapd.conf file containing just the include directives, run slaptest, and then hack the output so that it can be loaded into cn=config using ldapadd.
His step 1 and 2 were fine. Everything after that is garbage.
1: schemaConvert.conf #### include /etc/ldap/schema/core.schema include /etc/ldap/schema/cosine.schema include /etc/ldap/schema/nis.schema include /etc/ldap/schema/inetorgperson.schema include samba.schema ####
2: slaptest mkdir config&& slaptest -f schemaConvert.conf -F config
3: slapcat
slapcat -F config -n0 -s cn=schema,cn=config
and all of your converted schema will pop out, ready to be slapadd'd or ldapadd'd anywhere else.
Hi Howard,
Thanks for the response - this makes a bit more sense now. Just to clarify another point: when you generate schemaConvert.conf, I guess that you need to include *all* schemas in your current cn=config matching the existing order, as well as the new one you are trying to add?
Obviously the config file has to be valid; any schema that the ones you're converting depend on must be loaded.
Also that begs another question: what happens if you want to modify an existing schema, e.g. if I need to hack a schema by hand and reload it into openldap so that it takes effect? Normally I would change the schema file on disk, restart slapd and it would just work.
I frankly can't believe that you just asked that question. cn=config is an LDAP database. When you want to change its contents, you use ldapmodify. It takes effect immediately and there is no need to restart the server.
On 29/06/11 12:59, Howard Chu wrote:
Thanks for the response - this makes a bit more sense now. Just to clarify another point: when you generate schemaConvert.conf, I guess that you need to include *all* schemas in your current cn=config matching the existing order, as well as the new one you are trying to add?
Obviously the config file has to be valid; any schema that the ones you're converting depend on must be loaded.
I understand that part; however when installing the packages for Debian Squeeze the post-installation process preloads the following schemas into cn=schema,cn=config:
{0}core {1}cosine {2}nis {3}inetorgperson
So what I wanted to verify was that if I want to add a new sirius-custom.schema file into the directory I would need to setup schemaConvert.conf to look like this:
include /etc/openldap/schema/core.schema include /etc/openldap/schema/cosine.schema include /etc/openldap/schema/nis.schema include /etc/openldap/schema/inetorgperson.schema include /etc/openldap/schema/sirius-custom.schema
The previous entries are required to ensure that the sirius-custom.schema LDIF would be generated as {4}sirius-custom ready for using ldapadd to load into the directory.
However when I then run:
mkdir config && slaptest -f slapd.conf.tmp -F config
Then I get my {4}sirius-custom.ldif but the top lines look like this:
dn: cn={4}sirius-custom ... cn: {4}sirius-custom
...which then means I still can't add it directly using ldapadd without further processing.
Also that begs another question: what happens if you want to modify an existing schema, e.g. if I need to hack a schema by hand and reload it into openldap so that it takes effect? Normally I would change the schema file on disk, restart slapd and it would just work.
I frankly can't believe that you just asked that question. cn=config is an LDAP database. When you want to change its contents, you use ldapmodify. It takes effect immediately and there is no need to restart the server.
Oh I understand that part - my question was perhaps badly asked. What I really want to know is what should my workflow be when designing new LDAP schemas to be distributed as .schema files? Based upon what you are saying, I guess it should be either:
1) Update .schema file 2) Generate .ldif using slaptest 3) Manually update the relevant attribute in cn=schema using an LDAP browser to be the new value generated within the .ldif 4) Test
or:
1) Update .schema file 2) Generate .ldif using slaptest 3) Hack the dn within the .ldif to add cn=schema,cn=config suffix 4) Remove the entire cn={x}sirius-custom section 5) Reload the entire cn={x}sirius-custom section using ldapadd 6) Test
Does that sound correct?
ATB,
Mark.
Mark Cave-Ayland wrote:
On 29/06/11 12:59, Howard Chu wrote:
Thanks for the response - this makes a bit more sense now. Just to clarify another point: when you generate schemaConvert.conf, I guess that you need to include *all* schemas in your current cn=config matching the existing order, as well as the new one you are trying to add?
Obviously the config file has to be valid; any schema that the ones you're converting depend on must be loaded.
I understand that part; however when installing the packages for Debian Squeeze the post-installation process preloads the following schemas into cn=schema,cn=config:
{0}core {1}cosine {2}nis {3}inetorgperson
So what I wanted to verify was that if I want to add a new sirius-custom.schema file into the directory I would need to setup schemaConvert.conf to look like this:
include /etc/openldap/schema/core.schema include /etc/openldap/schema/cosine.schema include /etc/openldap/schema/nis.schema include /etc/openldap/schema/inetorgperson.schema include /etc/openldap/schema/sirius-custom.schema
The previous entries are required to ensure that the sirius-custom.schema LDIF would be generated as {4}sirius-custom ready for using ldapadd to load into the directory.
You only need to load those 4 schema files if your sirius-custom.schema file actually depends on all of them. The ordering prefix only needs to be {4} if you really need those others to be parsed first. Otherwise the prefix can be deleted and the config backend will generate it automatically. This is all in the documentation. You should try reading it sometime.
http://www.openldap.org/doc/admin24/slapdconf2.html
However when I then run:
mkdir config&& slaptest -f slapd.conf.tmp -F config
Then I get my {4}sirius-custom.ldif but the top lines look like this:
No.
No.
No.
You're not paying attention. You skipped step 3 of my reply.
And apparently you need to be flamed too. http://www.openldap.org/lists/openldap-technical/201106/msg00085.html
I frankly can't believe that you just asked that question. cn=config is an LDAP database. When you want to change its contents, you use ldapmodify. It takes effect immediately and there is no need to restart the server.
Oh I understand that part - my question was perhaps badly asked. What I really want to know is what should my workflow be when designing new LDAP schemas to be distributed as .schema files? Based upon what you are saying, I guess it should be either:
- Update .schema file
- Generate .ldif using slaptest
- Manually update the relevant attribute in cn=schema using an LDAP
browser to be the new value generated within the .ldif 4) Test
or:
- Update .schema file
- Generate .ldif using slaptest
- Hack the dn within the .ldif to add cn=schema,cn=config suffix
- Remove the entire cn={x}sirius-custom section
- Reload the entire cn={x}sirius-custom section using ldapadd
- Test
Does that sound correct?
No.
As soon as you use the word "hack" you should realize you're doing something wrong. As soon as you use the words "manually update" you should realize you're doing something wrong.
On 29/06/11 14:42, Howard Chu wrote:
You only need to load those 4 schema files if your sirius-custom.schema file actually depends on all of them. The ordering prefix only needs to be {4} if you really need those others to be parsed first. Otherwise the prefix can be deleted and the config backend will generate it automatically. This is all in the documentation. You should try reading it sometime.
(cut)
You're not paying attention. You skipped step 3 of my reply.
And apparently you need to be flamed too. http://www.openldap.org/lists/openldap-technical/201106/msg00085.html
Okay - I consider myself flamed on this occasion :) But that still doesn't quite get me there:
echo "include /etc/ldap/schema/sirius-custom.schema" > slapd.conf.tmp mkdir config && slaptest -f slapd.conf.tmp -F config slapcat -F config/ -n0 -s cn=schema,cn=config > sirius-custom.ldif
That gives me a sirius-custom.ldif which contains both "dn: cn=schema,cn=config" and "dn: cn={0}sirius-custom,cn=schema,cn=config" which is better - but it's still not something I can directly run through ldapadd:
zeno:/tmp# ldapadd -D 'cn=admin,dc=siriusit,dc=co,dc=uk' -f sirius-custom.ldif -xW Enter LDAP Password: adding new entry "cn=schema,cn=config" ldap_add: Constraint violation (19) additional info: structuralObjectClass: no user modification allowed
I don't get much further even if I remove cn=schema,cn=config leaving just "dn: cn={0}sirius-custom,cn=schema,cn=config" within the output.
And also the slapcat command above always generates a {0} index for the output, so how can I alter the above command so that it omits the index so that slapd generates it automatically? And what happens if I accidentally add the same schema but with a different index? Does slapd consider them to be two different entries?
Does that sound correct?
No.
As soon as you use the word "hack" you should realize you're doing something wrong. As soon as you use the words "manually update" you should realize you're doing something wrong.
I have been honest enough during this thread to admit that I felt I may have missed something obvious. But I have to point out that all of the Google searches I have done on this topic have returned posts similar to the one I pointed you to, which you are saying are wrong.
There is a lot of mis-information out there regarding cn=config but I at least I understand that I need to come to the project itself to get some answers.
I understand your frustration here, but it will also take some effort on behalf of the project to ensure that the documentation can answer questions such as mine - I've been using openldap for 3 years now, and cn=config still takes some getting used to.
Once we have figured out a basic schema workflow, I am happy to submit a documentation patch to help other people such as myself in return for your assistance.
ATB,
Mark.
On 29/06/2011 16:46, Mark Cave-Ayland wrote:
I have been honest enough during this thread to admit that I felt I may have missed something obvious. But I have to point out that all of the Google searches I have done on this topic have returned posts similar to the one I pointed you to, which you are saying are wrong.
There is a lot of mis-information out there regarding cn=config but I at least I understand that I need to come to the project itself to get some answers.
I understand your frustration here, but it will also take some effort on behalf of the project to ensure that the documentation can answer questions such as mine - I've been using openldap for 3 years now, and cn=config still takes some getting used to.
I think that putting the cn=config backend in some other directory instead of /etc/ldap/slapd.d (that's for Debian, don't know for other distributions) could help a lot.
Most sysadmin expect to find text configuration files under /etc, not a kind of testual DB. I already filed a bug for Debian because that choice is like looking for trouble.
Having it in a binary format could be even better, so people will not mislead thinking they are some kind of split configuration files.
Regards Simone
On 29/06/11 16:50, Simone Piccardi wrote:
I think that putting the cn=config backend in some other directory instead of /etc/ldap/slapd.d (that's for Debian, don't know for other distributions) could help a lot.
Most sysadmin expect to find text configuration files under /etc, not a kind of testual DB. I already filed a bug for Debian because that choice is like looking for trouble.
Having it in a binary format could be even better, so people will not mislead thinking they are some kind of split configuration files.
Hmmm - that's an interesting thought. It probably should be under /var/lib/slapd-config or similar as it is effectively a database rather than a configuration file.
My understanding is that touching the files in /etc/ldap/slapd.d is a definite no-no once the LDAP directory has been initialised, which is why I'm interested in Howard showing us his workflow for working on schemas so I can document it.
ATB,
Mark.
On 11-06-29 12:28 PM, Mark Cave-Ayland wrote:
On 29/06/11 16:50, Simone Piccardi wrote:
I think that putting the cn=config backend in some other directory instead of /etc/ldap/slapd.d (that's for Debian, don't know for other distributions) could help a lot.
Most sysadmin expect to find text configuration files under /etc, not a kind of testual DB. I already filed a bug for Debian because that choice is like looking for trouble.
Having it in a binary format could be even better, so people will not mislead thinking they are some kind of split configuration files.
Hmmm - that's an interesting thought. It probably should be under /var/lib/slapd-config or similar as it is effectively a database rather than a configuration file.
My understanding is that touching the files in /etc/ldap/slapd.d is a definite no-no once the LDAP directory has been initialised, which is why I'm interested in Howard showing us his workflow for working on schemas so I can document it.
ATB,
Mark.
Question is why the /etc/openldap/slapd.d file structure is there for users to edit in the first place? Wouldn't it be even more misleading if the running one is modified on the fly while the one in slapd.d/ is not? A bunch of online tutorials and howtos are actually asking people to touch those files, such as cn=config file at least for initial setup.
--On Wednesday, June 29, 2011 12:51 PM -0400 Daniel Qian daniel@up247solution.com wrote:
Question is why the /etc/openldap/slapd.d file structure is there for users to edit in the first place? Wouldn't it be even more misleading if the running one is modified on the fly while the one in slapd.d/ is not? A bunch of online tutorials and howtos are actually asking people to touch those files, such as cn=config file at least for initial setup.
The OpenLDAP project has no control over how Linux distributions decide to deploy their configurations, or any control over online tutorials or how-NOT-to's. Feel free to file bugs with Debian, etc, to fix their cn=config layout. Feel free to email people who have posted online tutorials and how-NOT-tos to either correct them or take them offline.
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
zeno:/tmp# ldapadd -D 'cn=admin,dc=siriusit,dc=co,dc=uk' -f sirius-custom.ldif -xW Enter LDAP Password: adding new entry "cn=schema,cn=config" ldap_add: Constraint violation (19) additional info: structuralObjectClass: no user modification allowed
Hi Mark,
How did you generate your schema LDIF again? It's quite easy to do it without slapcat etc.
See http://www.openldap.org/lists/openldap-software/200504/msg00195.html about above. You've got some entries in there you shouldn't.
I don't get much further even if I remove cn=schema,cn=config leaving just "dn: cn={0}sirius-custom,cn=schema,cn=config" within the output.
And also the slapcat command above always generates a {0} index for the output, so how can I alter the above command so that it omits the index so that slapd generates it automatically? And what happens if I accidentally add the same schema but with a different index? Does slapd consider them to be two different entries?
Just the same as any LDIF. If it's repeated/exists it will fail.
Does that sound correct?
No.
As soon as you use the word "hack" you should realize you're doing something wrong. As soon as you use the words "manually update" you should realize you're doing something wrong.
I have been honest enough during this thread to admit that I felt I may have missed something obvious. But I have to point out that all of the Google searches I have done on this topic have returned posts similar to the one I pointed you to, which you are saying are wrong.
There is a lot of mis-information out there regarding cn=config but I at least I understand that I need to come to the project itself to get some answers. I understand your frustration here, but it will also take some effort on behalf of the project to ensure that the documentation can answer questions such as mine - I've been using openldap for 3 years now, and cn=config still takes some getting used to.
I think it depends on how you look at it as it's in LDIF. It's been 7 for me (2004) and it depends on how the question is asked and the prior knowledge, which sometimes gets in the way.
Once we have figured out a basic schema workflow, I am happy to submit a documentation patch to help other people such as myself in return for your assistance.
What would you like to see? We have a wiki now that could be more tutorial like. The current Admin Guide covers it well, but we could break it down more with comparison to slapd.conf on the wiki for people with less than 5 years experience :-)
Is there anything I can learn from the PostgreSQL docs project using your experience and contacts?
Thanks,
Gavin.
ATB,
Mark.
-- Mark Cave-Ayland - Senior Technical Architect PostgreSQL - PostGIS Sirius Corporation plc - control through freedom http://www.siriusit.co.uk t: +44 870 608 0063
Sirius Labs: http://www.siriusit.co.uk/labs
On 11-06-29 9:26 AM, Mark Cave-Ayland wrote:
On 29/06/11 12:59, Howard Chu wrote:
Thanks for the response - this makes a bit more sense now. Just to clarify another point: when you generate schemaConvert.conf, I guess that you need to include *all* schemas in your current cn=config matching the existing order, as well as the new one you are trying to add?
Obviously the config file has to be valid; any schema that the ones you're converting depend on must be loaded.
I understand that part; however when installing the packages for Debian Squeeze the post-installation process preloads the following schemas into cn=schema,cn=config:
{0}core {1}cosine {2}nis {3}inetorgperson
So what I wanted to verify was that if I want to add a new sirius-custom.schema file into the directory I would need to setup schemaConvert.conf to look like this:
include /etc/openldap/schema/core.schema include /etc/openldap/schema/cosine.schema include /etc/openldap/schema/nis.schema include /etc/openldap/schema/inetorgperson.schema include /etc/openldap/schema/sirius-custom.schema
The previous entries are required to ensure that the sirius-custom.schema LDIF would be generated as {4}sirius-custom ready for using ldapadd to load into the directory.
However when I then run:
mkdir config && slaptest -f slapd.conf.tmp -F config
Then I get my {4}sirius-custom.ldif but the top lines look like this:
dn: cn={4}sirius-custom ... cn: {4}sirius-custom
...which then means I still can't add it directly using ldapadd without further processing.
Not sure if it is the same on Debian but on Fedora I only copied the "workplace" output schema file (autofs.schema in my case) to /etc/openldap/slapd.d/cn=config/cn=schema/ without modifying anything. I restarted slapd after that and everything worked for me. Surprisingly easy although I had to do a restart.
On 29/06/11 15:15, Daniel Qian wrote:
Not sure if it is the same on Debian but on Fedora I only copied the "workplace" output schema file (autofs.schema in my case) to /etc/openldap/slapd.d/cn=config/cn=schema/ without modifying anything. I restarted slapd after that and everything worked for me. Surprisingly easy although I had to do a restart.
Right, that is one option. But indeed the primary advantage of cn=config is that it doesn't need a restart, and that's really where I want to be :)
ATB,
Mark.
On 11-06-29 10:49 AM, Mark Cave-Ayland wrote:
On 29/06/11 15:15, Daniel Qian wrote:
Not sure if it is the same on Debian but on Fedora I only copied the "workplace" output schema file (autofs.schema in my case) to /etc/openldap/slapd.d/cn=config/cn=schema/ without modifying anything. I restarted slapd after that and everything worked for me. Surprisingly easy although I had to do a restart.
Right, that is one option. But indeed the primary advantage of cn=config is that it doesn't need a restart, and that's really where I want to be :)
ATB,
Mark.
In that case you can slapcat it into the db after trimming the schema file I guess. But then the cn=config directory structure on file system will be inconsistent with the actual running config. Actually I have been wondering for a while if this will be an issue and would like to hear some words if anyone have about this.
On Wednesday, 29 June 2011 16:15:54 Daniel Qian wrote:
On 11-06-29 9:26 AM, Mark Cave-Ayland wrote:
On 29/06/11 12:59, Howard Chu wrote:
Thanks for the response - this makes a bit more sense now. Just to clarify another point: when you generate schemaConvert.conf, I guess that you need to include *all* schemas in your current cn=config matching the existing order, as well as the new one you are trying to add?
Obviously the config file has to be valid; any schema that the ones you're converting depend on must be loaded.
I understand that part; however when installing the packages for Debian Squeeze the post-installation process preloads the following schemas into cn=schema,cn=config:
{0}core {1}cosine {2}nis {3}inetorgperson
So what I wanted to verify was that if I want to add a new sirius-custom.schema file into the directory I would need to setup schemaConvert.conf to look like this:
include /etc/openldap/schema/core.schema include /etc/openldap/schema/cosine.schema include /etc/openldap/schema/nis.schema include /etc/openldap/schema/inetorgperson.schema include /etc/openldap/schema/sirius-custom.schema
The previous entries are required to ensure that the sirius-custom.schema LDIF would be generated as {4}sirius-custom ready for using ldapadd to load into the directory.
However when I then run:
mkdir config && slaptest -f slapd.conf.tmp -F config
Then I get my {4}sirius-custom.ldif but the top lines look like this:
dn: cn={4}sirius-custom ... cn: {4}sirius-custom
...which then means I still can't add it directly using ldapadd without further processing.
Not sure if it is the same on Debian but on Fedora I only copied the "workplace" output schema file (autofs.schema in my case) to /etc/openldap/slapd.d/cn=config/cn=schema/ without modifying anything. I restarted slapd after that and everything worked for me. Surprisingly easy although I had to do a restart.
But, this isn't how it is intended to be used, and gets you into bad habits. You should treat the config database as if it is any other database. The closest you should get to manual changes to it is 'slapadd', just like hdb or bdb etc.
You should just have been able to ldapadd the ldif file. And it wouldn't have required a restart.
For example, in Mandriva we ship pre-converted ldif schema files, that can just be ldapadd'ed.
http://svn.mandriva.com/cgi-bin/viewvc.cgi/packages/cooker/openldap-extra- schemas/current/SOURCES/
(viewvc seems to have changed, you used to be able to download the files raw before, which made this more useful)
Yes, for this package I do manually remove the ordering from the DN/naming attribute, purely to allow easy post-initial-configuration addition.
Regards, Buchan
On 29/06/2011 16:15, Daniel Qian wrote:
Not sure if it is the same on Debian but on Fedora I only copied the "workplace" output schema file (autofs.schema in my case) to /etc/openldap/slapd.d/cn=config/cn=schema/ without modifying anything. I restarted slapd after that and everything worked for me. Surprisingly easy although I had to do a restart.
Then I think you must be flamed too...
But if you did so, then why just not to stay with slapd.conf?
It still work and is far easier to understand and manage.
Regards Simone
On 11-06-29 11:56 AM, Simone Piccardi wrote:
On 29/06/2011 16:15, Daniel Qian wrote:
Not sure if it is the same on Debian but on Fedora I only copied the "workplace" output schema file (autofs.schema in my case) to /etc/openldap/slapd.d/cn=config/cn=schema/ without modifying anything. I restarted slapd after that and everything worked for me. Surprisingly easy although I had to do a restart.
Then I think you must be flamed too...
But if you did so, then why just not to stay with slapd.conf?
It still work and is far easier to understand and manage.
Regards Simone
I want to stay with the cn=config but to be frank there are some items that need to be clarified, perhaps in the documentation of the distribution so that end users like myself won't has so much frustration when it comes to setting it up the new way.
On 29/06/11 16:56, Simone Piccardi wrote:
Then I think you must be flamed too...
But if you did so, then why just not to stay with slapd.conf?
It still work and is far easier to understand and manage.
Because slapd.conf is deprecated and going away in openldap 2.5. So I'd like to get familiar with it now before this happens :)
Although I'm not sure exactly how this is going to work since elements of slapd.conf are still required to bootstrap the directory from LDAP .schema files. Hmmm.
ATB,
Mark.
--On Wednesday, June 29, 2011 5:30 PM +0100 Mark Cave-Ayland mark.cave-ayland@siriusit.co.uk wrote:
On 29/06/11 16:56, Simone Piccardi wrote:
Then I think you must be flamed too...
But if you did so, then why just not to stay with slapd.conf?
It still work and is far easier to understand and manage.
Because slapd.conf is deprecated and going away in openldap 2.5. So I'd like to get familiar with it now before this happens :)
Although I'm not sure exactly how this is going to work since elements of slapd.conf are still required to bootstrap the directory from LDAP .schema files. Hmmm.
Zimbra puts cn=config in the same location as any other LDAP database we use (/opt/zimbra/data/ldap/, which has cn=config, hdb, and accesslog databases).
As for the *.ldif schema, OpenLDAP ships both .ldif and .schema files for the schemas it has always shipped. If a project is not shipping a .ldif version, file a bug with the project (I did this for Amavis, for example).
If you have your own schema, change your schema generation process to generate an LDIF formatted schema instead of the old .schema format. That's also what we did at Zimbra (In fact, we generate both .schema and .ldif versions of the ZCS schema).
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
On 29/06/11 18:05, Quanah Gibson-Mount wrote:
Although I'm not sure exactly how this is going to work since elements of slapd.conf are still required to bootstrap the directory from LDAP .schema files. Hmmm.
Zimbra puts cn=config in the same location as any other LDAP database we use (/opt/zimbra/data/ldap/, which has cn=config, hdb, and accesslog databases).
Right - so the Debian guys have definitely got this wrong then :(
As for the *.ldif schema, OpenLDAP ships both .ldif and .schema files for the schemas it has always shipped. If a project is not shipping a .ldif version, file a bug with the project (I did this for Amavis, for example).
If you have your own schema, change your schema generation process to generate an LDIF formatted schema instead of the old .schema format. That's also what we did at Zimbra (In fact, we generate both .schema and .ldif versions of the ZCS schema).
Whoa - wait second. Now I get it - so not only is slapd.conf deprecated, but also the distribution of .schema files. If we're supposed to be distributing schemas in .ldif format from now on, then the conversion problem just goes away. So I was missing something obvious after all...
As a long-term follower of this project, can I just summarise what I believe the results of this thread are:
1) Distribution of .schema files is deprecated. Projects should be distributing .ldif files containing schema changes instead.
2) If a project is distributing a .schema file, ask them politely to move to .ldif format instead. If they refuse, you need to convert from .schema to .ldif yourself using slaptest.
Finally, just as a general observation it's obvious that this message is NOT getting through to distributions and developers. I would advise that you add an explicit deprecation warning related to the distribution of .schema files with applications (as it's not clear that the .schema files are directly considered part of the configuration) to the main documentation and point people towards that when (like me) they start to ask questions.
ATB,
Mark.
--On Thursday, June 30, 2011 9:59 AM +0100 Mark Cave-Ayland mark.cave-ayland@siriusit.co.uk wrote:
On 29/06/11 18:05, Quanah Gibson-Mount wrote:
Although I'm not sure exactly how this is going to work since elements of slapd.conf are still required to bootstrap the directory from LDAP .schema files. Hmmm.
Zimbra puts cn=config in the same location as any other LDAP database we use (/opt/zimbra/data/ldap/, which has cn=config, hdb, and accesslog databases).
Right - so the Debian guys have definitely got this wrong then :(
As for the *.ldif schema, OpenLDAP ships both .ldif and .schema files for the schemas it has always shipped. If a project is not shipping a .ldif version, file a bug with the project (I did this for Amavis, for example).
If you have your own schema, change your schema generation process to generate an LDIF formatted schema instead of the old .schema format. That's also what we did at Zimbra (In fact, we generate both .schema and .ldif versions of the ZCS schema).
Whoa - wait second. Now I get it - so not only is slapd.conf deprecated, but also the distribution of .schema files. If we're supposed to be distributing schemas in .ldif format from now on, then the conversion problem just goes away. So I was missing something obvious after all...
Right.
As a long-term follower of this project, can I just summarise what I believe the results of this thread are:
- Distribution of .schema files is deprecated. Projects should be
distributing .ldif files containing schema changes instead.
Correct. Or ship both until 2.5 is out. ;)
- If a project is distributing a .schema file, ask them politely to move
to .ldif format instead. If they refuse, you need to convert from .schema to .ldif yourself using slaptest.
Correct.
Finally, just as a general observation it's obvious that this message is NOT getting through to distributions and developers. I would advise that you add an explicit deprecation warning related to the distribution of .schema files with applications (as it's not clear that the .schema files are directly considered part of the configuration) to the main documentation and point people towards that when (like me) they start to ask questions.
Unfortunately, the people who maintain various distributions don't necessarily participate (by joining the OpenLDAP mailing lists) in the project. Others (RedHat, Mandriva, possibly others) do. The OpenLDAP project obviously cannot force them to do things in any particular way.
Again, I urge people to file bug reports etc with distributions when they find issues like these. I do this fairly often with Debian/Ubuntu.
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
On 30/06/2011 19:18, Quanah Gibson-Mount wrote:
Again, I urge people to file bug reports etc with distributions when they find issues like these. I do this fairly often with Debian/Ubuntu.
As I said I already filed a bug for Debian, is here:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=626911
That's about using /etc/ldap/slapd.d.
I will not file any bugs about passing to .ldap files for schemas, because I think .schema files are far more readable, so they are still useful, and there is a clear way to get LDIF files from them where the reverse is not true.
Regards Simone
On 29/06/2011 18:30, Mark Cave-Ayland wrote:
On 29/06/11 16:56, Simone Piccardi wrote:
Then I think you must be flamed too...
But if you did so, then why just not to stay with slapd.conf?
It still work and is far easier to understand and manage.
Because slapd.conf is deprecated and going away in openldap 2.5. So I'd like to get familiar with it now before this happens :)
I understood that this was not the case, but I can be wrong. And if yes I hope this decision could be reversed, but I'm not a developer so I can just state my uneasiness about such a choice.
As I already said I'm just an old style Unix admin, and as such I don't like at all to be forced to use any kind of program different from just an editor to do a service configuration.
Having text configuration files (that can be commented and put under a revision system) is a major strength for an Unix system, and giving it away is very bad. I can afford a restart or a reload in any of the tens of installation I have, but reworking the method we have to take trace of configuration changes made in production servers will be a real pain.
I support the choice to make possible configuration change without restarting the service, what I do not support is the choice to remove the use of textual configuration files.
Simone
What stops you from checking the ldif of cn=config into revision control before and after a change?
On Fri, Jul 01, 2011 at 07:20:14PM +0200, Michael Ströder wrote:
Simone Piccardi wrote:
Having text configuration files (that can be commented and put under a revision system) is a major strength for an Unix system, and giving it away is very bad.
Amen.
Ciao, Michael.
Il 02/07/2011 00:47, Christopher Wood ha scritto:
What stops you from checking the ldif of cn=config into revision control before and after a change?
Complexity, because you have to add more steps to have less informations. I don't think there are comments inside cn=config, and those are very important when you have to document and track tens of different installations.
Simone
On 2/7/2011 2:53 μμ, Simone Piccardi wrote:
... I don't think there are comments inside cn=config, and those are very important when you have to document and track tens of different installations.
Indeed, it seems that the config schema does not include a "description" (or similar) attribute which can be used freely in any config DIT entries for documentation purposes.
If this is truly the case, wouldn't it be advisable to add such support to the config schema? Of course, this means that all objectclass definitions should include: ...MAY (... $ description ) ...
Or, are there any other suggestions on how to include documentation in the config DIT?
Documenting entries has proved very valuable in practice.
Usually, in our DIT (which includes mainly people, mail alias, other accounts, and DNS data) we use single-valued or multi-valued (per entry) "description" attributes (as defined in RFC 4519) which serve very well our documentation needs. These are already included in the objectClasses we use (based on account, person, domain etc. objectClasses ).
In some cases, we format these description values in very specific ways so as to allow automatic processing of various attributes from external scripts, based on data provided in the description attribute values. This use of description attribute values might be unconventional (?) but it suits our needs well and it has been a quite flexible tool.
Regards, Nick
Nick Milas wrote:
On 2/7/2011 2:53 μμ, Simone Piccardi wrote:
I don't think there are comments inside cn=config, and those are very important when you have to document and track tens of different installations.
Indeed, it seems that the config schema does not include a "description" (or similar) attribute which can be used freely in any config DIT entries for documentation purposes.
If this is truly the case, wouldn't it be advisable to add such support to the config schema? Of course, this means that all objectclass definitions should include: ...MAY (... $ description ) ...
While description may help a little bit you cannot add comments for each configuration directive for example ACLs.
Usually, in our DIT (which includes mainly people, mail alias, other accounts, and DNS data) we use single-valued or multi-valued (per entry) "description" attributes (as defined in RFC 4519) which serve very well our documentation needs. These are already included in the objectClasses we use (based on account, person, domain etc. objectClasses ).
I also use 'description' quite a lot especially since my web2ldap displays it as bubble help in the browser. But it's not sufficient in this case.
Ciao, Michael.
Michael Ströder wrote:
Nick Milas wrote:
On 2/7/2011 2:53 μμ, Simone Piccardi wrote:
I don't think there are comments inside cn=config, and those are very important when you have to document and track tens of different installations.
Indeed, it seems that the config schema does not include a "description" (or similar) attribute which can be used freely in any config DIT entries for documentation purposes.
If this is truly the case, wouldn't it be advisable to add such support to the config schema? Of course, this means that all objectclass definitions should include: ...MAY (... $ description ) ...
While description may help a little bit you cannot add comments for each configuration directive for example ACLs.
We've been discussing this problem for quite a while. My current thinking is that somehow we can use attribute options to help. Visually it might be better to associate the option with the original attribute, e.g. olcAccess: olcAccess;x-comment: This would require defining a new (and strange) type of attribute option though, since the value with the option has no relation (syntactically) to the original attribute type.
The other alternative is to add a generic description attribute, and tag it with the attribute that a comment refers to: description;x-olcAccess: blah blah blah
This is a lot simpler for us to implement.
Usually, in our DIT (which includes mainly people, mail alias, other accounts, and DNS data) we use single-valued or multi-valued (per entry) "description" attributes (as defined in RFC 4519) which serve very well our documentation needs. These are already included in the objectClasses we use (based on account, person, domain etc. objectClasses ).
I also use 'description' quite a lot especially since my web2ldap displays it as bubble help in the browser. But it's not sufficient in this case.
Ciao, Michael.
On 3/7/2011 2:18 πμ, Howard Chu wrote:
We've been discussing this problem for quite a while. My current thinking is that somehow we can use attribute options to help. Visually it might be better to associate the option with the original attribute, e.g. olcAccess: olcAccess;x-comment: This would require defining a new (and strange) type of attribute option though, since the value with the option has no relation (syntactically) to the original attribute type.
Thank you for this discussion.
I think it might not matter which of the two is the implementation, as long as the "description" (either "olcWhatever;x-comment:" or "description;x-olcWhatever:") is available and multi-valued for each and every attribute and if it can easily stay together with the entry concerned, and easily displayed in relation to it as well (yes, visual behavior for comments is important!). For example, for olcAccess, modifying an example from: http://www.openldap.org/doc/admin24/access-control.html, if we create the attribute values:
olcAccess: to attrs=member,entry by dnattr=member selfwrite description;x-olcAccess: (0) Access rules for attrs: member,entry description;x-olcAccess: (0) Entered by Nick on 7/12/2012 olcAccess: to dn.children="dc=example,dc=com" by * search description;x-olcAccess: (1) Allow search on children of example.com description;x-olcAccess: (1) Entered by Nick on 7/12/2012 olcAccess: to dn.children="dc=com" by * read description;x-olcAccess: (2) Allow only read on children of com description;x-olcAccess: (2) Entered by Nick on 7/12/2012
we should be able to easily display:
olcAccess: {0}to attrs=member,entry by dnattr=member selfwrite description;x-olcAccess: (0) Access rules for attrs: member,entry description;x-olcAccess: (0) Entered by Nick on 7/12/2012 olcAccess: {1}to dn.children="dc=example,dc=com" by * search description;x-olcAccess: (1) Allow search on children of example.com description;x-olcAccess: (1) Entered by Nick on 7/12/2012 olcAccess: {2}to dn.children="dc=com" by * read description;x-olcAccess: (2) Allow only read on children of com description;x-olcAccess: (2) Entered by Nick on 7/12/2012
I assume, all the above should be easier and more intuitive to achieve using the "olcAccess;x-comment:" approach (even if it's more difficult - as you say - to implement).
But, I am sure there are many experts in this list, far more experienced than me, whose feedback would be most valuable.
In any case, I believe a solution should be outlined soon and scheduled for development, because it is a real need.
Regards, Nick
Nick Milas wrote:
On 3/7/2011 2:18 πμ, Howard Chu wrote:
We've been discussing this problem for quite a while. My current thinking is that somehow we can use attribute options to help. Visually it might be better to associate the option with the original attribute, e.g. olcAccess: olcAccess;x-comment: This would require defining a new (and strange) type of attribute option though, since the value with the option has no relation (syntactically) to the original attribute type.
Thank you for this discussion.
I think it might not matter which of the two is the implementation, as long as the "description" (either "olcWhatever;x-comment:" or "description;x-olcWhatever:") is available and multi-valued for each and every attribute and if it can easily stay together with the entry concerned, and easily displayed in relation to it as well (yes, visual behavior for comments is important!). For example, for olcAccess, modifying an example from: http://www.openldap.org/doc/admin24/access-control.html, if we create the attribute values:
olcAccess: to attrs=member,entry by dnattr=member selfwrite description;x-olcAccess: (0) Access rules for attrs: member,entry description;x-olcAccess: (0) Entered by Nick on 7/12/2012 olcAccess: to dn.children="dc=example,dc=com" by * search description;x-olcAccess: (1) Allow search on children of example.com description;x-olcAccess: (1) Entered by Nick on 7/12/2012 olcAccess: to dn.children="dc=com" by * read description;x-olcAccess: (2) Allow only read on children of com description;x-olcAccess: (2) Entered by Nick on 7/12/2012
Interleaved values such as above may never be returned by an LDAP Search operation. Whether it's desirable for a client to reorganize the data to display as such, I have no opinion.
I note that the accesslog overlay already allows you to track the history of individual changes to attributes, so recording an "Entered by <name> on <date>" comment seems both redundant and vague.
we should be able to easily display:
olcAccess: {0}to attrs=member,entry by dnattr=member selfwrite description;x-olcAccess: (0) Access rules for attrs: member,entry description;x-olcAccess: (0) Entered by Nick on 7/12/2012 olcAccess: {1}to dn.children="dc=example,dc=com" by * search description;x-olcAccess: (1) Allow search on children of example.com description;x-olcAccess: (1) Entered by Nick on 7/12/2012 olcAccess: {2}to dn.children="dc=com" by * read description;x-olcAccess: (2) Allow only read on children of com description;x-olcAccess: (2) Entered by Nick on 7/12/2012I assume, all the above should be easier and more intuitive to achieve using the "olcAccess;x-comment:" approach (even if it's more difficult - as you say - to implement).
On 4/7/2011 1:00 μμ, Howard Chu wrote:
Interleaved values such as above may never be returned by an LDAP Search operation. Whether it's desirable for a client to reorganize the data to display as such, I have no opinion.
I know, that's why I mentioned the issue, so that a solution might possibly be found at design time. In long/complex ACLs (ours is already about 1000 lines and growing - in standard format, not RTC), if comments are decoupled from the statements concerned, then they lose a most of their power;
I note that the accesslog overlay already allows you to track the history of individual changes to attributes, so recording an "Entered by <name> on <date>" comment seems both redundant and vague.
Of course; this was just an example to illustrate the use of two comments per statement.
I hope other people will provide feedback on this as well.
Nick.
Howard Chu wrote:
I note that the accesslog overlay already allows you to track the history of individual changes to attributes, so recording an "Entered by <name> on <date>" comment seems both redundant and vague.
I really like slapo-accesslog and the audit trail it provides. Feature request: Sometimes it would be handy to send along a extended control which notes *why* a certain modification was done.
Ciao, Michael.
On 04/07/2011 00:32, Nick Milas wrote:
olcAccess: to attrs=member,entry by dnattr=member selfwrite description;x-olcAccess: (0) Access rules for attrs: member,entry description;x-olcAccess: (0) Entered by Nick on 7/12/2012 olcAccess: to dn.children="dc=example,dc=com" by * search description;x-olcAccess: (1) Allow search on children of example.com description;x-olcAccess: (1) Entered by Nick on 7/12/2012 olcAccess: to dn.children="dc=com" by * read description;x-olcAccess: (2) Allow only read on children of com description;x-olcAccess: (2) Entered by Nick on 7/12/2012
we should be able to easily display:
olcAccess: {0}to attrs=member,entry by dnattr=member selfwrite description;x-olcAccess: (0) Access rules for attrs: member,entry description;x-olcAccess: (0) Entered by Nick on 7/12/2012 olcAccess: {1}to dn.children="dc=example,dc=com" by * search description;x-olcAccess: (1) Allow search on children of example.com description;x-olcAccess: (1) Entered by Nick on 7/12/2012 olcAccess: {2}to dn.children="dc=com" by * read description;x-olcAccess: (2) Allow only read on children of com description;x-olcAccess: (2) Entered by Nick on 7/12/2012
I use comments to clarify things, so to be easy readable is a major requirement. This doesn't seems so readable to me (personal opinion).
Simone
On 3/7/2011 2:18 πμ, Howard Chu wrote:
We've been discussing this problem (ACL rule descriptions) for quite a while. My current thinking is that somehow we can use attribute options to help. Visually it might be better to associate the option with the original attribute, e.g. olcAccess: olcAccess;x-comment: This would require defining a new (and strange) type of attribute option though, since the value with the option has no relation (syntactically) to the original attribute type.
The other alternative is to add a generic description attribute, and tag it with the attribute that a comment refers to: description;x-olcAccess: blah blah blah
This is a lot simpler for us to implement.
Sorry for jumping back to this issue again, but I think that this is an important issue from an administrator's point of view and AFAIK it has not been tackled yet.
I would like to suggest two possible (alternative) changes on the olcAccess attribute:
1. In order to achieve proper display order of olcAccess rules (since most clients automatically sort values based on characters), I would like to ask that the default numbering is not {0}...{n}, but {00000} ... {n} where n is a 5-digit integer (I will depict this as DDDDD). This scheme would support 100.000 (0 - 99999) rules and I believe that it would be sufficient for all practical needs. (If not, it could be made a 6 digit figure; if it is too much already, it could be a 4-digit figure.) Please note that proper ordering/listing of these rules is important in everyday administration!
As an example, instead of:
olcAccess: {0}to dn.subtree=... olcAccess: {1}to dn.regex=...
use:
olcAccess: {00000}to dn.subtree=... olcAccess: {00001}to dn.regex=...
2. On comments about ACL rules: numbering could provide a possible solution (if it's technically feasible and software architects consider it appropriate). olcAccess could take two possible values, one for comments and one for the rule itself; "numbering" could in this case be: {DDDDDc} and {DDDDDr} where D is a digit, c refers to a comment and r refers to the rule itself. This approach would have the advantage that most ldap clients would display rules and comments together in proper sequence. Of course {DDDDDc} would be optional. OpenLDAP software would:
1. Automatically add the {DDDDDr} tag (for new rules), as it now adds an {X} numeric index. 2. Allow admins to manually add olcAccess values starting with a {DDDDDc} and it would ignore them when evaluating rules. 3. This scheme would allow the automatic inclusion of existing ACL comments when converting from slapd.conf, by including {DDDDDc} values for the olcAccess attribute, as needed.
As an example:
olcAccess: {00000c}*** Provide access to subtree xxxx *** olcAccess: {00000r}to dn.subtree=... olcAccess: {00001c}*** Provide access to whatever *** olcAccess: {00001r}to dn.regex=...
To take this a bit further, multiple {DDDDDc} could be allowed and it would be up to the administrator(s) to use an initial string to allow proper sorting of such comments (for example {00000c}[00]...{00000c}[01] etc.). OpenLDAP software would ignore these values in evaluating rules anyway.
Just my 2c on an everyday maintenance problem, or a seed for further or other proposals.
By the way, a question: in case someone manually deletes (accidentally or intentionally) a rule so that a gap in numbering occurs, will OpenLDAP continue to evaluate subsequent rules? For example, if someone deletes existing rule {12} (current numbering scheme), will the system evaluate ACLs after {11} (like {13}, {14} etc.)?
Thanks for your time, Nick
Nick Milas wrote:
By the way, a question: in case someone manually deletes (accidentally or intentionally) a rule so that a gap in numbering occurs, will OpenLDAP continue to evaluate subsequent rules? For example, if someone deletes existing rule {12} (current numbering scheme), will the system evaluate ACLs after {11} (like {13}, {14} etc.)?
Read http://highlandsun.com/hyc/drafts/draft-chu-ldap-xordered-xx.html
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 10/19/2011 12:50 PM, Howard Chu wrote:
Read http://highlandsun.com/hyc/drafts/draft-chu-ldap-xordered-xx.html
I would like to point out that while back-ldif handles the ordering prefix fine, bconfig's (bconfig.c:4726) implementation of the same maintains one joint sibling ordering for all AttributeTypes under the same parent, instead of one ordering per each AttributeType as in the draft.
Lately, I've been hacking together a solution to this, but my code so far is too ugly and fragile as DN handling inside OpenLDAP has been a Terra Incognita for me until now.
This is the issue behind ITS#6987 I reported some time ago. Thank you Howard for linking to the draft. It reminded me that I should have done a better job at describing the issue than I did back when reporting the ITS. (Should I post the more descriptive explanation to the ITS too?)
Also, the draft does not handle multivalued RDN's nor does it explicitely say they are out of scope.
- -- Ondrej Kuznik
This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you.
Ondrej Kuznik wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 10/19/2011 12:50 PM, Howard Chu wrote:
Read http://highlandsun.com/hyc/drafts/draft-chu-ldap-xordered-xx.html
I would like to point out that while back-ldif handles the ordering prefix fine, bconfig's (bconfig.c:4726) implementation of the same maintains one joint sibling ordering for all AttributeTypes under the same parent, instead of one ordering per each AttributeType as in the draft.
Lately, I've been hacking together a solution to this, but my code so far is too ugly and fragile as DN handling inside OpenLDAP has been a Terra Incognita for me until now.
This is the issue behind ITS#6987 I reported some time ago. Thank you Howard for linking to the draft. It reminded me that I should have done a better job at describing the issue than I did back when reporting the ITS. (Should I post the more descriptive explanation to the ITS too?)
Sounds like that would be a good idea.
Also, the draft does not handle multivalued RDN's nor does it explicitely say they are out of scope.
Well, certainly they are out of the scope of the current implementation. I thought the draft specified that only single-valued RDNs were supported, but looking again, it's not there.
Ondrej Kuznik -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk6euU0ACgkQ9GWxeeH+cXuDTwCfULpYqmU+MWgJBG1QFCqImjdt h2kAoK+ork9Ycgy3+hy8TFSCAyNTG7TA =F5O4 -----END PGP SIGNATURE-----
This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you.
Christopher Wood wrote:
What stops you from checking the ldif of cn=config into revision control before and after a change?
Nothing. But it's more cumbersome. Also comments are lost. Normally I comment everything and describe for which requirement in the concept the ACLs, constraints, etc. are for.
Also the schema with X-ORDERED is cumbersome. There's no LDAP client which really supports it. You have to do too much manually.
There's no mechanism like include.
Etc.
Ciao, Michael.
On Fri, Jul 01, 2011 at 07:20:14PM +0200, Michael Ströder wrote:
Simone Piccardi wrote:
Having text configuration files (that can be commented and put under a revision system) is a major strength for an Unix system, and giving it away is very bad.
Amen.
Ciao, Michael.
Michael Ströder wrote:
Christopher Wood wrote:
What stops you from checking the ldif of cn=config into revision control before and after a change?
Nothing. But it's more cumbersome. Also comments are lost. Normally I comment everything and describe for which requirement in the concept the ACLs, constraints, etc. are for.
Also the schema with X-ORDERED is cumbersome. There's no LDAP client which really supports it. You have to do too much manually.
All of the X-ORDERED stuff can be ignored in the general case. If you find it cumbersome, you're doing something wrong. Obviously the ldapmodify commandline tool supports it.
There's no mechanism like include.
False. OpenLDAP's LDIF library supports include directives, and they're used frequently in the test suite. The include statement has been documented in the ldif(5) manpage since 2006.
You're blaming the tools when it's your own ignorance at fault.
Etc.
Ciao, Michael.
On Fri, Jul 01, 2011 at 07:20:14PM +0200, Michael Ströder wrote:
Simone Piccardi wrote:
Having text configuration files (that can be commented and put under a revision system) is a major strength for an Unix system, and giving it away is very bad.
Amen.
Ciao, Michael.
Howard Chu wrote:
Michael Ströder wrote:
There's no mechanism like include.
False. OpenLDAP's LDIF library supports include directives, and they're used frequently in the test suite. The include statement has been documented in the ldif(5) manpage since 2006.
How do you use this proprieatry include-statement in cn=config?
From my understanding this can only be used when editing LDIF files outside
back-config and slapadd the whole stuff at once. But this defeats the goal of cn=config because slapadd needs a restart.
You're blaming the tools when it's your own ignorance at fault.
Ah, yes...again you don't want to hear the defencencies of the current approach which get obvious again in this thread.
Ciao, Michael.
Michael Ströder wrote:
Howard Chu wrote:
Michael Ströder wrote:
There's no mechanism like include.
False. OpenLDAP's LDIF library supports include directives, and they're used frequently in the test suite. The include statement has been documented in the ldif(5) manpage since 2006.
How do you use this proprieatry include-statement in cn=config?
The full discussion of include: was on the ldapext mailing list, there's nothing proprietary about it. If you don't like the fact that there's no new version of RFC2849 with the relevant text, feel free to spend the time to polish up a new version of that document.
From my understanding this can only be used when editing LDIF files outside
back-config and slapadd the whole stuff at once. But this defeats the goal of cn=config because slapadd needs a restart.
This conversation was about checking a config into a version control system. If you're actually doing that, then you should be checking in a complete slapcat'd LDIF as a single file. The only way to make use of such a version controlled file is by slapadd of course, but most likely the reason you're digging an old version out of your VCS in the first place is because you need to restore a backup, and slapd would already be down in that case.
You're blaming the tools when it's your own ignorance at fault.
Ah, yes...again you don't want to hear the defencencies of the current approach which get obvious again in this thread.
What you call a deficiency(sp) is sheer nonsense. Tell me how you expect to provide read/write database access to a file included from any arbitrary filesystem location? Think about the safety of such a proposition as well; not everybody has AppArmor or other security mechanisms on their machines. You cry that it can't do "X" when in fact it would be irresponsible to ever allow it to do "X". You haven't thought it through, you're just babbling.
On 03/07/2011 14:21, Howard Chu wrote:
What you call a deficiency(sp) is sheer nonsense. Tell me how you expect to provide read/write database access to a file included from any arbitrary filesystem location? Think about the safety of such a proposition as well; not everybody has AppArmor or other security mechanisms on their machines. You cry that it can't do "X" when in fact it would be irresponsible to ever allow it to do "X". You haven't thought it through, you're just babbling.
In my use cases I don't need read/write database access to a file for configuration.
I can afford to restart slapd to make it reload the configuration file, as it always did until now.
What I'm advocating is not losing this feature.
Simone
Does anyone have an example of using the pcache overlay with back-sql? Here's the relevant section of my slapd.config: database sqlsuffix "......"rootdn "......."rootpw ........dbname MySQLdbuser .........subtree_cond "ldap_entries.dn like CONCAT('%',?)"insentry_stmt "INSERT INTO ldap_entries (dn,oc_map_id,parent,keyval) VALUES (?,?,?,?)"has_ldapinfo_dn_ru nobaseobjectupper_func UPPERautocommit yes pcache bdb 10000 1 50 100pcacheAttrset 0 * +pcacheTemplate (objectClass=) 0 60...pcache-directory /var/tmp/cachepcache-cachesize 100 What I'm unclear about is how to configure the private bdb database used for the cache. When use this config, slapd starts up ok, but crashes with a SEGV when (I assume) it tries to cache a result: (I am running slapd built from the latest code in HEAD from git) slap_queue_csn: queing 0xb7a52efa 20110629132038.554471Z#000000#000#000000 Program received signal SIGSEGV, Segmentation fault.[Switching to Thread 0xb7a55b90 (LWP 26226)]0x080f3844 in slap_queue_csn (op=0xb7a53378, csn=0xb7a52f60) at ctxcsn.c:199199 LDAP_TAILQ_INSERT_TAIL( be->be_pending_csn_list,(gdb) where#0 0x080f3844 in slap_queue_csn (op=0xb7a53378, csn=0xb7a52f60) at ctxcsn.c:199#1 0x080f38e0 in slap_get_csn (op=0xb7a53378, csn=0xb7a52f60, manage_ctxcsn=1) at ctxcsn.c:214#2 0x08079f33 in slap_add_opattrs (op=0xb7a53378, text=0xb7a53214, textbuf=0xb7a53074 "", textlen=256, manage_ctxcsn=1) at add.c:609#3 0x0814187b in bdb_add (op=0xb7a53378, rs=0xb7a53200) at add.c:107#4 0x081a144e in merge_entry (op=0xb7a53378, e=0x9f56e4c, dup=0, query_uuid=0xa040848) at pcache.c:865#5 0x081a4d1c in cache_entries (op=0x9fdbfc8, query_uuid=0xa040848) at pcache.c:2325#6 0x081a5244 in pcache_op_cleanup (op=0x9fdbfc8, rs=0xb7a550e4) at pcache.c:2430#7 0x08084697 in slap_cleanup_play (op=0x9fdbfc8, rs=0xb7a550e4) at result.c:539#8 0x08084dd3 in send_ldap_response (op=0x9fdbfc8, rs=0xb7a550e4) at result.c:724#9 0x0808554e in slap_send_ldap_result (op=0x9fdbfc8, rs=0xb7a550e4) at result.c:851#10 0x0813e4a3 in backsql_search (op=0x9fdbfc8, rs=0xb7a550e4) at search.c:2493#11 0x0807402e in fe_op_search (op=0x9fdbfc8, rs=0xb7a550e4) at search.c:402#12 0x080f1aaa in overlay_op_walk (op=0x9fdbfc8, rs=0xb7a550e4, which=op_search, oi=0x9f3dc80, on=0x0) at backover.c:671#13 0x080f1c5f in over_op_func (op=0x9fdbfc8, rs=0xb7a550e4, which=op_search) at backover.c:723#14 0x080f1d0e in over_op_search (op=0x9fdbfc8, rs=0xb7a550e4) at backover.c:750#15 0x0807397d in do_search (op=0x9fdbfc8, rs=0xb7a550e4) at search.c:247#16 0x08070704 in connection_operation (ctx=0xb7a551d0, arg_v=0x9fdbfc8) at connection.c:1138#17 0x08070c44 in connection_read_thread (ctx=0xb7a551d0, argv=0xd) at connection.c:1274#18 0x081e04a5 in ldap_int_thread_pool_wrapper (xpool=0x9f11f78) at tpool.c:685#19 0x004be5ab in start_thread () from /lib/libpthread.so.0#20 0x003b2cfe in clone () from /lib/libc.so.6 It crashes because be->be_pending_csn_list is zero, presumably because backend_startup_one() has not been called for this backend. This leads me to believe that I need something in slapd.conf to get the private bdb database initialised, but what? Chris
On Wed, Jun 29, 2011 at 5:26 AM, Mark Cave-Ayland < mark.cave-ayland@siriusit.co.uk> wrote:
Hi all,
Having started to look at the changes required to migrate from a slapd.conf setup to a cn=config setup, one of things I'm struggling with is how to load new LDAP schemas into cn=config.
There's a certain mindset to be achieved in understanding cn=config / RTC, for sure.
I'm happening to be building up an updated config for an N-way multi-master setup, and having the occasion to revisit the OpenLDAP manual, the Zytrax LDAP book/site, and the ever-useful test suites in the source, I can say there's lots of good documentation out there... but it could always be better.
One item I can highly recommend is that once you convert to RTC format, don't ever touch the files on disk. Oh, you can, for sure, but if you ever think about replication, there's so many lovely ways to shoot your servers it's crazy. Been there, learnt that.
Regarding bootstrap -- the test suites show nicely how minimal it can be -- one slapadd to create the base cn=config and olcDatabase{0}config,cn=config, and then start up the server and do the remaining configuration, including adding the schemas (presuming you've got them in .ldif format already.) One rough edge not so obviously logged is the server (as of 2.4.25) needs a restart after adding an (hdb) database object/branch. Replication has its own pointy edges as well along with TLS; I've spent hours chasing obscure TLS issues due to O/S dependent behaviors.
Regarding schemas -- it's a straightforward perl 1-liner to translate from .schema to .ldif format. Then import and edit via ldapmodify as needed.
Hope this helps,
--Chan
I've seen the guides similar to this one here: http://blogger.ziesemer.com/**2011/01/ldap-authentication-**for-samba.htmlhttp://blogger.ziesemer.com/2011/01/ldap-authentication-for-samba.htmlwhich suggest hacking together a temporary slapd.conf file containing just the include directives, run slaptest, and then hack the output so that it can be loaded into cn=config using ldapadd.
Given that this is a quite a common task, is there no way of generating the LDIF directly to be loaded into the directory, e.g.
slaptest -s /etc/ldap/schema/myschema.**schema [ -n <schemanum> ] -l myschema.ldif
Or then again, is this functionality already there but I just haven't managed to find it yet? I'd be grateful if someone could point me in the right direction and/or give me some hints as to the best way to manage schemas in the new regime.
Many thanks,
Mark.
-- Mark Cave-Ayland - Senior Technical Architect PostgreSQL - PostGIS Sirius Corporation plc - control through freedom http://www.siriusit.co.uk t: +44 870 608 0063
Sirius Labs: http://www.siriusit.co.uk/labs
Chan Wilson wrote:
On Wed, Jun 29, 2011 at 5:26 AM, Mark Cave-Ayland <mark.cave-ayland@siriusit.co.uk mailto:mark.cave-ayland@siriusit.co.uk> wrote:
Hi all, Having started to look at the changes required to migrate from a slapd.conf setup to a cn=config setup, one of things I'm struggling with is how to load new LDAP schemas into cn=config.There's a certain mindset to be achieved in understanding cn=config / RTC, for sure.
I'm happening to be building up an updated config for an N-way multi-master setup, and having the occasion to revisit the OpenLDAP manual, the Zytrax LDAP book/site, and the ever-useful test suites in the source, I can say there's lots of good documentation out there... but it could always be better.
One item I can highly recommend is that once you convert to RTC format, don't ever touch the files on disk. Oh, you can, for sure, but if you ever think about replication, there's so many lovely ways to shoot your servers it's crazy. Been there, learnt that.
Which is one (of many) reasons why I continue to flame people for ever suggesting it.
Let me be clear - I've been administering Unix installations for over 25 years. I am the last person anyone needs to convince about the value of human-readable and editable config files. In fact this was one of my principle requirements for cn=config when it was being developed, and I stated it many times during the early days: http://www.openldap.org/lists/openldap-devel/200503/msg00038.html
Those of you who pipe up about this topic today without having done your homework and read the background of this design are, to be blunt, both ignorant and simply preaching to the choir; neither of which is ever appreciated.
The ability to do manual disaster-recovery/emergency config changes is crucial. But you also have to understand that going this route is for *emergencies* - it is not the way to go for routine administrative tasks.
Regarding bootstrap -- the test suites show nicely how minimal it can be -- one slapadd to create the base cn=config and olcDatabase{0}config,cn=config, and then start up the server and do the remaining configuration, including adding the schemas (presuming you've got them in .ldif format already.)
One rough edge not so obviously logged is the server (as of 2.4.25) needs a restart after adding an (hdb) database object/branch.
That is certainly not true, as test050 does this and much more.
Replication has its own pointy edges as well along with TLS; I've spent hours chasing obscure TLS issues due to O/S dependent behaviors.
Regarding schemas -- it's a straightforward perl 1-liner to translate from .schema to .ldif format. Then import and edit via ldapmodify as needed.
Hope this helps,
--Chan
I've seen the guides similar to this one here: http://blogger.ziesemer.com/__2011/01/ldap-authentication-__for-samba.html <http://blogger.ziesemer.com/2011/01/ldap-authentication-for-samba.html> which suggest hacking together a temporary slapd.conf file containing just the include directives, run slaptest, and then hack the output so that it can be loaded into cn=config using ldapadd. Given that this is a quite a common task, is there no way of generating the LDIF directly to be loaded into the directory, e.g. slaptest -s /etc/ldap/schema/myschema.__schema [ -n <schemanum> ] -l myschema.ldif Or then again, is this functionality already there but I just haven't managed to find it yet? I'd be grateful if someone could point me in the right direction and/or give me some hints as to the best way to manage schemas in the new regime. Many thanks, Mark. -- Mark Cave-Ayland - Senior Technical Architect PostgreSQL - PostGIS Sirius Corporation plc - control through freedom http://www.siriusit.co.uk t: +44 870 608 0063 <tel:%2B44%20870%20608%200063> Sirius Labs: http://www.siriusit.co.uk/labs
The ability to do manual disaster-recovery/emergency config changes is crucial. But you also have to understand that going this route is for *emergencies* - it is not the way to go for routine administrative tasks.
One item I've always wondered about the RTC design -- with all the other details being tracked, where's the checksum of the entry?
And recognizing that emergencies *do* happen, and having a Plan of Action is key to responding to emergencies correctly, what *is* the proper way to manually edit cn=config? Particularly in a replicated RTC setup? I figure it's something like
- shut off slapd - edit the ldif. - remove | edit one or more of the operational attributes - turn on slapd
One
rough edge not so obviously logged is the server (as of 2.4.25) needs a restart after adding an (hdb) database object/branch.
That is certainly not true, as test050 does this and much more.
Oddly, I followed the test050 procedure in creating my bootstraps and based from that did expect it to work.
More accurately, under my particular build of 2.4.25 (based off the RH rpm spec file from 2.x) under Centos5.6 using the -r chroot environment, with db-5.1.19. I found it necessary to restart slapd in order for the database to be properly opened and available for ldapadds.
--Chan
Replication has its own
pointy edges as well along with TLS; I've spent hours chasing obscure TLS issues due to O/S dependent behaviors.
Regarding schemas -- it's a straightforward perl 1-liner to translate from .schema to .ldif format. Then import and edit via ldapmodify as needed.
Hope this helps,
--Chan
I've seen the guides similar to this one here: http://blogger.ziesemer.com/__**2011/01/ldap-authentication-__** for-samba.htmlhttp://blogger.ziesemer.com/__2011/01/ldap-authentication-__for-samba.html <http://blogger.ziesemer.com/**2011/01/ldap-authentication-** for-samba.htmlhttp://blogger.ziesemer.com/2011/01/ldap-authentication-for-samba.html
which suggest hacking together a temporary slapd.conf file containing just the include directives, run slaptest, and then hack the output so that it can be loaded into cn=config using ldapadd.
Given that this is a quite a common task, is there no way of generating the LDIF directly to be loaded into the directory, e.g.
slaptest -s /etc/ldap/schema/myschema.__**schema [ -n <schemanum> ] -l myschema.ldif
Or then again, is this functionality already there but I just haven't managed to find it yet? I'd be grateful if someone could point me in the right direction and/or give me some hints as to the best way to manage schemas in the new regime.
Many thanks,
Mark.
-- Mark Cave-Ayland - Senior Technical Architect PostgreSQL - PostGIS Sirius Corporation plc - control through freedom http://www.siriusit.co.uk t: +44 870 608 0063 tel:%2B44%20870%20608%200063
Sirius Labs: http://www.siriusit.co.uk/labs
-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/**project/http://www.openldap.org/project/
--On Saturday, July 02, 2011 1:36 PM -0500 Chan Wilson chanwilson@gmail.com wrote:
And recognizing that emergencies *do* happen, and having a Plan of Action is key to responding to emergencies correctly, what *is* the proper way to manually edit cn=config? Particularly in a replicated RTC setup? I figure it's something like
- shut off slapd
- edit the ldif.
- remove | edit one or more of the operational attributes - turn on slapd
slapmodify would be the correct thing to use (Currently only in HEAD).
Oddly, I followed the test050 procedure in creating my bootstraps and based from that did expect it to work.
I do this routinely to set up replication, and it works fine for me without a restart. That's with 2.4.21, 2.4.23, 2.4.25 and 2.4.26.
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
Chan Wilson wrote:
The ability to do manual disaster-recovery/emergency config changes is crucial. But you also have to understand that going this route is for *emergencies* - it is not the way to go for routine administrative tasks.One item I've always wondered about the RTC design -- with all the other details being tracked, where's the checksum of the entry?
See 2.4.26.
openldap-technical@openldap.org
-
Buchan Milne -
Chan Wilson -
Chris Card -
Christopher Wood -
Daniel Qian -
Gavin Henry -
Howard Chu -
Mark Cave-Ayland -
Michael Ströder -
Nick Milas -
Ondrej Kuznik -
Quanah Gibson-Mount -
Simone Piccardi