Hello list,
I was wondering lately if I could use OpenLDAP as a 'generic' hierarchical data store for the application I'm planning, and tried to get any hint about feasibility. I found a blog post about OpenDS (http://blogs.sun.com/treydrake/entry/ldap_vs_relational_database), where the author states about Read/write performance parity ... That was the kind of news I just wanted to hear because I'm thinking about a persistence back-end, not just an almost static dataset, but I'm in a genuine PHP environment and Java is not an option; so I naturally tried to get some infos regarding OpenLDAP performance but found nothing recent, and the references I could find are quite discouraging on that matter ...
However since 4 to 8 years have passed since the tests I've been reading, I'm still in the hope of a 'a pleasant surprise' as the sun guy sayz ...
Does someone has an experience to share about that kind of (mis-)usage of OpenLDAP ?
Any pointers to some good and recent reference on the matter are of course well come.
Thanks in advance for any information.
LP
Lorenzo Pastrana - Happy End Vision -------------------------- Design web Conception multimédia Communication visuelle et édition -------------------------- Tél. : 01 42 47 83 09 Fax : 01 47 70 70 19 E-mail : lorenzo.pastrana@happyend.fr
--On Tuesday, December 09, 2008 7:18 PM +0100 Lorenzo Pastrana lorenzo.pastrana@happyend.fr wrote:
Hello list,
I was wondering lately if I could use OpenLDAP as a 'generic' hierarchical data store for the application I'm planning, and tried to get any hint about feasibility. I found a blog post about OpenDS (http://blogs.sun.com/treydrake/entry/ldap_vs_relational_database), where the author states about Read/write performance parity ... That was the kind of news I just wanted to hear because I'm thinking about a persistence back-end, not just an almost static dataset, but I'm in a genuine PHP environment and Java is not an option; so I naturally tried to get some infos regarding OpenLDAP performance but found nothing recent, and the references I could find are quite discouraging on that matter ...
However since 4 to 8 years have passed since the tests I've been reading, I'm still in the hope of a 'a pleasant surprise' as the sun guy sayz ...
Does someone has an experience to share about that kind of (mis-)usage of OpenLDAP ?
Any pointers to some good and recent reference on the matter are of course well come.
I suggest you read the material available on the OpenLDAP website. There's no other LDAP server that I've found that performs as well as OpenLDAP, period.
http://www.openldap.org/pub/hyc/scale2007.pdf
You may also wish to read:
http://www.symas.com/benchmark.shtml
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration
Quanah Gibson-Mount wrote:
--On Tuesday, December 09, 2008 7:18 PM +0100 Lorenzo Pastrana lorenzo.pastrana@happyend.fr wrote:
Hello list,
I was wondering lately if I could use OpenLDAP as a 'generic' hierarchical data store for the application I'm planning, and tried to get any hint about feasibility. I found a blog post about OpenDS (http://blogs.sun.com/treydrake/entry/ldap_vs_relational_database), where the author states about Read/write performance parity ... That was the kind of news I just wanted to hear because I'm thinking about a persistence back-end, not just an almost static dataset, but I'm in a genuine PHP environment and Java is not an option; so I naturally tried to get some infos regarding OpenLDAP performance but found nothing recent, and the references I could find are quite discouraging on that matter ...
However since 4 to 8 years have passed since the tests I've been reading, I'm still in the hope of a 'a pleasant surprise' as the sun guy sayz ...
Does someone has an experience to share about that kind of (mis-)usage of OpenLDAP ?
Any pointers to some good and recent reference on the matter are of course well come.
I suggest you read the material available on the OpenLDAP website. There's no other LDAP server that I've found that performs as well as OpenLDAP, period.
http://www.openldap.org/pub/hyc/scale2007.pdf
You may also wish to read:
Let me warn you that LDAP support in PHP is poor and, AFAIK, little maintained, not to mention development.
p.
Ing. Pierangelo Masarati OpenLDAP Core Team
SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it ----------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Fax: +39 0382 476497 Email: ando@sys-net.it -----------------------------------
Let me warn you that LDAP support in PHP is poor and, AFAIK, little
maintained, not to mention development.
Beeing a complete new comer I'm not sure to get how 'poor' you feel PHP support is ...
Could yo ube a little more specific ?
Does this mean almost unusable / broken ? Or does this mean so archaic / low level you can't use it without spending some time baking a more coder friendly API ?
Thx LP
Lorenzo Pastrana - Happy End Vision -------------------------- Design web Conception multimédia Communication visuelle et édition -------------------------- Tél. : 01 42 47 83 09 Fax : 01 47 70 70 19 E-mail : lorenzo.pastrana@happyend.fr
Lorenzo Pastrana wrote:
Let me warn you that LDAP support in PHP is poor and, AFAIK, little
maintained, not to mention development.
Beeing a complete new comer I'm not sure to get how 'poor' you feel PHP support is ...
Could yo ube a little more specific ?
Does this mean almost unusable / broken ? Or does this mean so archaic / low level you can't use it without spending some time baking a more coder friendly API ?
- you can't perform LDAP modify operations on multiple operations with multiple operation types (e.g. combinations of add, replace, delete); this is crucial for atomic modifications - braindead handling of binary vs. string data - limited, undocumented access to response structure - no support for asynchronous operations - no support for LDAPv3 extensibility (exops, controls)
Note that most of the above has always been available in a decent manner in the corresponding C API.
p.
Ing. Pierangelo Masarati OpenLDAP Core Team
SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it ----------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Fax: +39 0382 476497 Email: ando@sys-net.it -----------------------------------
- you can't perform LDAP modify operations on multiple operations with
multiple operation types (e.g. combinations of add, replace, delete); this is crucial for atomic modifications
I've been reading much more lately so I can weight this answer correctly now ... :(
What do you think could be a solution to (auto)incremented id then ? Would there be an alternative to the use of incremented ids ? Is my brain stuck in RDBMS mode ?
Thanks.
Lorenzo Pastrana - Happy End Vision -------------------------- Design web Conception multimédia Communication visuelle et édition -------------------------- Tél. : 01 42 47 83 09 Fax : 01 47 70 70 19 E-mail : lorenzo.pastrana@happyend.fr
On Tue, Dec 30, 2008 at 04:46:32PM +0100, Lorenzo Pastrana wrote:
- you can't perform LDAP modify operations on multiple operations with
multiple operation types (e.g. combinations of add, replace, delete); this is crucial for atomic modifications
I've been reading much more lately so I can weight this answer correctly now ... :(
Rember that Pierangelo was referring to specific limitations of the PHP interface to LDAP, not the LDAP protocol itself.
What do you think could be a solution to (auto)incremented id then ? Would there be an alternative to the use of incremented ids ? Is my brain stuck in RDBMS mode ?
It depends on what you are really trying to do. LDAP does not have things like autoincremented IDs (although you could probably write a plugin for a specific LDAP server to do it if you really wanted to). On the other hand, LDAP does provide a way to get safe incrementing values. To do this, you store the value in a designated entry and increment it like this:
repeat read current value ( delete current value; add current value + 1 ) until success
This works because you do the delete and add in a single LDAP operation, which will succeed or fail as a whole. This is the only atomic operation available in standard LDAP.
Pierangelo's comment (quoted above) is saying that the PHP interface to LDAP does not allow you to express the delete and add as a single operation, so you cannot do this from PHP.
There may be other ways to achieve what you want. If for example you just want to generate a unique name for a new entry and it does not matter what it is, you could try this:
repeat generate random entry name create entry until success
If the random name is sufficiently long and sufficiently random then this is a much more efficient method than trying to maintain a safely-incrementing counter to generate names from.
Andrew
Andrew Findlay wrote:
On Tue, Dec 30, 2008 at 04:46:32PM +0100, Lorenzo Pastrana wrote:
- you can't perform LDAP modify operations on multiple operations with
multiple operation types (e.g. combinations of add, replace, delete); this is crucial for atomic modifications
I've been reading much more lately so I can weight this answer correctly now ... :(
Rember that Pierangelo was referring to specific limitations of the PHP interface to LDAP, not the LDAP protocol itself.
What do you think could be a solution to (auto)incremented id then ? Would there be an alternative to the use of incremented ids ? Is my brain stuck in RDBMS mode ?
It depends on what you are really trying to do. LDAP does not have things like autoincremented IDs (although you could probably write a plugin for a specific LDAP server to do it if you really wanted to). On the other hand, LDAP does provide a way to get safe incrementing values. To do this, you store the value in a designated entry and increment it like this:
repeat read current value ( delete current value; add current value + 1 ) until success
This works because you do the delete and add in a single LDAP operation, which will succeed or fail as a whole. This is the only atomic operation available in standard LDAP.
Pierangelo's comment (quoted above) is saying that the PHP interface to LDAP does not allow you to express the delete and add as a single operation, so you cannot do this from PHP.
There may be other ways to achieve what you want. If for example you just want to generate a unique name for a new entry and it does not matter what it is, you could try this:
repeat generate random entry name create entry until success
If the random name is sufficiently long and sufficiently random then this is a much more efficient method than trying to maintain a safely-incrementing counter to generate names from.
My comment also applies to the fact that were the interface providing extensions like the modify increment extension (RFC4525) and the LDAP (postread) Read Entry control (RFC4527), one could atomically increment a counter and read back the incremented value.
p.
Ing. Pierangelo Masarati OpenLDAP Core Team
SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it ----------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Fax: +39 0382 476497 Email: ando@sys-net.it -----------------------------------
On Thu, 2009-01-08 at 19:44 +0000, Andrew Findlay wrote:
On Tue, Dec 30, 2008 at 04:46:32PM +0100, Lorenzo Pastrana wrote:
- you can't perform LDAP modify operations on multiple operations with
multiple operation types (e.g. combinations of add, replace, delete); this is crucial for atomic modifications
I've been reading much more lately so I can weight this answer correctly now ... :(
Rember that Pierangelo was referring to specific limitations of the PHP interface to LDAP, not the LDAP protocol itself.
Yep and that's my specific problem here ... I perfectly understood how this is done thru LDAP protocol but, my app is still stuck in PHP land :(
So, what I understood is PHP guys never built a multi-user writable DIT, so that they never faced the necessity of an atomic increment or flag setter.
That's a pity ...
Lo.
Lorenzo Pastrana - Happy End Vision -------------------------- Web Shop Multimédia Design Visual Communication & Publishing -------------------------- Tél.: +33 1 42 47 83 09 Fax.: +33 1 47 70 70 19 E-mail : lorenzo.pastrana@happyend.ff
Hi Lorenzo,
There are many companies that are developing product that uses directory servers as a 'generic' hierarchical data store. Not later than yesterday I was discussing with a company that replaced a SQL database with an LDAP directory server (not OpenLDAP though). The main benefit for them was to have the replication of the data in all of their boxes, simplifying their development and the deployments for their customers. They are even considering leveraging replication as a signaling mechanisms for specific events between the different components of their complex system.
Regards,
Ludovic.
On Dec 9, 2008, at 7:18 PM, Lorenzo Pastrana wrote:
Hello list,
I was wondering lately if I could use OpenLDAP as a 'generic' hierarchical data store for the application I'm planning, and tried to get any hint about feasibility. I found a blog post about OpenDS (http://blogs.sun.com/treydrake/entry/ldap_vs_relational_database), where the author states about Read/write performance parity ... That was the kind of news I just wanted to hear because I'm thinking about a persistence back-end, not just an almost static dataset, but I'm in a genuine PHP environment and Java is not an option; so I naturally tried to get some infos regarding OpenLDAP performance but found nothing recent, and the references I could find are quite discouraging on that matter ...
However since 4 to 8 years have passed since the tests I've been reading, I'm still in the hope of a 'a pleasant surprise' as the sun guy sayz ...
Does someone has an experience to share about that kind of (mis-)usage of OpenLDAP ?
Any pointers to some good and recent reference on the matter are of course well come.
Thanks in advance for any information.
LP
Lorenzo Pastrana - Happy End Vision
Design web Conception multimédia Communication visuelle et édition
Tél. : 01 42 47 83 09 Fax : 01 47 70 70 19 E-mail : lorenzo.pastrana@happyend.fr
Ludovic Poitou Sun Microsystems Inc. OpenDS Community Lead Directory Services http://blogs.sun.com/Ludo/ Grenoble Engineering Center - France
Sun Microsystems requires the following notice: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ NOTICE: This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
openldap-technical@openldap.org