Hello,
Why , when we use the back-sql , the search performance reduces. Is it because , all operations to database is through odbc manager
Can I get some more details regarding the above?
Thanks in advance,
Arunachalam.
**************************************************************************** **************************** This e-mail and attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient's) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it!
Why , when we use the back-sql , the search performance reduces.
Compared to back-bdb/back-hdb? Because SQL databases can't hold a candle to the performance you get via these backends. They are just faster.
Is it because , all operations to database is through odbc manager
No. And the ODBC manager has little to no effect on performance. Your specific ODBC driver does, so it depends on the quality and version of your ODBC driver. And you should check that the queries back-sql is generated aren't just slow. Capture then and use EXPLAIN (or your databases equivalent) to analyze performance [as with any database application that seems too slow].
Can I get some more details regarding the above?
This is so loaded. Postgresql is entirely configurable and in the long run, when you're dealing with hundred's of thousands of records, postgresql is going to out perform any local text based db file.
If you're postgresql database system is configured correctly you'll see no difference with small amounts of records between the two methods, it's when your data becomes significantly large, that postgresql will blow away the local db file.
On Fri, 22 Jun 2007, Adam Tauno Williams wrote:
Why , when we use the back-sql , the search performance reduces.
Compared to back-bdb/back-hdb? Because SQL databases can't hold a candle to the performance you get via these backends. They are just faster.
Is it because , all operations to database is through odbc manager
No. And the ODBC manager has little to no effect on performance. Your specific ODBC driver does, so it depends on the quality and version of your ODBC driver. And you should check that the queries back-sql is generated aren't just slow. Capture then and use EXPLAIN (or your databases equivalent) to analyze performance [as with any database application that seems too slow].
Can I get some more details regarding the above?
This is so loaded. Postgresql is entirely configurable and in the long
Thanks, been a PostgreSQL DBA for a decade now.
run, when you're dealing with hundred's of thousands of records, postgresql is going to out perform any local text based db file.
Only back-hdb/back-dbd are *NOT* "local text based db file". They use Berkley DB, and *YES* they are faster than PostgreSQL. Period. OpenLDAP back-hdb is about the fastest thing you can find, anywhere.
Of course, back-hdb/back-dbd don't need to support every crazy thing that a full-blown SQL RDBMS does.
If you're postgresql database system is configured correctly you'll see no difference with small amounts of records between the two methods,
Disagree; BUT performance can be entirely acceptable. You also loosing performance in the transport between the driver and the database.
it's when your data becomes significantly large, that postgresql will blow away the local db file.
No, it won't. If your OpenLDAP database gets slow as its DB gets big then you need to attend to your DB_CONFIG file.
gonzales@linuxlouis.net wrote:
This is so loaded. Postgresql is entirely configurable and in the long run, when you're dealing with hundred's of thousands of records, postgresql is going to out perform any local text based db file.
If you're postgresql database system is configured correctly you'll see no difference with small amounts of records between the two methods, it's when your data becomes significantly large, that postgresql will blow away the local db file.
Do you have any figures (e.g. fair benchmarks) to support your words? Do you have any direct experience on the topic, or are you just reporting boatos?
p.
Ing. Pierangelo Masarati OpenLDAP Core Team
SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati@sys-net.it ---------------------------------------
That's funny that you would ask that, seeing as how the initial statements had no such quantification either... did I make my point?
On Fri, 22 Jun 2007, Pierangelo Masarati wrote:
gonzales@linuxlouis.net wrote:
This is so loaded. Postgresql is entirely configurable and in the long run, when you're dealing with hundred's of thousands of records, postgresql is going to out perform any local text based db file.
If you're postgresql database system is configured correctly you'll see no difference with small amounts of records between the two methods, it's when your data becomes significantly large, that postgresql will blow away the local db file.
Do you have any figures (e.g. fair benchmarks) to support your words? Do you have any direct experience on the topic, or are you just reporting boatos?
p.
Ing. Pierangelo Masarati OpenLDAP Core Team
SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it
Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati@sys-net.it
gonzales@linuxlouis.net wrote:
That's funny that you would ask that, seeing as how the initial statements had no such quantification either... did I make my point?
For a general discussion of why it is unlikely that an RDBMS based database can possibly outperform an embedded database, see http://www.openldap.org/faq/data/cache/378.html.
For real numbers, and comparisons with other products, see, for example http://www.symas.com/benchmark.shtml and http://www.openldap.org/lists/openldap-software/200705/msg00014.html.
Obviously, you can object, there's no comparison with any Directory Server based on an RDBMS data store.
p.
Ing. Pierangelo Masarati OpenLDAP Core Team
SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati@sys-net.it ---------------------------------------
No kidding! Embedded database systems - 'local' files only with no other overhead of course, from the superficial standpoint will 'out perform' another DBMS that is being used in a client/server mode. These are obvious facts. Once the SQL engine receives the data in an RDBMS vs. the embedded database format, then you can start comparing apples to apples.
Based on what I said initially, looks like there are many liberties being taken to what was implied. Of course I was thinking that we would all be agreeing on the context, which didn't happen; that said, I would like to state clearer, that when in a client/server model naturally there are going to be greater overhead in process boundaries and naming contexts that will not be able to compete with an embedded database. They each have their own market of applicability. For contexts sake from the moment of 'true' processing SQL engine - once the RDBMS system has received it's data - vs. local embedded DBMS, this brings the context and comparison much closer. I too can go out and google benchmarks, but if you want to impress me with truth and facts, let me know how we can bencharmk engine vs. engine, not a Hummer running down the road at 85mph with air resistance for mpg, vs. an engine sitting on a concete block running for same mpg performance... c'mon.
On Fri, 22 Jun 2007, Pierangelo Masarati wrote:
gonzales@linuxlouis.net wrote:
That's funny that you would ask that, seeing as how the initial statements had no such quantification either... did I make my point?
For a general discussion of why it is unlikely that an RDBMS based database can possibly outperform an embedded database, see http://www.openldap.org/faq/data/cache/378.html.
For real numbers, and comparisons with other products, see, for example http://www.symas.com/benchmark.shtml and http://www.openldap.org/lists/openldap-software/200705/msg00014.html.
Obviously, you can object, there's no comparison with any Directory Server based on an RDBMS data store.
p.
Ing. Pierangelo Masarati OpenLDAP Core Team
SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it
Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati@sys-net.it
You brought the discussion on this ground. As soon as you manage to "google out" something real, I mean: something so "superficial" that has full LDAP semantics and that people can use to look up directories and authenticate with better performances, please let us know.
p.
gonzales@linuxlouis.net wrote:
No kidding! Embedded database systems - 'local' files only with no other overhead of course, from the superficial standpoint will 'out perform' another DBMS that is being used in a client/server mode. These are obvious facts. Once the SQL engine receives the data in an RDBMS vs. the embedded database format, then you can start comparing apples to apples.
Based on what I said initially, looks like there are many liberties being taken to what was implied. Of course I was thinking that we would all be agreeing on the context, which didn't happen; that said, I would like to state clearer, that when in a client/server model naturally there are going to be greater overhead in process boundaries and naming contexts that will not be able to compete with an embedded database. They each have their own market of applicability. For contexts sake from the moment of 'true' processing SQL engine - once the RDBMS system has received it's data - vs. local embedded DBMS, this brings the context and comparison much closer. I too can go out and google benchmarks, but if you want to impress me with truth and facts, let me know how we can bencharmk engine vs. engine, not a Hummer running down the road at 85mph with air resistance for mpg, vs. an engine sitting on a concete block running for same mpg performance... c'mon.
On Fri, 22 Jun 2007, Pierangelo Masarati wrote:
gonzales@linuxlouis.net wrote:
That's funny that you would ask that, seeing as how the initial statements had no such quantification either... did I make my point?
For a general discussion of why it is unlikely that an RDBMS based database can possibly outperform an embedded database, see http://www.openldap.org/faq/data/cache/378.html.
For real numbers, and comparisons with other products, see, for example http://www.symas.com/benchmark.shtml and http://www.openldap.org/lists/openldap-software/200705/msg00014.html.
Obviously, you can object, there's no comparison with any Directory Server based on an RDBMS data store.
p.
Ing. Pierangelo Masarati OpenLDAP Core Team
SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it
Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati@sys-net.it
Ing. Pierangelo Masarati OpenLDAP Core Team
SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati@sys-net.it ---------------------------------------
(top-posting as seems to be the style of this thread)
If your context was that back-sql would be faster than back-passwd or back-ldif on a large dataset, I think you are right. Fortunately, bdb can rush into memory and go really fast. Or even if your database is larger than your memory (as in my case) it can "index" into memory and go pretty fast. This can probably be achieved with an rdbms, but would require a lot of tuning and work on the part of the dba. Good luck trying to get there. :)
On 6/22/07, gonzales@linuxlouis.net gonzales@linuxlouis.net wrote:
No kidding! Embedded database systems - 'local' files only with no other overhead of course, from the superficial standpoint will 'out perform' another DBMS that is being used in a client/server mode. These are obvious facts. Once the SQL engine receives the data in an RDBMS vs. the embedded database format, then you can start comparing apples to apples.
Based on what I said initially, looks like there are many liberties being taken to what was implied. Of course I was thinking that we would all be agreeing on the context, which didn't happen; that said, I would like to state clearer, that when in a client/server model naturally there are going to be greater overhead in process boundaries and naming contexts that will not be able to compete with an embedded database. They each have their own market of applicability. For contexts sake from the moment of 'true' processing SQL engine - once the RDBMS system has received it's data - vs. local embedded DBMS, this brings the context and comparison much closer. I too can go out and google benchmarks, but if you want to impress me with truth and facts, let me know how we can bencharmk engine vs. engine, not a Hummer running down the road at 85mph with air resistance for mpg, vs. an engine sitting on a concete block running for same mpg performance... c'mon.
On Fri, 22 Jun 2007, Pierangelo Masarati wrote:
gonzales@linuxlouis.net wrote:
That's funny that you would ask that, seeing as how the initial statements had no such quantification either... did I make my point?
For a general discussion of why it is unlikely that an RDBMS based database can possibly outperform an embedded database, see http://www.openldap.org/faq/data/cache/378.html.
For real numbers, and comparisons with other products, see, for example http://www.symas.com/benchmark.shtml and http://www.openldap.org/lists/openldap-software/200705/msg00014.html.
Obviously, you can object, there's no comparison with any Directory Server based on an RDBMS data store.
p.
Ing. Pierangelo Masarati OpenLDAP Core Team
SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it
Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati@sys-net.it
-- Louis Gonzales louis.gonzales@linuxlouis.net http://www.linuxlouis.net
openldap-software@openldap.org
-
Adam Tauno Williams -
Arunachalam Parthasarathy -
gonzales＠linuxlouis.net -
matthew sporleder -
Pierangelo Masarati