openldap-devel

openldap-devel@openldap.org

1418 discussions

Fwd: [Berkeley DB Announce] Berkeley DB 11gR2 (11.2.5.0.21) - New Release, Major Revision
by Howard Chu 01 Apr '10

01 Apr '10

Looks like the major addition is the SQLite work-alike API. Not much of interest to us here, possibly some of the lock bugs they fixed may be relevant. -------- Original Message -------- Subject: [Berkeley DB Announce] Berkeley DB 11gR2 (11.2.5.0.21) - New Release, Major Revision Date: Wed, 31 Mar 2010 14:17:56 -0700 (PDT) From: Announcements related to the Berkeley DB product family. <bdb(a)oracle.com> Reply-To: bdb(a)oss.oracle.com To: bdb(a)oss.oracle.com Berkeley DB 11gR2 (11.2.5.0.21) http://www.oracle.com/database/berkeley-db/ http://www.oracle.com/technology/products/berkeley-db/ http://www.oracle.com/technology/products/berkeley-db/sql.html Berkeley DB 11gR2 (11.2.5.0.21) is a major new release. Berkeley DB now includes a SQL API for relational data storage based on SQLite. This release uses the Oracle standard marketing naming (11gR2), however the revision number's last three components map to the historic versioning pattern (5.0.21) to help maintain consistency. NEW IN THIS RELEASE: * SQL * SQL/Relational access via integration with SQLite * Support for SQL92, ODBC and JDBC * Drop-in replacement for any SQLite-based solution * New Documentation covering the SQL API * SQL Performance * 10s of thousands of INSERTS/UPDATES/DELETES per-second * 100s of thousands of SELECTs per-second * Approximately as fast as SQLite for reads (within 10%) * About 3X faster than SQLite for updates/writes * Has fine grained locking which leads to better read/write concurrency * 6-8x more transactions/second compared to SQLite when using multiple threads * Other Features * Support for DTrace/SystemTap performance monitoring * Automatic resource management: close open database and cursor handles on environment close * Transaction priorities * Compact to reclaim unused spaces in hash databases to match similar existing feature for btrees * Log file portability across different endian machines * High Availability * Client-to-client synchronization * Read-after-write consistency on replicas * Integration with other Oracle Products * Synchronization with Oracle Database Server using Oracle Mobile Sync Server The complete list of changes can also be found in the change log page. http://www.oracle.com/technology/documentation/berkeley-db/db/installation/… QUOTES: "Oracle Berkeley DB 11gR2 combines seamlessly the benefits of SQLite's ubiquitous API with Berkeley DB's concurrency and performance into an easy-to-use product. SQLite users can now benefit from enhanced concurrency and performance, and Oracle Berkeley DB users can now benefit from an easy-to-use SQL API." - Dr. Richard Hipp, architect, SQLite "Berkeley DB's new SQLite-compatible SQL API offers powerful features not only for its users but to the SQLite community at large. Berkeley DB developers now have a proven embedded SQL engine to make application development even easier. SQLite users now have the option of a powerful, industry-supported storage engine that is built from the ground up for high-concurrency, and which includes powerful features like built-in replication and hot backups. The combination of the two technologies provides all developers with tremendous flexibility in that a single SQL/API can now be used across a broad spectrum of applications ranging from the embedded space all the way up to large-scale transaction processing. Oracle has done a fantastic job of not only integrating the technologies, but also being an exemplary participant in the open source community by contributing time, expertise and resources to the SQLite project." - Mike Owens, author of "The Definitive Guide to SQLite" "Berkeley DB 11gR2 is a very interesting product mainly because it combines the familiar SQLite sqlite3() API with the powerful, proven Berkeley DB data storage technology. I was able to re-compile my SQLiteManager product against the Berkeley DB library without any major issues in very little time. I have now a product that is fully able to manage Berkeley DB databases without changing a single line of my code. So far performance and concurrency are excellent and I am thrilled to think about the future improvements of this great product." - Marco Bambini, CEO of SQLabs "Berkley DB 11gR2 is, in my experience, a fully drop-in replacement for SQLite3. I was hugely surprised at just how easy it was; I'd had zero experience in using Berkeley DB in products before, but within a couple hours I had my application powered by Berkeley DB. From there onwards, you can enjoy all the benefits of Berkeley DB including concurrency and long term stability compared to SQLite3 from an API you're already familiar with. It's quite simply amazing." - Liam McLoughlin, Chromium OS independent developer DOWNLOAD: http://www.oracle.com/technology/software/products/berkeley-db http://www.oracle.com/technology/software/products/berkeley-db/db Berkeley DB .tar.gz, with AES encryption http://download.oracle.com/berkeley-db/db-5.0.21.tar.gz Berkeley DB .zip, with AES encryption http://download.oracle.com/berkeley-db/db-5.0.21.zip Berkeley DB .msi Windows installer, with AES encryption http://download.oracle.com/berkeley-db/db-5.0.21.msi Berkeley DB NC.tar.gz , without encryption http://download.oracle.com/berkeley-db/db-5.0.21.NC.tar.gz Berkeley DB .NC.zip, without encryption http://download.oracle.com/berkeley-db/db-5.0.21.NC.zip EDUCATION: Documentation page: http://www.oracle.com/technology/documentation/berkeley-db/db You may find an answer on our product FAQ page: http://www.oracle.com/technology/products/berkeley-db/faq/db_faq.html Join OTN today and participate in the Berkeley DB Support Forum and discussions. General Questions http://forums.oracle.com/forums/forum.jspa?forumID=271 Questions on HA/Replication http://forums.oracle.com/forums/forum.jspa?forumID=272 For further information, please contact us at: mailto:berkeleydb-info_us@oracle.com Thank you for your support of Berkeley DB. -greg Greg Burd | Product Manager | Oracle Berkeley DB _______________________________________________ BDB mailing list BDB(a)oss.oracle.com http://oss.oracle.com/mailman/listinfo/bdb

1 0

User-level network stack
by Howard Chu 30 Mar '10

30 Mar '10

Quite a while back I talked about optimizing select/epoll/whatever using mmap'd memory shared between user space and the kernel for the event table. http://www.openldap.org/lists/openldap-devel/200411/msg00088.html These guys have gone even further, and use mapped memory to bring the entire network stack up to user level: http://www.openonload.org/openonload-google-talk.pdf Their paper talks about the work they did with two particular 10Gbit NICs. I find this interesting because in these benchmarks I ran http://connexitor.com/blog/pivot/entry.php?id=191 the NIC interrupt overhead was 100% of a CPU core, and that was only on 1Gbit ethernet. Clearly for heavy-load deployments a better network solution is needed. It sounds like maybe the OpenOnload guys have a better approach. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

logging "reqEntryUUID" in accesslog?
by Michael Ströder 29 Mar '10

29 Mar '10

HI! In web2ldap when displaying a single entry I have a button which directly forms a filter for searching the audit log (accesslog DB) for the entry based on the attribute 'reqDN'. This is handy but does not work anymore when entries were renamed. So how about logging the value of entryUUID of an entry affected by a write operation in a separate attribute 'reqEntryUUID'? Ciao, Michael.

2 2

pcache logging
by masarati＠aero.polimi.it 26 Mar '10

26 Mar '10

I'd like to enhance pcache logging by adding the conn= op= prefix. Objections? p.

2 1

liblutil/ultils.c: lutil_localtime() <g-differential> timestamp format bug?
by Daniel 20 Mar '10

20 Mar '10

Hi, there seems to be a small problem in "lutil_localtime()" regarding the creation of <g-differential> timestamp formated bervals: The returned length "ret" (of type: size_t) of a timestamp value in <g-differential> format contains the terminating "\0" which leads to base64 encoded presentation... For demonstration I've attached a minimal (and probably very dirty) patch which fixed this issue for me, but I'm quite unsure whether: - it's really a bug or I'm doing something completely wrong - my patch represents a clean solution. Therefore I've not opened an ITS. First of all I would appreciate your valuable feedback, whether lutil_localtime() creates <g-differential> timestamps like intended? Thanks a lot! Best regards Daniel Index: libraries/liblutil/utils.c =================================================================== RCS file: /repo/OpenLDAP/pkg/ldap/libraries/liblutil/utils.c,v retrieving revision 1.70 diff -u -r1.70 utils.c --- libraries/liblutil/utils.c 20 Nov 2009 19:43:50 -0000 1.70 +++ libraries/liblutil/utils.c 20 Mar 2010 17:15:12 -0000 @@ -147,7 +147,7 @@ snprintf( p, smax - 15, "%02ld%02ld", delta / 3600, ( delta % 3600 ) / 60 ); - return ret + 5; + return ret + 4; } int lutil_tm2time( struct lutil_tm *tm, struct lutil_timet *tt )

1 0

Altering the behavior of pcache to improve cache hit/miss ratio
by Ryan Steele 10 Mar '10

10 Mar '10

Hey folks, I've been playing with pcache for about a week (in combination with back-ldap), testing various things and trying to create a configuration that helps preserve system stability if access to the LDAP server disappears. Many things seem to work well, but I noticed that quite a good number of Unix commands, like "id", request a user's full entry (i.e., no filter string). In its current implementation, this will always fail to hit the cache, because (as the man page states) you cannot specify an empty list of attributes. This means that even if you have all the information required to return what was requested in the local cache, you will never be able to leverage that information. Given that, I wanted to ask if it could be considered to change the design of the overlay a bit, such that if an empty attribute list is specified, pcache will return every attribute *in the cache* for that entry, instead of blindly assuming it doesn't have the information to accomplish the task. I understand the shortcomings this could introduce, most notably if one did not want to cache every attribute for a particular entry and still wanted to be able to omit the filter string and get the entry's full attribute list, but I think the benefits would make it worthwhile. I say that for a few reasons: if you're using pcache and are interested in truly optimizing performance or keeping a system that is LDAP-dependent from having major issues if the network disappears, it would make sense to cache every attribute for entries used in common system tasks (e.g., using utilities like "id"). I believe this could also significantly improve the hit/miss ratios, especially if you take a look at how many of these common system utilities make queries without specifying any kind of filter. I would think that this could be accomplished by adding a case for the empty filter string, such that it would take a unison of each set of attributes corresponding to the cached entry for that particular LDAP filter, and returning the result. If the same attribute is present in more than one cache entry matching the specified LDAP filter, the most recent would be favored and returned, thereby preventing pcache from having choose between several query-string-dependent entries for that entry, if more than one are present. I'm interested to hear what you guys think of this. Even if my implementation ideas aren't 100% spot-on, I would think that the wiser minds could adapt the general idea in to something more artful, provided they agree with the idea in principle. Thanks! -Ryan

2 2

Re: commit: openldap-guide/admin backends.sdf overlays.sdf
by Gavin Henry 06 Mar '10

06 Mar '10

----- hyc(a)OpenLDAP.org wrote: > Update of /repo/OpenLDAP/pkg/openldap-guide/admin > > Modified Files: > backends.sdf 1.17 -> 1.18 > overlays.sdf 1.47 -> 1.48 > > Log Message: > ITS#6486 - avoid explicit moduleload in examples > > > CVS Web URLs: > > http://www.openldap.org/devel/cvsweb.cgi/admin/?cvsroot=OpenLDAP-guide > > http://www.openldap.org/devel/cvsweb.cgi/admin/backends.sdf?cvsroot=OpenLDA… > > http://www.openldap.org/devel/cvsweb.cgi/admin/overlays.sdf?cvsroot=OpenLDA… > Thanks! -- Kind Regards, Gavin Henry. OpenLDAP Engineering Team. E ghenry(a)OpenLDAP.org Community developed LDAP software. http://www.openldap.org/project/

1 0

back-config delete support (syncprov overlay)
by Ralf Haferkamp 02 Mar '10

02 Mar '10

Hi, while taking up some loose ends on my work on delete support for back- config (enabled with -DSLAP_CONFIG_DELETE) I wondered how we should deal with the deletion of the syncprov overlay when there are active refreshAndPersist sessions. What errorcode should we sent when closing such a connection? To me LDAP_UNAVAILABLE sound like the best choice. LDAP_UNWILLING_TO_PERFORM might also be ok. -- Ralf

4 9

SASL OTP and syncrepl
by manu＠netbsd.org 21 Feb '10

21 Feb '10

Hello After exchanging a few private messages with Pierangelo Masarati, I just posted ITD#6475: > When binding using SASL OTP to a replica, the bind works, but the > cmusaslsecretOTP attribute is modified on the replica and fail to be > propagated to the master. On the next modification, the master will > overwrite the replica's updated cmusaslsecretOTP value. > > Here is a script that exhibit the behaviour: > ftp://ftp.openldap.org/incoming/ldapotp.tgz > That require SASL enabled OpenLDAP, with the OTP plugin installed. The > PATH in run.sh must probably be adjusted. The problem is in sasl_auxprop_store(), who bypass the replication process. The easier fix to me seems to send a referal to the master on any SASL OTP bind, Any other idea? -- Emmanuel Dreyfus http://hcpnet.free.fr/pubz manu(a)netbsd.org

6 8

syncprov mmr propagation
by Howard Chu 02 Feb '10

02 Feb '10

Currently we still send too many updates in a fully connected mesh. The provider currently knows not to resend an update to the originating server, and also not to resend an update to the server that just relayed the update to it. We should also filter out updates that we know another server is likely to receive from an alternate path. E.g. in a 4-way mesh: 1 - 2 |\ /| | X | |/ \| 3 - 4 When server 1 receives an update from a client, it will propagate it to each of 2, 3, and 4. These servers will also attempt to propagate it to each other: 2 to 3,4; 3 to 2,4; and 4 to 2,3. If these further updates are not quite simultaneous, it is possible that one or two redundant updates will be pruned out given the existing code. But that's just based on luck and variations in server and network load. Once any node has received any updates from a given sid, it should be able to remember which neighbors have sent it identical updates, and filter them out from further propagation attempts. Of course, this may turn out to be a bad idea; if one of the nodes loses one connection, it might stop receiving updates altogether even though other servers could forward to it. I guess we can't protect against this case without at least some redundant messages still being generated. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-devel