Does anyone have opinions about questions 2 and 3?
Thanks
----- Forwarded by Mark Cooper/Poughkeepsie/IBM on 08/01/2013 10:27 PM -----
From: Mark Cooper/Poughkeepsie/IBM To: Howard Chu hyc@symas.com, Cc: openldap-technical@openldap.org, openldap-technical-bounces@openldap.org Date: 07/30/2013 09:58 PM Subject: Re: OpenLDAP (using BDB) stalls adding 65,536th entry
Not sure exactly how I am supposed to respond (reply all, or just to openldap-technical@openldap.org). So, excuse me if I did it wrong doing a reply all.
I appreciate all the quick responses. I've been working on this issue for a couple of weeks, so it's great to be able to post a question and get back the answer so rapidly.
I do have some follow-up questions based on the responses: 1) Quanah wrote: "This is indeed a BDB bit." Not sure I understand what that means. Is part of the delay due to BDB? It does seem like a whole bunch (maybe 30) log files getting written to file before processing resumes.
2) Index lists get collapsed into ranges - I assume this done to make processing more efficient. Is this a one time event? As I mentioned, I did not see it happen again.
3)If I know I'm going to have greater than 64K elements, is there any way to force it to use ranges at server startup and avoid the delay? I didn't see any such option.
Quanah - Yes, I understood that 65,536 is a power of 2 and it's importance in computer processing :)
Thanks again for the answers
From: Howard Chu hyc@symas.com To: Mark Cooper/Poughkeepsie/IBM@IBMUS, openldap-technical@openldap.org, Date: 07/30/2013 05:11 PM Subject: Re: OpenLDAP (using BDB) stalls adding 65,536th entry Sent by: openldap-technical-bounces@openldap.org
Mark Cooper wrote:
I've been doing some testing using OpenLDAP with BDB on a couple of
different
platforms. I noticed a similar situation. When I sit in a loop doing
adds,
at the 65,536th added entry the process stalls for a short period of
time.
After a minute or two, the add succeeds. My first thought is that this
is a
BDB issue, so I posted this question to Oracle's BDB forum. But I have
yet to
receive any answer.
This is all known/expected behavior. One (or more) of your index slots hit its maxsize of 65535 elements and was collapsed into a range. This typically happens with the objectClass index first, if you're adding a bunch of objects all of the same classes.
Taking a minute or two is abnormal, but I suppose is possible if multiple indices hit the condition at the same time.
This situation seems to happen when I have around 43 10MB log files.
During
the stall, I notice many log files are being written (another 25 or so),
which
is a much quicker rate than was being written prior to the stall.
The stall only happens once. I added another 350,000 entries and no more stalls. I ran a few other tests. Added 65,535 entries. All is fine.
As
soon as the next entry is add, even if I recycle the server, I hit the condition. I even tried deleting 1,000 entries. I would then need to
add
1,0001 to get to 65,536 entries in the database and then hit the delay.
-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
