https://bugs.openldap.org/show_bug.cgi?id=10275
Issue ID: 10275
Summary: mdb_load could use a parameter to pass batch size
Product: LMDB
Version: 0.9.21
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: tools
Assignee: bugs(a)openldap.org
Reporter: gkwicker(a)amazon.com
Target Milestone: ---
Created attachment 1037
--> https://bugs.openldap.org/attachment.cgi?id=1037&action=edit
Patch to mdb_load.c in version 0.9.21
mdb_load is very slow when ingesting a large db. The attached patch (0.9.21
but will apply to other versions with modification) allows the user to pass a
batch size to be used instead of the default value of 100. Changing the batch
size to a larger value improves ingestion speed.
I, Gary Wicker, hereby place the following modifications to OpenLDAP Software
(and only these modifications) into the public domain. Hence, these
modifications may be freely used and/or redistributed for any purpose with or
without attribution and/or other notice.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10024
Issue ID: 10024
Summary: MDB_PREVSNAPSHOT broken
Product: LMDB
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: markus(a)objectbox.io
Target Milestone: ---
It seems that the patch #9496 had a negative side effect on MDB_PREVSNAPSHOT.
In certain cases, when opening the DB using MDB_PREVSNAPSHOT, the previous (2nd
latest) commit is not selected. Instead, reads show that the latest commit was
selected voiding the effect of MDB_PREVSNAPSHOT.
I observed this in our test cases a while back. Today, I was finally able to
reproduce it and debug into it.
When creating the transaction to read the data, I debugged into mdb_txn_renew0.
Here, ti (MDB_txninfo; env->me_txns) was non-NULL. However, ti->mti_txnid was 0
(!) and thus txn->mt_txnid was set to 0. That's the reason for always selecting
the first (index 0) meta page inside mdb_txn_renew0:
meta = env->me_metas[txn->mt_txnid & 1];
This line occurs twice (once for read txn and once for write txn; it affects
both txn types).
Thus, the chances of MDB_PREVSNAPSHOT selecting the correct meta page is 50-50.
It's only correct if the first meta page (index 0) is the older one.
I believe that this is related to #9496 because the patch, that was provided
there, removed the initialization of "env->me_txns->mti_txnid" in
mdb_env_open2. This would explain why txn->mt_txnid inside mdb_txn_renew0 was
set to 0.
I can confirm that adding back the following two lines back in fixes
MDB_PREVSNAPSHOT:
if (env->me_txns)
env->me_txns->mti_txnid = meta.mm_txnid;
The said patch including the removal of these two lines was applied in the
commit(s) "ITS#9496 fix mdb_env_open bug from #8704" (Howard Chu on 09.04.21).
I hope this information is useful to find a suitable fix. Please let me know if
you have questions. Also, I'd be happy to help confirming a potential fix with
our test suite.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10274
Issue ID: 10274
Summary: Replication issue on MMR configuration
Product: OpenLDAP
Version: 2.5.14
Hardware: All
OS: Linux
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: slapd
Assignee: bugs(a)openldap.org
Reporter: falgon.comp(a)gmail.com
Target Milestone: ---
Created attachment 1036
--> https://bugs.openldap.org/attachment.cgi?id=1036&action=edit
In this attachment you will find 2 openldap configurations for 2 instances +
slamd conf exemple + 5 screenshots to show the issue and one text file to
explain what you see
Hello we are openning this issue further to the initial post in technical :
https://lists.openldap.org/hyperkitty/list/openldap-technical@openldap.org/…
Issue :
We are working on a project and we've come across an issue with the replication
after performance testing :
*Configuration :*
RHEL 8.6
OpenLDAP 2.5.14
*MMR-delta *configuration on multiple servers attached
300,000 users configured and used for tests
*olcLastBind: TRUE*
Use of SLAMD (performance shooting)
*Problem description:*
We are currently running performance and resilience tests on our infrastructure
using the SLAMD tool configured to perform BINDs and MODs on a defined range of
accounts.
We use a load balancer (VIP) to poll all of our servers equally. (but it is
possible to do performance tests directly on each of the directories)
With our current infrastructure we're able to perform approximately 300
MOD/BIND/s. Beyond that, we start to generate delays and can randomly come
across one issue.
However, when we run performance tests that exceed our write capacity, our
replication between servers can randomly create an incident with directories
being unable to catch up with their replication delay.
The directories update their contextCSNs, but extremely slowly (like freezing).
From then on, it's impossible for the directories to catch again. (even with no
incoming traffic)
A restart of the instance is required to perform a full refresh and solve the
incident.
We have enabled synchronization logs and have no error or refresh logs to
indicate a problem ( we can provide you with logs if necessary).
We suspect a write collision or a replication conflict but this is never write
in our sync logs.
We've run a lot of tests.
For example, when we run a performance test on a single live server, we don't
reproduce the problem.
Anothers examples: if we define different accounts ranges for each server with
SALMD, we don't reproduce the problem either.
If we use only one account for the range, we don't reproduce the problem
either.
______________________________________________________________________
I have add some screenshots on attachement to show you the issue and all the
explanations.
______________________________________________________________________
*Symptoms :*
One or more directories can no longer be replicated normally after performance
testing ends.
No apparent error logs.
Need a restart of instances to solve the problem.
*How to reproduce the problem:*
Have at least two servers in MMR mode
Set LastBind to TRUE
Perform a SLAMD shot from a LoadBalancer in bandwidth mode OR start multiple
SLAMD test on same time for each server with the same account range.
Exceed the maximum write capacity of the servers.
*SLAMD configuration :*
authrate.sh --hostname ${HOSTNAME} --port ${PORTSSL} \
--useSSL --trustStorePath ${CACERTJKS} \
--trustStorePassword ${CACERTJKSPW} --bindDN "${BINDDN}" \
--bindPassword ${BINDPW} --baseDN "${BASEDN}" \
--filter "(uid=[${RANGE}])" --credentials ${USERPW} \
--warmUpIntervals ${WARMUP} \
--numThreads ${NTHREADS} ${ARGS}
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10286
Issue ID: 10286
Summary: ldap_pvt_gettime may result in "not new enough csn"
problems in multi-thread case.
Product: OpenLDAP
Version: 2.6.6
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: libraries
Assignee: bugs(a)openldap.org
Reporter: 971748261(a)qq.com
Target Milestone: ---
Created attachment 1041
--> https://bugs.openldap.org/attachment.cgi?id=1041&action=edit
the log of adding two entry which shows the time sequence.
I used openldap as krb5's database, and openldap was deploymented in
mirrormode.
I tried to add kerberos principals via kadmin.local -q "addprinc -randkey
principal.
slapd log showed that the entry of kadmin/admin was added earlier than the
entry of ossuser. But the csn of kadmin/admin was greater than ossuser.
In this case, when the two entry began to sync to the other slapd server,
ossuser was ignored because of "csn not new enough"
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10289
Issue ID: 10289
Summary: myldap
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: build
Assignee: bugs(a)openldap.org
Reporter: cmp(a)tutanota.de
Target Milestone: ---
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10288
Issue ID: 10288
Summary: autoca Attribute olcAutoCAserverClass
Product: OpenLDAP
Version: 2.6.9
Hardware: All
OS: Linux
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: overlays
Assignee: bugs(a)openldap.org
Reporter: stefan(a)kania-online.de
Target Milestone: ---
I try to add the autoca overlay with the following ldif:
--------------
dn: olcOverlay=autoca,olcDatabase={2}mdb,cn=config
objectClass: olcAutoCAConfig
objectClass: olcOverlayConfig
olcOverlay: autoca
olcAutoCADays: 3652
olcAutoCAKeybits: 4096
olcAutoCAserverClass: ipHost
olcAutoCAserverDays: 1826
olcAutoCAserverKeybits: 4096
olcAutoCAuserClass: person
olcAutoCAuserDays: 365
olcAutoCAuserKeybits: 4096
--------------
ldapadd gives me:
adding new entry "olcOverlay=autoca,olcDatabase={2}mdb,cn=config"
ldap_add: Other (e.g., implementation specific) error (80)
additional info: <olcAutoCAserverClass> handler exited with 1
If I remove the attribute from my ldif, it works.
What is wrong with the olcAutoCAserverClass attribute in my ldif? I try to look
it up in the admin handbook but I could not find anything. I looked in the
source code and found:
------------
{ "serverClass", "objectclass", 2, 2, 0,
ARG_STRING|ARG_MAGIC|ACA_SRVCLASS, autoca_cf,
"( OLcfgOvAt:22.2 NAME 'olcAutoCAserverClass' "
"DESC 'ObjectClass of server entries' "
"EQUALITY caseIgnoreMatch "
"SYNTAX OMsDirectoryString SINGLE-VALUE )", NULL, NULL },
------------
For me it looks the same as the attribute olcAutoCAuserclass.
-------------
{ "userClass", "objectclass", 2, 2, 0,
ARG_STRING|ARG_MAGIC|ACA_USRCLASS, autoca_cf,
"( OLcfgOvAt:22.1 NAME 'olcAutoCAuserClass' "
"DESC 'ObjectClass of user entries' "
"EQUALITY caseIgnoreMatch "
"SYNTAX OMsDirectoryString SINGLE-VALUE )", NULL, NULL },
-------------
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10229
Issue ID: 10229
Summary: ldap_result, when invoked with MSG_RECEIVED and a
timeout value set to 0 (polling), does not return all
available messages until it is called again
Product: OpenLDAP
Version: 2.6.8
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: libraries
Assignee: bugs(a)openldap.org
Reporter: nivanova(a)symas.com
Target Milestone: ---
The issue is noticeable when ldap_result is used by the proxy back-ends. It has
not affected back-meta behavior, because when a first call is unsuccessful, it
retries with a small timeout. back-asyncmeta will also usually call it twice on
the same connection from different threads, although this is not a desired
behavior.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10235
Issue ID: 10235
Summary: Configure without --enable-nestgroup=no enables
feature
Product: OpenLDAP
Version: 2.6.8
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: slapd
Assignee: bugs(a)openldap.org
Reporter: stacey.marshall(a)gmail.com
Target Milestone: ---
OpenLDAP 2.6.8 configured with --enable-overlays.
Some openldap commands output a Duplicate attributeType notice, for example
# /sbin/slapcat -b cn=config -H ldap:///???olcTLSCertificateKeyFile=\* | grep
olcTLSCertificateKeyFile
register_at: AttributeType "( 1.2.840.113556.1.2.102 NAME 'memberOf' DESC
'Group that the entry belongs to' SYNTAX '1.3.6.1.4.1.1466.115.121.1.12'
EQUALITY distinguishedNameMatch USAGE dSAOperation NO-USER-MODIFICATION
X-ORIGIN 'iPlanet Delegated Administrator' )": Duplicate attributeType,
1.2.840.113556.1.2.102
olcTLSCertificateKeyFile: /etc/certs/localhost/host.key
Specifying configure option --enable-nestgroup=no prevents nestgroup feature
and the message from being displayed.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10234
Issue ID: 10234
Summary: syncrepl does not reset the retrynum
Product: OpenLDAP
Version: 2.6.8
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: slapd
Assignee: bugs(a)openldap.org
Reporter: hamano(a)osstech.co.jp
Target Milestone: ---
```
syncrepl
retry="5 10 30 +"
```
When replication fails with the above settings, syncrepl retries "10 times at 5
second intervals". Then, the retry count should be reset on the next
replication failure.
In actual, it does not reset. The behavior is as follows:
```
(first time replication failure)
do_syncrepl: rid=001 rc -1 retrying (9 retries left)
do_syncrepl: rid=001 rc -1 retrying (8 retries left)
(resume replication)
(second time replication failure)
do_syncrepl: rid=001 rc -1 retrying (7 retries left)
do_syncrepl: rid=001 rc -1 retrying (6 retries left)
```
--
You are receiving this mail because:
You are on the CC list for the issue.