https://bugs.openldap.org/show_bug.cgi?id=10274
Issue ID: 10274
Summary: Replication issue on MMR configuration
Product: OpenLDAP
Version: 2.5.14
Hardware: All
OS: Linux
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: slapd
Assignee: bugs(a)openldap.org
Reporter: falgon.comp(a)gmail.com
Target Milestone: ---
Created attachment 1036
--> https://bugs.openldap.org/attachment.cgi?id=1036&action=edit
In this attachment you will find 2 openldap configurations for 2 instances +
slamd conf exemple + 5 screenshots to show the issue and one text file to
explain what you see
Hello we are openning this issue further to the initial post in technical :
https://lists.openldap.org/hyperkitty/list/openldap-technical@openldap.org/…
Issue :
We are working on a project and we've come across an issue with the replication
after performance testing :
*Configuration :*
RHEL 8.6
OpenLDAP 2.5.14
*MMR-delta *configuration on multiple servers attached
300,000 users configured and used for tests
*olcLastBind: TRUE*
Use of SLAMD (performance shooting)
*Problem description:*
We are currently running performance and resilience tests on our infrastructure
using the SLAMD tool configured to perform BINDs and MODs on a defined range of
accounts.
We use a load balancer (VIP) to poll all of our servers equally. (but it is
possible to do performance tests directly on each of the directories)
With our current infrastructure we're able to perform approximately 300
MOD/BIND/s. Beyond that, we start to generate delays and can randomly come
across one issue.
However, when we run performance tests that exceed our write capacity, our
replication between servers can randomly create an incident with directories
being unable to catch up with their replication delay.
The directories update their contextCSNs, but extremely slowly (like freezing).
From then on, it's impossible for the directories to catch again. (even with no
incoming traffic)
A restart of the instance is required to perform a full refresh and solve the
incident.
We have enabled synchronization logs and have no error or refresh logs to
indicate a problem ( we can provide you with logs if necessary).
We suspect a write collision or a replication conflict but this is never write
in our sync logs.
We've run a lot of tests.
For example, when we run a performance test on a single live server, we don't
reproduce the problem.
Anothers examples: if we define different accounts ranges for each server with
SALMD, we don't reproduce the problem either.
If we use only one account for the range, we don't reproduce the problem
either.
______________________________________________________________________
I have add some screenshots on attachement to show you the issue and all the
explanations.
______________________________________________________________________
*Symptoms :*
One or more directories can no longer be replicated normally after performance
testing ends.
No apparent error logs.
Need a restart of instances to solve the problem.
*How to reproduce the problem:*
Have at least two servers in MMR mode
Set LastBind to TRUE
Perform a SLAMD shot from a LoadBalancer in bandwidth mode OR start multiple
SLAMD test on same time for each server with the same account range.
Exceed the maximum write capacity of the servers.
*SLAMD configuration :*
authrate.sh --hostname ${HOSTNAME} --port ${PORTSSL} \
--useSSL --trustStorePath ${CACERTJKS} \
--trustStorePassword ${CACERTJKSPW} --bindDN "${BINDDN}" \
--bindPassword ${BINDPW} --baseDN "${BASEDN}" \
--filter "(uid=[${RANGE}])" --credentials ${USERPW} \
--warmUpIntervals ${WARMUP} \
--numThreads ${NTHREADS} ${ARGS}
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10280
Issue ID: 10280
Summary: Combining positive & negated filters doesn't work with
dynlist
Product: OpenLDAP
Version: 2.5.18
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: overlays
Assignee: bugs(a)openldap.org
Reporter: code(a)pipoprods.org
Target Milestone: ---
The directory contains 3 users & 2 groups.
user1 is in group1, user2 is in group2, user3 isn't is any group.
Filter [1] matches users that are either:
- member of group1
- member of group2
✅ It returns user1 & user2
Filter [2] matches user that are:
- not member of group1 nor group2
✅ It returns user3
Filter [3] should match users that are either:
- member of group1
- member of group2
- not member of group1 nor group2
❌ It should return the 3 users but only returns users matched by the first part
of the filter (whatever the first part, if we swap both parts we get the
complementary search results)
Filter [1]:
(|(memberOf=cn=group1,ou=example-groups,dc=example,dc=com)(memberOf=cn=group2,ou=example-groups,dc=example,dc=com))
Filter [2]:
(!(|(memberOf=cn=group1,ou=example-groups,dc=example,dc=com)(memberOf=cn=group2,ou=example-groups,dc=example,dc=com)))
Filter [3]:
(|(memberOf=cn=group1,ou=example-groups,dc=example,dc=com)(memberOf=cn=group2,ou=example-groups,dc=example,dc=com)(!(|(memberOf=cn=group1,ou=example-groups,dc=example,dc=com)(memberOf=cn=group2,ou=example-groups,dc=example,dc=com))))
Here's my dynlist config:
```
dn: olcOverlay={2}dynlist,olcDatabase={1}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcDynListConfig
olcOverlay: {2}dynlist
olcDynListAttrSet: {0}groupOfURLs memberURL member+memberOf@groupOfNames
structuralObjectClass: olcDynListConfig
entryUUID: 7df8328a-fd72-103e-82df-6fed25d5f6c8
creatorsName: cn=config
createTimestamp: 20240902122741Z
entryCSN: 20240902122741.257759Z#000000#000#000000
modifiersName: cn=config
modifyTimestamp: 20240902122741Z
```
Here's a LDIF to initialise directory contents:
```
dn: ou=example-groups,dc=example,dc=com
changetype: add
objectClass: organizationalUnit
ou: example-groups
dn: ou=example-users,dc=example,dc=com
changetype: add
objectClass: organizationalUnit
ou: example-users
dn: uid=user1,ou=example-users,dc=example,dc=com
changetype: add
objectClass: inetOrgPerson
cn: User
sn: One
uid: user1
dn: uid=user2,ou=example-users,dc=example,dc=com
changetype: add
objectClass: inetOrgPerson
cn: User
sn: Two
uid: user2
dn: uid=user3,ou=example-users,dc=example,dc=com
changetype: add
objectClass: inetOrgPerson
cn: User
sn: Three
uid: user3
dn: cn=group1,ou=example-groups,dc=example,dc=com
changetype: add
objectClass: groupOfNames
cn: group1
member: uid=user1,ou=example-users,dc=example,dc=com
dn: cn=group2,ou=example-groups,dc=example,dc=com
changetype: add
objectClass: groupOfNames
cn: group2
member: uid=user2,ou=example-users,dc=example,dc=com
```
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10270
Issue ID: 10270
Summary: Issues with pcache when refresh/persistPcache used
Product: OpenLDAP
Version: 2.5.18
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: overlays
Assignee: bugs(a)openldap.org
Reporter: aweits(a)rit.edu
Target Milestone: ---
Greetings, OpenLDAP-folk.
We've been running with the pcache overlay in 2.5.18
with both query refresh and pcachePersist for a bit
and have observed some oddities:
1.) Negative queries don't get refreshed
2.) Queries don't seem to be persisted
These behaviors are all exhibited from the current
git version as well - code/patches below for clarity
of communication:
Thanks!
Andy
commit c0b4fe92c8df746c0e6a777f93f1687135114eb9
Author: Andrew Elble <aweits(a)rit.edu>
Date: Fri Oct 11 08:43:47 2024 -0400
negative cache entries are not loaded when pcachePersist is on
diff --git a/servers/slapd/overlays/pcache.c b/servers/slapd/overlays/pcache.c
index 9ef78fd6bf43..9fd72e6d7261 100644
--- a/servers/slapd/overlays/pcache.c
+++ b/servers/slapd/overlays/pcache.c
@@ -802,7 +802,11 @@ url2query(
goto error;
}
- cq = add_query( op, qm, &query, qt, PC_POSITIVE, 0 );
+ if (BER_BVISNULL( &uuid )) {
+ cq = add_query( op, qm, &query, qt, PC_NEGATIVE, 0 );
+ } else {
+ cq = add_query( op, qm, &query, qt, PC_POSITIVE, 0 );
+ }
if ( cq != NULL ) {
cq->expiry_time = expiry_time;
cq->refresh_time = refresh_time;
commit 8f7b50dfcec69fa01f8cf0a4b77f3dee8ef9f0f6
Author: Andrew Elble <aweits(a)rit.edu>
Date: Fri Oct 11 08:38:36 2024 -0400
queries with ttr/x-refresh are not loaded when pcachePersist is on
diff --git a/servers/slapd/overlays/pcache.c b/servers/slapd/overlays/pcache.c
index 40c1f9673776..9ef78fd6bf43 100644
--- a/servers/slapd/overlays/pcache.c
+++ b/servers/slapd/overlays/pcache.c
@@ -749,7 +749,7 @@ url2query(
}
}
- if ( got != GOT_ALL ) {
+ if ( (got & GOT_ALL) != GOT_ALL) {
rc = 1;
goto error;
}
commit c7e52c90192a43876d40b9776a58db951d27937c
Author: Andrew Elble <aweits(a)rit.edu>
Date: Fri Oct 11 08:37:13 2024 -0400
ttr was not being applied to negatively cached entries
diff --git a/servers/slapd/overlays/pcache.c b/servers/slapd/overlays/pcache.c
index 1d6e4ba4edcf..40c1f9673776 100644
--- a/servers/slapd/overlays/pcache.c
+++ b/servers/slapd/overlays/pcache.c
@@ -1580,6 +1580,8 @@ add_query(
case PC_NEGATIVE:
ttl = templ->negttl;
+ if ( templ->ttr )
+ ttr = now + templ->ttr;
break;
case PC_SIZELIMIT:
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10275
Issue ID: 10275
Summary: mdb_load could use a parameter to pass batch size
Product: LMDB
Version: 0.9.21
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: tools
Assignee: bugs(a)openldap.org
Reporter: gkwicker(a)amazon.com
Target Milestone: ---
Created attachment 1037
--> https://bugs.openldap.org/attachment.cgi?id=1037&action=edit
Patch to mdb_load.c in version 0.9.21
mdb_load is very slow when ingesting a large db. The attached patch (0.9.21
but will apply to other versions with modification) allows the user to pass a
batch size to be used instead of the default value of 100. Changing the batch
size to a larger value improves ingestion speed.
I, Gary Wicker, hereby place the following modifications to OpenLDAP Software
(and only these modifications) into the public domain. Hence, these
modifications may be freely used and/or redistributed for any purpose with or
without attribution and/or other notice.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10278
Issue ID: 10278
Summary: Move away from python-ldap0 in the test suite
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: test suite
Assignee: bugs(a)openldap.org
Reporter: ondra(a)mistotebe.net
Target Milestone: ---
While python-ldap0 has a superior API, it seems the module is no longer
receiving any development. We should move our python test suite over to another
module that can be supported long-term.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10277
Issue ID: 10277
Summary: How to deal with desync between cn=config and
back-ldif DNs
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: slapd
Assignee: bugs(a)openldap.org
Reporter: ondra(a)mistotebe.net
Target Milestone: ---
If someone deletes a cn=config entry offline (or through bugs in cn=config,
they exist, will file as I isolate), the X-ORDERED RDNs will not be contiguous.
cn=config papers over this internally at a cost of never being able to modify
the entries affected.
Right now the only remedy is slapcat+slapadd of the whole config DB, is that
the best we can do? When we detect this (doesn't always happen), should we fix
the on-disk copy on startup?
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10276
Issue ID: 10276
Summary: crash using pcache overlay on mdb backend
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: overlays
Assignee: bugs(a)openldap.org
Reporter: chris.paul(a)rexconsulting.net
Target Milestone: ---
Created attachment 1038
--> https://bugs.openldap.org/attachment.cgi?id=1038&action=edit
backtrace
When using the pcache overlay with a local mdb backend, slapd crashes. The
intention is to use pcache to reduce lookup times for dynamic (dynlist) groups
and memberOf values.
GDB backtrace attached. Config available upon request.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10279
Issue ID: 10279
Summary: add debug notice also to client tools
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: client tools
Assignee: bugs(a)openldap.org
Reporter: rossi.f(a)inwind.it
Target Milestone: ---
Created attachment 1040
--> https://bugs.openldap.org/attachment.cgi?id=1040&action=edit
openldap-2.6.4-debug-notice.patch
The command line -d option, when used for debugging, does nothing if openldap
was not compiled byth --enable-debug option. For the server part there is a
notice to the user regarding this, I propose to add the same also to client
tools.
Here is attached the simple patch.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10268
Issue ID: 10268
Summary: Operation rate limiting
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: slapd
Assignee: bugs(a)openldap.org
Reporter: chris.paul(a)rexconsulting.net
Target Milestone: ---
Please consider this request for enhancement. It would be very useful for slapd
to have some basic rate limiting per connection or per IP. The
monitorConnectionsOpsCompleted counts are available in cn=monitor. A dependency
of cn=monitor seems reasonable.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10272
Issue ID: 10272
Summary: translucent regression when asking for attributes
Product: OpenLDAP
Version: 2.6.8
Hardware: x86_64
OS: Linux
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: overlays
Assignee: bugs(a)openldap.org
Reporter: mike(a)nolta.net
Target Milestone: ---
Hi,
Attached please find a translucent overlay regression testcase.
The test works in version 2.4.59, but fails in versions 2.5.5 and 2.6.8.
Basically, `ldapsearch "employeeType=fulltime"` returns the expected entries,
but `ldapsearch "employeeType=fulltime" cn` returns nothing.
-Mike
--
You are receiving this mail because:
You are on the CC list for the issue.