https://bugs.openldap.org/show_bug.cgi?id=10169
Issue ID: 10169
Summary: Add support for token only authentication with otp
overlay
Product: OpenLDAP
Version: 2.6.6
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: overlays
Assignee: bugs(a)openldap.org
Reporter: quanah(a)openldap.org
Target Milestone: ---
Currently the OTP overlay is password + token. It would be nice to be able to
configure it so it can run in a token only mode, similar to the slapo-totp
overlay in contrib. This would allow us to have a project supported solution
and retire that contrib module.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9564
Issue ID: 9564
Summary: Race condition with freeing the spilled pages from
transaction
Product: LMDB
Version: 0.9.29
Hardware: Other
OS: Mac OS
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: kriszyp(a)gmail.com
Target Milestone: ---
Created attachment 825
--> https://bugs.openldap.org/attachment.cgi?id=825&action=edit
Free the spilled pages and dirty overflows before unlocking the write mutex
The spilled pages (a transaction's mt_spill_pgs) is freed *after* a write
transaction's mutex is unlocked (in mdb.master3). This can result in a race
condition where a second transaction can start and subsequently assign a new
mt_spill_pgs list to the transaction structure that it reuses. If this occurs
after the first transaction unlocks the mutex, but before it performs the free
operation on mt_spill_pgs, then the first transaction will end up calling a
free on the same spilled page list as the second transaction, resulting in a
double free (and crash).
It would seem to be an extremely unlikely scenario to actually happen, since
the free call is literally the next operation after the mutex is unlocked, and
the second transaction would need to make it all the way to the point of saving
the freelist before a page spill list is likely to be allocated. Consequently,
this probably has rarely happened. However, one of our users (see
https://github.com/DoctorEvidence/lmdb-store/issues/48 for the discussion) has
noticed this occurring, and it seems that it may be particularly likely to
happen on MacOS on the new M1 silicon. Perhaps there is some peculiarity to how
the threads are more likely to yield execution after a mutex unlock, I am not
sure. I was able to reproduce the issue by intentionally manipulating the
timing (sleeping before the free) to verify that the race condition is
technically feasible, and apparently this can happen "in the wild" on MacOS, at
least with an M1.
It is also worth noting that this is due to (or a regression from) the fix for
ITS#9155
(https://github.com/LMDB/lmdb/commit/cb256f409bb53efeda4ac69ee8165a0b4fc1a277)
where the free call was moved outside the conditional (for having a parent)
that had previously never been executed after the mutex is unlocked, but now is
called after the unlock.
Anyway, the solution is relatively simple, the free call simply has to be moved
above the unlock. In my patch, I also moved the free call for mt_dirty_ovs. I
am not sure what OVERFLOW_NOTYET/mt_dirty_ovs is for, but presumably it should
be handled the same. This could alternately be solved by saving the reference
to these lists before unlocking, and freeing after unlocking, which would
slightly decrease the amount of processing within the mutex guarded code. Let
me know if you prefer a patch that does that.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10162
Issue ID: 10162
Summary: Fix for binary attributes data corruption in back-sql
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: backends
Assignee: bugs(a)openldap.org
Reporter: dex.tracers(a)gmail.com
Target Milestone: ---
Created attachment 1006
--> https://bugs.openldap.org/attachment.cgi?id=1006&action=edit
Fix for binary attributes corruption on backed-sql
I've configured slapd to use back-sql (mariadb through odbc) and observed
issues with the BINARY data retrievals from the database. The length of the
attributes was properly reported, but the correct data inside was always 16384
bytes and after that point - some junk (usually filled-up with AAAAAAAA and
some other attributes data from memory).
During the debugging - I've noticed that:
- The MAX_ATTR_LEN (16384 bytes) is used to set the length of the data for
BINARY columns when SQLBindCol is done inside of the
"backsql_BindRowAsStrings_x" function
- After SQLFetch is done - data in row->cols[i] is fetched up to the specified
MAX_ATTR_LEN
- After SQLFetch is done - the correct data size (greater than MAX_ATTR_LEN) is
represented inside of the row->value_len
I'm assuming that slapd allocates the pointer in memory (row->cols[i]), fills
it with the specified amount of data (MAX_ATTR_LEN), but when forming the
actual attribute data - uses the length from row->value_len and so everything
from 16384 bytes position till row->value_len is just a junk from the memory
(uninitialized, leftovers, data from other variables).
After an investigation, I've find-out that:
- for BINARY or variable length fields - SQLGetData should be used
- SQLGetData supports chunked mode (if length is unknown) or full-read mode if
the length is known
- it could be used in pair with SQLBindCol after SQLFetch (!)
Since we have the correct data length inside of row->value_len, I've just added
the code to the backsql_get_attr_vals() function to overwrite the corrupted
data with the correct data by issuing SQLGetData request. And it worked -
binary data was properly retrieved and reported over LDAP!
My current concerns / help needed - I'm not very familiar with the memory
allocation/deallocation mechanisms, so I'm afraid that mentioned change can
lead to memory corruption (so far not observed).
Please review attached patch (testing was done on OPENLDAP_REL_ENG_2_5_13, and
applied on the master branch for easier review/application).
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10404
Issue ID: 10404
Summary: slapd uses all available memory and starts using swap
Product: OpenLDAP
Version: 2.6.10
Hardware: x86_64
OS: Linux
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: slapd
Assignee: bugs(a)openldap.org
Reporter: tomas.bjorklund(a)su.se
Target Milestone: ---
We have a problem with slapd using more and more memory until in starts using
swap.
We are using mdb and our maxsize is set to 8192000000.
The server has 32GB of ram.
data.mdb is around 2.6 GB
Some settings from our slapd.conf:
sizelimit 5000
timelimit 600
checkpoint 1024 15
loglevel sync stats
This is how it looks with top:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
314486 root 20 0 31.6g 26.2g 2.6g S 16.3 83.7 8,09 slapd
Depending on the load of the server it usually takes 2 weeks until all memory
has been used.
Things that we have tried:
Changed swapiness to 1
Changed slapd service to use this:
[Service]
Environment="LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libtcmalloc_minimal.so.4"
LimitCORE=infinity
LimitNOFILE=4096
Restart=on-failure
TimeoutStopSec=900
Everything worked fine with bdb and cache settings like:
cachesize 100000
idlcachesize 100000
But since switching to mdb we run out of memory.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10402
Issue ID: 10402
Summary: Feature request: parameter for mdb_env_copy to exclude
databases
Product: LMDB
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: jeffro256(a)tutanota.com
Target Milestone: ---
## Desired behavior
New overload or API modification to `mdb_env_copy{...}()` which allows passing
a list of names of databases to exclude from the environment copy.
## Example use-case
The Monero blockchain database [1] has the option to derived a "pruned"
database for space-saving purposes. This removes some ~60% of data which some
users may find non-essential. Currently, the pruning code [2] copies each
non-pruned table manually, specifying key ordering functions, DB flags, etc. It
then drops some entries from the relevant "prunable tables". This, however,
adds technical maintenance debt when databases are added/updated. A preferable
alternative would be to write high-level modification code once and use an
overload of `mdb_env_copy` which excludes copying database that we know in
advance we don't want to copy. Then our pruning utility would work agnostic to
database changes.
## Why
Adding this functionality ourselves would involve re-writing large portions of
`mdb_env_copyfd{0,1}` and `mdb_env_cwalk`, which requires either A) vendoring
LMDB, or B) possibly breaking in future updates.
You may understandably be of the opinion that the maintenance burden is an "us
problem", and not deem pursuing this feature request worth it, but hopefully
you seem the value in this utility. Thanks for y'alls hard work on LMDB.
## Links
[1]
https://github.com/monero-project/monero/blob/d32b5bfe18e2f5b979fa8dc3a8966…
[2]
https://github.com/monero-project/monero/blob/master/src/blockchain_utiliti…
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10274
Issue ID: 10274
Summary: Replication issue on MMR configuration
Product: OpenLDAP
Version: 2.5.14
Hardware: All
OS: Linux
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: slapd
Assignee: bugs(a)openldap.org
Reporter: falgon.comp(a)gmail.com
Target Milestone: ---
Created attachment 1036
--> https://bugs.openldap.org/attachment.cgi?id=1036&action=edit
In this attachment you will find 2 openldap configurations for 2 instances +
slamd conf exemple + 5 screenshots to show the issue and one text file to
explain what you see
Hello we are openning this issue further to the initial post in technical :
https://lists.openldap.org/hyperkitty/list/openldap-technical@openldap.org/…
Issue :
We are working on a project and we've come across an issue with the replication
after performance testing :
*Configuration :*
RHEL 8.6
OpenLDAP 2.5.14
*MMR-delta *configuration on multiple servers attached
300,000 users configured and used for tests
*olcLastBind: TRUE*
Use of SLAMD (performance shooting)
*Problem description:*
We are currently running performance and resilience tests on our infrastructure
using the SLAMD tool configured to perform BINDs and MODs on a defined range of
accounts.
We use a load balancer (VIP) to poll all of our servers equally. (but it is
possible to do performance tests directly on each of the directories)
With our current infrastructure we're able to perform approximately 300
MOD/BIND/s. Beyond that, we start to generate delays and can randomly come
across one issue.
However, when we run performance tests that exceed our write capacity, our
replication between servers can randomly create an incident with directories
being unable to catch up with their replication delay.
The directories update their contextCSNs, but extremely slowly (like freezing).
From then on, it's impossible for the directories to catch again. (even with no
incoming traffic)
A restart of the instance is required to perform a full refresh and solve the
incident.
We have enabled synchronization logs and have no error or refresh logs to
indicate a problem ( we can provide you with logs if necessary).
We suspect a write collision or a replication conflict but this is never write
in our sync logs.
We've run a lot of tests.
For example, when we run a performance test on a single live server, we don't
reproduce the problem.
Anothers examples: if we define different accounts ranges for each server with
SALMD, we don't reproduce the problem either.
If we use only one account for the range, we don't reproduce the problem
either.
______________________________________________________________________
I have add some screenshots on attachement to show you the issue and all the
explanations.
______________________________________________________________________
*Symptoms :*
One or more directories can no longer be replicated normally after performance
testing ends.
No apparent error logs.
Need a restart of instances to solve the problem.
*How to reproduce the problem:*
Have at least two servers in MMR mode
Set LastBind to TRUE
Perform a SLAMD shot from a LoadBalancer in bandwidth mode OR start multiple
SLAMD test on same time for each server with the same account range.
Exceed the maximum write capacity of the servers.
*SLAMD configuration :*
authrate.sh --hostname ${HOSTNAME} --port ${PORTSSL} \
--useSSL --trustStorePath ${CACERTJKS} \
--trustStorePassword ${CACERTJKSPW} --bindDN "${BINDDN}" \
--bindPassword ${BINDPW} --baseDN "${BASEDN}" \
--filter "(uid=[${RANGE}])" --credentials ${USERPW} \
--warmUpIntervals ${WARMUP} \
--numThreads ${NTHREADS} ${ARGS}
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10403
Issue ID: 10403
Summary: Add a configuration directive that uses the OpenSSL
CONF API to allow openldap config files to set any
configuration supported by that API, and to get new
OpenSSL configuration capabilities through that API
with no changes in openldap.
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: slapd
Assignee: bugs(a)openldap.org
Reporter: stephen.wall(a)redcom.com
Target Milestone: ---
Created attachment 1090
--> https://bugs.openldap.org/attachment.cgi?id=1090&action=edit
Add a configuration directive that uses the OpenSSL CONF API to allow openldap
config files to set any configuration supported by that API, and to get new
OpenSSL configuration capabilities through th
I am submitting a patch to create a new directive for OpenLDAP config files
that uses the OpenSSL SSL_CONF API to allow configuration of any aspect of
OpenSSL that the API supports without adding specific directives to OpenLDAP
for them. When OpenSSL extends that API, all versions of OpenLDAP with thie
patch will also support those extensions with no additional code.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=7901
OndÅ™ej KuznÃk <ondra(a)mistotebe.net> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|UNCONFIRMED |IN_PROGRESS
Ever confirmed|0 |1
--- Comment #4 from OndÅ™ej KuznÃk <ondra(a)mistotebe.net> ---
https://git.openldap.org/openldap/openldap/-/merge_requests/800
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10398
Issue ID: 10398
Summary: memberof and refint clash on subtree renames
Product: OpenLDAP
Version: 2.6.10
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: overlays
Assignee: bugs(a)openldap.org
Reporter: hyc(a)openldap.org
Target Milestone: ---
If a group and its members are under a subtree that got renamed, refint will
trigger, and try to update all the relevant DNs. When it processes the group
entry, it will issue Modifies to update the DNs of the group's members. The
memberof overlay will see these modifies and start trying to update the
corresponding memberof values but will only succeed halfway.
It will try to delete the old memberof value from the old member DN's entry,
which fails because the subtree has renamed all the entries. Then it will try
to add the new memberof value to the new member DN's entry, which succeeds.
Then eventually refint will try to process the member's. It will try to delete
the old memberof value from the new entry, and add the new memberof value to
the entry. This modify request fails because the new value is already present.
The entry is left with a memberof value that points to the obsolete group DN.
The solution is for refint to set the manageDsaIt control on its repair ops,
and for memberof to ignore Modify requests with this control set.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10396
Issue ID: 10396
Summary: LMDB: another issue with sorted duplicate DBs and
cursor delete
Product: LMDB
Version: 0.9.19
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: hyc(a)openldap.org
Target Milestone: ---
In mdb_cursor_del0, in the second "Adjust other cursors" section (after
mdb_rebalance was called), in these lines
if (m3->mc_xcursor && !(m3->mc_flags & C_EOF)) {
MDB_node *node = NODEPTR(m3->mc_pg[m3->mc_top],
m3->mc_ki[m3->mc_top]);
The node being referenced should be using mc->mc_top, not m3->mc_top. Most of
the time these will be identical, but sometimes due to rebalancing, the mc
cursor may be popped below its stack top when calling here.
This bug was introduced for ITS#8406 in commit
37081325f7356587c5e6ce4c1f36c3b303fa718c on 2016-04-18.
--
You are receiving this mail because:
You are on the CC list for the issue.