https://bugs.openldap.org/show_bug.cgi?id=10404
Issue ID: 10404
Summary: slapd uses all available memory and starts using swap
Product: OpenLDAP
Version: 2.6.10
Hardware: x86_64
OS: Linux
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: slapd
Assignee: bugs(a)openldap.org
Reporter: tomas.bjorklund(a)su.se
Target Milestone: ---
We have a problem with slapd using more and more memory until in starts using
swap.
We are using mdb and our maxsize is set to 8192000000.
The server has 32GB of ram.
data.mdb is around 2.6 GB
Some settings from our slapd.conf:
sizelimit 5000
timelimit 600
checkpoint 1024 15
loglevel sync stats
This is how it looks with top:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
314486 root 20 0 31.6g 26.2g 2.6g S 16.3 83.7 8,09 slapd
Depending on the load of the server it usually takes 2 weeks until all memory
has been used.
Things that we have tried:
Changed swapiness to 1
Changed slapd service to use this:
[Service]
Environment="LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libtcmalloc_minimal.so.4"
LimitCORE=infinity
LimitNOFILE=4096
Restart=on-failure
TimeoutStopSec=900
Everything worked fine with bdb and cache settings like:
cachesize 100000
idlcachesize 100000
But since switching to mdb we run out of memory.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10383
Issue ID: 10383
Summary: slapd-meta ignores olcDbIDAssertBind if olcDbURI
defined after it
Product: OpenLDAP
Version: 2.6.9
Hardware: x86_64
OS: Linux
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: slapd
Assignee: bugs(a)openldap.org
Reporter: david.frickert(a)protonmail.com
Target Milestone: ---
Hi,
We are using slapd-meta to connect an OpenLDAP server to another external LDAP
server and it works well on first configuration.
However, if we want to update any info, e.g. the external LDAP URI, we must
replace the olcDbURI attribute. This means that the ordering of the attributes
change and this attribute is now defined after olcDbIDAssertBind.
Didn't think this would be important, but after this change the "meta"
connection stops working and upon enabling debugging i can see that the
external LDAP server is responding with:
"ldap_bind: Inappropriate authentication (48)
additional info: Anonymous Simple Bind Disabled"
This seems to imply that the olcDbIDAssertBind attribute is being ignored,
likely due to being defined before olcDbURI (my assumption).
Is this intended? If so, what can we do to mitigate this problem?
Do we need to perform a replace on all attributes of the object to ensure
correct ordering, or is there any way to perform an in-place attribute
modification without making it shift its position in the object?
Example:
First configuration (OK):
# {0}uri, {2}meta, config
dn: olcMetaSub={0}uri,olcDatabase={2}meta,cn=config
objectClass: olcMetaTargetConfig
olcMetaSub: {0}uri
--> olcDbURI: ldap://REDACTED/ou=users,REDACTED
olcDbIDAssertBind: bindmethod=simple starttls=yes tls_reqcert=demand
binddn="REDACTED" credentials="REDACTED"
olcDbRewrite: {0}suffixmassage REDACTED REDACTED
olcDbKeepalive: 0:0:0
olcDbBindTimeout: 100000
olcDbCancel: abandon
After URI update (NOK):
# {0}uri, {2}meta, config
dn: olcMetaSub={0}uri,olcDatabase={2}meta,cn=config
objectClass: olcMetaTargetConfig
olcMetaSub: {0}uri
olcDbIDAssertBind: bindmethod=simple starttls=yes tls_reqcert=demand
binddn="REDACTED" credentials="REDACTED"
olcDbRewrite: {0}suffixmassage REDACTED REDACTED
olcDbKeepalive: 0:0:0
olcDbBindTimeout: 100000
olcDbCancel: abandon
--> olcDbURI: ldap://REDACTED/ou=users,REDACTED
The olcDbURI attribute is shifted to the bottom after a modify operation, and
seems to cause these issues.
Best Regards,
David
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10162
Issue ID: 10162
Summary: Fix for binary attributes data corruption in back-sql
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: backends
Assignee: bugs(a)openldap.org
Reporter: dex.tracers(a)gmail.com
Target Milestone: ---
Created attachment 1006
--> https://bugs.openldap.org/attachment.cgi?id=1006&action=edit
Fix for binary attributes corruption on backed-sql
I've configured slapd to use back-sql (mariadb through odbc) and observed
issues with the BINARY data retrievals from the database. The length of the
attributes was properly reported, but the correct data inside was always 16384
bytes and after that point - some junk (usually filled-up with AAAAAAAA and
some other attributes data from memory).
During the debugging - I've noticed that:
- The MAX_ATTR_LEN (16384 bytes) is used to set the length of the data for
BINARY columns when SQLBindCol is done inside of the
"backsql_BindRowAsStrings_x" function
- After SQLFetch is done - data in row->cols[i] is fetched up to the specified
MAX_ATTR_LEN
- After SQLFetch is done - the correct data size (greater than MAX_ATTR_LEN) is
represented inside of the row->value_len
I'm assuming that slapd allocates the pointer in memory (row->cols[i]), fills
it with the specified amount of data (MAX_ATTR_LEN), but when forming the
actual attribute data - uses the length from row->value_len and so everything
from 16384 bytes position till row->value_len is just a junk from the memory
(uninitialized, leftovers, data from other variables).
After an investigation, I've find-out that:
- for BINARY or variable length fields - SQLGetData should be used
- SQLGetData supports chunked mode (if length is unknown) or full-read mode if
the length is known
- it could be used in pair with SQLBindCol after SQLFetch (!)
Since we have the correct data length inside of row->value_len, I've just added
the code to the backsql_get_attr_vals() function to overwrite the corrupted
data with the correct data by issuing SQLGetData request. And it worked -
binary data was properly retrieved and reported over LDAP!
My current concerns / help needed - I'm not very familiar with the memory
allocation/deallocation mechanisms, so I'm afraid that mentioned change can
lead to memory corruption (so far not observed).
Please review attached patch (testing was done on OPENLDAP_REL_ENG_2_5_13, and
applied on the master branch for easier review/application).
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10026
Issue ID: 10026
Summary: Refresh handling can skip entries (si_dirty not
managed properly)
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: overlays
Assignee: bugs(a)openldap.org
Reporter: ondra(a)mistotebe.net
Target Milestone: ---
Take MPR plain syncrepl with 3+ providers.
When a provider's own syncrepl session transitions to persist and a it starts a
new parallel session towards another host, that session always has to start as
a refresh. If that refresh serves entries to us, our handling of si_dirty is
not consistent:
- if the existing persist session serves some of these entries to us, we can
"forget" to pass the others to a newly connected consumer
- same if the refresh is abandoned and we start refreshing from a different
provider that might be behind what we were being served (again our consumers
could suffer)
- if we restart, si_dirty is forgotten and our consumers suffer even worse
We might need to be told (at the beginning of the refresh?) what the end state
we're going for is, so we can keep si_dirty on until then. And somehow persist
that knowledge in the DB...
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10358
Issue ID: 10358
Summary: syncrepl can revert an entry's CSN
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: slapd
Assignee: bugs(a)openldap.org
Reporter: ondra(a)mistotebe.net
Target Milestone: ---
Created attachment 1080
--> https://bugs.openldap.org/attachment.cgi?id=1080&action=edit
Debug log of an instance of this happening
There is a sequence of operations which can force a MPR node to apply changes
out of order (essentially reverting an operation). Currently investigating
which part of the code that should have prevented this has let it slip.
A sample log showing how this happened is attached.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10375
Issue ID: 10375
Summary: [patch] minor patch to const up oids array
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: libraries
Assignee: bugs(a)openldap.org
Reporter: caolanm(a)gmail.com
Target Milestone: ---
Created attachment 1084
--> https://bugs.openldap.org/attachment.cgi?id=1084&action=edit
minor patch to const up oids array
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10379
Issue ID: 10379
Summary: lastbind change prevents ppolicy response from
reaching accesslog
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: slapd
Assignee: bugs(a)openldap.org
Reporter: ondra(a)mistotebe.net
Target Milestone: ---
When "lastbind on" and ppolicy are configured together, the pwdLastSuccess
update triggers an accesslog entry (using op->o_time, op->o_tincr), then
ppolicy_bind_response issues its own modification and since the time was copied
in lastbind, an entry of the same name already exists. This means the ppolicy
change is lost (and e.g. won't replicate).
Note that slapo-lastbind (=the contrib overlay) probably has the same impact.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10370
Issue ID: 10370
Summary: result.c:930: try_read1msg: Assertion `!BER_BVISEMPTY(
&resoid )' failed.
Product: OpenLDAP
Version: 2.6.10
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: libraries
Assignee: bugs(a)openldap.org
Reporter: daniel(a)haxx.se
Target Milestone: ---
When using curl built with OpenLDAP to access a broken/malicious ldap server,
OpenLDAP will abort on this assert.
It seems it should rather return a proper error code?
A full reproducer that unfortunately uses curl is available here:
https://hackerone.com/reports/3258022 together with more details about this
problem.
(I'm forwarding this information, I did not discover this.)
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10371
Issue ID: 10371
Summary: tools don't print useful error codes
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: client tools
Assignee: bugs(a)openldap.org
Reporter: hyc(a)openldap.org
Target Milestone: ---
In tools/common.c, if ldap_result() returns an error, this return value is used
directly as an LDAP error code in the subsequent call to tool_perror(). That is
incorrect; ldap_result() always returns -1 on errors. The actual return code
must be retrieved using ldap_get_option(ld, LDAP_OPT_RESULT_CODE,...). So up
till now the tools have never printed the actual error message that's relevant
to whatever failure occurred.
Other applications appear to have copied this erroneous behavior. E.g., in
investigating ITS#10370 I see that curl's code does the same thing.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10344
Issue ID: 10344
Summary: Potential memory leak in function
firstComponentNormalize and objectClassPretty.
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: slapd
Assignee: bugs(a)openldap.org
Reporter: alexguo1023(a)gmail.com
Target Milestone: ---
Created attachment 1071
--> https://bugs.openldap.org/attachment.cgi?id=1071&action=edit
Patch: Ensure the first argument passed to `ber_dupbv_x` is not `NULL`.
In `firstComponentNormalize`, the code calls `ber_dupbv_x` but ignores its
return value.
```c
ber_dupbv_x(normalized, val, ctx);
```
When `normalized` is `NULL`, `ber_dupbv_x` allocates a new `struct berval` and
returns it; failing to capture that pointer means we lose ownership and leak
the allocation. The same issue arises in `objectClassPretty`. We should follow
the pattern in function `hexNormalize`, which asserts that its `normalized`
argument is non-NULL before use:
```c
assert(normalized != NULL);
```
--
You are receiving this mail because:
You are on the CC list for the issue.