https://bugs.openldap.org/show_bug.cgi?id=9753
Issue ID: 9753 Summary: back-mdb index incorrect on modify/replace Product: OpenLDAP Version: 2.4.6 Hardware: All OS: All Status: UNCONFIRMED Keywords: needs_review Severity: normal Priority: --- Component: backends Assignee: bugs@openldap.org Reporter: hyc@openldap.org Target Milestone: ---
When doing a modify/replace on an indexed attribute, back-mdb only deletes the old values from the index and doesn't add the new values into the index.
https://bugs.openldap.org/show_bug.cgi?id=9753
Howard Chu hyc@openldap.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|UNCONFIRMED |IN_PROGRESS Ever confirmed|0 |1
--- Comment #1 from Howard Chu hyc@openldap.org --- https://git.openldap.org/openldap/openldap/-/merge_requests/452
https://bugs.openldap.org/show_bug.cgi?id=9753
Quanah Gibson-Mount quanah@openldap.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Target Milestone|--- |2.5.10 Keywords|needs_review |
--- Comment #2 from Quanah Gibson-Mount quanah@openldap.org --- • 739081f2 by Howard Chu at 2021-11-23T17:29:31+00:00 ITS#9753 back-mdb: Fix index updating for replace ops
https://bugs.openldap.org/show_bug.cgi?id=9753
Quanah Gibson-Mount quanah@openldap.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|IN_PROGRESS |RESOLVED Resolution|--- |TEST
--- Comment #3 from Quanah Gibson-Mount quanah@openldap.org --- RE26:
• be85886d by Howard Chu at 2021-11-23T22:18:45+00:00 ITS#9753 back-mdb: Fix index updating for replace ops
https://bugs.openldap.org/show_bug.cgi?id=9753
--- Comment #4 from Quanah Gibson-Mount quanah@openldap.org --- RE25:
• 15631011 by Howard Chu at 2021-11-23T22:19:46+00:00 ITS#9753 back-mdb: Fix index updating for replace ops
https://bugs.openldap.org/show_bug.cgi?id=9753
--- Comment #5 from Quanah Gibson-Mount quanah@openldap.org --- Note: Also affects back-bdb and back-hdb. There will be no fix from the OpenLDAP project for those backends.
https://bugs.openldap.org/show_bug.cgi?id=9753
--- Comment #6 from Michael Ströder michael@stroeder.com --- What is the impact of this issue?
Given that admin guide advices to index entryCSN and entryCSN is updated with a MOD_REPLACE (according to reqMod values in accesslog DB) this issue could cause various replication issues hard to track down.
Also the fix has not been back-ported to RE24 branch yet. Are there any plans to add the fix also to release 2.4.60?
https://bugs.openldap.org/show_bug.cgi?id=9753
--- Comment #7 from Howard Chu hyc@openldap.org --- (In reply to Michael Ströder from comment #6)
What is the impact of this issue?
Given that admin guide advices to index entryCSN and entryCSN is updated with a MOD_REPLACE (according to reqMod values in accesslog DB) this issue could cause various replication issues hard to track down.
This is precisely the impact we observed.
Also the fix has not been back-ported to RE24 branch yet. Are there any plans to add the fix also to release 2.4.60?
Not at the moment. 2.4 EOL was announced at the beginning of this year. Officially 2.4 is dead. There's some debate about whether to resurrect it for this. I will state up front: non-contributors get no vote.
https://bugs.openldap.org/show_bug.cgi?id=9753
--- Comment #8 from Michael Ströder michael@stroeder.com --- On 11/24/21 23:12, openldap-its@openldap.org wrote:
Are there any plans to add the fix also to release 2.4.60?
Not at the moment. 2.4 EOL was announced at the beginning of this year. Officially 2.4 is dead. There's some debate about whether to resurrect it for this. I will state up front: non-contributors get no vote.
Hmm, the functional model is so seriously broken that all the 2.4 installations out there are affected.
Even if you're not releasing 2.4.60 downstream packagers should have a fair chance to easily pick up the back-port patch from RE24 branch.
https://bugs.openldap.org/show_bug.cgi?id=9753
--- Comment #9 from Michael Ströder michael@stroeder.com --- Thinking about this some more: It also seems to have pretty large security impact for all systems/components relying on index attributes for access control decisions, probably also slapd processing ACLs, slapo-unique, slapo-constraint.
IMO it would be worth a public warning via openldap-announce list.
https://bugs.openldap.org/show_bug.cgi?id=9753
--- Comment #10 from Michael Ströder michael@stroeder.com --- A customer running Æ-DIR confirmed that the fix for RE26 also fixes searching with pwdChangedTime as assertion type (used in Æ-DIR for sending password expiration warnings).
https://bugs.openldap.org/show_bug.cgi?id=9753
--- Comment #12 from Howard Chu hyc@openldap.org --- (In reply to Michael Ströder from comment #9)
Thinking about this some more: It also seems to have pretty large security impact for all systems/components relying on index attributes for access control decisions, probably also slapd processing ACLs, slapo-unique, slapo-constraint.
ACL evaluation performs no search operations; it only compares a filter against the entry currently being checked. There is no security impact from this.
IMO it would be worth a public warning via openldap-announce list.
https://bugs.openldap.org/show_bug.cgi?id=9753
--- Comment #13 from Michael Ströder michael@stroeder.com --- (In reply to Howard Chu from comment #12)
ACL evaluation performs no search operations; it only compares a filter against the entry currently being checked. There is no security impact from this.
What about set-based ACLs with ldap://... parts?
And IMO correct functional behaviour of slapo-unique and slapo-constraint is also security-relevant.
https://bugs.openldap.org/show_bug.cgi?id=9753
--- Comment #14 from Quanah Gibson-Mount quanah@openldap.org --- (In reply to Michael Ströder from comment #13)
(In reply to Howard Chu from comment #12)
ACL evaluation performs no search operations; it only compares a filter against the entry currently being checked. There is no security impact from this.
What about set-based ACLs with ldap://... parts?
And IMO correct functional behaviour of slapo-unique and slapo-constraint is also security-relevant.
Did you enable 64-bit indexing in your configuration?
https://bugs.openldap.org/show_bug.cgi?id=9753
--- Comment #15 from Quanah Gibson-Mount quanah@openldap.org --- slapo-dynlist would be another module that could potentially be affected by this bug, depending on how it was configured.
https://bugs.openldap.org/show_bug.cgi?id=9753
Howard Chu hyc@openldap.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Version|2.4.6 |2.5.9
--- Comment #16 from Howard Chu hyc@openldap.org --- Upon review, this bug was misreported. It does not occur at all in 2.4. It hits in 2.5 when 64bit index hashes are configured, on a single valued attribute without substring indexing.
https://bugs.openldap.org/show_bug.cgi?id=9753
--- Comment #17 from Michael Ströder michael@stroeder.com --- Quanah Gibson-Mount quanah@openldap.org wrote:
Did you enable 64-bit indexing in your configuration?
I did not set directive index_hash64 at all, so it uses 32-bit indexing by default.
https://bugs.openldap.org/show_bug.cgi?id=9753
--- Comment #18 from Shawn McKinney smckinney@symas.com --- (In reply to Michael Ströder from comment #17)
Quanah Gibson-Mount quanah@openldap.org wrote:
Did you enable 64-bit indexing in your configuration?
I did not set directive index_hash64 at all, so it uses 32-bit indexing by default.
Hey Michael,
I see that a pwdChangedTime search was broke, then worked after the patch. Can you provide specifics as to a test case to repeat this error?
Anything that would help me to verify this in a test env.
Thanks
-- Shawn
https://bugs.openldap.org/show_bug.cgi?id=9753
--- Comment #19 from Michael Ströder michael@stroeder.com ---
--- Comment #18 from Shawn McKinney smckinney@symas.com --- I see that a pwdChangedTime search was broke, then worked after the patch. Can you provide specifics as to a test case to repeat this error?
It was the usual search for entries with passwords which will expire soon (password expiry warning period).
Example filter:
(&(pwdChangedTime>=20210609160107Z)(pwdChangedTime<=20210708160107Z))
The search did not return all the expected results.
https://bugs.openldap.org/show_bug.cgi?id=9753
--- Comment #20 from Shawn McKinney smckinney@symas.com --- (In reply to Michael Ströder from comment #19)
--- Comment #18 from Shawn McKinney smckinney@symas.com --- I see that a pwdChangedTime search was broke, then worked after the patch. Can you provide specifics as to a test case to repeat this error?
It was the usual search for entries with passwords which will expire soon (password expiry warning period).
Example filter:
(&(pwdChangedTime>=20210609160107Z)(pwdChangedTime<=20210708160107Z))
The search did not return all the expected results.
This helps. More questions, what version? Also, can you supply an (ldif) example of an entry that fit the criteria but wasn't returned by the search?
Is the condition repeatable or intermittent? Can you recreate it?
Thanks
-- Shawn
https://bugs.openldap.org/show_bug.cgi?id=9753
--- Comment #21 from Shawn McKinney smckinney@symas.com --- (In reply to Shawn McKinney from comment #20)
(In reply to Michael Ströder from comment #19)
--- Comment #18 from Shawn McKinney smckinney@symas.com --- I see that a pwdChangedTime search was broke, then worked after the patch. Can you provide specifics as to a test case to repeat this error?
It was the usual search for entries with passwords which will expire soon (password expiry warning period).
Example filter:
(&(pwdChangedTime>=20210609160107Z)(pwdChangedTime<=20210708160107Z))
The search did not return all the expected results.
This helps. More questions, what version? Also, can you supply an (ldif) example of an entry that fit the criteria but wasn't returned by the search?
Is the condition repeatable or intermittent? Can you recreate it?
Thanks
-- Shawn
Setup test env, running 2.5 before the patch. 64-bit indexing not enabled.
Inserted 100K users, then updated their passwords.
Running this search pulls back all 100K. At this point, don't know how to recreate this error.
``` [root@tx01 ~]# ldapsearch -x -LLL -H ldap://tx01 -D "dc=example,dc=com" -w -s sub -b 'dc=example,dc=com' "(&(pwdChangedTime>=20211206230000Z)(pwdChangedTime<=20211206231000Z))" numsubordinates | grep -w -c "dn:" 100000 ```
-- Shawn
https://bugs.openldap.org/show_bug.cgi?id=9753
--- Comment #22 from Shawn McKinney smckinney@symas.com --- (In reply to Shawn McKinney from comment #21)
Setup test env, running 2.5 before the patch. 64-bit indexing not enabled.
Inserted 100K users, then updated their passwords.
Oh, and added this index:
index pwdChangedTime
https://bugs.openldap.org/show_bug.cgi?id=9753
--- Comment #23 from Shawn McKinney smckinney@symas.com --- Knowing this defect triggered when replacing an indexed attribute, I added another 100K users, changed their passwords, and then changed their passwords again, effectively replacing pwdchangedtime.
Still pulling back all on search, as expected. Currently, don't have enough to pursue, but keeping an open mind.
Input welcome.
-- Shawn
https://bugs.openldap.org/show_bug.cgi?id=9753
--- Comment #24 from Michael Ströder michael@stroeder.com --- (In reply to Shawn McKinney from comment #20)
More questions, what version? Also, can you supply an (ldif) example of an entry that fit the criteria but wasn't returned by the search?
Is the condition repeatable or intermittent? Can you recreate it?
Sorry, I can't easily reproduce it because I don't have access to the system where this happened.
The customer is running my Debian packages which already has a longer history:
https://code.stroeder.com/AE-DIR/debian-openldap-ms/src/branch/master/debian...
There could also have been an issue when upgrading from 2.4.59 to 2.5.x which did not have compat fixes for 2.4 databases yet. At the time when I received the report the customer was running 2.6.0 but without back-port patch for ITS#9753.
Sorry, at the moment I'm not able to provide more enlightning information.
https://bugs.openldap.org/show_bug.cgi?id=9753
Quanah Gibson-Mount quanah@openldap.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|RESOLVED |VERIFIED Resolution|TEST |FIXED
-
openldap-its＠openldap.org