https://bugs.openldap.org/show_bug.cgi?id=9880
Issue ID: 9880 Summary: reqStart filter with trailing zeros is truncated, which breaks certain searches Product: OpenLDAP Version: 2.5.12 Hardware: All OS: All Status: UNCONFIRMED Keywords: needs_review Severity: normal Priority: --- Component: slapd Assignee: bugs@openldap.org Reporter: subbarao@computer.org Target Milestone: ---
Note: ITS#9358 seems to address this very issue, but it doesn't seem to help this particular case.
Certain range searches with reqStart on the changelog don't work as expected. For example:
(!(reqStart<=20220707123456.000000Z))
The idea here is to grab all entries strictly greater than the timestamp. But slapd truncates zeros in this filter, rewriting it to:
(!(reqStart<=20220707123456Z))
As a result, the reqStart=20220707123456.000000Z entry is the first match since it is greater than 20220707123456Z, which is not the desired behavior.
I was able to reproduce this issue on 2.5.12 as follows:
1) Start the test043-delta-syncrepl test, let it run almost to the end so that it makes many changes, and then hit ^Z to suspend the script. I waited until one of the last occurrences of "Waiting 7 seconds for syncrepl to receive changes".
2) pkill -CONT -f slapd to restart slapd (but not the test script)
3) ldapsearch -x -h localhost:9011 -b cn=log objectclass=top | grep '^dn:.*.000000Z'
Look for a change with all trailing zeros. It's not as rare as one might think, I saw at least one trailing-zeros change in two consecutive runs of the test script. I suppose you could also just create an entry with all trailing zeros in the accesslog :-)
4) Run a range search to only return changes after that change (I used -z 1 and -s one so that it would only give me one result):
ldapsearch -x -z 1 -h localhost:9011 -b cn=log -s one '(!(reqStart<=20220708012121.000000Z))'
If you see the same entry, then the problem is present.
5) Even if you can't have an accesslog entry with all trailing zeros, you can still do the above search verbatim and look at the slapd log file testrun/slapd.1.log:
62c7873f.145b4396 0x7fabf14c9700 conn=1011 op=1 SRCH base="cn=log" scope=1 deref =0 filter="(!(reqStart<=20220708012121Z))"
The filter being rewritten in the server log seems to indicate that trailing zeros are being truncated somewhere.
https://bugs.openldap.org/show_bug.cgi?id=9880
subbarao@computer.org subbarao@computer.org changed:
What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.openldap.org/s | |how_bug.cgi?id=9358 CC| |subbarao@computer.org
https://bugs.openldap.org/show_bug.cgi?id=9880
Howard Chu hyc@openldap.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|UNCONFIRMED |CONFIRMED Ever confirmed|0 |1
--- Comment #1 from Howard Chu hyc@openldap.org --- The generalizedTimeNormalize function truncates the fractional part if it's zero.
https://bugs.openldap.org/show_bug.cgi?id=9880
Quanah Gibson-Mount quanah@openldap.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Keywords|needs_review | Target Milestone|--- |2.7.0 Assignee|bugs@openldap.org |hyc@openldap.org
--- Comment #2 from Quanah Gibson-Mount quanah@openldap.org --- The fix will require rebuilding all timestamp based indexes, target is 2.7 release.
https://bugs.openldap.org/show_bug.cgi?id=9880
--- Comment #3 from Howard Chu hyc@openldap.org --- A less intrusive fix may be to just change accesslog to truncate the fractional part from reqStart when it would be all zeroes.
https://bugs.openldap.org/show_bug.cgi?id=9880
--- Comment #4 from Howard Chu hyc@openldap.org --- Ignore previous comment. As shown in ITS#9358 the reqStart timestamp must be constant length.
The question here is why the filter's zeroes are being truncated, since the normalizer for reqStart explicitly preserves them.
https://bugs.openldap.org/show_bug.cgi?id=9880
Howard Chu hyc@openldap.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|CONFIRMED |IN_PROGRESS
--- Comment #5 from Howard Chu hyc@openldap.org --- https://git.openldap.org/openldap/openldap/-/merge_requests/584
https://bugs.openldap.org/show_bug.cgi?id=9880
Howard Chu hyc@openldap.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Target Milestone|2.7.0 |2.5.14
https://bugs.openldap.org/show_bug.cgi?id=9880
Quanah Gibson-Mount quanah@openldap.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|IN_PROGRESS |RESOLVED Resolution|--- |FIXED
--- Comment #6 from Quanah Gibson-Mount quanah@openldap.org --- head:
• 1f251fa4 by Howard Chu at 2022-12-07T14:58:35+00:00 ITS#9880 slapo-accesslog: fix reqStart ordering filter
RE26:
• 54cff1c6 by Howard Chu at 2023-01-09T18:37:12+00:00 ITS#9880 slapo-accesslog: fix reqStart ordering filter
RE25:
• 84830fdc by Howard Chu at 2023-01-09T18:38:15+00:00 ITS#9880 slapo-accesslog: fix reqStart ordering filter
https://bugs.openldap.org/show_bug.cgi?id=9880
Quanah Gibson-Mount quanah@openldap.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|RESOLVED |VERIFIED