Petteri.Stenius@ubisecure.com wrote:
Hello,
Further investigation shows this issue is caused by operator LE search failing with indexed attributes. Also this indexed search issue is NOT limited to DDS.=20
I have reproduced the issue with integerOrderingMatch and generalizedTimeOrderingMatch.=20
The piece of code I find suspicious is in servers/back-bdb/idl.c, somewhere in the middle it reads
/* skip presence key on range inequality lookups */ while (rc =3D=3D 0&& kptr->size !=3D len) { rc =3D cursor->c_get( cursor, kptr,&data, flags | DB_NEXT_NODUP ); }
If I remove this block then LE search works as expected with indexed attributes. The key here seems to be the DB_NEXT_NODUP flag. This flag causes the iterator block a few lines below to return partial matches.
That implies that there's something else corrupt in the index, because the presence key will never be the same size as an equality key.
Thanks, Petteri
-----Original Message----- From: openldap-bugs-bounces@OpenLDAP.org [mailto:openldap-bugs-bounces@OpenLDAP.org] On Behalf Of petteri.stenius@ubisecure.com Sent: Monday, October 25, 2010 1:35 PM To: openldap-its@openldap.org Subject: (ITS#6683) DDS fails with expired branches
Full_Name:=20 Version: 2.4.23 OS: Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (195.197.205.34)
Hello,
I have a directory with branches of dynamicObject entries. It looks like if the entryExpireTimestamp value is the same on objects within a branch then DDS search for expired objects will only find the top-most object. This results in remove failing with message
DDS dn=3D"cn=3Dtop,cn=3Droot,dc=3Dtest" is non-leaf; deferring.
To reproduce
OpenLDAP 2.4.23, Berkeley DB 4.6.21
Use slapadd to prepare directory with following
dn: cn=3DRoot,dc=3Dtest objectClass: top objectClass: applicationProcess cn: Root
dn: cn=3Dtop,cn=3DRoot,dc=3Dtest objectClass: top objectClass: device objectClass: dynamicObject entryTTL: 60 entryExpireTimestamp: 20101024113626Z cn: top
dn: cn=3Dleaf1,cn=3Dtop,cn=3DRoot,dc=3Dtest objectClass: top objectClass: device objectClass: dynamicObject entryTTL: 60 entryExpireTimestamp: 20101024113626Z cn: leaf1
dn: cn=3Dleaf2,cn=3Dtop,cn=3DRoot,dc=3Dtest objectClass: top objectClass: device objectClass: dynamicObject entryTTL: 60 entryExpireTimestamp: 20101024113626Z cn: leaf2
dn: cn=3Dleaf3,cn=3Dtop,cn=3DRoot,dc=3Dtest objectClass: top objectClass: device objectClass: dynamicObject entryTTL: 60 entryExpireTimestamp: 20101024113626Z cn: leaf3
Relevant slapd.conf entries
database bdb suffix "cn=3DRoot,dc=3Dtest" rootdn "cn=3DRoot,dc=3Dtest" rootpw "password"
overlay dds dds-default-ttl 3600 dds-min-ttl 60 dds-interval 60 dds-state true index entryExpireTimestamp eq,pres
access to dn.subtree=3D"cn=3DRoot,dc=3Dtest" by users write by * read
Running "slapd -d 1 -d 256" produces following
put_filter: "(&(objectClass=3DdynamicObject)(entryExpireTimestamp<=3D20101025082446Z)= )" put_filter: AND put_filter_list "(objectClass=3DdynamicObject)(entryExpireTimestamp<=3D20101025082446Z)" put_filter: "(objectClass=3DdynamicObject)" put_filter: simple put_simple_filter: "objectClass=3DdynamicObject" put_filter: "(entryExpireTimestamp<=3D20101025082446Z)" put_filter: simple put_simple_filter: "entryExpireTimestamp<=3D20101025082446Z" ber_scanf fmt ({mm}) ber: ber_scanf fmt ({mm}) ber: =3D> bdb_search bdb_dn2entry("cn=3Droot,dc=3Dtest") =3D> bdb_dn2id("cn=3Droot,dc=3Dtest") <=3D bdb_dn2id: got id=3D0x1 entry_decode: "cn=3DRoot,dc=3Dtest" <=3D entry_decode(cn=3DRoot,dc=3Dtest) search_candidates: base=3D"cn=3Droot,dc=3Dtest" (0x00000001) scope=3D2 =3D> bdb_dn2idl("cn=3Droot,dc=3Dtest") =3D> bdb_equality_candidates (objectClass) =3D> key_read <=3D bdb_index_read: failed (-30989) <=3D bdb_equality_candidates: id=3D0, first=3D0, last=3D0 =3D> bdb_equality_candidates (objectClass) =3D> key_read <=3D bdb_index_read 4 candidates <=3D bdb_equality_candidates: id=3D4, first=3D2, last=3D5 =3D> bdb_inequality_candidates (entryExpireTimestamp) =3D> key_read <=3D bdb_index_read 1 candidates =3D> key_read <=3D bdb_index_read: failed (-30989) <=3D bdb_inequality_candidates: id=3D1, first=3D2, last=3D2 bdb_search_candidates: id=3D1 first=3D2 last=3D2 entry_decode: "cn=3Dtop,cn=3DRoot,dc=3Dtest" <=3D entry_decode(cn=3Dtop,cn=3DRoot,dc=3Dtest) =3D> bdb_dn2id("cn=3Dtop,cn=3Droot,dc=3Dtest") <=3D bdb_dn2id: got id=3D0x2 send_ldap_result: conn=3D-1 op=3D0 p=3D0 bdb_dn2entry("cn=3Dtop,cn=3Droot,dc=3Dtest") =3D> bdb_dn2id_children("cn=3Dtop,cn=3Droot,dc=3Dtest") <=3D bdb_dn2id_children("cn=3Dtop,cn=3Droot,dc=3Dtest"): (0) send_ldap_result: conn=3D-1 op=3D0 p=3D0 DDS dn=3D"cn=3Dtop,cn=3Droot,dc=3Dtest" is non-leaf; deferring. DDS expired=3D0
ldapsearch "(entryExpireTimestamp=3D*)" produces
dn: cn=3Dtop,cn=3DRoot,dc=3Dtest entryExpireTimestamp: 20101024113626Z
dn: cn=3Dleaf1,cn=3Dtop,cn=3DRoot,dc=3Dtest entryExpireTimestamp: 20101024113626Z
dn: cn=3Dleaf2,cn=3Dtop,cn=3DRoot,dc=3Dtest entryExpireTimestamp: 20101024113626Z
dn: cn=3Dleaf3,cn=3Dtop,cn=3DRoot,dc=3Dtest entryExpireTimestamp: 20101024113626Z
where ldapsearch "(entryExpireTimestamp<=3D20101024113626Z)" only finds
dn: cn=3Dtop,cn=3DRoot,dc=3Dtest entryExpireTimestamp: 20101024113626Z
If I change all timestamps to distinct values then expiration of complete branches works as expected.
Thanks, Petteri