(ITS#7266) back-mdb deletion failures
by quanah＠OpenLDAP.org
Full_Name: Quanah Gibson-Mount
Version: 2.4.31
OS: Linux 2.6
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (75.108.184.39)
Create an MDB database with approximately 130,000 entries.
Delete the entries sequentially. Deletes will sporadically fail in blocks:
deleting entry "uid=testuser44968,ou=people,dc=zre-ldap004,dc=eng,dc=vmware,dc=com"
deleting entry "uid=testuser44969,ou=people,dc=zre-ldap004,dc=eng,dc=vmware,dc=com"
ldap_delete: Other (e.g., implementation specific) error (80)
additional info: entry index delete failed
deleting entry "uid=testuser44970,ou=people,dc=zre-ldap004,dc=eng,dc=vmware,dc=com"
ldap_delete: Other (e.g., implementation specific) error (80)
additional info: entry index delete failed
...
continues to fail up to:
deleting entry "uid=testuser45093,ou=people,dc=zre-ldap004,dc=eng,dc=vmware,dc=com"
ldap_delete: Other (e.g., implementation specific) error (80)
additional info: entry index delete failed
deleting entry "uid=testuser45094,ou=people,dc=zre-ldap004,dc=eng,dc=vmware,dc=com"
ldap_delete: Other (e.g., implementation specific) error (80)
additional info: entry index delete failed
deleting entry "uid=testuser45095,ou=people,dc=zre-ldap004,dc=eng,dc=vmware,dc=com"
ldap_delete: Other (e.g., implementation specific) error (80)
additional info: entry index delete failed
deleting entry "uid=testuser45096,ou=people,dc=zre-ldap004,dc=eng,dc=vmware,dc=com"
deleting entry "uid=testuser45097,ou=people,dc=zre-ldap004,dc=eng,dc=vmware,dc=com"
deleting entry "uid=testuser45098,ou=people,dc=zre-ldap004,dc=eng,dc=vmware,dc=com"
deleting entry "uid=testuser45099,ou=people,dc=zre-ldap004,dc=eng,dc=vmware,dc=com"
then deletes work for a while again, then fail again for a while, etc.
11 years, 4 months
Re: (ITS#7249) slapd segfault with memberof overlay on frontend db
by jvcelak＠redhat.com
--nextPart2505210.xvcxic42sH
Content-Transfer-Encoding: 7Bit
Content-Type: text/plain; charset="us-ascii"
On Friday 27 of April 2012 12:26:22, Howard Chu wrote:
> jvcelak(a)redhat.com wrote:
> >> What happens is that the memberof overlay, when stacked on the frontend
> >> database, keeps looping until the stack is exhausted, since internal
> >> modifications keep calling the frontend's modify hook rather than the
> >> actual one that needs to be called.
> >
> > I have no idea how to fix it. I probably do not understand the
> > manipulation
> > with BackedInfo structures fully.
> >
> > It is obious that it is caused by doing a search from the modrdn operation
> > callback memberof_res_modrdn() which is set up in memberof_op_modrdn().
> > Other backends does not call any backend operation at the same moment
> > (op->callback).
> >
> > Frontend fe_op_search() is called. It tries to choose the right backend
> > with select_backend(). Which returns the overlay backend (bi_type ==
> > "over"), over_op_walk() then selects fe_op_search() again, and we are
> > looping infinitely.
> >
> > I have no clue how to hide or temporarily deactivate the overlay in
> > memberof_res_modrdn() to make the select_backend() function to choose the
> > right underlaying backend. Or maybe I have choosen a wrong way of fixing
> > it.
> >
> > All ideas are welcomed. :-)
>
> The attached diff fixes this particular problem. I haven't spent any time to
> see if the other be_* invocations need to be protected the same way.
Thank you. I have added this check to another places (attaching patch). But
it seems that it is not sufficient. In certain situations I can still trigger
the segfault - from memberof_res_modrdn() by calling backend_attribute().
At a first glance, the circumstances are a little bit different but I see
no difference in the structures when calling the function with overlay
set on frontend and backend database. I hope I will have some time to look
at it soon.
(gdb) bt -30
#35697 0x00000000004d9368 in overlay_entry_get_ov (op=0x7fffe40028f0, dn=0x7ffff23d9050, oc=0x0, ad=0x9e8450, rw=0, e=0x7ffff23d8d28,
on=0x0) at ../../../servers/slapd/backover.c:364
#35698 0x00000000004d9460 in over_entry_get_rw (op=0x7fffe40028f0, dn=0x7ffff23d9050, oc=0x0, ad=0x9e8450, rw=0, e=0x7ffff23d8d28)
at ../../../servers/slapd/backover.c:396
#35699 0x00000000004dd448 in fe_entry_get_rw (op=0x7fffe40028f0, ndn=0x7ffff23d9050, oc=0x0, at=0x9e8450, rw=0, e=0x7ffff23d8d28)
at ../../../servers/slapd/frontend.c:62
#35700 0x00000000004d9368 in overlay_entry_get_ov (op=0x7fffe40028f0, dn=0x7ffff23d9050, oc=0x0, ad=0x9e8450, rw=0, e=0x7ffff23d8d28,
on=0x0) at ../../../servers/slapd/backover.c:364
#35701 0x00000000004d9460 in over_entry_get_rw (op=0x7fffe40028f0, dn=0x7ffff23d9050, oc=0x0, ad=0x9e8450, rw=0, e=0x7ffff23d8d28)
at ../../../servers/slapd/backover.c:396
#35702 0x00000000004dd448 in fe_entry_get_rw (op=0x7fffe40028f0, ndn=0x7ffff23d9050, oc=0x0, at=0x9e8450, rw=0, e=0x7ffff23d8d28)
at ../../../servers/slapd/frontend.c:62
#35703 0x00000000004d9368 in overlay_entry_get_ov (op=0x7fffe40028f0, dn=0x7ffff23d9050, oc=0x0, ad=0x9e8450, rw=0, e=0x7ffff23d8d28,
on=0x0) at ../../../servers/slapd/backover.c:364
#35704 0x00000000004d9460 in over_entry_get_rw (op=0x7fffe40028f0, dn=0x7ffff23d9050, oc=0x0, ad=0x9e8450, rw=0, e=0x7ffff23d8d28)
at ../../../servers/slapd/backover.c:396
#35705 0x00000000004dd448 in fe_entry_get_rw (op=0x7fffe40028f0, ndn=0x7ffff23d9050, oc=0x0, at=0x9e8450, rw=0, e=0x7ffff23d8d28)
at ../../../servers/slapd/frontend.c:62
#35706 0x00000000004d9368 in overlay_entry_get_ov (op=0x7fffe40028f0, dn=0x7ffff23d9050, oc=0x0, ad=0x9e8450, rw=0, e=0x7ffff23d8d28,
on=0x0) at ../../../servers/slapd/backover.c:364
#35707 0x00000000004d9460 in over_entry_get_rw (op=0x7fffe40028f0, dn=0x7ffff23d9050, oc=0x0, ad=0x9e8450, rw=0, e=0x7ffff23d8d28)
at ../../../servers/slapd/backover.c:396
#35708 0x000000000044fc46 in be_entry_get_rw (op=0x7fffe40028f0, ndn=0x7ffff23d9050, oc=0x0, at=0x9e8450, rw=0, e=0x7ffff23d8d28)
at ../../../servers/slapd/backend.c:1366
#35709 0x0000000000450990 in fe_acl_attribute (op=0x7fffe40028f0, target=0x0, edn=0x7ffff23d9050, entry_at=0x9e8450,
vals=0x7ffff23d9088, access=ACL_READ) at ../../../servers/slapd/backend.c:1658
#35710 0x00000000004d9cdb in over_acl_attribute (op=0x7fffe40028f0, target=0x0, entry_ndn=0x7ffff23d9050, entry_at=0x9e8450,
vals=0x7ffff23d9088, access=ACL_READ) at ../../../servers/slapd/backover.c:589
#35711 0x0000000000450ed5 in backend_attribute (op=0x7fffe40028f0, target=0x0, edn=0x7ffff23d9050, entry_at=0x9e8450,
vals=0x7ffff23d9088, access=ACL_READ) at ../../../servers/slapd/backend.c:1778
#35712 0x00000000005a4c9f in memberof_res_modrdn (op=0x7fffe40028f0, rs=0x7ffff23d9a60)
at ../../../../servers/slapd/overlays/memberof.c:1558
#35713 0x000000000045252c in slap_response_play (op=0x7fffe40028f0, rs=0x7ffff23d9a60) at ../../../servers/slapd/result.c:507
#35714 0x0000000000452768 in send_ldap_response (op=0x7fffe40028f0, rs=0x7ffff23d9a60) at ../../../servers/slapd/result.c:582
#35715 0x0000000000453952 in slap_send_ldap_result (op=0x7fffe40028f0, rs=0x7ffff23d9a60) at ../../../servers/slapd/result.c:860
#35716 0x000000000050dfa9 in bdb_modrdn (op=0x7fffe40028f0, rs=0x7ffff23d9a60) at ../../../../servers/slapd/back-bdb/modrdn.c:789
#35717 0x00000000004625be in fe_op_modrdn (op=0x7fffe40028f0, rs=0x7ffff23d9a60) at ../../../servers/slapd/modrdn.c:314
#35718 0x00000000004d9ed4 in overlay_op_walk (op=0x7fffe40028f0, rs=0x7ffff23d9a60, which=op_modrdn, oi=0x7fffe41050e0, on=0x0)
at ../../../servers/slapd/backover.c:671
#35719 0x00000000004da125 in over_op_func (op=0x7fffe40028f0, rs=0x7ffff23d9a60, which=op_modrdn)
at ../../../servers/slapd/backover.c:723
#35720 0x00000000004da2b2 in over_op_modrdn (op=0x7fffe40028f0, rs=0x7ffff23d9a60) at ../../../servers/slapd/backover.c:768
#35721 0x0000000000461d20 in do_modrdn (op=0x7fffe40028f0, rs=0x7ffff23d9a60) at ../../../servers/slapd/modrdn.c:186
#35722 0x000000000043a8df in connection_operation (ctx=0x7ffff23d9ba0, arg_v=0x7fffe40028f0)
at ../../../servers/slapd/connection.c:1150
#35723 0x000000000043ae9c in connection_read_thread (ctx=0x7ffff23d9ba0, argv=0x15) at ../../../servers/slapd/connection.c:1286
#35724 0x00000000005d12ce in ldap_int_thread_pool_wrapper (xpool=0x9b0bf0) at ../../../libraries/libldap_r/tpool.c:688
#35725 0x00007ffff73e4d90 in start_thread () from /lib64/libpthread.so.0
#35726 0x00007ffff5cf5f5d in clone () from /lib64/libc.so.6
--nextPart2505210.xvcxic42sH
Content-Disposition: attachment; filename="memberof_partial.patch"
Content-Transfer-Encoding: 7Bit
Content-Type: text/x-patch; charset="UTF-8"; name="memberof_partial.patch"
diff --git a/servers/slapd/overlays/memberof.c b/servers/slapd/overlays/memberof.c
index 502cb46..356c1e6 100644
--- a/servers/slapd/overlays/memberof.c
+++ b/servers/slapd/overlays/memberof.c
@@ -190,7 +190,16 @@ typedef struct memberof_cbinfo_t {
BerVarray memberof;
memberof_is_t what;
} memberof_cbinfo_t;
-
+
+static void
+memberof_set_backend(Operation *op_target, Operation *op, slap_overinst *on)
+{
+ BackendInfo *bi = op->o_bd->bd_info;
+
+ if (bi->bi_type == memberof.on_bi.bi_type)
+ op_target->o_bd->bd_info = (BackendInfo *)on->on_info;
+}
+
static int
memberof_isGroupOrMember_cb( Operation *op, SlapReply *rs )
{
@@ -285,7 +294,7 @@ memberof_isGroupOrMember( Operation *op, memberof_cbinfo_t *mci )
op2.ors_filterstr = mo->mo_groupFilterstr;
op2.ors_filter = &mo->mo_groupFilter;
- op2.o_bd->bd_info = (BackendInfo *)on->on_info;
+ memberof_set_backend(&op2, op, on);
(void)op->o_bd->be_search( &op2, &rs2 );
op2.o_bd->bd_info = bi;
@@ -307,7 +316,7 @@ memberof_isGroupOrMember( Operation *op, memberof_cbinfo_t *mci )
op2.ors_filterstr = mo->mo_memberFilterstr;
op2.ors_filter = &mo->mo_memberFilter;
- op2.o_bd->bd_info = (BackendInfo *)on->on_info;
+ memberof_set_backend(&op2, op, on);
(void)op->o_bd->be_search( &op2, &rs2 );
op2.o_bd->bd_info = bi;
@@ -407,7 +416,7 @@ memberof_value_modify(
oex.oe_key = (void *)&memberof;
LDAP_SLIST_INSERT_HEAD(&op2.o_extra, &oex, oe_next);
- op2.o_bd->bd_info = (BackendInfo *)on->on_info;
+ memberof_set_backend(&op2, op, on);
(void)op->o_bd->be_modify( &op2, &rs2 );
op2.o_bd->bd_info = bi;
LDAP_SLIST_REMOVE(&op2.o_extra, &oex, OpExtra, oe_next);
@@ -449,7 +458,7 @@ memberof_value_modify(
oex.oe_key = (void *)&memberof;
LDAP_SLIST_INSERT_HEAD(&op2.o_extra, &oex, oe_next);
- op2.o_bd->bd_info = (BackendInfo *)on->on_info;
+ memberof_set_backend(&op2, op, on);
(void)op->o_bd->be_modify( &op2, &rs2 );
op2.o_bd->bd_info = bi;
LDAP_SLIST_REMOVE(&op2.o_extra, &oex, OpExtra, oe_next);
--nextPart2505210.xvcxic42sH--
11 years, 4 months
Re: (ITS#7255) back-mdb hangs during mdb_tool_entry_modify
by hyc＠symas.com
ondrej.kuznik(a)acision.com wrote:
> Full_Name: Ondrej Kuznik
> Version: master
> OS: Linux
> URL: ftp://ftp.openldap.org/incoming/Ondrej-Kuznik-20120423-slapmodify-support...
> Submission from: (NULL) (62.168.56.1)
>
>
> When calling mdb_tool_entry_modify, a new transaction is being set up and this
> hangs the process on a mutex that the thread already holds.
>
> For a possible patch see the last patch in the series at the above URL.
Thanks for the report. Your fix is incomplete, a working fix is in git master now.
>
> The attached file is derived from OpenLDAP Software. All of the
> modifications to OpenLDAP Software represented in the following
> patch(es) were developed by Acision. Acision has not assigned rights
> and/or interest in this work to any party. I, Ondrej Kuznik am
> authorized by Acision, my employer, to release this work under the
> following terms.
>
> The attached modifications to OpenLDAP Software are subject to the
> following notice:
> Copyright 2012 Acision
> Redistribution and use in source and binary forms, with or without
> modification, are permitted only as authorized by the OpenLDAP Public
> License.
>
>
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
11 years, 4 months
(ITS#7262) ppolicy overlay cannot use a policy stored in a different backend from the account that it controls
by andrew.findlay＠skills-1st.co.uk
Full_Name: Andrew Findlay
Version: 2.4.26
OS: OpenSuSE 11.4
URL: ftp://ftp.openldap.org/incoming/andrew-findlay-2012050201.tar
Submission from: (NULL) (2a01:348:28c:1::94)
I have a setup where several OUs share a server, with one backend database per
OU.
Config data like password policies and service accounts is in another backend
DB.
The password policies do not work unless they are copied into each backend DB.
The attached tar contains a full test to demonstrate the problem.
Here is a summary of the README:
There are two databases, for suffixes dc=a,dc=example,dc=org and
dc=zzz,dc=example,dc=org (note different length, which helps later)
There are two accounts - a1 and zzz1 - each stored in a different backend
database.
Both accounts are locked with 'pwdAccountLockedTime: 00000101000000Z'
Each database uses the ppolicy overlay, and both have
cn=ppol-a,dc=a,dc=example,dc=org
as the default policy.
The run-test script does this:
echo "Binding as a1"
ldapwhoami -x -D uid=a1,dc=a,dc=example,dc=org -w "secret"
echo "Binding as zzz1"
ldapwhoami -x -D uid=zzz1,dc=zzz,dc=example,dc=org -w "secret"
Both accounts are locked so both should fail to bind.
In practice a1 fails correctly, but zzz1 binds.
If you run slapd with debug:
./start-slapd -d 65535
you can see some clues:
=> bdb_entry_get: found entry: "uid=zzz1,dc=zzz,dc=example,dc=org"
bdb_entry_get: rc=0
=> bdb_entry_get: ndn: "cn=ppol-a,dc=a,dc=example,dc=org"
=> bdb_entry_get: oc: "(null)", at: "(null)"
bdb_dn2entry("cn=ppol-a,dc=a,dc=example,dc=org")
=> hdb_dn2id("a,dc=a,dc=example,dc=org")
--------------^^
INVALID DN
<= hdb_dn2id: get failed: DB_NOTFOUND: No matching key/data pair found (-30988)
=> bdb_entry_get: cannot find entry: "cn=ppol-a,dc=a,dc=example,dc=org"
The corresponding lines for a1 are:
=> bdb_entry_get: found entry: "uid=a1,dc=a,dc=example,dc=org"
bdb_entry_get: rc=0
=> bdb_entry_get: ndn: "cn=ppol-a,dc=a,dc=example,dc=org"
=> bdb_entry_get: oc: "(null)", at: "(null)"
bdb_dn2entry("cn=ppol-a,dc=a,dc=example,dc=org")
=> hdb_dn2id("cn=ppol-a,dc=a,dc=example,dc=org")
The big clue here is this line:
=> hdb_dn2id("a,dc=a,dc=example,dc=org")
The invalid DN has the same length as the suffix of the other DB:
a,dc=a,dc=example,dc=org
dc=zzz,dc=example,dc=org
I think the overlay is looking for cn=ppol-a,dc=a,dc=example,dc=org in the DB
containing dc=zzz,dc=example,dc=org
Andrew
11 years, 4 months
