(ITS#6720) back-ldap core dump
by olivier.chirossel＠sfr.com
Full_Name: olivier chirossel
Version: 2.4.23
OS: linux
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (62.39.9.251)
Hi
I have to migrate from a openldap 2.3.x to openldap 2.4. During this migration
phase i want my openldap 2.3.x master replicate with slurpd to my new openldap
2.4.x master mirror mode architecture with some rewriting rules for split my big
directory.
I configure a proxy ldap using back-ldap with slapo-rwm overlay, but
operational attributes related to entry creation and modification are
proxied, even when i put « lastmod off » in the conf, contrary of the man of
slapd-ldap
So i put rw-map directives in the conf for strip operationnal attributes but
this kind of operation generate a core dump
dn: neufDhcpIP=10.145.250.119,suffix=9,o=auth,dc=neuf,dc=fr
changetype: modify
delete: neufDhcpOption
neufDhcpOption: 1;255.255.255.0
-
replace: entryCSN
entryCSN: 20100511215647Z#000024#00#000000
-
replace: modifiersName
modifiersName: cn=manager,dc=neuf,dc=fr
-
replace: modifyTimestamp
modifyTimestamp: 20100511215647Z
dn: neufDhcpIP=10.145.250.119,suffix=9,o=auth,dc=neuf,dc=fr
changetype: modify
add: neufDhcpOption
neufDhcpOption: 1;255.255.255.128
-
replace: entryCSN
entryCSN: 20100511215647Z#000024#00#000000
-
replace: modifiersName
modifiersName: cn=manager,dc=neuf,dc=fr
-
replace: modifyTimestamp
modifyTimestamp: 20100511215647Z
Core was generated by `openldap-2.4.23/servers/slapd/slapd -F
/etc/openldap/slapd-ldap.d -h ldap://10.'.
Program terminated with signal 6, Aborted.
[New process 7995]
[New process 7994]
[New process 7990]
[New process 7989]
#0 0x00007f01373f9ed5 in raise () from /lib/libc.so.6
(gdb) bt
#0 0x00007f01373f9ed5 in raise () from /lib/libc.so.6
#1 0x00007f01373fb3f3 in abort () from /lib/libc.so.6
#2 0x00007f0137436408 in __libc_message () from /lib/libc.so.6
#3 0x00007f013743b9a8 in malloc_printerr () from /lib/libc.so.6
#4 0x00007f013743dab6 in free () from /lib/libc.so.6
#5 0x000000000055b8e2 in ber_bvarray_free_x (a=0x26de840, ctx=0x0) at
memory.c:737
#6 0x000000000046e3a1 in slap_mod_free (mod=0x26d07f0, freeit=0) at mods.c:461
#7 0x000000000046e42d in slap_mods_free (ml=0x26d07f0, freevals=1) at
mods.c:481
#8 0x0000000000437960 in do_modify (op=0x25d02e0, rs=0x41d24cb0) at
modify.c:187
#9 0x000000000041f6ef in connection_operation (ctx=0x41d24e00, arg_v=<value
optimized out>) at connection.c:1109 #10 0x00000000004203dd in
connection_read_thread (ctx=0x41d24e00, argv=<value optimized out>) at
connection.c:1245
#11 0x0000000000534080 in ldap_int_thread_pool_wrapper (xpool=<value optimized
out>) at tpool.c:685
#12 0x00007f0137f21fc7 in start_thread () from /lib/libpthread.so.0
#13 0x00007f013749764d in clone () from /lib/libc.so.6
#14 0x0000000000000000 in ?? ()
(gdb)
My conf :
database ldap
suffix "dc=neuf,dc=fr"
rootdn "cn=manager,dc=neuf,dc=fr"
rootpw secret
updatedn "cn=manager,dc=neuf,dc=fr"
uri "ldap://127.0.0.1/"
idassert-bind bindmethod=simple binddn="cn=manager,dc=neuf,dc=fr"
credentials=secret
overlay rwm
rwm-rewriteEngine on
rwm-rewriteContext default
rwm-rewriteRule "neufDhcpIP=([0-9\\.]+)([0-9]{1}),ou=People,o=auth,dc=neuf,dc=fr$"
"neufDhcpIP=$1$2,suffix=$2,o=auth,dc=neuf,dc=fr"
rwm-map attribute neufDhcpIP neufDhcpIP
rwm-map attribute neufDhcp82 neufDhcp82
rwm-map attribute neufDhcpOption neufDhcpOption
rwm-map attribute neufDhcp60 neufDhcp60
rwm-map attribute neufDhcpRelay neufDhcpRelay
rwm-map attribute neufDhcpComment neufDhcpComment
rwm-map attribute neufDhcpUnique neufDhcpUnique
rwm-map attribute neufDhcpProvider neufDhcpProvider
rwm-map attribute neufDhcpFQDN neufDhcpFQDN
rwm-map attribute neufDhcp82b neufDhcp82b
rwm-map attribute neufDhcpMac neufDhcpMac
rwm-map attribute neufDhcpSiaddr neufDhcpSiaddr
rwm-map attribute neufDhcpBootfilename neufDhcpBootfilename
rwm-map attribute neufBTID neufBTID
rwm-map attribute neufBTOption neufBTOption
rwm-map attribute neufTWINOption neufTWINOption
rwm-map attribute neufE28Option neufE28Option
rwm-map attribute circuitid circuitid
rwm-map attribute clientid clientid
rwm-map attribute device device
rwm-map attribute clientoffer clientoffer
rwm-map attribute ispid ispid
rwm-map attribute suspended suspended
rwm-map attribute neufDhcpIPb neufDhcpIPb
rwm-map attribute neufDhcpOptionb neufDhcpOptionb
rwm-map attribute idworkflow idworkflow
rwm-map attribute neufDhcpRelayIP neufDhcpRelayIP
rwm-map attribute interceptID interceptID
rwm-map attribute interceptEndpoint interceptEndpoint
rwm-map attribute neufDhcpRadiusOption neufDhcpRadiusOption
rwm-map attribute suffix suffix
rwm-map attribute dc dc
rwm-map attribute o o
rwm-map attribute ou ou
rwm-map attribute cn cn
rwm-map objectclass neufDhcpOption1 neufDhcpOption1
rwm-map objectclass neufDhcpOption5 neufDhcpOption5
rwm-map objectclass neufDhcpOption1R neufDhcpOption1R
rwm-map objectclass neufDhcpOption35 neufDhcpOption35
rwm-map objectclass neufBTOption neufBTOption
rwm-map objectclass neufDhcpOptionwifi neufDhcpOptionwifi
rwm-map objectclass Dhcp82 Dhcp82
rwm-map objectclass ClientDevice ClientDevice
rwm-map objectclass DhcpDevice DhcpDevice
rwm-map objectclass suffixobj suffixobj
rwm-map objectclass OrganizationalUnit OrganizationalUnit
rwm-map objectclass Organization Organization
rwm-map objectclass dcObject dcObject
rwm-map objectclass domain domain
rwm-map objectclass device device
rwm-map attribute *
rwm-map objectclass *
Thanks in avance for your help
Regards,
12 years, 6 months
Re: (ITS#6700) memberOf overlay does not handle MODRDN correctly
by masarati＠aero.polimi.it
raphael.ouazana(a)linagora.com wrote:
> When using memberOf overlay, a MODRDN request on a member does not update
> correctly the matching group, even if the memberof-refint option is set to
> true.
>
> In my little understanding, memberof_value_modify calls op->o_bd->be_modify
> which calls back the overlay code, and finally never reach the backend.
Guess what, modrdn is the only operation test052 does not check :(
WOrking at it. Thanks, p.
12 years, 6 months
Re: (ITS#6625) draft-zeilenga-ldap-c-api-concurrency support
by h.b.furuseth＠usit.uio.no
Some issues with tests/progs/slapd-mtread.c:
- snprintf format "%04.4d" is not standard. How about just "%04d"?
- However, shouldn't 'me' be 'myidx' in those snprintfs and when
computing 'thisconn = (me + j) % noconns;' ?
- A thread ID can be a struct, in which case whoami() will fail.
OpenLDAP (the TLS code) already fails in that case, but still, no
need to add do that. Could pass 'myidx' around, or create a
thread-local variable for it with ldap_pvt_thread_key_create().
- It says "initially developed by Kurt Spanier", shouldn't that be
Doug Leavitt?
- scripts/test060-mt-hot takes a long time and can generate 2G of logs.
Maybe it should run with a lesser loglevel by default? Put
: ${SLAPD_DEBUG=0x4100} -- by default, avoid 1.-2 gigabytes of logs
before sourcing defines.sh.
--
Hallvard
12 years, 6 months
(ITS#6719) syncrepl_add_glue_ancestors() uninitialized return
by h.b.furuseth＠usit.uio.no
Full_Name: Hallvard B Furuseth
Version: 2.4.23, HEAD
OS:
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (129.240.6.233)
Submitted by: hallvard
syncrepl_add_glue_ancestors() returns 'rc' uninitialized if the 'else'
branch of 'if ( ndn.bv_len > be->be_nsuffix[0].bv_len )' is taken: Then
the loop is not entered.
This also means the 'else' branch is useless. Should it be doing
something more?
Found with gcc -O -Wuninitialized . 'make test' does not seem to enter
the 'else'.
12 years, 6 months
Re: (ITS#6710) Mods already refreshed on a forwarding server is lost by its consumer
by hyc＠symas.com
rein(a)OpenLDAP.org wrote:
> On 11/15/10 16:39 , rein(a)OpenLDAP.org wrote:
>> New syncprov consumers connecting to a forwarding server and presenting an
>> apperently up-to-date cookie will loose any mods that have already taken place
>> on the forwarding server if it itself is refreshing from its provider. This
>> should not be a problem if the forwarding server have a sufficiently large
>> syncprov log, but a fix for servers without is coming.
> The currently committed fix for this its leaves one problem open. If a
> forwarding server restarts in the middle of the refresh phase, after
> making some changes but before updating the csn, new consumers
> connecting with an apparently up to date csn set after the server comes
> up again will not know that the context is dirty and will loose these
> changes. The same problem arise if the server restarts between the time
> a locally initiated delete operation is performed in the database and
> the accompanying csn set is saved.
>
> A fix could be to always assume the context is dirty after start up, and
> thereby forcing all clients to undergo the refresh phase even if they
> are in sync until some operation that updates the csn set is performed.
> An unnecessary refresh is probably better than loosing changes..
I haven't had the opportunity to review these patches yet, but the bug
description sounds a little flaky to me.
The original design is this: when a consumer requests a refresh from the
provider, the provider uses a snapshot of its current contextCSN. All changes
from the consumer's cookie to this snapshot (inclusive) are sent to the
consumer. Any changes currently in progress that have not yet been committed
will be skipped until the next refresh. Nothing is lost, it's simply delayed,
and that's in accordance with syncrepl's "eventual convergence" model.
Likewise, the provider only updates its contextCSN when a change is fully
committed. syncprov should NOT need to defer any consumer while it has
outstanding mods. There is no reason that would be needed by the sync protocol.
Servers only update their contextCSN after an entire refresh has completed. If
a downstream consumer connects while a forwarder is still refreshing, the
consumer should receive nothing. (Or, it should only receive the changes
between its cookie and the server's committed contextCSN, if any.)
You may very well have found bugs in the implementation, but it sounds to me
like you've changed the overall behavior to something outside of the original
design.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
12 years, 6 months
(ITS#6718) Send new-cookie messages to all consumers
by rein＠OpenLDAP.org
Full_Name: Rein Tollevik
Version: CVS head
OS: Irrelevant
URL:
Submission from: (NULL) (84.215.59.121)
Submitted by: rein
When syncprov updates its csn as a result of syncrepl updating its, syncprov
currently sends NEW_COOKIE messages only to those consumers that replicate the
entry where the csn is stored. It should be sent to all consumers. A fix is
coming.
--
Rein Tollevik
Basefarm AS
12 years, 6 months
Re: (ITS#6710) Mods already refreshed on a forwarding server is lost by its consumer
by rein＠OpenLDAP.org
On 11/15/10 16:39 , rein(a)OpenLDAP.org wrote:
> New syncprov consumers connecting to a forwarding server and presenting an
> apperently up-to-date cookie will loose any mods that have already taken place
> on the forwarding server if it itself is refreshing from its provider. This
> should not be a problem if the forwarding server have a sufficiently large
> syncprov log, but a fix for servers without is coming.
The currently committed fix for this its leaves one problem open. If a
forwarding server restarts in the middle of the refresh phase, after
making some changes but before updating the csn, new consumers
connecting with an apparently up to date csn set after the server comes
up again will not know that the context is dirty and will loose these
changes. The same problem arise if the server restarts between the time
a locally initiated delete operation is performed in the database and
the accompanying csn set is saved.
A fix could be to always assume the context is dirty after start up, and
thereby forcing all clients to undergo the refresh phase even if they
are in sync until some operation that updates the csn set is performed.
An unnecessary refresh is probably better than loosing changes..
Rein
12 years, 6 months
(ITS#6717) syncprov performs unnecessary presentations/refreshes.
by cmikk＠qwest.net
Full_Name: Chris Mikkelson
Version: 2.4.23
OS: FreeBSD
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (204.147.85.37)
I have a multimaster setup, and some read-only slaves mirroring the data with
syncrepl. In some circumstances, I've noticed that restarting a slave server
leads to a full presentation, even though the masters have an adequately sized
sessionlog. This makes refreshOnly synchronization prohibitively expensive.
It appears that the problem is as follows:
The syncprov overlay uses the minimum CSN from the client cookie to decide where
to start synchronization, regardless of whether that CSN has changed relative to
the same SID's CSN on the provider. For example:
Provider state:
CSN: <1 second ago>...#001#...
CSN: <1 year ago>.....#002#...
sessionlog mincsn: <1 hour ago>...#001#...
Consumer state:
CSN: <1 minute ago>...#001#...
CSN: <1 year ago>.....#002#... <-- same as SID 2 CSN on provider
The syncprov overly will take the mincsn from the consumer cookie (the <1 year
ago>
one above) and compare it to the sessionlog mincsn when deciding whether to do a
full presentation. Since the SID 2 CSN hasn't changed in the above example, the
consumer is up to date on changes recorded on SID 2, so the SID 1 CSN from the
cookie should be used.
To reproduce:
* Set up slapd servers ID 1 and 2 as multimaster nodes, and a third slapd server
replicating from server 1.
* Insert one entry into server ID 2, and restart servers 1 and 2.
* Insert entries into Server ID 1.
* Perform enough modifications / deletions on Server ID 1 to cause entries to be
purged from the sessionlog (see ITS #6716).
* Wait for slave to catch up, then stop the slave.
* Do any modification on Server ID 1 (only need to change the SID 1 contextCSN)
* Restart slave. Server will present entries to the slave rather than replaying
the session log.
12 years, 6 months
