Re: Re: (ITS#8223) Replication error when slave is down and insert on master are made
by Frank.Offermanns＠caseris.de
Dies ist eine mehrteilige Nachricht im MIME-Format.
--=_alternative 00253B99C1257EAE_=
Content-Type: text/plain; charset="US-ASCII"
Hi Quanah,
as stated above I have the same problem with 2.4.41 (and lmdb).
Unfortunately, I am currently not able to build 2.4.42. I will report,
when I have a running 2.4.42.
But due to the fact, that the problem was in 2.4.25 and is still in
2.4.41, I would think, that it will also be in 2.4.42.
Best regards,
Frank
--=_alternative 00253B99C1257EAE_=
Content-Type: text/html; charset="US-ASCII"
<font size=2 face="sans-serif">Hi Quanah,</font>
<br>
<br><font size=2 face="sans-serif">as stated above I have the same problem
with 2.4.41 (and lmdb).</font>
<br>
<br><font size=2 face="sans-serif">Unfortunately, I am currently not able
to build 2.4.42. I will report, when I have a running 2.4.42.</font>
<br><font size=2 face="sans-serif">But due to the fact, that the problem
was in 2.4.25 and is still in 2.4.41, I would think, that it will also
be in 2.4.42.</font>
<br>
<br><font size=2 face="sans-serif">Best regards,</font>
<br><font size=2 face="sans-serif">Frank</font>
<br>
--=_alternative 00253B99C1257EAE_=--
8 years, 1 month
(ITS#8224) configure --disable-slapd doesn't ignore every slapd option
by ryan＠nardis.ca
Full_Name: Ryan Tandy
Version: RE24
OS: Debian
URL:
Submission from: (NULL) (24.68.37.4)
Submitted by: ryan
The configure script forcibly disables most slapd options if slapd is disabled,
but a few are missing: --enable-cleartext --enable-crypt --enable-lmpasswd
--enable-spasswd --enable-slp
Actually that whole section seems like it could be cleaned up/de-duplicated a
bit...
8 years, 1 month
Re: (ITS#8223) Replication error when slave is down and insert on master are made
by quanah＠zimbra.com
--On Wednesday, August 26, 2015 8:33 AM +0000 Frank.Offermanns(a)caseris.de
wrote:
> Full_Name: Frank Offermanns
> Version: 2.4.25 and 2.4.41
> OS: Windows
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (217.7.149.50)
2.4.25 is known to have numerous replication bugs. Why are you wasting
your time doing testing with it?
Run the /same/ version on both master and replica, preferably 2.4.42, and
then test.
--Quanah
--
Quanah Gibson-Mount
Platform Architect
Zimbra, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
8 years, 1 month
(ITS#8223) Replication error when slave is down and insert on master are made
by Frank.Offermanns＠caseris.de
Full_Name: Frank Offermanns
Version: 2.4.25 and 2.4.41
OS: Windows
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (217.7.149.50)
Hello,
when I have two ldapserver (master/slave with delta syncrepl) and do insert on
the master while the slave is down and then start the slave, only some entries
get replicated.
This problem is reproducable (every time). I did my test with a build of 2.4.25
and mingw and with 2.4.41 build with visual studio. The version 2.4.25 is 32 bit
and the 2.4.41 is 64 bit. Operation system is Windows. Version 2.4.25 runs with
hdb, version 2.4.41 with lmdb.
Here is a description in detail:
Master and slave are fully synchronized. Then I stop the slave. Add 100 users to
the master LDAP. I wait a few seconds, then I start the service at the slave.
Now only some users get replicated.
Here is my configuration of master (of the hdb try):
######################################################################
database config
rootdn cn=config
rootpw secret
# Accesslog database definitions
database hdb
suffix cn=accesslog
checkpoint 1024 5
cachesize 10000
directory "c:/mydir/accessdata"
dbconfig set_cachesize 0 30000000 1
dbconfig set_flags DB_LOG_AUTOREMOVE
dbconfig set_lg_regionmax 1048576
dbconfig set_lg_max 10485760
dbconfig set_lg_bsize 2097152
rootdn cn=accesslog
index default eq
index entryCSN,objectClass,reqEnd,reqResult,reqStart
overlay syncprov
syncprov-nopresent TRUE
syncprov-reloadhint TRUE
limits dn.exact="cn=Replicator,ou=admins,o=caesar" time.soft=unlimited
time.hard=unlimited size.soft=unlimited size.hard=unlimited
database hdb
suffix "o=caesar"
checkpoint 1024 5
cachesize 10000
idlcachesize 30000
rootdn "cn=Administrator,o=caesar"
rootpw secret
directory "c:/mydir/data"
dbconfig set_cachesize 0 100000000 1
dbconfig set_flags DB_LOG_AUTOREMOVE
dbconfig set_lg_regionmax 1048576
dbconfig set_lg_max 10485760
dbconfig set_lg_bsize 2097152
# syncprov specific indexing
index sn pres,eq
...
index objectClass eq
overlay syncprov
syncprov-checkpoint 100 10
syncprov-sessionlog 10000
overlay accesslog
logdb cn=accesslog
logops writes
logsuccess TRUE
logpurge 07+00:00 01+00:00
sizelimit size.soft=100 size.hard=1000 size.prtotal=unlimited
limits dn.exact="cn=Replicator,ou=admins,o=caesar" time.soft=unlimited
time.hard=unlimited size.soft=unlimited size.hard=unlimited
Here is my config of the slave (of the hdb test):
access to * by dn.one="ou=Admins,o=caesar" write
by anonymous auth
####################################################################"3%0
database config
rootdn cn=config
rootpw secret
database hdb
suffix "o=caesar"
checkpoint 1024 5
cachesize 10000
idlcachesize 30000
rootdn "cn=Administrator,o=caesar"
rootpw secret
directory "C:/mydir/data"
dbconfig set_cachesize 0 100000000 1
dbconfig set_flags DB_LOG_AUTOREMOVE
dbconfig set_lg_regionmax 1048576
dbconfig set_lg_max 10485760
dbconfig set_lg_bsize 2097152
# syncprov specific indexing
index sn pres,eq
...
index objectClass eq
sizelimit size.soft=100 size.hard=1000 size.prtotal=unlimited
# Let the replica DN have limitless searches
limits dn.exact="cn=Replicator,ou=admins,o=caesar" time.soft=unlimited
time.hard=unlimited size.soft=unlimited size.hard=unlimited
syncrepl rid=001
provider="ldap://masterserver.mydomain"
searchbase="o=caesar"
type=refreshAndPersist
retry="5 3 15 +"
binddn="cn=Replicator,ou=admins,o=caesar"
bindmethod=simple
credtitials="secret"
logbase="cn=accesslog"
logfilter="(&(objectClass=auditWriteObject)(reqResult=0))"
schemachecking=on
syncdata=accesslog
8 years, 1 month
Re: (ITS#7537) ppolicy hangs slapd on 64bit version debian
by ryan＠nardis.ca
I reproduced this with current RE24. It seems to be caused by setting
pwdMaxAge to a very large value.
In Marek's previous debug output you can see his policy object as
returned from a search. Among other attrs it has:
pwdMaxAge: 2592000000
I reproduced the report with the following setup:
### slapd.conf
include ../openldap/servers/slapd/schema/core.schema
include ../openldap/servers/slapd/schema/ppolicy.schema
moduleload back_hdb
moduleload ppolicy
database hdb
directory db
suffix dc=example,dc=com
rootdn cn=root,dc=example,dc=com
rootpw secret
access to *
by dn="cn=admin,dc=example,dc=com" manage
by * read
index objectClass eq
overlay ppolicy
ppolicy_default cn=default,ou=policies,dc=example,dc=com
### init.ldif
dn: dc=example,dc=com
objectClass: domain
dn: cn=admin,dc=example,dc=com
objectClass: organizationalRole
objectClass: simpleSecurityObject
userPassword: secret
dn: ou=policies,dc=example,dc=com
objectClass: organizationalUnit
dn: cn=default,ou=policies,dc=example,dc=com
objectClass: organizationalRole
objectClass: pwdPolicy
pwdAttribute: userPassword
pwdMaxAge: 2592000000
### mod.ldif
dn: cn=default,ou=policies,dc=example,dc=com
add: pwdMaxFailure
pwdMaxFailure: 7
Launched slapd and triggered via:
$ ldapmodify -x -D cn=admin,dc=example,dc=com -W -f mod.ldif
slapd hangs, ldapmodify is still waiting for it, and gdb says:
Program received signal SIGINT, Interrupt.
0x00007ffff75f24db in pthread_join (threadid=140737286698752, thread_return=0x0) at pthread_join.c:92
92 pthread_join.c: No such file or directory.
(gdb) thread apply all bt
Thread 3 (Thread 0x7ffff3faf700 (LWP 25350)):
#0 0x00007ffff7326653 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81
#1 0x00000000004257b9 in slapd_daemon_task (ptr=0xa83a10) at daemon.c:2539
#2 0x00007ffff75f10a4 in start_thread (arg=0x7ffff3faf700) at pthread_create.c:309
#3 0x00007ffff732607d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
Thread 2 (Thread 0x7ffff37ae700 (LWP 25353)):
#0 pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1 0x00007ffff7a4e79d in __db_pthread_mutex_lock () from /usr/lib/x86_64-linux-gnu/libdb-5.3.so
#2 0x00007ffff7af74e0 in __lock_get_internal () from /usr/lib/x86_64-linux-gnu/libdb-5.3.so
#3 0x00007ffff7af8719 in __lock_vec () from /usr/lib/x86_64-linux-gnu/libdb-5.3.so
#4 0x00007ffff7af8e9f in __lock_vec_pp () from /usr/lib/x86_64-linux-gnu/libdb-5.3.so
#5 0x000000000052f7ed in hdb_cache_entry_db_relock (bdb=0x934920, txn=0x7fffe4102810, ei=0x7fffe4101e80, rw=1,
tryOnly=0, lock=0x7ffff37ac340) at cache.c:198
#6 0x0000000000531a43 in hdb_cache_modify (bdb=0x934920, e=0x9aca18, newAttrs=0x9c0690, txn=0x7fffe4102810,
lock=0x7ffff37ac340) at cache.c:1231
#7 0x00000000004d85f6 in hdb_modify (op=0x7fffe4000aa0, rs=0x7ffff37adae0) at modify.c:764
#8 0x00000000004b740c in overlay_op_walk (op=0x7fffe4000aa0, rs=0x7ffff37adae0, which=op_modify, oi=0x936420, on=0x0)
at backover.c:677
#9 0x00000000004b7637 in over_op_func (op=0x7fffe4000aa0, rs=0x7ffff37adae0, which=op_modify) at backover.c:730
#10 0x00000000004b776b in over_op_modify (op=0x7fffe4000aa0, rs=0x7ffff37adae0) at backover.c:769
#11 0x0000000000449541 in fe_op_modify (op=0x7fffe4000aa0, rs=0x7ffff37adae0) at modify.c:303
#12 0x0000000000448e13 in do_modify (op=0x7fffe4000aa0, rs=0x7ffff37adae0) at modify.c:177
#13 0x0000000000429a3f in connection_operation (ctx=0x7ffff37adc10, arg_v=0x7fffe4000aa0) at connection.c:1155
#14 0x0000000000429fdb in connection_read_thread (ctx=0x7ffff37adc10, argv=0xe) at connection.c:1291
#15 0x000000000058415d in ldap_int_thread_pool_wrapper (xpool=0x909fd0) at tpool.c:696
#16 0x00007ffff75f10a4 in start_thread (arg=0x7ffff37ae700) at pthread_create.c:309
#17 0x00007ffff732607d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
Thread 1 (Thread 0x7ffff7fec700 (LWP 25346)):
#0 0x00007ffff75f24db in pthread_join (threadid=140737286698752, thread_return=0x0) at pthread_join.c:92
#1 0x00000000005855f3 in ldap_pvt_thread_join (thread=140737286698752, thread_return=0x0) at thr_posix.c:197
#2 0x0000000000426986 in slapd_daemon () at daemon.c:2932
#3 0x000000000040602d in main (argc=8, argv=0x7fffffffe598) at main.c:1017
If I change the config to use back-mdb instead, I get a crash:
55dce2bf conn=1000 op=1 MOD dn="cn=default,ou=policies,dc=example,dc=com"
55dce2bf conn=1000 op=1 MOD attr=pwdMaxFailure
slapd: id2entry.c:520: mdb_opinfo_get: Assertion `!rc' failed.
[New Thread 0x7ffff2caf700 (LWP 25374)]
[New Thread 0x7ffff34b0700 (LWP 25371)]
Program received signal SIGABRT, Aborted.
[Switching to Thread 0x7ffff2caf700 (LWP 25374)]
0x00007ffff7275107 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0 0x00007ffff7275107 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x00007ffff72764e8 in __GI_abort () at abort.c:89
#2 0x00007ffff726e226 in __assert_fail_base (fmt=0x7ffff73a4d08 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",
assertion=assertion@entry=0x5f29e4 "!rc", file=file@entry=0x5f2975 "id2entry.c", line=line@entry=520,
function=function@entry=0x5f2b30 <__PRETTY_FUNCTION__.12200> "mdb_opinfo_get") at assert.c:92
#3 0x00007ffff726e2d2 in __GI___assert_fail (assertion=0x5f29e4 "!rc", file=0x5f2975 "id2entry.c", line=520,
function=0x5f2b30 <__PRETTY_FUNCTION__.12200> "mdb_opinfo_get") at assert.c:101
#4 0x0000000000553a71 in mdb_opinfo_get (op=0x7fffe4000aa0, mdb=0x7ffff7f2a010, rdonly=1, moip=0x7ffff2cacf18)
at id2entry.c:520
#5 0x000000000055306b in mdb_entry_get (op=0x7fffe4000aa0, ndn=0x7fffe4000ad8, oc=0x0, at=0x0, rw=0,
ent=0x7ffff2cad2c8) at id2entry.c:327
#6 0x00000000004b6a41 in overlay_entry_get_ov (op=0x7fffe4000aa0, dn=0x7fffe4000ad8, oc=0x0, ad=0x0, rw=0,
e=0x7ffff2cad2c8, on=0x0) at backover.c:364
#7 0x00000000004b6b11 in over_entry_get_rw (op=0x7fffe4000aa0, dn=0x7fffe4000ad8, oc=0x0, ad=0x0, rw=0,
e=0x7ffff2cad2c8) at backover.c:396
#8 0x000000000043ca86 in be_entry_get_rw (op=0x7fffe4000aa0, ndn=0x7fffe4000ad8, oc=0x0, at=0x0, rw=0,
e=0x7ffff2cad2c8) at backend.c:1366
#9 0x0000000000561e6c in ppolicy_modify (op=0x7fffe4000aa0, rs=0x7ffff2caeae0) at ppolicy.c:1626
#10 0x00000000004b7362 in overlay_op_walk (op=0x7fffe4000aa0, rs=0x7ffff2caeae0, which=op_modify, oi=0x935700,
on=0x9358e0) at backover.c:661
#11 0x00000000004b7637 in over_op_func (op=0x7fffe4000aa0, rs=0x7ffff2caeae0, which=op_modify) at backover.c:730
#12 0x00000000004b776b in over_op_modify (op=0x7fffe4000aa0, rs=0x7ffff2caeae0) at backover.c:769
#13 0x0000000000449541 in fe_op_modify (op=0x7fffe4000aa0, rs=0x7ffff2caeae0) at modify.c:303
#14 0x0000000000448e13 in do_modify (op=0x7fffe4000aa0, rs=0x7ffff2caeae0) at modify.c:177
#15 0x0000000000429a3f in connection_operation (ctx=0x7ffff2caec10, arg_v=0x7fffe4000aa0) at connection.c:1155
#16 0x0000000000429fdb in connection_read_thread (ctx=0x7ffff2caec10, argv=0xb) at connection.c:1291
#17 0x000000000058415d in ldap_int_thread_pool_wrapper (xpool=0x909fd0) at tpool.c:696
#18 0x00007ffff75f10a4 in start_thread (arg=0x7ffff2caf700) at pthread_create.c:309
#19 0x00007ffff732607d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
It seems like the crash does not happen if I bind as the rootdn, only
when using the admin DN. The hang with back-hdb happens with both.
Like I mentioned above, the problem seems to be related to the large
pwdMaxAge value. If I drop a few 0s off the end of it, everything works
properly. That might be related to Marek's statement that he only
experienced the problem on 64-bit...
8 years, 1 month
(ITS#8222) memory administration in ldap_parse_result()
by lawrence.newman＠alcatel-lucent.com
Full_Name: Larry Newman
Version: 2.4.40
OS: Linux
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (135.245.10.2)
My application uses OpenLDAP as a client with long duration connections used to
send many requests - multiple TCP connections are pre-established and then
ldap_init_fd() is invoked for each of them. After a while (and many
request/response exchanges) memory exhaustion is observed, and this report
relates to the possibility that ldap_parse_result() might be leaking memory. I
believe the ber_scanf() calls in ldap_parse_result() that search for matched
DN/error message/referrals can dynamically allocate memory (in my case, for an
error message) that is being linked to the LDAP structure. This memory doesn't
appear to be freed before ldap_parse_result() returns and the allocation isn't
visible to the caller (the LDAP structure is supposed to be "opaque"). This
appears to be different memory from that passed back to the caller via the
ldap_parse_result() pointer parameters (that memory is conditionally created by
LDAP_STRDUP() and can/should be freed by the caller if requested) E Early in
ldap_parse_result(), there is some code that invokes LDAP_FREE()/LDAP_VFREE()
for these dynamic memory pointers in the LDAP structure - but it doesn't seem
like that will happen until the next invocation of ldap_parse_result() (if
ever). The implementation of ldap_parse_extended_result() seems similar.
Shouldn't memory allocated (for internal use only) by these functions be freed
before the functions return?
8 years, 1 month
(ITS#8221) getting MDB_PAGE_FULL using mdb_cursor_del
by sergej.jurecko＠gmail.com
Full_Name: Sergej Jure&#269;ko
Version: lmdb 0.9.16
OS: osx
URL: ftp://ftp.openldap.org/incoming/delerror.zip
Submission from: (NULL) (193.189.172.218)
Code and lmdb file is uploaded to ftp. Code is also here:
https://gist.github.com/SergejJurecko/68979ff6460806581ad5
If you run the code on the uploaded lmdb file, it will result in MDB_PAGE_FULL
error out of mdb_cursor_del.
I use dupsorts a lot in my app. Every value in the dupsort has 2 64bit integers
and 1 u8 in the beginning, which are used in the custom comparison function to
sort the values.
During normal operation values are added to the end of the dupsort and
eventually a cleanup operation occurs that deletes unused values.
The example app tries to delete the first value in the dupsort. But it will fail
the same if it tries to delete the next one instead.
8 years, 1 month
Re: (ITS#8198) pw-pbkdf2: optionally use libnettle for crypto
by ryan＠nardis.ca
On Mon, Jul 13, 2015 at 08:44:49PM +0000, luca.bruno(a)rocket-internet.de wrote:
>This is a followup to ITS#7977.
>Here below are two patches against the pw-pbkdf2 contrib module to:
> 1) fix an always-true check
> 2) optionally make use of libnettle
These are in git master now, along with the #else->#elif fixup. Thank
you for the contribution!
8 years, 1 month
Re: (ITS#8185) Clarification/enhancement request: purging stale pwdFailureTime attributes
by subbarao＠computer.org
On 08/14/2015 10:25 AM, Howard Chu wrote:
> I've added a pwdMaxRecordedFailure attribute to the policy schema.
> Overloading pwdMaxFailure would be a mistake.
>
> MaxRecordedFailure will default to MaxFailure if that is set. It
> defaults to 5 if nothing is set. There's no good reason to allow the
> timestamps to accumulate without bound.
>
> This is now available for testing in git master.
Tested on Ubuntu 14.04, works great. Thanks again Howard!
-Kartik
8 years, 1 month
Re: (ITS#8185) Clarification/enhancement request: purging stale pwdFailureTime attributes
by subbarao＠computer.org
On 08/14/2015 10:25 AM, Howard Chu wrote:
> I've added a pwdMaxRecordedFailure attribute to the policy schema.
> Overloading pwdMaxFailure would be a mistake.
>
> MaxRecordedFailure will default to MaxFailure if that is set. It
> defaults to 5 if nothing is set. There's no good reason to allow the
> timestamps to accumulate without bound.
>
> This is now available for testing in git master.
Howard, I just saw this message from you today, when I happened to be
looking through my gmail spam folder -- no idea why it ended up there!
On Friday, I only saw your subsequent message and responded to it
without knowing that you had already implemented this enhancement. So I
didn't fully understand the context in which you had written that message.
Thanks very much for implementing this enhancement! I will check out the
code.
Regards,
-Kartik
8 years, 1 month
