openldap-bugs January 2019

openldap-bugs@openldap.org

21 participants
77 discussions

Re: (ITS#8953) Windows lmdb warnings and a memory error
by hyc＠symas.com 04 Jan '19

04 Jan '19

robin.rowe(a)cinepaint.org wrote: > Full_Name: Robin Rowe > Version: --branch OPENLDAP_REL_ENG_2_4 > OS: Windows 10 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (23.242.20.216) > > > Thanks for lmdb! > > Building lmdb in Visual Studio 2017 produces warnings a memory error due to > returning address of local variable from function mdb_strerror. Many warnings > for integer truncation without using a cast. None of these seem severe. Will > anyone clean this up? None of these are meaningful, so no. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8953) Windows lmdb warnings and a memory error
by robin.rowe＠cinepaint.org 04 Jan '19

04 Jan '19

Full_Name: Robin Rowe Version: --branch OPENLDAP_REL_ENG_2_4 OS: Windows 10 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (23.242.20.216) Thanks for lmdb! Building lmdb in Visual Studio 2017 produces warnings a memory error due to returning address of local variable from function mdb_strerror. Many warnings for integer truncation without using a cast. None of these seem severe. Will anyone clean this up? I used these switches in my cmake file to cut down the warnings: add_definitions(-D_CRT_SECURE_NO_DEPRECATE) add_definitions(-D_CRT_NONSTDC_NO_DEPRECATE) # Necessary for building mdb_dkey for mtest6: add_definitions(-DMDB_DEBUG) Visual Studio 2017 Build Output: 2>mdb.c liblmdb\mdb.c(1640): warning C4267: '=': conversion from 'size_t' to 'unsigned int', possible loss of data liblmdb\mdb.c(1831): warning C4267: 'initializing': conversion from 'size_t' to 'unsigned int', possible loss of data liblmdb\mdb.c(2014): warning C4267: '+=': conversion from 'size_t' to 'unsigned int', possible loss of data liblmdb\mdb.c(2053): warning C4267: '=': conversion from 'size_t' to 'unsigned int', possible loss of data liblmdb\mdb.c(2028): warning C4267: 'initializing': conversion from 'size_t' to 'unsigned int', possible loss of data liblmdb\mdb.c(2258): warning C4267: '=': conversion from 'size_t' to 'unsigned int', possible loss of data liblmdb\mdb.c(2278): warning C4267: '=': conversion from 'size_t' to 'unsigned int', possible loss of data liblmdb\mdb.c(2164): warning C4267: 'initializing': conversion from 'size_t' to 'unsigned int', possible loss of data liblmdb\mdb.c(2439): warning C4267: '=': conversion from 'size_t' to 'unsigned short', possible loss of data liblmdb\mdb.c(2893): warning C4267: '=': conversion from 'size_t' to 'int', possible loss of data liblmdb\mdb.c(2894): warning C4267: 'function': conversion from 'size_t' to 'int', possible loss of data liblmdb\mdb.c(3360): warning C4267: '=': conversion from 'size_t' to 'DWORD', possible loss of data liblmdb\mdb.c(3361): warning C4267: 'function': conversion from 'size_t' to 'DWORD', possible loss of data liblmdb\mdb.c(3300): warning C4267: 'initializing': conversion from 'size_t' to 'int', possible loss of data liblmdb\mdb.c(3510): warning C4267: '=': conversion from 'size_t' to 'unsigned int', possible loss of data liblmdb\mdb.c(3514): warning C4267: '=': conversion from 'size_t' to 'unsigned int', possible loss of data liblmdb\mdb.c(3550): warning C4267: '=': conversion from 'size_t' to 'unsigned int', possible loss of data liblmdb\mdb.c(3553): warning C4267: '=': conversion from 'size_t' to 'unsigned int', possible loss of data liblmdb\mdb.c(3568): warning C4267: '=': conversion from 'size_t' to 'unsigned int', possible loss of data liblmdb\mdb.c(3875): warning C4244: '+=': conversion from '__int64' to 'off_t', possible loss of data liblmdb\mdb.c(3988): warning C4267: '=': conversion from 'size_t' to 'LONG', possible loss of data liblmdb\mdb.c(4334): warning C4996: 'GetVersion': was declared deprecated 2>C:\Program Files (x86)\Windows Kits\10\Include\10.0.16299.0\um\sysinfoapi.h(184): note: see declaration of 'GetVersion' liblmdb\mdb.c(4719): warning C4244: 'function': conversion from 'mdb_hash_t' to 'unsigned long', possible loss of data liblmdb\mdb.c(5226): warning C4267: '=': conversion from 'size_t' to 'unsigned int', possible loss of data liblmdb\mdb.c(5229): warning C4267: '=': conversion from 'size_t' to 'unsigned int', possible loss of data liblmdb\mdb.c(5234): warning C4244: 'return': conversion from 'ssize_t' to 'int', possible loss of data liblmdb\mdb.c(5260): warning C4244: 'return': conversion from 'ssize_t' to 'int', possible loss of data liblmdb\mdb.c(5708): warning C4267: '=': conversion from 'size_t' to 'unsigned int', possible loss of data liblmdb\mdb.c(5716): warning C4267: '=': conversion from 'size_t' to 'unsigned int', possible loss of data liblmdb\mdb.c(5728): warning C4267: '=': conversion from 'size_t' to 'unsigned int', possible loss of data liblmdb\mdb.c(5729): warning C4267: '=': conversion from 'size_t' to 'unsigned int', possible loss of data liblmdb\mdb.c(6559): warning C4267: '=': conversion from 'size_t' to 'unsigned int', possible loss of data liblmdb\mdb.c(6674): warning C4267: '=': conversion from 'size_t' to 'uint16_t', possible loss of data liblmdb\mdb.c(6756): warning C4267: '=': conversion from 'size_t' to 'uint16_t', possible loss of data liblmdb\mdb.c(6762): warning C4267: '=': conversion from 'size_t' to 'indx_t', possible loss of data liblmdb\mdb.c(6818): warning C4267: '=': conversion from 'size_t' to 'unsigned int', possible loss of data liblmdb\mdb.c(6898): warning C4146: unary minus operator applied to unsigned type, result still unsigned liblmdb\mdb.c(6906): warning C4267: '=': conversion from 'size_t' to 'unsigned short', possible loss of data liblmdb\mdb.c(7027): warning C4267: '=': conversion from 'size_t' to 'int', possible loss of data liblmdb\mdb.c(6854): warning C4267: 'initializing': conversion from 'size_t' to 'int', possible loss of data liblmdb\mdb.c(7299): warning C4267: '-=': conversion from 'size_t' to 'indx_t', possible loss of data liblmdb\mdb.c(7339): warning C4267: '=': conversion from 'size_t' to 'indx_t', possible loss of data liblmdb\mdb.c(7347): warning C4267: '=': conversion from 'size_t' to 'unsigned short', possible loss of data liblmdb\mdb.c(7350): warning C4267: '=': conversion from 'size_t' to 'unsigned short', possible loss of data liblmdb\mdb.c(7352): warning C4267: '=': conversion from 'size_t' to 'unsigned short', possible loss of data liblmdb\mdb.c(7312): warning C4267: 'initializing': conversion from 'size_t' to 'int', possible loss of data liblmdb\mdb.c(7413): warning C4267: '+=': conversion from 'size_t' to 'indx_t', possible loss of data liblmdb\mdb.c(7475): warning C4333: '>>': right shift by too large amount, data loss liblmdb\mdb.c(7811): warning C4267: '=': conversion from 'size_t' to 'unsigned short', possible loss of data liblmdb\mdb.c(7934): warning C4267: 'function': conversion from 'size_t' to 'int', possible loss of data liblmdb\mdb.c(8697): warning C4267: '-=': conversion from 'size_t' to 'indx_t', possible loss of data liblmdb\mdb.c(8705): warning C4267: '-=': conversion from 'size_t' to 'indx_t', possible loss of data liblmdb\mdb.c(8713): warning C4267: '=': conversion from 'size_t' to 'int', possible loss of data liblmdb\mdb.c(8715): warning C4267: '=': conversion from 'size_t' to 'int', possible loss of data liblmdb\mdb.c(9297): warning C4267: '=': conversion from 'size_t' to 'unsigned short', possible loss of data liblmdb\mdb.c(9484): warning C4267: '=': conversion from 'size_t' to 'DWORD', possible loss of data liblmdb\mdb.c(9520): warning C4267: '=': conversion from 'size_t' to 'DWORD', possible loss of data 2>midl.c liblmdb\midl.c(44): warning C4267: 'initializing': conversion from 'size_t' to 'unsigned int', possible loss of data liblmdb\midl.c(146): warning C4267: '+=': conversion from 'size_t' to 'unsigned int', possible loss of data liblmdb\midl.c(176): warning C4267: 'function': conversion from 'size_t' to 'int', possible loss of data 2>Generating Code... 2>c:\code\github\libunistd\lmdb\liblmdb\mdb.c(1515): warning C4172: returning address of local variable or temporary: buf 2>Db.cpp 2>liblmdb.vcxproj -> C:\Code\github\libunistd\build\lmdb\Debug\liblmdb.lib 2>Done building project "liblmdb.vcxproj". ========== Rebuild All: 2 succeeded, 0 failed, 0 skipped =========

1 0

Re: (ITS#8952) olcIdleTimeout < 4 causes high CPU usage on some systems
by hyc＠symas.com 02 Jan '19

02 Jan '19

openldap-bugs(a)PaulSD.com wrote: > Both epoll and the solaris /dev/poll infrastructure support only millisecond > resolution, not microsecond resolution, so I assume the microseconds were simply > dropped for simplicity. > > Could (tvp)->tv_sec*1000 be changed to (tvp)->tv_sec*1000+(tvp)->tv_usec/1000 to > correct this issue? > https://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=blob;f=servers/s… > https://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=blob;f=servers/s… Thanks for the report. Changed in git master, please test. > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8952) olcIdleTimeout < 4 causes high CPU usage on some systems
by openldap-bugs＠PaulSD.com 02 Jan '19

02 Jan '19

Full_Name: Paul Donohue Version: 2.4.44-20.el7 OS: RHEL7 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (172.58.184.58) After upgrading from OpenLDAP 2.2 to OpenLDAP 2.4, we noticed much higher than expected CPU usage on our LDAP servers, particularly on our master server (which only accepts connections from our slave servers, and therefore should generally be idle). After doing lots of profiling and debugging, we managed to determine that the problem was caused by our olcIdleTimeout setting, which we had set to 3. slapd_daemon_task divides olcIdleTimeout by 4 and sets the seconds and microseconds values of a timeval structure appropriately: https://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=blob;f=servers/s… https://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=blob;f=servers/s… In OpenLDAP 2.2, this timeval structure is then passed directly as the timeout parameter to a select() call: https://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=blob;f=servers/s… However, in OpenLDAP 2.4, this timeval structure is passed to SLAP_EVENT_WAIT. There are four different implementations of SLAP_EVENT_WAIT. The kqueue and winsock implementations handle this timeout properly, but the epoll and solaris implementations drop the microseconds value and use only the seconds value. (We are using the epoll implementation.) Therefore, if olcIdleTimeout is less than 4 and there are any open connections to the server, then epoll_wait is always called with a zero timeout, which causes the slapd_daemon_task while loop to spin continuously and consume CPU. It looks like this issue has been present since epoll support was first added to OpenLDAP: https://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=commit;h=b7d4e1a… Both epoll and the solaris /dev/poll infrastructure support only millisecond resolution, not microsecond resolution, so I assume the microseconds were simply dropped for simplicity. Could (tvp)->tv_sec*1000 be changed to (tvp)->tv_sec*1000+(tvp)->tv_usec/1000 to correct this issue? https://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=blob;f=servers/s… https://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=blob;f=servers/s…

1 0

(ITS#8951) Replication stalls
by whm＠dropbox.com 02 Jan '19

02 Jan '19

Full_Name: Bill MacAllister Version: 2.4.46 OS: Ubuntu 16.04 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (50.247.112.108) We have been seeing replication stall on many of our replicas. Once replication stalls is never recovers on its own and the slapd process on the replica needs to be restarted. As soon as slapd is restarted on a replica it catches up to the master. By examining the ContextCSNs it appears that all stalls at exactly the same point. Our configuration is a single master with 20 replicas. Most replicas are deployed in pairs at remote site. Generally replicas are deployed to address latency issues at remote sites. The LDAP infrastructure provides DHCP and authorization services. The replicas and the master are all running Ubuntu 16.04 with a custom built slapd using 2.4.46 source. Our build of slapd starts with Ryan Tandys ppa source and has the following changes. - Build with OpenSSL - ITS patches 8054, 8752, and 8727 We have been seeing this problem intermittently for months now, and the problem just recently gotten worse, going from once every other month to once or twice a week. One anomaly that we have seen with the last two stalls is that only 18 of the 20 replicas stalled. The replicas that did not stall are in the same network as the master. I think that is a red herring since the stalling hosts are widely dispersed and have a variety of network paths. We use GSSAPI authentication and the directory holds no passwords. We have logs and thread dumps of the most recent stall. If you would like to see them let me know where to send them. Bill

1 0

(ITS#8950) segfault with TXN+accesslog
by ondra＠openldap.org 02 Jan '19

02 Jan '19

Full_Name: Ondrej Kuznik Version: master OS: Linux URL: Submission from: (NULL) (82.10.24.68) Trying to reproduce a potential lockup between TXN support and accesslog, I have instead come across a segfault in TXN handling. With the following config: database mdb suffix cn=log directory ./log database mdb suffix cn=test directory ./db overlay accesslog logdb cn=log logops writes Make sure cn=test entry exists and issue ldapmodify -E '!txn=commit' with dn: cn=test changetype: modify slapd segfaults with the following picked up by valgrind: ==10599== Invalid read of size 8 ==10599== at 0x509ACD: txn_end_extop (txn.c:243) ==10599== by 0x49A7D9: fe_extended (extended.c:222) ==10599== by 0x49A505: do_extended (extended.c:177) ==10599== by 0x44F36D: connection_operation (connection.c:1169) ==10599== by 0x44D52F: connection_read_thread (connection.c:1326) ==10599== by 0x485869E: ldap_int_thread_pool_wrapper (tpool.c:1048) ==10599== by 0x6886FA2: start_thread (pthread_create.c:486) ==10599== by 0x699988E: clone (clone.S:95) ==10599== Address 0xa552330 is on thread 3's stack ==10599== 4112 bytes below stack pointer ==10599== ==10599== Invalid read of size 8 ==10599== at 0x509AD0: txn_end_extop (txn.c:243) ==10599== by 0x49A7D9: fe_extended (extended.c:222) ==10599== by 0x49A505: do_extended (extended.c:177) ==10599== by 0x44F36D: connection_operation (connection.c:1169) ==10599== by 0x44D52F: connection_read_thread (connection.c:1326) ==10599== by 0x485869E: ldap_int_thread_pool_wrapper (tpool.c:1048) ==10599== by 0x6886FA2: start_thread (pthread_create.c:486) ==10599== by 0x699988E: clone (clone.S:95) ==10599== Address 0x20333d706f203108 is not stack'd, malloc'd or (recently) free'd ==10599== ==10599== ==10599== Process terminating with default action of signal 11 (SIGSEGV) ==10599== General Protection Fault ==10599== at 0x509AD0: txn_end_extop (txn.c:243) ==10599== by 0x49A7D9: fe_extended (extended.c:222) ==10599== by 0x49A505: do_extended (extended.c:177) ==10599== by 0x44F36D: connection_operation (connection.c:1169) ==10599== by 0x44D52F: connection_read_thread (connection.c:1326) ==10599== by 0x485869E: ldap_int_thread_pool_wrapper (tpool.c:1048) ==10599== by 0x6886FA2: start_thread (pthread_create.c:486) ==10599== by 0x699988E: clone (clone.S:95) I doesn't seem to happen without accesslog configured.

1 0

(ITS#8949) slapd crashes when adding first olcPlugin value
by ryan＠openldap.org 01 Jan '19

01 Jan '19

Full_Name: Ryan Tandy Version: 2.4, master OS: Debian URL: Submission from: (NULL) (70.66.128.207) Submitted by: ryan Initial configuration, no existing value for olcPlugin: ldapmodify -H ldap://:9000 -x -D cn=config -w secret << eof dn: olcDatabase={1}mdb,cn=config add: olcPlugin olcPlugin: preoperation ./libtestpreop-plugin.so testpreop_init eof Thread 4 "slapd" received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7ffff6020700 (LWP 17326)] 0x00005555556ee351 in slapi_int_free_object_extensions (objecttype=1, object=0x7fffec000cb0) at slapi_ext.c:298 298 if ( eblock->extensions != NULL ) { (gdb) bt #0 0x00005555556ee351 in slapi_int_free_object_extensions (objecttype=1, object=0x7fffec000cb0) at slapi_ext.c:298 #1 0x00005555555c14f3 in slap_op_free (op=0x7fffec000cb0, ctx=0x7ffff601fb90) at operation.c:113 #2 0x00005555555a62c6 in connection_operation (ctx=0x7ffff601fb90, arg_v=0x7fffec000cb0) at connection.c:1246 #3 0x00005555555a6577 in connection_read_thread (ctx=0x7ffff601fb90, argv=0xb) at connection.c:1326 #4 0x00005555556a2dff in ldap_int_thread_pool_wrapper (xpool=0x55555580cc40) at tpool.c:1048 #5 0x00007ffff7f7ffa3 in start_thread (arg=<optimized out>) at pthread_create.c:486 #6 0x00007ffff7eb089f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 It looks like we went through slap_op_alloc() with slapi_plugins_used = 0, so slapi_int_create_object_extensions() was never called, but in slap_op_free() we have added a plugin and have called slapi_int_free_object_extensions(). Can probably be fixed just by short-circuiting when eblock == NULL?

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs January 2019