- openldap-bugs - openldap.org

Re: (ITS#8070) test023 failures
by quanah＠zimbra.com 03 Mar '15

03 Mar '15

--On Monday, March 02, 2015 10:17 PM +0000 richton(a)nbcs.rutgers.edu wrote: > Full_Name: Aaron Richton > Version: OPENLDAP_REL_ENG_2_4_40-178-gba9a739 > OS: Linux > URL: https://www.nbcs.rutgers.edu/~richton/testrun-test023-fail.tar.bz2 > Submission from: (NULL) (128.6.31.135) > > > Seeing occasional failures on test023 using mdb. Takes a few runs (~30 > for me, recently) but seems to be more than a one-off... > Confirmed with Howard this is a timing dependent script, and it's affected by recent changes to the code base. refint used to do all its work inline, but now it uses a separate thread, so on a slow enough machine, it may not be done by the time the script runs its ldapsearch. I.e., doesn't appear to be a code bug. The test needs tweaking at some point to add some sleeps to handle this. :P --Quanah -- Quanah Gibson-Mount Platform Architect Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#8070) test023 failures
by quanah＠zimbra.com 03 Mar '15

03 Mar '15

--On Monday, March 02, 2015 10:17 PM +0000 richton(a)nbcs.rutgers.edu wrote: > Full_Name: Aaron Richton > Version: OPENLDAP_REL_ENG_2_4_40-178-gba9a739 > OS: Linux > URL: https://www.nbcs.rutgers.edu/~richton/testrun-test023-fail.tar.bz2 > Submission from: (NULL) (128.6.31.135) > > > Seeing occasional failures on test023 using mdb. Takes a few runs (~30 > for me, recently) but seems to be more than a one-off... Doesn't happen for me with 4129e246a85a6d1f0033466ed2c7fb51c6dee508 (Feb 6th), so it is something recently introduced. --Quanah -- Quanah Gibson-Mount Platform Architect Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

(ITS#8070) test023 failures
by richton＠nbcs.rutgers.edu 02 Mar '15

02 Mar '15

Full_Name: Aaron Richton Version: OPENLDAP_REL_ENG_2_4_40-178-gba9a739 OS: Linux URL: https://www.nbcs.rutgers.edu/~richton/testrun-test023-fail.tar.bz2 Submission from: (NULL) (128.6.31.135) Seeing occasional failures on test023 using mdb. Takes a few runs (~30 for me, recently) but seems to be more than a one-off...

1 0

(ITS#8069) MDb library patch: Use explicit Windows API functions for char* strings
by pmedvedev＠gmail.com 27 Feb '15

27 Feb '15

Full_Name: Pavel Medvedev Version: OS: Windows 7 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (217.25.225.40) Most Windows API functions such as CreateFile(), CreateMutex(), OpenMutex() that accept string arguments, are defined with suffix W or A for appropriate wchar_t* or char* strings, depending of _UNICODE preprocessor define. Calling explicit functions with A suffix would eliminate compile errors when _UNICODE is defined. As a further enhancement, I think path argument encoding should be documented (probably UTF-8?) for such functions as mdb_env_open(), mdb_env_copy(), mdb_env_copy2(), and mdb_env_get_path(). This prevent possible problems with non-latin characters in path names on Windows. In this case, LMDB implementation on Windows should perform path conversion to UTF-16 and explicit call to CreateFileW() in mdb_env_open() and mdb_env_copy2() functions. --- libraries/liblmdb/mdb.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/libraries/liblmdb/mdb.c b/libraries/liblmdb/mdb.c index a54ce83..a669da6 100644 --- a/libraries/liblmdb/mdb.c +++ b/libraries/liblmdb/mdb.c @@ -1409,7 +1409,7 @@ mdb_strerror(int err) ; } buf[0] = 0; - FormatMessage(FORMAT_MESSAGE_FROM_SYSTEM | + FormatMessageA(FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_IGNORE_INSERTS, NULL, err, 0, ptr, sizeof(buf), (va_list *)pad); return ptr; @@ -4413,7 +4413,7 @@ mdb_env_setup_locks(MDB_env *env, char *lpath, int mode, int *excl) off_t size, rsize; #ifdef _WIN32 - env->me_lfd = CreateFile(lpath, GENERIC_READ|GENERIC_WRITE, 909env->me_lfd = CreateFileA(lpath, GENERIC_READ|GENERIC_WRITE, FILE_SHARE_READ|FILE_SHARE_WRITE, NULL, OPEN_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL); #else @@ -4517,9 +4517,9 @@ mdb_env_setup_locks(MDB_env *env, char *lpath, int mode, int xcxcl) mdb_hash_enc(&val, encbuf); sprintf(env->me_txns->mti_rmname, "Global\\MDBr%s", encbuf); sprintf(env->me_txns->mti_wmname, "Global\\MDBw%s", encbuf); - env->me_rmutex = CreateMutex(&mdb_all_sa, FALSE, env->me_txns->mti_rmname); + env->me_rmutex = CreateMutexA(&mdb_all_sa, FALSE, env->me_txns->mti_rmname); if (!env->me_rmutex) goto fail_errno; - env->me_wmutex = CreateMutex(&mdb_all_sa, FALSE, env->me_txns->mti_wmname); + env->me_wmutex = CreateMutexA(&mdb_all_sa, FALSE, env->me_txns->mti_wmname); if (!env->me_wmutex) goto fail_errno; #elif defined(MDB_USE_SYSV_SEM) unsigned short vals[2] = {1, 1}; @@ -4569,9 +4569,9 @@ mdb_env_setup_locks(MDB_env *env, char *lpath, int mode, int *excl) goto fail; %%D #ifdef _WIN32 - env->me_rmutex = OpenMutex(SYNCHRONIZE, FALSE, env->me_txns->mti_rmname); + env->me_rmutex = OpenMutexA(SYNCHRONIZE, FALSE, env->me_txns->mti_rmname); if (!env->me_rmutex) goto fail_errno; - env->me_wmutex =pepenMutex(SYNCHRONIZE, FALSE, env->me_txns->mti_wmname); + env->me_wmutex = OpenMutexA(SYNCHRONIZE, FALSE, env->me_txns->mti_wmname); if (!env->me_wmutex) goto fail_errno; #elif defined(MDB_USE_SYSV_SEM) semid = env->me_txns->mti_semid; @@ -4686,7 +4686,7 @@ mdb_env_open(MDB_env *env, const char *path, unsigned int flags, mdb_mode_t mode len = OPEN_ALWAYS; } mode = FILE_ATTRIBUTE_NORMAL; - env->me_fd = CreateFile(dpath, oflags, FILE_SHARE_READ|FILE_SHARE_WRITE, + env->me_fd = CreateFileA(dpath, oflags, FILE_SHARE_READ|FILE_SHARE_WRITE, NULL, len, mode, NULL); #else if (F_ISSET(flags, MDB_RDONLY)) @@ -4716,7 +4716,7 @@ mdb_env_open(MDB_env *env, const char *path, unsigned int flags, mdb_mode_t mode */ #ifdef _WIN32 len = OPEN_EXISTING; - % e9env->me_mfd = CreateFile(dpath, oflags, + env->me_mfd = CreateFileA(dpath, oflags, FILE_SHARE_READ|FILE_SHARE_WRITE, NULL, len, mode | FILE_FLAG_WRITE_THROUGH, NULL); #else @@ -8981,7 +8981,7 @@ mdb_env_copy2(MDB_env *env, const char *path, unsigned int flags) * already in the OS cache. */ #ifdef _WIN32 - newfd = CreateFile(lpath, GENERIC_WRITE, 0, NULL, CREATE_NEW, + newfd = CreateFileA(lpath, GENERIC_WRITE, 0, NULL, CREATE_NEW, FILE_FLAG_NO_BUFFERING|FILE_FLAG_WRITE_THROUGH, NULL); #else newfd = open(lpath, O_WRONLY|O_CREAT|O_EXCL, 0666); -- 1.9.5.msysgit.0

1 0

(ITS#8068) MDB library patch: Using IsWindowsVersionOrGreater() instead of obsolete GetVersion()
by pmedvedev＠gmail.com 27 Feb '15

27 Feb '15

Full_Name: Pavel Medvedev Version: OS: Windows 7 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (217.25.225.40) --- libraries/liblmdb/mdb.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libraries/liblmdb/mdb.c b/libraries/liblmdb/mdb.c index 65293f4..a54ce83 100644 --- a/libraries/liblmdb/mdb.c +++ b/libraries/liblmdb/mdb.c @@ -239,6 +239,7 @@ union semun { #endif #ifdef _WIN32 +#include <VersionHelpers.h> #define MDB_USE_HASH 1 #define MDB_PIDLOCK 0 #define THREAD_RET DWORD @@ -3998,8 +3999,7 @@ mdb_env_open2(MDB_env *env) #ifdef _WIN32 /* See if we should use QueryLimited */ - rc = GetVersion(); - if ((rc & 0xff) > 5) + if (IsWindowsVersionOrGreater(6, 0, 0)) env->me_pidquery = MDB_PROCESS_QUERY_LIMITED_INFORMATION; else env->me_pidquery = PROCESS_QUERY_INFORMATION; -- 1.9.5.msysgit.0

1 0

(ITS#8067) MDB library patch: Added ssize_t typedef for Visual C++
by pmedvedev＠gmail.com 27 Feb '15

27 Feb '15

Full_Name: Pavel Medvedev Version: OS: Windows 7 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (217.25.225.40) --- libraries/liblmdb/mdb.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/libraries/liblmdb/mdb.c b/libraries/liblmdb/mdb.c index 984a5bd..65293f4 100644 --- a/libraries/liblmdb/mdb.c +++ b/libraries/liblmdb/mdb.c%%0 @@ -96,7 +96,13 @@ extern int cacheflush(char *addr, int nbytes, int cache); #include <stdlib.h> #include <string.h> #include <time.h> + +#ifdef _MSC_VER +#include <io.h> +typedef SSIZE_T ssize_t; +#else #include <unistd.h> +#endif % A #if defined(__sun) || defined(ANDROID) /* Most platforms have posix_memalign, older may only have memalign */

1 0

Re: (ITS#8066) mdb_load truncates long values when resizing buffer
by hyc＠symas.com 26 Feb '15

26 Feb '15

catwell(a)archlinux.us wrote: > Full_Name: Pierre Chapuis > Version: master > OS: GNU/Linux > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (82.238.40.212) > > > Description: > > Starting from 2048, the first input line larger than any power of two N > is truncated to N-1. This results in truncated values in the database. Thanks for the report. Fixed now in mdb.master > > Explanation: > > In the code that resizes the input buffer, fgets() is used. > fgets(*, n, *) reads a maximum of n-1 characters and 0-terminates > the string. When the next chunk is read, the '\0' remains in the > string. Later, strlen() is used and the string is truncated. > > Proposed fix: > > A patch against the current OpenLDAP master llllows. > > --- > > libraries/liblmdb/mdb_load.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/libraries/liblmdb/mdb_load.c b/libraries/liblmdb/mdb_load.c > index f626692..e86b6fd 100644 > --- a/libraries/liblmdb/mdb_load.c > +++ b/libraries/liblmdb/mdb_load.c > @@ -218,7 +218,7 @@ badend: > } > c1 = buf->mv_data; > c1 += buf->mv_size; > - if (fgets((char *)c1, buf->mv_size, stdin) == NULL) {D%D > + if (fgets((char *)c1-1, buf->mv_size+1, stdin) == NULL) { > Eof = 1; > badend(); > return EOF; > > --- > > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8066) mdb_load truncates long values when resizing buffer
by catwell＠archlinux.us 26 Feb '15

26 Feb '15

For what it's worth, here is a slightly better patch. The previous one would start the strlen() one character too far. I don't think it would have much consequence in practice, but this should be safer. --- libraries/liblmdb/mdb_load.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/libraries/liblmdb/mdb_load.c b/libraries/liblmdb/mdb_load.c index f626692..960d23d 100644 --- a/libraries/liblmdb/mdb_load.c +++ b/libraries/liblmdb/mdb_load.c @@ -216,9 +216,8 @@ badend: prog, lineno); return EOF; } - c1 = buf->mv_data; - c1 += buf->mv_size; - if (fgets((char *)c1, buf->mv_size, stdin) == NULL) { + c1 = buf->mv_data + buf->mv_size - 1; + if (fgets((char *)c1, buf->mv_size+1, stdin) == NULL) { Eof = 1; badend(); return EOF; -- Pierre Chapuis

1 0

(ITS#8066) mdb_load truncates long values when resizing buffer
by catwell＠archlinux.us 25 Feb '15

25 Feb '15

Full_Name: Pierre Chapuis Version: master OS: GNU/Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (82.238.40.212) Description: Starting from 2048, the first input line larger than any power of two N is truncated to N-1. This results in truncated values in the database. Explanation: In the code that resizes the input buffer, fgets() is used. fgets(*, n, *) reads a maximum of n-1 characters and 0-terminates the string. When the next chunk is read, the '\0' remains in the string. Later, strlen() is used and the string is truncated. Proposed fix: A patch against the current OpenLDAP master llllows. --- libraries/liblmdb/mdb_load.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libraries/liblmdb/mdb_load.c b/libraries/liblmdb/mdb_load.c index f626692..e86b6fd 100644 --- a/libraries/liblmdb/mdb_load.c +++ b/libraries/liblmdb/mdb_load.c @@ -218,7 +218,7 @@ badend: } c1 = buf->mv_data; c1 += buf->mv_size; - if (fgets((char *)c1, buf->mv_size, stdin) == NULL) {D%D + if (fgets((char *)c1-1, buf->mv_size+1, stdin) == NULL) { Eof = 1; badend(); return EOF; ---

1 0

Re: (ITS#8064) SIGSEGV in monitor-shutdown code
by leo＠yuriev.ru 25 Feb '15

25 Feb '15

Yes, Howard you are right. In our CI-process 'make test' crashes only for 2.5 branch. I was noted 2.4 in the ITS, it's my mistake. To fix this bug seems a de-registration should be provided by monitor-backend. 2015-02-25 17:47 GMT+03:00 Howard Chu <hyc(a)symas.com>: > pierangelo.masarati(a)polimi.it wrote: >> >> back-monitor was designed before back-config, without considering the >> possibility of database removal during regular operations. There might >> be several flaws of that kind. > > > Removal is not supported in 2.4. All of that is work-in-progress for 2.5. > Patches welcome, but this is not a 2.4 bug. > > >> My suggestion is that back-monitor is never shut down (it should be >> inhibited) except at slapd shutdown. Meanwhile, the code initialization >> and shutdown could be redesigned to properly handle those cases. >> >> On 25/02/2015 15:18, leo(a)yuriev.ru wrote: >>> >>> Full_Name: Leonid Yuriev >>> Version: 2.4-HEAD >>> OS: RHEL7 >>> URL: ftp://ftp.openldap.org/incoming/ >>> Submission from: (NULL) (31.130.36.33) >>> >>> >>> Monitor-backend shutdown code could call a callback from registered >>> subsystem >>> after it is already destroyed. >>> >>> For instance, currently ldap-backend frees its own registered context >>> (lmi_mss >>> from ldap_monitor_info_t) before than monitor_back_db_destroy() will be >>> called. >>> >>> Therefore SIGSEGV would be occur on monitor shutdown if any ldap-backend >>> database is configured and such freed context will be overwrited. >>> >>> This bug could be reproduced by a filling-memory-with-non-zero before >>> calling >>> free() from glibc. >>> >>> >>> >> >> > > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8065) autogroup+accesslog problems
by hyc＠openldap.org 25 Feb '15

25 Feb '15

Full_Name: Howard Chu Version: 2.4 OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (199.188.194.66) Submitted by: hyc The autogroup overlay doesn't work well with accesslog. This is the same problem we've had in the past with e.g. memberof, refint, and other overlays that do internal modifications as a side-effect of a main user operation. In particular, autogroup's internal ops use the same timestamp as the original op, so accesslog will log one of these internal ops and then none of the others because it uses the timestamp for the RDN of its log entries, and that RDN is already in use. This means the main user onevever gets logged, nor do any side-effect modifications after the first one. The solution in previous cases has been to turn off all accesslog activity for internal operations. In the case of accesslog used for replication, accesslog shoulononly log the main user operation, and it's up to replicas to perform the same internal operations themselves as needed.

1 0

Re: (ITS#8064) SIGSEGV in monitor-shutdown code
by hyc＠symas.com 25 Feb '15

25 Feb '15

pierangelo.masarati(a)polimi.it wrote: > back-monitor was designed before back-config, without considering the > possibility of database removal during regular operations. There might > be several flaws of that kind. Removal is not supported in 2.4. All of that is work-in-progress for 2.5. Patches welcome, but this is not a 2.4 bug. > My suggestion is that back-monitor is never shut down (it should be > inhibited) except at slapd shutdown. Meanwhile, the code initialization > and shutdown could be redesigned to properly handle those cases. > > On 25/02/2015 15:18, leo(a)yuriev.ru wrote: >> Full_Name: Leonid Yuriev >> Version: 2.4-HEAD >> OS: RHEL7 >> URL: ftp://ftp.openldap.org/incoming/ >> Submission from: (NULL) (31.130.36.33) >> >> >> Monitor-backend shutdown code could call a callback from registered subsystem >> after it is already destroyed. >> >> For instance, currently ldap-backend frees its own registered context (lmi_mss >> from ldap_monitor_info_t) before than monitor_back_db_destroy() will be called. >> >> Therefore SIGSEGV would be occur on monitor shutdown if any ldap-backend >> database is configured and such freed context will be overwrited. >> >> This bug could be reproduced by a filling-memory-with-non-zero before calling >> free() from glibc. >> >> >> > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8064) SIGSEGV in monitor-shutdown code
by pierangelo.masarati＠polimi.it 25 Feb '15

25 Feb '15

back-monitor was designed before back-config, without considering the possibility of database removal during regular operations. There might be several flaws of that kind. My suggestion is that back-monitor is never shut down (it should be inhibited) except at slapd shutdown. Meanwhile, the code initialization and shutdown could be redesigned to properly handle those cases. On 25/02/2015 15:18, leo(a)yuriev.ru wrote: > Full_Name: Leonid Yuriev > Version: 2.4-HEAD > OS: RHEL7 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (31.130.36.33) > > > Monitor-backend shutdown code could call a callback from registered subsystem > after it is already destroyed. > > For instance, currently ldap-backend frees its own registered context (lmi_mss > from ldap_monitor_info_t) before than monitor_back_db_destroy() will be called. > > Therefore SIGSEGV would be occur on monitor shutdown if any ldap-backend > database is configured and such freed context will be overwrited. > > This bug could be reproduced by a filling-memory-with-non-zero before calling > free() from glibc. > > > -- Pierangelo Masarati Associate Professor Dipartimento di Scienze e Tecnologie Aerospaziali Politecnico di Milano

1 0

(ITS#8064) SIGSEGV in monitor-shutdown code
by leo＠yuriev.ru 25 Feb '15

25 Feb '15

Full_Name: Leonid Yuriev Version: 2.4-HEAD OS: RHEL7 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (31.130.36.33) Monitor-backend shutdown code could call a callback from registered subsystem after it is already destroyed. For instance, currently ldap-backend frees its own registered context (lmi_mss from ldap_monitor_info_t) before than monitor_back_db_destroy() will be called. Therefore SIGSEGV would be occur on monitor shutdown if any ldap-backend database is configured and such freed context will be overwrited. This bug could be reproduced by a filling-memory-with-non-zero before calling free() from glibc.

1 0

(ITS#8063) syncprov+autogroup deadlock
by hyc＠openldap.org 24 Feb '15

24 Feb '15

Full_Name: Howard Chu Version: 2.4 OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (199.188.194.66) Submitted by: hyc syncprov can deadlock itself if a persistent search is active while a group entry controlled by autogroup is being modified. Fix coming shortly.

1 0

Re: (ITS#8062) Assertion 'IS_LEAF(mp)' failed in mdb_cursor_next()
by g＠wenarab.com 24 Feb '15

24 Feb '15

--bcaec53d526febcac5050fd27aa4 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable > > Thanks for the report. Unfortunately, it appears the DB was constructed > incorrectly at some time in the past, so even updating to later revisions > of the code won't avoid the crash. It's more important now to find the > sequence of steps that recreates the invalid DB in the first place, or > verify that the same sequence doesn't crash using the 0.9.15 release > candidate. OK. I'll try to reproduce the steps. Thanks. G *=E2=80=94 http://gawen.me/ <http://gawen.me/> **- +33 (0) 6 65 42 31 83* On Tue, Feb 24, 2015 at 1:49 AM, Howard Chu <hyc(a)symas.com> wrote: > g(a)wenarab.com wrote: > >> Full_Name: Gawen Arab >> Version: 0.9.14 >> OS: OSX >> URL: http://gawen.me/pub/lmdb_crash/gawen-150223.tar.xz >> Submission from: (NULL) (77.151.59.7) >> >> >> Hello, >> >> I'm currently developing a virtual filesystem whose metadata are stored >> in a >> LMDB 0.9.14 database. This software can currently run over Android, iOS, >> Linux, >> OSX and Windows. >> >> This software is currently used by 50 users and I received 11 crash >> reports of >> SIGSEV raised in LMDB. All reports come from OSX computers. >> > > Thanks for the report. Unfortunately, it appears the DB was constructed > incorrectly at some time in the past, so even updating to later revisions > of the code won't avoid the crash. It's more important now to find the > sequence of steps that recreates the invalid DB in the first place, or > verify that the same sequence doesn't crash using the 0.9.15 release > candidate. > > >> mdb.c:5382: Assertion 'IS_LEAF(mp)' failed in mdb_cursor_next() >> >> This comes from this assertion >> https://gitorious.org/mdb/mdb/source/2f587ae081d076e3707360c5db0865 >> 20c219d3ea:libraries/liblmdb/mdb.c#L5382 >> . >> >> This happens when the software iterates over keys in the database, the >> same way >> mdb_dump does, calling in loop mdb_cursor_get(..., MDB_NEXT). >> >> I managed to get back the LMDB db folder from one user which runs on OSX >> 64bits. >> I performed a mdb_dump of it with my Linux 64bits. mdb_dump crashes the >> same >> way. I tried with the lastest mdb_dump from the mdb.master branch (commi= t >> 3368d1f5e243225cba4d730fba19ff600798ebe3), but I have the same assertion= . >> >> mdb.c:5599: Assertion 'IS_LEAF(mp)' failed in mdb_cursor_next() >> >> I attached to this issue the faulty database 'db/'. >> >> The output of mdb_dump on my machine is attached to this issue named as >> the >> file >> 'mdb_dump.log'. I runned back mdb_dump with MDB_DEBUG defined to 2 and >> mdb_debug >> true. I attached the output of this new dump as the file >> 'mdb_dump.debug.log'. >> The interesting part: >> >> mdb_cursor_next:5574 cursor_next: top page is 1946 in cursor 0x1897350 >> mdb_cursor_next:5591 =3D=3D> cursor points to page 1946 with 50 keys, ke= y >> index 48 >> b5f1ff0700000000 >> 0700000000000000bc10000000000000000e000000000000860d00000000 >> 0000e308000000000000ee07000000000000a5070000000000000307000000000000 >> mdb_cursor_next:5574 cursor_next: top page is 1946 in cursor 0x1897350 >> mdb_cursor_next:5591 =3D=3D> cursor points to page 1946 with 50 keys, ke= y >> index 49 >> b6f1ff0700000000 >> 0700000000000000820e000000000000310e000000000000770d00000000 >> 0000660c000000000000cb0a0000000000002a080000000000000008000000000000 >> mdb_cursor_next:5574 cursor_next: top page is 1946 in cursor 0x1897350 >> mdb_cursor_next:5579 =3D=3D=3D=3D=3D> move to next sibling page mdb_curs= or_pop:5095 >> popped >> page 1946 off db 1 cursor 0x1897350 mdb_cursor_sibling:5503 parent page >> is page >> 4643, index 16 mdb_cursor_sibling:5521 just moving to right index key 17 >> mdb_cursor_push:5104 pushing page 704 on db 1 cursor 0x1897350 >> mdb_cursor_next:5585 next page is 704, key index 0 mdb_cursor_next:5591 >> =3D=3D> >> cursor points to page 704 with 7 keys, key index 0 mdb.c:5599: Assertion >> 'IS_LEAF(mp)' failed in mdb_cursor_next() >> %%0 >> I do not know much about internal LMDB design for now, so I'm struggling >> to >> understand the debug lines... >> >> Unfortunately, I do not know how to reproduce this bug, but it is a >> recurring >> one. >> >> Maybe the following information can help you, >> >> - My software opens the database in MDB_NOSYNC mode, and performs a >> mdb_env_sync(env, 1) every second or less. A mdb_env_sync can happen >> at the >> same time of any other operation (i.e. mdb_txn_commit). I didn't put >> any lock >> mechanism. Could this expln n such situation ? >> >> - mapsize is currently 512MiB. Previous versions of the software set to >> smaller >> values (the database had not been rebuilt since). >> >> I'd be happy to provide more information if you need. >> >> Thank you for your support and this amazing IP. >> >> Regards >> >> >> >> > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/ > --bcaec53d526febcac5050fd27aa4 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px = 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-l= eft-style:solid;padding-left:1ex">Thanks for the report. Unfortunately, it appears the DB was constructed= incorrectly at some time in the past, so even updating to later revisions = of the code won't avoid the crash. It's more important now to find = the sequence of steps that recreates the invalid DB in the first place, or = verify that the same sequence doesn't crash using the 0.9.15 release ca= ndidate.</blockquote><div> </div><div>OK. I'll try to reprodu= ce the steps.</div><div> </div><div>Thanks.</div></div><div class=3D"gma= il_extra"> <div><div class=3D"gmail_signature"><div dir=3D= "ltr"><div><div><div><div><div>G</spa= n> <sp= an style=3D"font-weight:normal">=E2=80=94=C2=A0 </= span><spa= n style=3D"color:rgb(68,68,68)"><a href=3D"http://gawen.me/" target=3D"_bla= nk">http://gawen.me/</a> - +33 (0) 6 65 42 31 83</= span></div></sp= an><div><= span style=3D"border-collapse:collapse"></div></div></div></d= iv></div> </div></div></div> <div class=3D"gmail_quote">On Tue, Feb 24, 2015 at 1:49 AM, Howard Chu = <<a href=3D"mailto:hyc@symas.com" target=3D"_blank">hy= c(a)symas.com</a>> wrote: <blockquote class=3D"gmail_quote" styl= e=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><a href= =3D"mailto:g@wenarab.com" target=3D"_blank">g(a)wenarab.com</a> wrote: <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> Full_Name: Gawen Arab Version: 0.9.14 OS: OSX URL: <a href=3D"http://gawen.me/pub/lmdb_crash/gawen-150223.tar.xz" target= =3D"_blank">http://gawen.me/pub/lmdb_crash/gawen-150223.tar.xz</a><b= r> Submission from: (NULL) (77.151.59.7) Hello, I'm currently developing a virtual filesystem whose metadata are stored= in a LMDB 0.9.14 database. This software can currently run over Android, iOS, Li= nux, OSX and Windows. This software is currently used by 50 users and I received 11 crash reports= of SIGSEV raised in LMDB. All reports come from OSX computers. </blockquote> Thanks for the report. Unfortunately, it appears the DB was constructed inc= orrectly at some time in the past, so even updating to later revisions of t= he code won't avoid the crash. It's more important now to find the = sequence of steps that recreates the invalid DB in the first place, or veri= fy that the same sequence doesn't crash using the 0.9.15 release candid= ate. <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> mdb.c:5382: Assertion 'IS_LEAF(mp)' failed in mdb_cursor_next() This comes from this assertion <a href=3D"https://gitorious.org/mdb/mdb/source/2f587ae081d076e3707360c5db0= 86520c219d3ea:libraries/liblmdb/mdb.c#L5382" target=3D"_blank">https://gito= rious.org/mdb/mdb/source/2f587ae081d076e3707360c5db0865</u= >20c219d3ea:libraries/liblmdb/mdb.c#L5382</a> . This happens when the software iterates over keys in the database, the same= way mdb_dump does, calling in loop mdb_cursor_get(..., MDB_NEXT). I managed to get back the LMDB db folder from one user which runs on OSX<br= > 64bits. I performed a mdb_dump of it with my Linux 64bits. mdb_dump crashes the sam= e way. I tried with the lastest mdb_dump from the mdb.master branch (commit<b= r> 3368d1f5e243225cba4d730fba19ff600798ebe3), but I have the same asser= tion. mdb.c:5599: Assertion 'IS_LEAF(mp)' failed in mdb_cursor_next() I attached to this issue the faulty database 'db/'. The output of mdb_dump on my machine is attached to this issue named as the= file 'mdb_dump.log'. I runned back mdb_dump with MDB_DEBUG defined to 2 = and mdb_debug true. I attached the output of this new dump as the file 'mdb_dump.debu= g.log'. The interesting part: mdb_cursor_next:5574 cursor_next: top page is 1946 in cursor 0x1897350 mdb_cursor_next:5591 =3D=3D> cursor points to page 1946 with 50 keys, ke= y index 48 b5f1ff0700000000 0700000000000000bc10000000000000000e000000000000860d000000000= 000e308000000000000ee07000000000000a5070000000000000307000000= 000000 mdb_cursor_next:5574 cursor_next: top page is 1946 in cursor 0x1897350 mdb_cursor_next:5591 =3D=3D> cursor points to page 1946 with 50 keys, ke= y index 49 b6f1ff0700000000 0700000000000000820e000000000000310e000000000000770d000000000= 000660c000000000000cb0a0000000000002a080000000000000008000000= 000000 mdb_cursor_next:5574 cursor_next: top page is 1946 in cursor 0x1897350 mdb_cursor_next:5579 =3D=3D=3D=3D=3D> move to next sibling page mdb_curs= or_pop:5095 popped page 1946 off db 1 cursor 0x1897350 mdb_cursor_sibling:5503 parent page is = page 4643, index 16 mdb_cursor_sibling:5521 just moving to right index key 17<br= > mdb_cursor_push:5104 pushing page 704 on db 1 cursor 0x1897350 mdb_cursor_next:5585 next page is 704, key index 0 mdb_cursor_next:5591 =3D= =3D> cursor points to page 704 with 7 keys, key index 0 mdb.c:5599: Assertion<br= > 'IS_LEAF(mp)' failed in mdb_cursor_next() %%0 I do not know much about internal LMDB design for now, so I'm strugglin= g to understand the debug lines... Unfortunately, I do not know how to reproduce this bug, but it is a recurri= ng one. Maybe the following information can help you, - My software opens the database in MDB_NOSYNC mode, and performs a =C2=A0 =C2=A0mdb_env_sync(env, 1) every second or less. A mdb_env_sync can = happen at the =C2=A0 =C2=A0same time of any other operation (i.e. mdb_txn_commit). I didn= 't put any lock =C2=A0 =C2=A0mechanism. Could this expln n such situation ? - mapsize is currently 512MiB. Previous versions of the software set to sma= ller =C2=A0 =C2=A0values (the database had not been rebuilt since). I'd be happy to provide more information if you need. Thank you for your support and this amazing IP. Regards </blockquote> -- =C2=A0 -- Howard Chu =C2=A0 CTO, Symas Corp.=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0<a href=3D"= http://www.symas.com" target=3D"_blank">http://www.symas.com</a> =C2=A0 Director, Highland Sun=C2=A0 =C2=A0 =C2=A0<a href=3D"http://highland= sun.com/hyc/" target=3D"_blank">http://highlandsun.com/hyc/</a> =C2=A0 Chief Architect, OpenLDAP=C2=A0 <a href=3D"http://www.openldap.org/p= roject/" target=3D"_blank">http://www.openldap.org/project/</a> </blockquote></div> </div> --bcaec53d526febcac5050fd27aa4--

1 0

Re: (ITS#8054) Operation duration logging
by clem.oudot＠gmail.com 24 Feb '15

24 Feb '15

2015-02-11 18:33 GMT+01:00 <h.b.furuseth(a)usit.uio.no>: > On 11. feb. 2015 12:38, Michael Str=C3=B6der wrote: >> h.b.furuseth(a)usit.uio.no wrote: >>> Cool feture, but please use fewer characteres in the already >>> bloated log. How about "time=3D<integer>us" or even omit the "us"? >> >> +1 >> >> But 'time' sounds a bit general. >> How about 'tdu' or even 'td' as abbreviation for "time duration"? > > I don't think so. You can't see from that what it means unless you > already know, so you have to look it up just to see if you're > interested. 'time' you can decide to look up if you are interested. > Seems other LDAP servers have choosen 'etime' (see http://ludopoitou.com/2015/02/24/about-auditing-ldap-operations/). Why not try to use the same word? Cl=C3=A9ment OUDOT.

1 0

Re: (ITS#8062) Assertion 'IS_LEAF(mp)' failed in mdb_cursor_next()
by hyc＠symas.com 23 Feb '15

23 Feb '15

g(a)wenarab.com wrote: > Full_Name: Gawen Arab > Version: 0.9.14 > OS: OSX > URL: http://gawen.me/pub/lmdb_crash/gawen-150223.tar.xz > Submission from: (NULL) (77.151.59.7) > > > Hello, > > I'm currently developing a virtual filesystem whose metadata are stored in a > LMDB 0.9.14 database. This software can currently run over Android, iOS, Linux, > OSX and Windows. > > This software is currently used by 50 users and I received 11 crash reports of > SIGSEV raised in LMDB. All reports come from OSX computers. Thanks for the report. Unfortunately, it appears the DB was constructed incorrectly at some time in the past, so even updating to later revisions of the code won't avoid the crash. It's more important now to find the sequence of steps that recreates the invalid DB in the first place, or verify that the same sequence doesn't crash using the 0.9.15 release candidate. > > mdb.c:5382: Assertion 'IS_LEAF(mp)' failed in mdb_cursor_next() > > This comes from this assertion > https://gitorious.org/mdb/mdb/source/2f587ae081d076e3707360c5db086520c219d3… > . > > This happens when the software iterates over keys in the database, the same way > mdb_dump does, calling in loop mdb_cursor_get(..., MDB_NEXT). > > I managed to get back the LMDB db folder from one user which runs on OSX > 64bits. > I performed a mdb_dump of it with my Linux 64bits. mdb_dump crashes the same > way. I tried with the lastest mdb_dump from the mdb.master branch (commit > 3368d1f5e243225cba4d730fba19ff600798ebe3), but I have the same assertion. > > mdb.c:5599: Assertion 'IS_LEAF(mp)' failed in mdb_cursor_next() > > I attached to this issue the faulty database 'db/'. > > The output of mdb_dump on my machine is attached to this issue named as the > file > 'mdb_dump.log'. I runned back mdb_dump with MDB_DEBUG defined to 2 and > mdb_debug > true. I attached the output of this new dump as the file 'mdb_dump.debug.log'. > The interesting part: > > mdb_cursor_next:5574 cursor_next: top page is 1946 in cursor 0x1897350 > mdb_cursor_next:5591 ==> cursor points to page 1946 with 50 keys, key index 48 > b5f1ff0700000000 > 0700000000000000bc10000000000000000e000000000000860d000000000000e308000000000000ee07000000000000a5070000000000000307000000000000 > mdb_cursor_next:5574 cursor_next: top page is 1946 in cursor 0x1897350 > mdb_cursor_next:5591 ==> cursor points to page 1946 with 50 keys, key index 49 > b6f1ff0700000000 > 0700000000000000820e000000000000310e000000000000770d000000000000660c000000000000cb0a0000000000002a080000000000000008000000000000 > mdb_cursor_next:5574 cursor_next: top page is 1946 in cursor 0x1897350 > mdb_cursor_next:5579 =====> move to next sibling page mdb_cursor_pop:5095 > popped > page 1946 off db 1 cursor 0x1897350 mdb_cursor_sibling:5503 parent page is page > 4643, index 16 mdb_cursor_sibling:5521 just moving to right index key 17 > mdb_cursor_push:5104 pushing page 704 on db 1 cursor 0x1897350 > mdb_cursor_next:5585 next page is 704, key index 0 mdb_cursor_next:5591 ==> > cursor points to page 704 with 7 keys, key index 0 mdb.c:5599: Assertion > 'IS_LEAF(mp)' failed in mdb_cursor_next() > %%0 > I do not know much about internal LMDB design for now, so I'm struggling to > understand the debug lines... > > Unfortunately, I do not know how to reproduce this bug, but it is a recurring > one. > > Maybe the following information can help you, > > - My software opens the database in MDB_NOSYNC mode, and performs a > mdb_env_sync(env, 1) every second or less. A mdb_env_sync can happen at the > same time of any other operation (i.e. mdb_txn_commit). I didn't put any lock > mechanism. Could this expln n such situation ? > > - mapsize is currently 512MiB. Previous versions of the software set to smaller > values (the database had not been rebuilt since). > > I'd be happy to provide more information if you need. > > Thank you for your support and this amazing IP. > > Regards > > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8062) Assertion 'IS_LEAF(mp)' failed in mdb_cursor_next()
by g＠wenarab.com 23 Feb '15

23 Feb '15

Full_Name: Gawen Arab Version: 0.9.14 OS: OSX URL: http://gawen.me/pub/lmdb_crash/gawen-150223.tar.xz Submission from: (NULL) (77.151.59.7) Hello, I'm currently developing a virtual filesystem whose metadata are stored in a LMDB 0.9.14 database. This software can currently run over Android, iOS, Linux, OSX and Windows. This software is currently used by 50 users and I received 11 crash reports of SIGSEV raised in LMDB. All reports come from OSX computers. mdb.c:5382: Assertion 'IS_LEAF(mp)' failed in mdb_cursor_next() This comes from this assertion https://gitorious.org/mdb/mdb/source/2f587ae081d076e3707360c5db086520c219d3… . This happens when the software iterates over keys in the database, the same way mdb_dump does, calling in loop mdb_cursor_get(..., MDB_NEXT). I managed to get back the LMDB db folder from one user which runs on OSX 64bits. I performed a mdb_dump of it with my Linux 64bits. mdb_dump crashes the same way. I tried with the lastest mdb_dump from the mdb.master branch (commit 3368d1f5e243225cba4d730fba19ff600798ebe3), but I have the same assertion. mdb.c:5599: Assertion 'IS_LEAF(mp)' failed in mdb_cursor_next() I attached to this issue the faulty database 'db/'. The output of mdb_dump on my machine is attached to this issue named as the file 'mdb_dump.log'. I runned back mdb_dump with MDB_DEBUG defined to 2 and mdb_debug true. I attached the output of this new dump as the file 'mdb_dump.debug.log'. The interesting part: mdb_cursor_next:5574 cursor_next: top page is 1946 in cursor 0x1897350 mdb_cursor_next:5591 ==> cursor points to page 1946 with 50 keys, key index 48 b5f1ff0700000000 0700000000000000bc10000000000000000e000000000000860d000000000000e308000000000000ee07000000000000a5070000000000000307000000000000 mdb_cursor_next:5574 cursor_next: top page is 1946 in cursor 0x1897350 mdb_cursor_next:5591 ==> cursor points to page 1946 with 50 keys, key index 49 b6f1ff0700000000 0700000000000000820e000000000000310e000000000000770d000000000000660c000000000000cb0a0000000000002a080000000000000008000000000000 mdb_cursor_next:5574 cursor_next: top page is 1946 in cursor 0x1897350 mdb_cursor_next:5579 =====> move to next sibling page mdb_cursor_pop:5095 popped page 1946 off db 1 cursor 0x1897350 mdb_cursor_sibling:5503 parent page is page 4643, index 16 mdb_cursor_sibling:5521 just moving to right index key 17 mdb_cursor_push:5104 pushing page 704 on db 1 cursor 0x1897350 mdb_cursor_next:5585 next page is 704, key index 0 mdb_cursor_next:5591 ==> cursor points to page 704 with 7 keys, key index 0 mdb.c:5599: Assertion 'IS_LEAF(mp)' failed in mdb_cursor_next() %%0 I do not know much about internal LMDB design for now, so I'm struggling to understand the debug lines... Unfortunately, I do not know how to reproduce this bug, but it is a recurring one. Maybe the following information can help you, - My software opens the database in MDB_NOSYNC mode, and performs a mdb_env_sync(env, 1) every second or less. A mdb_env_sync can happen at the same time of any other operation (i.e. mdb_txn_commit). I didn't put any lock mechanism. Could this expln n such situation ? - mapsize is currently 512MiB. Previous versions of the software set to smaller values (the database had not been rebuilt since). I'd be happy to provide more information if you need. Thank you for your support and this amazing IP. Regards

1 0

Re: (ITS#8044) openldap 2.4.39-8.el6: issue causing server unavailability
by Diana.Scannicchio＠cern.ch 23 Feb '15

23 Feb '15

I checked and the appropriate credentials are used, nothing in the configur= ation changed between this version (2.4.39-8) and the previous one(s). These are the corresponding lines of the slapd.conf on the consumer: overlay chain chain-rebind-as-user FALSE chain-uri "ldap://ldap_provider:389" chain-rebind-as-user TRUE chain-idassert-bind bindmethod=3D"simple" binddn=3D"cn=3DManager,ou=3Datlas,o=3Dcern,c=3Dc= h" credentials=3D"ldap_manager_pw" mode=3D"self" #chain-tls start chain-return-error TRUE Unless you spot something wrong in the configuration attached to the initia= l mail, there should be something else different between this non-working v= ersion and the previous ones, Thank you and best regards, Diana On 17 Feb 2015, at 12:55, hyc(a)symas.com wrote: > Diana.Scannicchio(a)cern.ch wrote: >> Is there anybody that could help on this issue?=3D20 >> this version of openldap is not usable, so I would like to understand wh= ich=3D >> is the problem and if can be fixed. >> Thank you and best regards, >=20 > The error message you're referring to was added in the patch for this ITS >=20 > http://www.openldap.org/its/index.cgi/Software%20Bugs?id=3D6851;selectid= =3D6851 >=20 > Check that you've configured appropriate credentials if you're using idas= sert on the target URI. >>=20 >> Diana >>=20 >>=20 >> On 02 Feb 2015, at 20:56, diana.scannicchio(a)cern.ch wrote: >>=20 >>> Should not, I did not enable it in the slapd.conf. >>> =3D20 >>> Diana >>> =3D20 >>> On 02 Feb 2015, at 20:34, <michael(a)stroeder.com> <michael(a)stroeder.com>= w=3D >> ro=3D3D >>> te: >>> =3D20 >>>> Is SSL/TLS part of the game? >>>> =3D3D20 >>>> Ciao, Michael. >>>> =3D3D20 >>>> =3D3D20 >>>> =3D3D20 >>>> =3D3D20 >>> =3D20 >>> - >>> Diana Scannicchio >>> University of California, Irvine >>> ATLAS TDAQ SysAdmin group >>> Office: +41 22 76 75240 >>> OnCall: 164851 >>> =3D20 >>> =3D20 >>> =3D20 >>> =3D20 >>> =3D20 >>> =3D20 >>> =3D20 >>> =3D20 >>> =3D20 >>=20 >> - >> Diana Scannicchio >> University of California, Irvine >> ATLAS TDAQ SysAdmin group >> Office: +41 22 76 75240 >> OnCall: 164851 >>=20 >>=20 >>=20 >>=20 >>=20 >>=20 >>=20 >>=20 >>=20 >>=20 >=20 >=20 > --=20 > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/ >=20 >=20 >=20 - Diana Scannicchio University of California, Irvine ATLAS TDAQ SysAdmin group Office: +41 22 76 75240 OnCall: 164851

1 0

Re: (ITS#8027) ldapsearch -E deref=member: crashes slapd
by Howard Chu 23 Feb '15

23 Feb '15

Of course not. But slapo-deref is not installed or enabled by default. And since it is undocumented, no one would enable it by accident - most don't even know that it exists.

1 0

Re: (ITS#7796) LDAP_DEBUG_TRACE for "not indexed" log messages
by michael＠stroeder.com 22 Feb '15

22 Feb '15

Could you please add this patch to RE24? The amount of unneeded log messages in my setup with lots of ACLs using non-indexed filters is huge and I hate to maintain local build patches. Ciao, Michael.

1 0

(ITS#8061
by frederic.jacquot＠insa-lyon.fr 18 Feb '15

18 Feb '15

I compiled a new instance of OpenLDAP from the git Master and the bug seems to be fixed. After parsing the git commits, I found a link back to a bug report : http://www.openldap.org/its/index.cgi/Software%20Bugs?id=7975;selectid=7975 It's similar to mine (but didn't come up in my searches, sorry) so my bug report can be closed. Frederic Jacquot INSA Lyon

1 0

(ITS#8061) Base object returned when searching with scope: singleLevel (1) when using mdb backend and paged results
by frederic.jacquot＠insa-lyon.fr 18 Feb '15

18 Feb '15

Full_Name: Frederic Jacquot Version: 2.4.40 OS: Ubuntu 14.04.1 LTS URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (134.214.252.20) Hi, I think I discovered a bug with OpenLDAP 2.4.40, using an mdb backend. If you do a simple search request with no pagination and a SingleLevel (1) scope, everything is ok. But if you enable pagination for the same search, the base will also be returned in the search results. This breaks RFC 4511, paragraph 4.5.1.2 (SearchRequest.scope) : "singleLevel: The scope is constrained to the immediate subordinates of the entry named by baseObject." To reproduce the problem, I compiled a fresh OpenLDAP 2.4.40 with --enable-mdb. I then created a root entry : dn: dc=my-domain,dc=com objectClass: dcObject objectClass: organization dc: my-domain o : my-domain You can now compare search results. This is for a simple search : ldapsearch -s one -h localhost -b "dc=my-domain,dc=com" -w secret -D "cn=Manager,dc=my-domain,dc=com" # extended LDIF # # LDAPv3 # base <dc=my-domain,dc=com> with scope oneLevel # filter: (objectclass=*) # requesting: ALL # # search result search: 2 result: 0 Success Which is ok. This is the same query with paged results enabled : ldapsearch -s one -h 134.214.182.252 -b "dc=my-domain,dc=com" -w secret -D "cn=Manager,dc=my-domain,dc=com" -E pr=100 # extended LDIF # # LDAPv3 # base <dc=my-domain,dc=com> with scope oneLevel # filter: (objectclass=*) # requesting: ALL # with pagedResults control: size=100 # # my-domain.com dn: dc=my-domain,dc=com objectClass: dcObject objectClass: organization dc: my-domain o: my-domain # search result search: 2 result: 0 Success control: 1.2.840.113556.1.4.319 false MAUCAQAEAA== pagedresults: cookie= # numResponses: 2 # numEntries: 1 The base object is returned, but shouldn't be. This behaviour shows an endless recursive directory hierarchy in some LDAP browsers (sometimes crashing them). It can also create endless loops in some applications querying the LDAP server (Canon Uniflow in my case). Regards, Frederic Jacquot INSA Lyon

1 0

(ITS#8060) mdb_from_db (bdb import)
by graehl＠gmail.com 17 Feb '15

17 Feb '15

Full_Name: Jonathan Graehl Version: n/a OS: n/a URL: ftp://ftp.openldap.org/incoming/0001-mdb_from_db-new-options-bugfix.patch Submission from: (NULL) (104.174.227.200) (see 0001-mdb_from_db-new-options-bugfix.patch) mdb_from_db utility for direct import from Berkeley DB to mdb Apparently this can only be distributed in binary form using older berkeley db versions, but building the utility from source for bulk import should be fine.

1 0

Re: (ITS#8044) openldap 2.4.39-8.el6: issue causing server unavailability
by hyc＠symas.com 17 Feb '15

17 Feb '15

Diana.Scannicchio(a)cern.ch wrote: > Is there anybody that could help on this issue?=20 > this version of openldap is not usable, so I would like to understand which= > is the problem and if can be fixed. > Thank you and best regards, The error message you're referring to was added in the patch for this ITS http://www.openldap.org/its/index.cgi/Software%20Bugs?id=6851;selectid=6851 Check that you've configured appropriate credentials if you're using idassert on the target URI. > > Diana > > > On 02 Feb 2015, at 20:56, diana.scannicchio(a)cern.ch wrote: > >> Should not, I did not enable it in the slapd.conf. >> =20 >> Diana >> =20 >> On 02 Feb 2015, at 20:34, <michael(a)stroeder.com> <michael(a)stroeder.com> w= > ro=3D >> te: >> =20 >>> Is SSL/TLS part of the game? >>> =3D20 >>> Ciao, Michael. >>> =3D20 >>> =3D20 >>> =3D20 >>> =3D20 >> =20 >> - >> Diana Scannicchio >> University of California, Irvine >> ATLAS TDAQ SysAdmin group >> Office: +41 22 76 75240 >> OnCall: 164851 >> =20 >> =20 >> =20 >> =20 >> =20 >> =20 >> =20 >> =20 >> =20 > > - > Diana Scannicchio > University of California, Irvine > ATLAS TDAQ SysAdmin group > Office: +41 22 76 75240 > OnCall: 164851 > > > > > > > > > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8059) In client code, set option LDAP_OPT_DEBUG_LEVEL LDAP_DEBUG_ANY does not set option.
by hyc＠symas.com 17 Feb '15

17 Feb '15

peter.driscoll(a)dionglobal.com wrote: > Full_Name: Peter John Driscoll > Version: openldap-2.4.40 > OS: Linux > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (203.3.133.17) > > > Set this option does not turn on debugging. > > #define LDAP_DEBUG_ANY 0xffff > const int optionValue = LDAP_DEBUG_ANY; > CHECK_RESULT(ldap_set_option(m_ld, LDAP_OPT_DEBUG_LEVEL, &optionValue), > "ldap_set_option debug level"); The ITS is for actual bug reports, not usage questions. Closing this ITS. Use the -technical mailing list for usage questions. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8044) openldap 2.4.39-8.el6: issue causing server unavailability
by Diana.Scannicchio＠cern.ch 17 Feb '15

17 Feb '15

Is there anybody that could help on this issue?=20 this version of openldap is not usable, so I would like to understand which= is the problem and if can be fixed. Thank you and best regards, Diana On 02 Feb 2015, at 20:56, diana.scannicchio(a)cern.ch wrote: > Should not, I did not enable it in the slapd.conf. >=20 > Diana >=20 > On 02 Feb 2015, at 20:34, <michael(a)stroeder.com> <michael(a)stroeder.com> w= ro=3D > te: >=20 >> Is SSL/TLS part of the game? >> =3D20 >> Ciao, Michael. >> =3D20 >> =3D20 >> =3D20 >> =3D20 >=20 > - > Diana Scannicchio > University of California, Irvine > ATLAS TDAQ SysAdmin group > Office: +41 22 76 75240 > OnCall: 164851 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 - Diana Scannicchio University of California, Irvine ATLAS TDAQ SysAdmin group Office: +41 22 76 75240 OnCall: 164851

1 0

Re: AW: (ITS#8058) OpenLDAP: module autogroup (memberof)
by michael＠stroeder.com 17 Feb '15

17 Feb '15

Tobias.Helfenstein(a)wald-rlp.de wrote: > sorry, I forgot to say that we have to use the version 2.4.39 from CentOS. > So we checked out the repo and compiled the latest autogroup source code > against the CentOS patched 2.4.39 slapd. Of course, mixing sources can lead to seg faults. Why didn't you compile the whole 2.4.40 release or even better the RE24 git branch? Export the RE24 git branch here: http://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=snapshot;h=refs/h… Ciao, Michael.

1 0

AW: (ITS#8058) OpenLDAP: module autogroup (memberof)
by Tobias.Helfenstein＠wald-rlp.de 17 Feb '15

17 Feb '15

Hello,=0A= =0A= sorry, I forgot to say that we have to use the version 2.4.39 from CentOS.= =0A= So we checked out the repo and compiled the latest autogroup source code ag= ainst the CentOS patched 2.4.39 slapd.=0A= =0A= When we tried to compile the autogroup source code provided with CentOS the= n the module triggers a segmentation fault.=0A= =0A= Is it possible that the problem occurs because of the CentOS patches?=0A= =0A= Best regards=0A= Tobias=0A= ________________________________________=0A= Von: Michael Str=F6der [michael(a)stroeder.com]=0A= Gesendet: Sonntag, 15. Februar 2015 20:16=0A= An: Helfenstein, Tobias; openldap-its(a)OpenLDAP.org=0A= Betreff: Re: (ITS#8058) OpenLDAP: module autogroup (memberof)=0A= =0A= tobias.helfenstein(a)wald-rlp.de wrote:=0A= > Version: 2.4.39=0A= >=0A= > we have problems when activating the module autogroup (memberof).=0A= =0A= There were several fixes after 2.4.39.=0A= =0A= Could you please try RE24 branch from git repo?=0A= =0A= Ciao, Michael.=0A= Mail wurde erfoglreich durch Sophos PureMessage auf Viren gepr=FCft.=

1 0

(ITS#8059) In client code, set option LDAP_OPT_DEBUG_LEVEL LDAP_DEBUG_ANY does not set option.
by peter.driscoll＠dionglobal.com 16 Feb '15

16 Feb '15

Full_Name: Peter John Driscoll Version: openldap-2.4.40 OS: Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (203.3.133.17) Set this option does not turn on debugging. #define LDAP_DEBUG_ANY 0xffff const int optionValue = LDAP_DEBUG_ANY; CHECK_RESULT(ldap_set_option(m_ld, LDAP_OPT_DEBUG_LEVEL, &optionValue), "ldap_set_option debug level"); Calling ldap_get_option confirms the change. But in libraries/libldap/sbind.c int ldap_simple_bind_s( LDAP *ld, LDAP_CONST char *dn, LDAP_CONST char *passwd ) The call, Debug( LDAP_DEBUG_TRACE, "ldap_simple_bind_s\n", 0, 0, 0 ); Does not log any debug information, because ldap_debug does not returns 0 instead of 0xFFFF. Debug is defined, #define Debug( level, fmt, arg1, arg2, arg3 ) \ Log3( (level), 0, (fmt), (arg1), (arg2), (arg3) ) #define Log3( level, severity, fmt, arg1, arg2, arg3 ) \ do { \ if ( ldap_debug & (level) ) \ lutil_debug( ldap_debug, (level), (fmt), (arg1), (arg2), (arg3) ); \ } while ( 0 ) #define ldap_debug ((LDAP_INT_GLOBAL_OPT())->ldo_debug) The option is set in, libraries/libldap/options.c int ldap_set_option( LDAP *ld, int option, LDAP_CONST void *invalue) starts with lo = LDAP_INT_GLOBAL_OPT(); but a few lines down, if(ld != NULL) { assert( LDAP_VALID( ld ) ); if( !LDAP_VALID( ld ) ) { return LDAP_OPT_ERROR; } lo = &ld->ld_options; } This code breaks the logic so that the option is not set in the right place. So later in the method, case LDAP_OPT_DEBUG_LEVEL: lo->ldo_debug = * (const int *) invalue; rc = LDAP_OPT_SUCCESS; break; is writing to the wrong place, because lo != LDAP_INT_GLOBAL_OPT() FYI my methods, doing the calling is, void NovaLdap::Connect() { NovaString ldaps = "ldap://"; if (m_SSL) { ldaps = "ldaps://"; } NovaString server = ldaps + m_IpAddress + ":" + ToNovaString(m_IpPort); JOURNAL(SECURITYSERVER,DTL) << "Security Server : Connect " << server << endl; #ifdef _WIN32 m_ld = ldap_sslinit((LDAP_PCHAR) m_IpAddress.data(), m_IpPort, m_SSL); #else CHECK_RESULT(ldap_initialize(&m_ld, server), "ldap_initialize(\2"22 + server + "\")"); #endif JOURNAL(SECURITYSERVER,DTL) << "Security Server : Connected - OK " << endl; if (!m_ld) { JOURNAL(SECURITYSERVER,DTL) << "Security Server : Connect - NULL LD " << endl;D0D throw NovaError(ISSFactory::Error_LDAP_INIT_NULL); } JOURNAL(SECURITYSERVER,DTL) << "Security Server : Connect - set option" << endl; int myVersion =LDAP_VERSION3; CHECK_RESULT(ldap_set_option(m_ld, LDAP_OPT_PROTOCOL_VERSION, &myVersion), "ldap_set_option version"); //CHECK_RESULT(ldap_set_option(m_ld, LDAP_OPT_TLS, &reqcert), "ldap_set_option TLS requires certificate"); #ifdef LDAP_OPT_DEBUG_LEVEL const int optionValue = LDAP_DEBUG_ANY; CHECK_RESULT(ldap_set_option(m_ld, LDAP_OPT_DEBUG_LEVEL, &optionValue), "ldap_set_option debug level"); int optionValueReturned = 0; CHECK_RESULT(ldap_get_option(m_ld, LDAP_OPT_DEBUG_LEVEL, &optionValueReturned), "ldap_set_option debug level"); JOURNAL(SECURITYSERVER,DTL) << "Security Server : Set debug level: " << optionValueReturned << endl; #endif #ifdef LDAP_OPT_CONNECT_ASYNC ldap_set_option( m_ld, LDAP_OPT_CONNECT_ASYNC, LDAP_OPT_OFF ); #endif ldap_set_option(m_ld, LDAP_OPT_REFERRALS, LDAP_OPT_OFF ); ldap_set_option(m_ld, LDAP_OPT_RESTART, LDAP_OPT_ON ); JOURNAL(SECURITYSERVER,DTL) << "Security Server : Connect - OK " << endl; #ifdef _WIN32 CHECK_RESULT(ldap_connect(m_ld, NULL), "ldap_connect"); #endif if (m_StartTLS) { JOURNAL(SECURITYSERVER,DTL) << "Security Server : Connect - Start TLS" << endl; #ifdef _WIN32 CHECK_RESULT(ldap_start_tls_s(m_ld, NULL, NULL, NULL, NULL), "ldap_start_tls_s"); // WINLDAPAPI ULONG LDAPAPI ldap_start_tls_sA ( // IN PLDAP ExternalHandle, // OUT PULONG ServerReturnValue, // OUT LDAPMessage **result, // IN PLDAPControlA *ServerControls, // IN PLDAPControlA *ClientControls // ); #else CHECK_RESULT(ldap_start_tls(m_ld, NULL, NULL, NULL), "ldap_start_tls_s"); #endif } } void NovaLdap::CheckConnection() { JOURNAL(SECURITYSERVER,DTL) << "NovaLdap::CheckConnection: Checking connection" << endl; Connect(); // See if can bind to the DN. if (!m_ServiceAccountUsername.isNull()) { JOURNAL(SECURITYSERVER,DTL) << "Security Server : GetDistinguishedNameForUserName Service service login " << m_ServiceAccountUsername << endl; puts("Security Server : GetDistinguishedNameForUserName Service service login\n"); LDAP_RESULT result = ldap_simple_bind_s(m_ld, (LDAP_PCHAR) m_ServiceAccountUsername.data(), (LDAP_PCHAR) m_ServiceAccountPassword.data()); if (result != (LDAP_RESULT) LDAP_SUCCESS) { NovaString errorMessage(ldap_err2string(result)); throw NovaError(ISSFactory::Error_LDAP_FAILURE, "bind as service user: ", errorMessage); char sevLevel; } ldap_unbind(m_ld); } JOURNAL(SECURITYSERVER,DTL) << "NovaLdap::ececkConnection: Success" << endl; }

1 0

Re: (ITS#8058) OpenLDAP: module autogroup (memberof)
by michael＠stroeder.com 15 Feb '15

15 Feb '15

tobias.helfenstein(a)wald-rlp.de wrote: > Version: 2.4.39 > > we have problems when activating the module autogroup (memberof). There were several fixes after 2.4.39. Could you please try RE24 branch from git repo? Ciao, Michael.

1 0

(ITS#8058) OpenLDAP: module autogroup (memberof)
by tobias.helfenstein＠wald-rlp.de 15 Feb '15

15 Feb '15

Full_Name: Tobias Helfenstein Version: 2.4.39 OS: CentOS 7.0 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (87.165.195.65) Hello, we have problems when activating the module autogroup (memberof). We configured the module like the described way in the README file. If we add a new entry to the LDAP tree, the entry is added to the desired group automatically. This is the expected behavior. We can add entries as much as possible and there is no problem. Now, if we delete one of the entries the slapd is very slow and hangs. The action have to be aborted to continue, there is no debug output. Because of that we have to restart the slapd daemon. But the restart takes a long time (3+ minutes). After the restart we can delete only one entry, then the described problem occurs again. We think it is a bug an hope you can help us. Thank you

1 0

(ITS#8057) slapo-unique can be bypassed by anyone
by ondra＠mistotebe.net 14 Feb '15

14 Feb '15

Full_Name: Ondrej Kuznik Version: master OS: URL: ftp://ftp.openldap.org/pub/Ondrej-Kuznik-20150214-ITS-8057-uniqueness-ACL.p… Submission from: (NULL) (86.177.93.243) This is caused by my fix for #6641. Since anyone can specify the manageDSAit control on an operation it is trivial to bypass the uniqueness check as it stands. The above patch is derived from OpenLDAP Software. All of the modifications to OpenLDAP Software represented in the above patches were developed by Ondrej Kuznik <ondra(a)mistotebe.net>. I have not assigned rights and/or interest in this work to any party. I, Ondrej Kuznik, hereby place the above modifications to OpenLDAP Software (and only these modifications) into the public domain. Hence, these modifications may be freely used and/or redistributed for any purpose with or without attribution and/or other notice.

1 0

Re: (ITS#8056) [PATCH] libdb version is not detected correctly when using gcc5
by jsynacek＠redhat.com 12 Feb '15

12 Feb '15

--=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Howard Chu <hyc(a)symas.com> writes: > Jan Synacek wrote: >> Howard Chu <hyc(a)symas.com> writes: >>> jsynacek(a)redhat.com wrote: >>>> Full_Name: Jan Synacek >>>> Version: 2.4.40 >>>> OS: GNU/Linux >>>> URL: https://jsynacek.fedorapeople.org/openldap/jsynacek-20150211-fix-= libdb-detection-with-gcc-5.patch >>>> Submission from: (NULL) (94.112.188.81) >>>> >>>> >>>> The old cpp version generated: >>>> __db_version 5 >>>> >>>> Configure then correctly picked up the version, since it greps for >>>> "__db_version". >>>> >>>> The new output: >>>> __db_version >>>> 5 >>>> >>>> Grepping no longer works and configure fails. >>>> >>>> Running cpp with -P (inhibit linemarkers generation) fixes this when u= sing gcc >>>> 5. >>>> Behavior with older versions of gcc is not changed. >>> >>> Your patch is simple enough but frankly this sounds like a bug in gcc >>> 5's cpp. It shouldn't be inserting gratuitous whitespace in the middle >>> of lines, or inserting line breaks arbitrarily. >> >> Yes, probably. But in the meantime, I needed the package to build, so I >> wrote this patch. > > Please file a bug with the gcc project and followup here with a link to=20 > that bug, for future reference. Thanks. Filed as https://gcc.gnu.org/bugzilla/show_bug.cgi?id=3D65037. > Adding this patch to git master. Thank you, =2D-=20 Jan Synacek Software Engineer, Red Hat --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJU3GWXAAoJEL3BmMJQOtjB6HQQAKF6PH2e6OACvDZCAmCRj4qG OT/izonRts5+ZEN645TB6oPjTN1hU43/Z/+W7eZQBQoTfQrngTzBwfAC3deFfFXU vhwf+oZR3vTsPjGev5EwM+vl7OCXN5AYhU1JNkeXM65cVQsZHWHLTIjkVImodXSs Gl258ydoURg29F1DNNfs4IA4ImpTuVZ9SZBLg7XCyJBN7Refgrbie7xpevkUh/5o ctvN4AQpUese60GE+OKiYhnyViG/pzxamMdKyOtbf5AwN+4u0HdndUqPU6iJHEQ6 66D8iJsrQgBH0dl8ggX8uQKAoq+8Pkf3hs339NUrEOAk6O91lixSWTV4UQsAJK9q 0aGMDc1f0JcNDFq1ycCcHnnJ+TbEJRi/s/RmD1Ku87vGXpYTzXUbYZE8S5pRlkLl Y5Nz0vfp76j9GvfFwrsKtvYBQAStWmV/HVeW5DbVXzRWE71StVeGfHOLRxViPoBp 3u2jaArPUMpA+ZgYfBZ7HMMvchBhgTW+f/nnZCudCVukWdd/VIK/JSe/r3q5SWv5 9QVnUy9er0t4yMsH+waSRCgRZ0BR14e7YNzPfzaqEIeFb/k5bbVtdAEhkjIREMHT 8oSF2/K+tCORx0lqpxoXZfjVEGAQUkobw/a9ABNnoUYeuy+/9nIkp970PK48AsCC wEULvjRGsq9wvJorGDJI =k/xN -----END PGP SIGNATURE----- --=-=-=--

1 0

Re: (ITS#8056) [PATCH] libdb version is not detected correctly when using gcc5
by h.b.furuseth＠usit.uio.no 11 Feb '15

11 Feb '15

On 11/02/15 14:25, hyc(a)symas.com wrote: > Your patch is simple enough but frankly this sounds like a bug in gcc > 5's cpp. It shouldn't be inserting gratuitous whitespace in the middle > of lines, or inserting line breaks arbitrarily. Not a gcc bug, the ITS report proved incomplete. gcc-5 inserts "# <file information>" lines (indicating where the macro expansion comes from, I think), so it needs the line breaks. This slightly different test program: #include <db.h> __db_version DB_VERSION_MAJOR+0000; ...produces this: # 2 "test.c" __db_version # 2 "test.c" 3 4 4 # 2 "test.c" +0000; -- Hallvard

1 0

Re: (ITS#8056) [PATCH] libdb version is not detected correctly when using gcc5
by hyc＠symas.com 11 Feb '15

11 Feb '15

Jan Synacek wrote: > Howard Chu <hyc(a)symas.com> writes: >> jsynacek(a)redhat.com wrote: >>> Full_Name: Jan Synacek >>> Version: 2.4.40 >>> OS: GNU/Linux >>> URL: https://jsynacek.fedorapeople.org/openldap/jsynacek-20150211-fix-libdb-dete… >>> Submission from: (NULL) (94.112.188.81) >>> >>> >>> The old cpp version generated: >>> __db_version 5 >>> >>> Configure then correctly picked up the version, since it greps for >>> "__db_version". >>> >>> The new output: >>> __db_version >>> 5 >>> >>> Grepping no longer works and configure fails. >>> >>> Running cpp with -P (inhibit linemarkers generation) fixes this when using gcc >>> 5. >>> Behavior with older versions of gcc is not changed. >> >> Your patch is simple enough but frankly this sounds like a bug in gcc >> 5's cpp. It shouldn't be inserting gratuitous whitespace in the middle >> of lines, or inserting line breaks arbitrarily. > > Yes, probably. But in the meantime, I needed the package to build, so I > wrote this patch. Please file a bug with the gcc project and followup here with a link to that bug, for future reference. Thanks. Adding this patch to git master. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8054) Operation duration logging
by h.b.furuseth＠usit.uio.no 11 Feb '15

11 Feb '15

On 11. feb. 2015 12:38, Michael Ströder wrote: > h.b.furuseth(a)usit.uio.no wrote: >> Cool feture, but please use fewer characteres in the already >> bloated log. How about "time=<integer>us" or even omit the "us"? > > +1 > > But 'time' sounds a bit general. > How about 'tdu' or even 'td' as abbreviation for "time duration"? I don't think so. You can't see from that what it means unless you already know, so you have to look it up just to see if you're interested. 'time' you can decide to look up if you are interested. -- Hallvard

1 0

Re: (ITS#8056) [PATCH] libdb version is not detected correctly when using gcc5
by jsynacek＠redhat.com 11 Feb '15

11 Feb '15

--=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Howard Chu <hyc(a)symas.com> writes: > jsynacek(a)redhat.com wrote: >> Full_Name: Jan Synacek >> Version: 2.4.40 >> OS: GNU/Linux >> URL: https://jsynacek.fedorapeople.org/openldap/jsynacek-20150211-fix-li= bdb-detection-with-gcc-5.patch >> Submission from: (NULL) (94.112.188.81) >> >> >> The old cpp version generated: >> __db_version 5 >> >> Configure then correctly picked up the version, since it greps for >> "__db_version". >> >> The new output: >> __db_version >> 5 >> >> Grepping no longer works and configure fails. >> >> Running cpp with -P (inhibit linemarkers generation) fixes this when usi= ng gcc >> 5. >> Behavior with older versions of gcc is not changed. > > Your patch is simple enough but frankly this sounds like a bug in gcc=20 > 5's cpp. It shouldn't be inserting gratuitous whitespace in the middle=20 > of lines, or inserting line breaks arbitrarily. Yes, probably. But in the meantime, I needed the package to build, so I wrote this patch. Cheers, =2D-=20 Jan Synacek Software Engineer, Red Hat --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJU22YfAAoJEL3BmMJQOtjBgz0P/0C6vOFzNeDmB/6gJApRB97j qQ+eAfs+VVi1nzanutQUZWIHM8F1wFFUm1ULp76z8dkxrquI9VSpTzWSj2kMPMqY iGZfK6fRM84Weog2iCR9LoVqXrNCkih1pOi3SBaQE8pLxZh0iLCg5ZEW3QEgoVam UMQz0fU5fFKtvKB85Xcy02drn64dTSn5zyF59H0CgVCTC3GBYP/tR2qp//ZPb2jj WsOMdSzx9ENf56Y/r6fmMRpKN3+c8jLHf7/zbStDIgVnCv6Haf8bLevgSYYejkus Av5U88rM9proraf6DvjwpJFfk6OCzvVo0eOcE63orw+uHyYQz/vWCLaK4/Wwo1e/ 84MekmHm1G9pQMpnrgOkUqQv/MYVQpeIgrL29DZC4y4nDa68iL7+WdNqova3s5ST 6onQTdIR8YBwVRXAfnV00SmJAEslzouhEH5MF7mcLCI+/06EnaKzqtMuTsQZvC1I 9oucRZRgdnax3jTyzDwyvxvei5/yvIESvj3NOx2vq35NxE2vqSpRqIGVucDM4cnN HVzk7fd6nFGhG9ZKWgv+2+kz6lZlH+3duIR0nJydgUJJPlRS8NxysNw439Q+Wu2D ql+Vfl9BglSrCwiE6+QrVmE9W5PLj1/I5AgE6Wrdl6zrAxYI2fj7LgiavM8tmWIp RvUpCjFNkI6AiD2KmmpV =bSRG -----END PGP SIGNATURE----- --=-=-=--

1 0

Re: (ITS#8056) [PATCH] libdb version is not detected correctly when using gcc5
by hyc＠symas.com 11 Feb '15

11 Feb '15

jsynacek(a)redhat.com wrote: > Full_Name: Jan Synacek > Version: 2.4.40 > OS: GNU/Linux > URL: https://jsynacek.fedorapeople.org/openldap/jsynacek-20150211-fix-libdb-dete… > Submission from: (NULL) (94.112.188.81) > > > The old cpp version generated: > __db_version 5 > > Configure then correctly picked up the version, since it greps for > "__db_version". > > The new output: > __db_version > 5 > > Grepping no longer works and configure fails. > > Running cpp with -P (inhibit linemarkers generation) fixes this when using gcc > 5. > Behavior with older versions of gcc is not changed. Your patch is simple enough but frankly this sounds like a bug in gcc 5's cpp. It shouldn't be inserting gratuitous whitespace in the middle of lines, or inserting line breaks arbitrarily. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8054) Operation duration logging
by michael＠stroeder.com 11 Feb '15

11 Feb '15

h.b.furuseth(a)usit.uio.no wrote: > Cool feture, but please use fewer characteres in the already > bloated log. How about "time=<integer>us" or even omit the "us"? +1 But 'time' sounds a bit general. How about 'tdu' or even 'td' as abbreviation for "time duration"? Ciao, Michael.

1 0

Re: (ITS#8054) Operation duration logging
by h.b.furuseth＠usit.uio.no 11 Feb '15

11 Feb '15

Cool feture, but please use fewer characteres in the already bloated log. How about "time=<integer>us" or even omit the "us"? When duration is of interest, what's usually interesting is probably which operations stand out as slower, not the actual speed. When duration is not of interest, then it's noise to keep small and boring. And we should document at least the default log format (Statslog). == About the code, ignoring the above suggestion for now: Once upon a time, floating-point was slow. Today, that extra code to avoid it is a pessimization at least on my Linux/x86. Try this: #define TIMEDIFF_MS(a,b) \ (((a)->tv_sec-(b)->tv_sec)*1000 + ((a)->tv_usec-(b)->tv_usec)/1000.0) OTOH floating-point printf is compex, you save a bit by doing msec = (int)t; printf("%d.%03d", msec, (int)((t-msec)*1000)); instead of printf("%4.3f", t); %d format requires 'int msec;' and (int)(duration.tv_usec % 1000). Since timersub() is use the same way each time, a macro could instead compute duration and msec, and maybe declare them too: DURATION_DECLARE(duration, msec, op->o_hr_time); For that matter, maybe we should get rid of the #ifdef-duplicated Statslog() statements and put the ugliness in macros instead. Like this, I think - an untested draft: DURATION_DECLARE(duration, msec, op->o_hr_time); StatslogDuration( LDAP_DEBUG_STATS, "%s DISCONNECT tag=%lu err=%d "DURATION_LOG_FMT" text=%s\n", op->o_log_prefix, rs->sr_tag, rs->sr_err, DURATION_LOG_ARG(duration, msec) /* no comma after this */ rs->sr_text ? rs->sr_text : "", 0 ); #ifdef HAVE_GETTIMEOFDAY #define DURATION_DECLARE(..blah..) <the duration code before Statslog> #define DURATION_LOG_FMT "duration=%d.%03dms " /* This is passed to StatslogDuration() without a comma before next arg. * So it is not an extra arg to StatslogDuration(), but it gives two * extra 'int' args to whatever StatslogDuration() passes it args on to. */ #define DURATION_LOG_ARG(duration,msec) (msec), (int)((duration).tv_usec%1000), #define StatslogDuration <like Statslog, but using Log7 instead of Log5> #else #define DURATION_DECLARE(..blah..) #define DURATION_LOG_FMT "" #define DURATION_LOG_ARG(duration, msec) #define StatslogDuration Statslog #endif

1 0

(ITS#8056) [PATCH] libdb version is not detected correctly when using gcc5
by jsynacek＠redhat.com 11 Feb '15

11 Feb '15

Full_Name: Jan Synacek Version: 2.4.40 OS: GNU/Linux URL: https://jsynacek.fedorapeople.org/openldap/jsynacek-20150211-fix-libdb-dete… Submission from: (NULL) (94.112.188.81) The old cpp version generated: __db_version 5 Configure then correctly picked up the version, since it greps for "__db_version". The new output: __db_version 5 Grepping no longer works and configure fails. Running cpp with -P (inhibit linemarkers generation) fixes this when using gcc 5. Behavior with older versions of gcc is not changed.

1 0

(ITS#8055) High CPU usage with syncreply and wrong port
by adam＠cassar.id.au 09 Feb '15

09 Feb '15

Full_Name: Adam Version: 2.4.40 OS: Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (115.70.231.55) Specifying a syncrepl provider with an unreachable port results in high CPU usage with slapd with slapd never terminating. Example configuration: syncrepl rid=1 provider=ldap://10.1.1.10:9292 type=refreshAndPersist interval=00:00:01:00 retry="60 +" searchbase="dc=howard,dc=chew" scope=sub schemachecking=on bindmethod=simple binddn="cn=admin,dc=howard,dc=chew" credentials=iwritecode

1 0

Re: (ITS#8054) Operation duration logging
by michael＠stroeder.com 09 Feb '15

09 Feb '15

hyc(a)symas.com wrote: > michael(a)stroeder.com wrote: >> ebackes(a)symas.com wrote: >>> Here is a patch to add duration logging for operations that appears in the >>> standard result messages of statslog. It's a reasonably small change with >>> negligible performance implications except under pathological benchmarking >>> conditions. It does change the quasi-machine-parsable statslog format, however, >>> so it might not be appropriate for general 2.4 release. Example line: >>> >>> 54d8298c conn=1000 op=1 SEARCH RESULT tag=101 err=0 duration=0.170ms nentries=1 >>> text= >> >> This is a very important feature. Highly appreciated! >> >> I'd love to see this in RE24. Yes, I know. No more features. :-( >> >> Would it be possible and make sense to have this configurable per database? > > No it would not make sense. There are ops like Abandon and Unbind which > have no associated database. Aren't Abandon and Unbind associated by authz-DN? Or how is this solved in slapo-accesslog? Ciao, Michael.

1 0

Re: (ITS#8054) Operation duration logging
by hyc＠symas.com 09 Feb '15

09 Feb '15

michael(a)stroeder.com wrote: > ebackes(a)symas.com wrote: >> Here is a patch to add duration logging for operations that appears in the >> standard result messages of statslog. It's a reasonably small change with >> negligible performance implications except under pathological benchmarking >> conditions. It does change the quasi-machine-parsable statslog format, however, >> so it might not be appropriate for general 2.4 release. Example line: >> >> 54d8298c conn=1000 op=1 SEARCH RESULT tag=101 err=0 duration=0.170ms nentries=1 >> text= > > This is a very important feature. Highly appreciated! > > I'd love to see this in RE24. Yes, I know. No more features. :-( > > Would it be possible and make sense to have this configurable per database? No it would not make sense. There are ops like Abandon and Unbind which have no associated database. Adding variation here is creating gratuitous complexity for no benefit. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8054) Operation duration logging
by michael＠stroeder.com 09 Feb '15

09 Feb '15

ebackes(a)symas.com wrote: > Here is a patch to add duration logging for operations that appears in the > standard result messages of statslog. It's a reasonably small change with > negligible performance implications except under pathological benchmarking > conditions. It does change the quasi-machine-parsable statslog format, however, > so it might not be appropriate for general 2.4 release. Example line: > > 54d8298c conn=1000 op=1 SEARCH RESULT tag=101 err=0 duration=0.170ms nentries=1 > text= This is a very important feature. Highly appreciated! I'd love to see this in RE24. Yes, I know. No more features. :-( Would it be possible and make sense to have this configurable per database? Ciao, Michael.

1 0

Re: (ITS#8054) Operation duration logging
by ebackes＠symas.com 08 Feb '15

08 Feb '15

This is a multi-part message in MIME format. --Multipart=_Sun__8_Feb_2015_19_56_17_-0800_yxgp4/9WqjBC20PY Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Here are the patches. -- Emily Backes Symas Corporation ebackes(a)symas.com --Multipart=_Sun__8_Feb_2015_19_56_17_-0800_yxgp4/9WqjBC20PY Content-Type: text/x-diff; name="master-operation-duration-logging.patch" Content-Disposition: attachment; filename="master-operation-duration-logging.patch" Content-Transfer-Encoding: 7bit >From 7582a21a4bc06271519e37cef11dcfb4348a34a2 Mon Sep 17 00:00:00 2001 From: Emily Backes <ebackes(a)symas.com> Date: Thu, 5 Feb 2015 18:52:19 -0800 Subject: [PATCH] operation duration logging --- include/ldap_log.h | 6 +++ servers/slapd/operation.c | 3 ++ servers/slapd/result.c | 98 +++++++++++++++++++++++++++++++++++++++++++++++ servers/slapd/slap.h | 19 +++++++++ 4 files changed, 126 insertions(+) diff --git a/include/ldap_log.h b/include/ldap_log.h index 9036060..289db00 100644 --- a/include/ldap_log.h +++ b/include/ldap_log.h @@ -192,6 +192,12 @@ extern void eb_syslog(int pri, const char *fmt, ...); #define Log5(level, severity, fmt, a1, a2, a3, a4, a5) \ LogExpand((level), (severity), (fmt) LogArg(a1) LogArg(a2) LogArg(a3) \ LogArg(a4) LogArg(a5)) +#define Log6(level, severity, fmt, a1, a2, a3, a4, a5, a6) \ + LogExpand((level), (severity), (fmt) LogArg(a1) LogArg(a2) LogArg(a3) \ + LogArg(a4) LogArg(a5) LogArg(a6)) +#define Log7(level, severity, fmt, a1, a2, a3, a4, a5, a6, a7) \ + LogExpand((level), (severity), (fmt) LogArg(a1) LogArg(a2) LogArg(a3) \ + LogArg(a4) LogArg(a5) LogArg(a6) LogArg(a7)) #define Debug(level, fmt, a1, a2, a3) \ LogExpand((level), ldap_syslog_level, (fmt) \ LogArg(a1) LogArg(a2) LogArg(a3)) diff --git a/servers/slapd/operation.c b/servers/slapd/operation.c index d16413f..1450718 100644 --- a/servers/slapd/operation.c +++ b/servers/slapd/operation.c @@ -204,6 +204,9 @@ slap_op_alloc( op->o_tag = tag; slap_op_time( &op->o_time, &op->o_tincr ); +#ifdef HAVE_GETTIMEOFDAY + (void) gettimeofday( &op->o_hr_time, NULL ); +#endif /* HAVE_GETTIMEOFDAY */ op->o_opid = id; #if defined( LDAP_SLAPI ) diff --git a/servers/slapd/result.c b/servers/slapd/result.c index 14bd6b7..20fc745 100644 --- a/servers/slapd/result.c +++ b/servers/slapd/result.c @@ -36,6 +36,19 @@ #include <ac/unistd.h> #include "slap.h" +#include "ldap_log.h" + +/* BSD seems to guarantee that this will be a macro, so... */ +#ifndef timersub +# define timersub(a,b,res) do { \ + (res)->tv_sec = (a)->tv_sec - (b)->tv_sec; \ + (res)->tv_usec = (a)->tv_usec - (b)->tv_usec; \ + if ((res)->tv_usec < 0) { \ + --(res)->tv_sec; \ + (res)->tv_usec += 1000000; \ + } \ + } while (0) +#endif const struct berval slap_dummy_bv = BER_BVNULL; @@ -819,10 +832,27 @@ send_ldap_disconnect( Operation *op, SlapReply *rs ) } if ( send_ldap_response( op, rs ) == SLAP_CB_CONTINUE ) { +#ifdef HAVE_GETTIMEOFDAY + struct timeval now; + struct timeval duration; + time_t msec; + + (void) gettimeofday( &now, NULL ); + timersub( &now, &op->o_hr_time, &duration ); + msec = duration.tv_usec / 1000 + duration.tv_sec * 1000; + + Statslog6( LDAP_DEBUG_STATS, + "%s DISCONNECT tag=%lu err=%d duration=%d.%03dms text=%s\n", + op->o_log_prefix, rs->sr_tag, rs->sr_err, + msec, + duration.tv_usec % 1000, + rs->sr_text ? rs->sr_text : "" ); +#else /* ! HAVE_GETTIMEOFDAY */ Statslog( LDAP_DEBUG_STATS, "%s DISCONNECT tag=%lu err=%d text=%s\n", op->o_log_prefix, rs->sr_tag, rs->sr_err, rs->sr_text ? rs->sr_text : "", 0 ); +#endif /* ! HAVE_GETTIMEOFDAY */ } } @@ -884,6 +914,37 @@ abandon: } if ( send_ldap_response( op, rs ) == SLAP_CB_CONTINUE ) { + +#ifdef HAVE_GETTIMEOFDAY + struct timeval now; + struct timeval duration; + time_t msec; + + (void) gettimeofday( &now, NULL ); + timersub( &now, &op->o_hr_time, &duration ); + msec = duration.tv_usec / 1000 + duration.tv_sec * 1000; + + if ( op->o_tag == LDAP_REQ_SEARCH ) { + Statslog7( LDAP_DEBUG_STATS, + "%s SEARCH RESULT tag=%lu err=%d duration=%d.%03dms nentries=%d text=%s\n", + op->o_log_prefix, + rs->sr_tag, + rs->sr_err, + msec, + duration.tv_usec % 1000, + rs->sr_nentries, + rs->sr_text ? rs->sr_text : ""); + } else { + Statslog6( LDAP_DEBUG_STATS, + "%s RESULT tag=%lu err=%d duration=%d.%03dms text=%s\n", + op->o_log_prefix, + rs->sr_tag, + rs->sr_err, + msec, + duration.tv_usec % 1000, + rs->sr_text ? rs->sr_text : ""); + } +#else /* ! HAVE_GETTIMEOFDAY */ if ( op->o_tag == LDAP_REQ_SEARCH ) { Statslog( LDAP_DEBUG_STATS, "%s SEARCH RESULT tag=%lu err=%d nentries=%d text=%s\n", @@ -895,6 +956,7 @@ abandon: op->o_log_prefix, rs->sr_tag, rs->sr_err, rs->sr_text ? rs->sr_text : "", 0 ); } +#endif /* ! HAVE_GETTIMEOFDAY */ } if( tmp != NULL ) ch_free(tmp); @@ -917,10 +979,27 @@ send_ldap_sasl( Operation *op, SlapReply *rs ) rs->sr_msgid = (rs->sr_tag != LBER_SEQUENCE) ? op->o_msgid : 0; if ( send_ldap_response( op, rs ) == SLAP_CB_CONTINUE ) { +#ifdef HAVE_GETTIMEOFDAY + struct timeval now; + struct timeval duration; + time_t msec; + + (void) gettimeofday( &now, NULL ); + timersub( &now, &op->o_hr_time, &duration ); + msec = duration.tv_usec / 1000 + duration.tv_sec * 1000; + + Statslog6( LDAP_DEBUG_STATS, + "%s RESULT tag=%lu err=%d duration=%d.%03dms text=%s\n", + op->o_log_prefix, rs->sr_tag, rs->sr_err, + msec, + duration.tv_usec % 1000, + rs->sr_text ? rs->sr_text : "" ); +#else /* ! HAVE_GETTIMEOFDAY */ Statslog( LDAP_DEBUG_STATS, "%s RESULT tag=%lu err=%d text=%s\n", op->o_log_prefix, rs->sr_tag, rs->sr_err, rs->sr_text ? rs->sr_text : "", 0 ); +#endif /* ! HAVE_GETTIMEOFDAY */ } } @@ -941,10 +1020,29 @@ slap_send_ldap_extended( Operation *op, SlapReply *rs ) rs->sr_msgid = (rs->sr_tag != LBER_SEQUENCE) ? op->o_msgid : 0; if ( send_ldap_response( op, rs ) == SLAP_CB_CONTINUE ) { +#ifdef HAVE_GETTIMEOFDAY + struct timeval now; + struct timeval duration; + time_t msec; + + (void) gettimeofday( &now, NULL ); + timersub( &now, &op->o_hr_time, &duration ); + msec = duration.tv_usec / 1000 + duration.tv_sec * 1000; + + Statslog6( LDAP_DEBUG_STATS, + "%s RESULT oid=%s err=%d duration=%d.%03dms text=%s\n", + op->o_log_prefix, + rs->sr_rspoid ? rs->sr_rspoid : "", + rs->sr_err, + msec, + duration.tv_usec % 1000, + rs->sr_text ? rs->sr_text : "" ); +#else /* ! HAVE_GETTIMEOFDAY */ Statslog( LDAP_DEBUG_STATS, "%s RESULT oid=%s err=%d text=%s\n", op->o_log_prefix, rs->sr_rspoid ? rs->sr_rspoid : "", rs->sr_err, rs->sr_text ? rs->sr_text : "", 0 ); +#endif /* ! HAVE_GETTIMEOFDAY */ } } diff --git a/servers/slapd/slap.h b/servers/slapd/slap.h index f89da27..17d09d9 100644 --- a/servers/slapd/slap.h +++ b/servers/slapd/slap.h @@ -2618,6 +2618,9 @@ struct Operation { #define o_log_prefix o_hdr->oh_log_prefix ber_tag_t o_tag; /* tag of the request */ +#ifdef HAVE_GETTIMEOFDAY + struct timeval o_hr_time; /* high-resolution time of op start*/ +#endif /* HAVE_GETTIMEOFDAY */ time_t o_time; /* time op was initiated */ int o_tincr; /* counter for multiple ops with same o_time */ @@ -2990,6 +2993,10 @@ struct Connection { #define Statslog( level, fmt, connid, opid, arg1, arg2, arg3 ) \ Log5( (level), ldap_syslog_level, (fmt), (connid), (opid), (arg1), (arg2), (arg3) ) +#define Statslog6( level, fmt, a1, a2, a3, a4, a5, a6 ) \ + Log6( (level), ldap_syslog_level, (fmt), (a1), (a2), (a3), (a4), (a5), (a6) ) +#define Statslog7( level, fmt, a1, a2, a3, a4, a5, a6, a7 ) \ + Log7( (level), ldap_syslog_level, (fmt), (a1), (a2), (a3), (a4), (a5), (a6), (a7) ) #define StatslogTest( level ) ((ldap_debug | ldap_syslog) & (level)) #else /* !LDAP_SYSLOG */ #define Statslog( level, fmt, connid, opid, arg1, arg2, arg3 ) \ @@ -2997,10 +3004,22 @@ struct Connection { if ( ldap_debug & (level) ) \ lutil_debug( ldap_debug, (level), (fmt), (connid), (opid), (arg1), (arg2), (arg3) );\ } while (0) +#define Statslog6( level, fmt, a1, a2, a3, a4, a5, a6 ) \ + do { \ + if ( ldap_debug & (level) ) \ + lutil_debug( ldap_debug, (level), (fmt), (a1), (a2), (a3), (a4), (a5), (a6) ); \ + } while (0) +#define Statslog7( level, fmt, a1, a2, a3, a4, a5, a6, a7 ) \ + do { \ + if ( ldap_debug & (level) ) \ + lutil_debug( ldap_debug, (level), (fmt), (a1), (a2), (a3), (a4), (a5), (a6), (a7) ); \ + } while (0) #define StatslogTest( level ) (ldap_debug & (level)) #endif /* !LDAP_SYSLOG */ #else /* !LDAP_DEBUG */ #define Statslog( level, fmt, connid, opid, arg1, arg2, arg3 ) ((void) 0) +#define Statslog6( level, fmt, a1, a2, a3, a4, a5, a6 ) ((void) 0) +#define Statslog7( level, fmt, a1, a2, a3, a4, a5, a6, a7 ) ((void) 0) #define StatslogTest( level ) (0) #endif /* !LDAP_DEBUG */ -- 2.1.4 --Multipart=_Sun__8_Feb_2015_19_56_17_-0800_yxgp4/9WqjBC20PY Content-Type: text/x-diff; name="RE24-operation-duration-logging.patch" Content-Disposition: attachment; filename="RE24-operation-duration-logging.patch" Content-Transfer-Encoding: 7bit >From bfabab23da6d0914f222a22f1cd0e28c48b4db90 Mon Sep 17 00:00:00 2001 From: Emily Backes <ebackes(a)symas.com> Date: Thu, 5 Feb 2015 18:52:19 -0800 Subject: [PATCH] operation duration logging --- include/ldap_log.h | 14 +++++++ servers/slapd/operation.c | 3 ++ servers/slapd/result.c | 98 +++++++++++++++++++++++++++++++++++++++++++++++ servers/slapd/slap.h | 19 +++++++++ 4 files changed, 134 insertions(+) diff --git a/include/ldap_log.h b/include/ldap_log.h index 94a73bd..50c7f4b 100644 --- a/include/ldap_log.h +++ b/include/ldap_log.h @@ -190,6 +190,20 @@ extern void eb_syslog(int pri, const char *fmt, ...); if ( ldap_syslog & (level) ) \ syslog( LDAP_LEVEL_MASK((severity)), (fmt), (arg1), (arg2), (arg3), (arg4), (arg5) ); \ } while ( 0 ) +#define Log6( level, severity, fmt, arg1, arg2, arg3, arg4, arg5, arg6 ) \ + do { \ + if ( ldap_debug & (level) ) \ + lutil_debug( ldap_debug, (level), (fmt), (arg1), (arg2), (arg3), (arg4), (arg5), (arg6) ); \ + if ( ldap_syslog & (level) ) \ + syslog( LDAP_LEVEL_MASK((severity)), (fmt), (arg1), (arg2), (arg3), (arg4), (arg5), (arg6) ); \ + } while ( 0 ) +#define Log7( level, severity, fmt, arg1, arg2, arg3, arg4, arg5, arg6, arg7 ) \ + do { \ + if ( ldap_debug & (level) ) \ + lutil_debug( ldap_debug, (level), (fmt), (arg1), (arg2), (arg3), (arg4), (arg5), (arg6), (arg7) ); \ + if ( ldap_syslog & (level) ) \ + syslog( LDAP_LEVEL_MASK((severity)), (fmt), (arg1), (arg2), (arg3), (arg4), (arg5), (arg6), (arg7) ); \ + } while ( 0 ) #define Debug( level, fmt, arg1, arg2, arg3 ) \ Log3( (level), ldap_syslog_level, (fmt), (arg1), (arg2), (arg3) ) #define LogTest(level) ( ( ldap_debug | ldap_syslog ) & (level) ) diff --git a/servers/slapd/operation.c b/servers/slapd/operation.c index cc09a22..1c712e0 100644 --- a/servers/slapd/operation.c +++ b/servers/slapd/operation.c @@ -204,6 +204,9 @@ slap_op_alloc( op->o_tag = tag; slap_op_time( &op->o_time, &op->o_tincr ); +#ifdef HAVE_GETTIMEOFDAY + (void) gettimeofday( &op->o_hr_time, NULL ); +#endif /* HAVE_GETTIMEOFDAY */ op->o_opid = id; #if defined( LDAP_SLAPI ) diff --git a/servers/slapd/result.c b/servers/slapd/result.c index 35cddc7..2bd910e 100644 --- a/servers/slapd/result.c +++ b/servers/slapd/result.c @@ -36,6 +36,19 @@ #include <ac/unistd.h> #include "slap.h" +#include "ldap_log.h" + +/* BSD seems to guarantee that this will be a macro, so... */ +#ifndef timersub +# define timersub(a,b,res) do { \ + (res)->tv_sec = (a)->tv_sec - (b)->tv_sec; \ + (res)->tv_usec = (a)->tv_usec - (b)->tv_usec; \ + if ((res)->tv_usec < 0) { \ + --(res)->tv_sec; \ + (res)->tv_usec += 1000000; \ + } \ + } while (0) +#endif const struct berval slap_dummy_bv = BER_BVNULL; @@ -794,10 +807,27 @@ send_ldap_disconnect( Operation *op, SlapReply *rs ) } if ( send_ldap_response( op, rs ) == SLAP_CB_CONTINUE ) { +#ifdef HAVE_GETTIMEOFDAY + struct timeval now; + struct timeval duration; + time_t msec; + + (void) gettimeofday( &now, NULL ); + timersub( &now, &op->o_hr_time, &duration ); + msec = duration.tv_usec / 1000 + duration.tv_sec * 1000; + + Statslog6( LDAP_DEBUG_STATS, + "%s DISCONNECT tag=%lu err=%d duration=%d.%03dms text=%s\n", + op->o_log_prefix, rs->sr_tag, rs->sr_err, + msec, + duration.tv_usec % 1000, + rs->sr_text ? rs->sr_text : "" ); +#else /* ! HAVE_GETTIMEOFDAY */ Statslog( LDAP_DEBUG_STATS, "%s DISCONNECT tag=%lu err=%d text=%s\n", op->o_log_prefix, rs->sr_tag, rs->sr_err, rs->sr_text ? rs->sr_text : "", 0 ); +#endif /* ! HAVE_GETTIMEOFDAY */ } } @@ -859,6 +889,37 @@ abandon: } if ( send_ldap_response( op, rs ) == SLAP_CB_CONTINUE ) { + +#ifdef HAVE_GETTIMEOFDAY + struct timeval now; + struct timeval duration; + time_t msec; + + (void) gettimeofday( &now, NULL ); + timersub( &now, &op->o_hr_time, &duration ); + msec = duration.tv_usec / 1000 + duration.tv_sec * 1000; + + if ( op->o_tag == LDAP_REQ_SEARCH ) { + Statslog7( LDAP_DEBUG_STATS, + "%s SEARCH RESULT tag=%lu err=%d duration=%d.%03dms nentries=%d text=%s\n", + op->o_log_prefix, + rs->sr_tag, + rs->sr_err, + msec, + duration.tv_usec % 1000, + rs->sr_nentries, + rs->sr_text ? rs->sr_text : ""); + } else { + Statslog6( LDAP_DEBUG_STATS, + "%s RESULT tag=%lu err=%d duration=%d.%03dms text=%s\n", + op->o_log_prefix, + rs->sr_tag, + rs->sr_err, + msec, + duration.tv_usec % 1000, + rs->sr_text ? rs->sr_text : ""); + } +#else /* ! HAVE_GETTIMEOFDAY */ if ( op->o_tag == LDAP_REQ_SEARCH ) { Statslog( LDAP_DEBUG_STATS, "%s SEARCH RESULT tag=%lu err=%d nentries=%d text=%s\n", @@ -870,6 +931,7 @@ abandon: op->o_log_prefix, rs->sr_tag, rs->sr_err, rs->sr_text ? rs->sr_text : "", 0 ); } +#endif /* ! HAVE_GETTIMEOFDAY */ } if( tmp != NULL ) ch_free(tmp); @@ -892,10 +954,27 @@ send_ldap_sasl( Operation *op, SlapReply *rs ) rs->sr_msgid = (rs->sr_tag != LBER_SEQUENCE) ? op->o_msgid : 0; if ( send_ldap_response( op, rs ) == SLAP_CB_CONTINUE ) { +#ifdef HAVE_GETTIMEOFDAY + struct timeval now; + struct timeval duration; + time_t msec; + + (void) gettimeofday( &now, NULL ); + timersub( &now, &op->o_hr_time, &duration ); + msec = duration.tv_usec / 1000 + duration.tv_sec * 1000; + + Statslog6( LDAP_DEBUG_STATS, + "%s RESULT tag=%lu err=%d duration=%d.%03dms text=%s\n", + op->o_log_prefix, rs->sr_tag, rs->sr_err, + msec, + duration.tv_usec % 1000, + rs->sr_text ? rs->sr_text : "" ); +#else /* ! HAVE_GETTIMEOFDAY */ Statslog( LDAP_DEBUG_STATS, "%s RESULT tag=%lu err=%d text=%s\n", op->o_log_prefix, rs->sr_tag, rs->sr_err, rs->sr_text ? rs->sr_text : "", 0 ); +#endif /* ! HAVE_GETTIMEOFDAY */ } } @@ -916,10 +995,29 @@ slap_send_ldap_extended( Operation *op, SlapReply *rs ) rs->sr_msgid = (rs->sr_tag != LBER_SEQUENCE) ? op->o_msgid : 0; if ( send_ldap_response( op, rs ) == SLAP_CB_CONTINUE ) { +#ifdef HAVE_GETTIMEOFDAY + struct timeval now; + struct timeval duration; + time_t msec; + + (void) gettimeofday( &now, NULL ); + timersub( &now, &op->o_hr_time, &duration ); + msec = duration.tv_usec / 1000 + duration.tv_sec * 1000; + + Statslog6( LDAP_DEBUG_STATS, + "%s RESULT oid=%s err=%d duration=%d.%03dms text=%s\n", + op->o_log_prefix, + rs->sr_rspoid ? rs->sr_rspoid : "", + rs->sr_err, + msec, + duration.tv_usec % 1000, + rs->sr_text ? rs->sr_text : "" ); +#else /* ! HAVE_GETTIMEOFDAY */ Statslog( LDAP_DEBUG_STATS, "%s RESULT oid=%s err=%d text=%s\n", op->o_log_prefix, rs->sr_rspoid ? rs->sr_rspoid : "", rs->sr_err, rs->sr_text ? rs->sr_text : "", 0 ); +#endif /* ! HAVE_GETTIMEOFDAY */ } } diff --git a/servers/slapd/slap.h b/servers/slapd/slap.h index 60db371..d22f1bf 100644 --- a/servers/slapd/slap.h +++ b/servers/slapd/slap.h @@ -2598,6 +2598,9 @@ struct Operation { #define o_log_prefix o_hdr->oh_log_prefix ber_tag_t o_tag; /* tag of the request */ +#ifdef HAVE_GETTIMEOFDAY + struct timeval o_hr_time; /* high-resolution time of op start*/ +#endif /* HAVE_GETTIMEOFDAY */ time_t o_time; /* time op was initiated */ int o_tincr; /* counter for multiple ops with same o_time */ @@ -2969,6 +2972,10 @@ struct Connection { #define Statslog( level, fmt, connid, opid, arg1, arg2, arg3 ) \ Log5( (level), ldap_syslog_level, (fmt), (connid), (opid), (arg1), (arg2), (arg3) ) +#define Statslog6( level, fmt, a1, a2, a3, a4, a5, a6 ) \ + Log6( (level), ldap_syslog_level, (fmt), (a1), (a2), (a3), (a4), (a5), (a6) ) +#define Statslog7( level, fmt, a1, a2, a3, a4, a5, a6, a7 ) \ + Log7( (level), ldap_syslog_level, (fmt), (a1), (a2), (a3), (a4), (a5), (a6), (a7) ) #define StatslogTest( level ) ((ldap_debug | ldap_syslog) & (level)) #else /* !LDAP_SYSLOG */ #define Statslog( level, fmt, connid, opid, arg1, arg2, arg3 ) \ @@ -2976,10 +2983,22 @@ struct Connection { if ( ldap_debug & (level) ) \ lutil_debug( ldap_debug, (level), (fmt), (connid), (opid), (arg1), (arg2), (arg3) );\ } while (0) +#define Statslog6( level, fmt, a1, a2, a3, a4, a5, a6 ) \ + do { \ + if ( ldap_debug & (level) ) \ + lutil_debug( ldap_debug, (level), (fmt), (a1), (a2), (a3), (a4), (a5), (a6) ); \ + } while (0) +#define Statslog7( level, fmt, a1, a2, a3, a4, a5, a6, a7 ) \ + do { \ + if ( ldap_debug & (level) ) \ + lutil_debug( ldap_debug, (level), (fmt), (a1), (a2), (a3), (a4), (a5), (a6), (a7) ); \ + } while (0) #define StatslogTest( level ) (ldap_debug & (level)) #endif /* !LDAP_SYSLOG */ #else /* !LDAP_DEBUG */ #define Statslog( level, fmt, connid, opid, arg1, arg2, arg3 ) ((void) 0) +#define Statslog6( level, fmt, a1, a2, a3, a4, a5, a6 ) ((void) 0) +#define Statslog7( level, fmt, a1, a2, a3, a4, a5, a6, a7 ) ((void) 0) #define StatslogTest( level ) (0) #endif /* !LDAP_DEBUG */ -- 2.1.4 --Multipart=_Sun__8_Feb_2015_19_56_17_-0800_yxgp4/9WqjBC20PY--

1 0

(ITS#8054) Operation duration logging
by ebackes＠symas.com 08 Feb '15

08 Feb '15

Full_Name: Emily Backes Version: master, 2.4 OS: URL: Submission from: (NULL) (50.113.67.84) Here is a patch to add duration logging for operations that appears in the standard result messages of statslog. It's a reasonably small change with negligible performance implications except under pathological benchmarking conditions. It does change the quasi-machine-parsable statslog format, however, so it might not be appropriate for general 2.4 release. Example line: 54d8298c conn=1000 op=1 SEARCH RESULT tag=101 err=0 duration=0.170ms nentries=1 text= As written, it uses a separate field in struct Operation and is entirely independent of the o_tincr functionality. Merging those has further portability problems, and is out of the scope of this feature. As a result, the overhead is two additional gettimeofday() calls per operation. The change is wrapped in #ifdef HAVE_GETTIMEOFDAY, so architectures lacking that will see no change. I have patches versus both the master branch (which likely covers 2.5), and OPENLDAP_REL_ENG_2_4. They'll be attached momentarily, outside jitterbug's web interface. %ThThese modifications are the sole work of myself (Emily Backes) on behalf of Symas Corporation based on existing log features of OpenLDAP. Therefore, these modifications are: Copyright 2014-2015 Symas Corporation Redistribution and use in source and binary forms, with or without modification, are permitted only as authorized by the OpenLDAP Public License.

1 0

Re: (ITS#8046) query caused slapd to stop
by ryan＠nardis.ca 06 Feb '15

06 Feb '15

This follow-up is just to add some missing details to the log, since this is getting some traffic now. A query that reproduces the crash is: ldapsearch -E 'mv=(cn={*)(sn=*)' It was introduced by "ITS#7942 plug leak in controls": commit 9d991339 on master, ffd0f03b on RE24. vrFilter_free was broken for a long time, but was never called until the leak was fixed in that ITS. hope that helps, Ryan

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs