openldap-bugs February 2019

openldap-bugs@openldap.org

19 participants
57 discussions

Re: (ITS#8979) The -l <timelimit> or -o nettimeout=<timelimit> don't limit DNS lookup time
by hyc＠symas.com 19 Feb '19

19 Feb '19

=C3=86var Arnfj=C3=B6r=C3=B0 Bjarmason wrote: > On Mon, Feb 18, 2019 at 5:18 PM Howard Chu <hyc(a)symas.com> wrote: >> >> avarab(a)gmail.com wrote: >>> Full_Name: .var Arnfj.r. Bjarmason >>> Version: 2.4.44-21 >>> OS: CentOS 7.6 >>> URL: ftp://ftp.openldap.org/incoming/ >>> Submission from: (NULL) (5.57.21.154) >>> >>> >>> On a setup where you have a blackholed DNS server: >>> >>> $ grep ^name /etc/resolv.conf >>> nameserver 1.2.3.4 >>> >>> Running e.g.: >>> >>> $ time ldapsearch -l 2 -o nettimeout=3D1 [...] >>> >>> Will (on my system) eventually return: >>> >>> ldap_sasl_bind(SIMPLE): Can't contact LDAP server (-1) >>> >>> real 0m24.039s >>> user 0m0.004s >>> sys 0m0.007s >>> >>> It'll take around 48 seconds if I have two DNS servers. Running strac= e(1) on it >>> reveals that it's sitting in a socket/connect/poll loop trying to loo= kup the >>> hostname of the LDAP server I'm trying to talk to. >>> >>> Instead one of these options should limit time spent on DNS lookups, = or there >>> should be another option, so that you can run ldapsearch with a combi= nation of >>> these options and be sure that it'll run in at most the <timeout> you= give it. >> >> The current code in libldap uses gethostbyname() and this API doesn't = offer >> any parameters for setting a timeout. You can configure a timeout in y= our >> system's /etc/resolv.conf or using RES_OPTIONS environment variable. R= ead your >> system's resolver(5) manpage. >> >> If you know of any widely available resolver API that allows specifyin= g a timeout >> on individual queries, you're welcome to submit a patch supporting thi= s feature. >> The standard -lresolv doesn't provide such an interface. >=20 > That resolv.conf had "options timeout:2". That it didn't kick in is > probably a bug somewhere else, but it would still be handy if > ldapsearch had this itself, since the timeout you use for general > resolutions might not be what you want for ldapsearch. If all you need is a timeout in the ldapsearch command, wrap it in a scri= pt that sets the RES_OPTIONS environment variable. > But implementing that is a mess. This SO post has a summary: > https://stackoverflow.com/questions/24403435/socket-hostname-lookup-tim= eout-how-to-implement-it >=20 > So e.g. there could be a probe for getaddrinfo_a() on GNU systems. >=20 >> Closing this ITS. >=20 > Fair enough, but is that a "we don't want this ever" or "if someone > submitted a patch for (possibly OS-specific) DNS timeout we'd be > interested". Generally not fond of features that aren't available cross-platform, part= icularly if that means it can only be tested on a particular OS. I guess for GNU e= xtensions that are easily detected and widely supported it may be OK. We already use getaddrinfo() if it's available, so adding getaddrinfo_a()= may not be too messy. --=20 -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8979) The -l <timelimit> or -o nettimeout=<timelimit> don't limit DNS lookup time
by quanah＠symas.com 19 Feb '19

19 Feb '19

--On Monday, February 18, 2019 4:32 PM +0000 avarab(a)gmail.com wrote: >> Closing this ITS. > > Fair enough, but is that a "we don't want this ever" or "if someone > submitted a patch for (possibly OS-specific) DNS timeout we'd be > interested". This was already answered in Howard's reply: > If you know of any widely available resolver API that allows specifying a timeout > on individual queries, you're welcome to submit a patch supporting this feature. > The standard -lresolv doesn't provide such an interface. I.e., the ITS can always be reopened if a working resolver API can be found and a patch is supplied. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Re: (ITS#8979) The -l <timelimit> or -o nettimeout=<timelimit> don't limit DNS lookup time
by avarab＠gmail.com 19 Feb '19

19 Feb '19

On Mon, Feb 18, 2019 at 5:18 PM Howard Chu <hyc(a)symas.com> wrote: > > avarab(a)gmail.com wrote: > > Full_Name: .var Arnfj.r. Bjarmason > > Version: 2.4.44-21 > > OS: CentOS 7.6 > > URL: ftp://ftp.openldap.org/incoming/ > > Submission from: (NULL) (5.57.21.154) > > > > > > On a setup where you have a blackholed DNS server: > > > > $ grep ^name /etc/resolv.conf > > nameserver 1.2.3.4 > > > > Running e.g.: > > > > $ time ldapsearch -l 2 -o nettimeout=1 [...] > > > > Will (on my system) eventually return: > > > > ldap_sasl_bind(SIMPLE): Can't contact LDAP server (-1) > > > > real 0m24.039s > > user 0m0.004s > > sys 0m0.007s > > > > It'll take around 48 seconds if I have two DNS servers. Running strace(1) on it > > reveals that it's sitting in a socket/connect/poll loop trying to lookup the > > hostname of the LDAP server I'm trying to talk to. > > > > Instead one of these options should limit time spent on DNS lookups, or there > > should be another option, so that you can run ldapsearch with a combination of > > these options and be sure that it'll run in at most the <timeout> you give it. > > The current code in libldap uses gethostbyname() and this API doesn't offer > any parameters for setting a timeout. You can configure a timeout in your > system's /etc/resolv.conf or using RES_OPTIONS environment variable. Read your > system's resolver(5) manpage. > > If you know of any widely available resolver API that allows specifying a timeout > on individual queries, you're welcome to submit a patch supporting this feature. > The standard -lresolv doesn't provide such an interface. That resolv.conf had "options timeout:2". That it didn't kick in is probably a bug somewhere else, but it would still be handy if ldapsearch had this itself, since the timeout you use for general resolutions might not be what you want for ldapsearch. But implementing that is a mess. This SO post has a summary: https://stackoverflow.com/questions/24403435/socket-hostname-lookup-timeout… So e.g. there could be a probe for getaddrinfo_a() on GNU systems. > Closing this ITS. Fair enough, but is that a "we don't want this ever" or "if someone submitted a patch for (possibly OS-specific) DNS timeout we'd be interested". > > As a workaround I'm using ldapsearch with /usr/bin/timeout, but since it kills > > it if it exceeds the timeout I don't get a meaningful error. > > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8979) The -l <timelimit> or -o nettimeout=<timelimit> don't limit DNS lookup time
by hyc＠symas.com 19 Feb '19

19 Feb '19

avarab(a)gmail.com wrote: > Full_Name: .var Arnfj.r. Bjarmason > Version: 2.4.44-21 > OS: CentOS 7.6 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (5.57.21.154) > > > On a setup where you have a blackholed DNS server: > > $ grep ^name /etc/resolv.conf > nameserver 1.2.3.4 > > Running e.g.: > > $ time ldapsearch -l 2 -o nettimeout=1 [...] > > Will (on my system) eventually return: > > ldap_sasl_bind(SIMPLE): Can't contact LDAP server (-1) > > real 0m24.039s > user 0m0.004s > sys 0m0.007s > > It'll take around 48 seconds if I have two DNS servers. Running strace(1) on it > reveals that it's sitting in a socket/connect/poll loop trying to lookup the > hostname of the LDAP server I'm trying to talk to. > > Instead one of these options should limit time spent on DNS lookups, or there > should be another option, so that you can run ldapsearch with a combination of > these options and be sure that it'll run in at most the <timeout> you give it. The current code in libldap uses gethostbyname() and this API doesn't offer any parameters for setting a timeout. You can configure a timeout in your system's /etc/resolv.conf or using RES_OPTIONS environment variable. Read your system's resolver(5) manpage. If you know of any widely available resolver API that allows specifying a timeout on individual queries, you're welcome to submit a patch supporting this feature. The standard -lresolv doesn't provide such an interface. Closing this ITS. > As a workaround I'm using ldapsearch with /usr/bin/timeout, but since it kills > it if it exceeds the timeout I don't get a meaningful error. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8979) The -l <timelimit> or -o nettimeout=<timelimit> don't limit DNS lookup time
by avarab＠gmail.com 19 Feb '19

19 Feb '19

Full_Name: .var Arnfj.r. Bjarmason Version: 2.4.44-21 OS: CentOS 7.6 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (5.57.21.154) On a setup where you have a blackholed DNS server: $ grep ^name /etc/resolv.conf nameserver 1.2.3.4 Running e.g.: $ time ldapsearch -l 2 -o nettimeout=1 [...] Will (on my system) eventually return: ldap_sasl_bind(SIMPLE): Can't contact LDAP server (-1) real 0m24.039s user 0m0.004s sys 0m0.007s It'll take around 48 seconds if I have two DNS servers. Running strace(1) on it reveals that it's sitting in a socket/connect/poll loop trying to lookup the hostname of the LDAP server I'm trying to talk to. Instead one of these options should limit time spent on DNS lookups, or there should be another option, so that you can run ldapsearch with a combination of these options and be sure that it'll run in at most the <timeout> you give it. As a workaround I'm using ldapsearch with /usr/bin/timeout, but since it kills it if it exceeds the timeout I don't get a meaningful error.

1 0

Re: (ITS#8978) Fix mdb_env_open2() failing when getting handle for NTDLL.dll
by hyc＠symas.com 18 Feb '19

18 Feb '19

khng300(a)gmail.com wrote: > Full_Name: Ka Ho Ng > Version: mdb.master > OS: Windows 10 > URL: ftp://ftp.openldap.org/incoming/Ka-Ho-Ng-190218.patch > Submission from: (NULL) (2001:470:fa95:1300::1) Thanks for the patch, committed to mdb.master > > Fix mdb_env_open2() failing when getting handle for NTDLL.dll > > Always call GetModuleHandleW() with Unicode string, as mdb_fopen() is > calling CreateFileW() already. Otherwise GetModuleHandle() will fail with > non-Unicode string. > > I, Ka Ho Ng, hereby place the following modifications to OpenLDAP Software (and > only these modifications) into the public domain. Hence, these modifications may > be freely used and/or redistributed for any purpose with or without attribution > and/or other notice. > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8978) Fix mdb_env_open2() failing when getting handle for NTDLL.dll
by khng300＠gmail.com 18 Feb '19

18 Feb '19

Full_Name: Ka Ho Ng Version: mdb.master OS: Windows 10 URL: ftp://ftp.openldap.org/incoming/Ka-Ho-Ng-190218.patch Submission from: (NULL) (2001:470:fa95:1300::1) Fix mdb_env_open2() failing when getting handle for NTDLL.dll Always call GetModuleHandleW() with Unicode string, as mdb_fopen() is calling CreateFileW() already. Otherwise GetModuleHandle() will fail with non-Unicode string. I, Ka Ho Ng, hereby place the following modifications to OpenLDAP Software (and only these modifications) into the public domain. Hence, these modifications may be freely used and/or redistributed for any purpose with or without attribution and/or other notice.

1 0

Re: (ITS#8976) LMDB > use posix_madvise(3) instead of madvise(3) for SmartOS
by matthieu.guegan＠smile-suisse.com 16 Feb '19

16 Feb '19

On Fri, Feb 15, 2019 at 5:16 PM Howard Chu <hyc(a)symas.com> wrote: > > matthieu.guegan(a)smile-suisse.com wrote: > > On Fri, Feb 15, 2019 at 4:33 PM Howard Chu <hyc(a)symas.com> wrote: > >> > >> Matthieu GUEGAN wrote: > >>> On Fri, Feb 15, 2019 at 3:56 PM Howard Chu <hyc(a)symas.com> wrote: > >>>> > >>>> mguegan(a)virtua.ch wrote: > >>>>> Full_Name: Matthieu Guegan > >>>>> Version: current > >>>>> OS: SmartOS > >>>>> URL: > >>>>> Submission from: (NULL) (109.190.148.77) > >>>>> > >>>>> > >>>>> Hi, > >>>>> > >>>>> In order to compile knot[1] on SmartOS, I have done a series of little patches. > >>>>> One of them is linked to the LMDB project, which redirects me here. > >>>>> > >>>>> The idea is to tell SmartOS (and I think the SunOS family) to make use of > >>>>> posix_madvise(3), instead of madvise(3) as implementation on this OS[2] is > >>>>> different than, for example, the Linux[3] one. > >>>>> > >>>>> So, I kindly ask you to take a look at the merge request of the knot project[4], > >>>>> in particular the `src/contrib/lmdb/mdb.c` patch (that's just a ifdef > >>>>> modification) : would it be possible to apply this patch directly in this > >>>>> repository ? > >>>> > >>>> I see no functional difference between your references [2] and [3]. > >>>> What is the benefit of this patch? > >>> > >>> Well, the Solaris/SunOS's madvise(3) is defined with an old `caddr_t` > >>> type whereas Linux (and *BSD OSes too) use a more modern approach with > >>> `void *`. > >>> On the `knot` project, which embed the lmdb part of openldap, > >>> compilation failed when using madvise(3) on SmartOS, so I did a > >>> replacement with posix_madvise(3) which uses the `void *` argument > >>> too. > >>> I don't know what's the best way to handle this case, that's why I ask here. > >> > >> A caddr_t is just a typedef of "char *". It should always be legal to > >> pass a void * to such a parameter. What is your compile error message? > >> > > > > ``` > > contrib/lmdb/mdb.c: In function 'mdb_env_map': > > contrib/lmdb/mdb.c:3998:3: error: implicit declaration of function > > 'madvise'; did you mean 'raise'? > > [-Werror=implicit-function-declaration] > > madvise(env->me_map, env->me_mapsize, MADV_RANDOM); > > ^~~~~~~ > > raise > > This has absolutely nothing to do with whether the parameter is a caddr_t or a void *. > Why would you even mention something totally irrelevant to the actual error? Sorry if I mislead you with that, that was not my intention. I was just trying to answer you when asking the difference using posix_madvise() vs madvise() on SmartOS. And that was the only thing I found. Indeed, it seems completely irrelevant to the error I get. > Strange that you have MADV_RANDOM defined but no declaration of madvise(). Indeed, searching in the header file `sys/mman.h` [5], it seems to be possible to have a condition that enable madvise() and not MADV_RANDOM or vice-versa. > Sounds like your system's header files are playing stupid games with definitions. > If you can find out what flags must be used to make the header files properly > declare the madvise function, a patch for that may be acceptable. I should say that I'm using the pkgsrc framework in order to compile the whole thing and found multiple packages on which was applied the same type of patch (excluding __sun when trying to use madvise(), thus using the posix_madvise variant). Flags can be overridden here, so it must be something related with the pkgsrc's Makefile variables. Anyway, maybe this should be done this way, instead of trying to change things upstream. Thanks for your help. [5] https://github.com/illumos/illumos-gate/blob/4e0c5eff9af325c80994e9527b7cb8…

1 0

Re: (ITS#8976) LMDB > use posix_madvise(3) instead of madvise(3) for SmartOS
by hyc＠symas.com 16 Feb '19

16 Feb '19

matthieu.guegan(a)smile-suisse.com wrote: > On Fri, Feb 15, 2019 at 4:33 PM Howard Chu <hyc(a)symas.com> wrote: >> >> Matthieu GUEGAN wrote: >>> On Fri, Feb 15, 2019 at 3:56 PM Howard Chu <hyc(a)symas.com> wrote: >>>> >>>> mguegan(a)virtua.ch wrote: >>>>> Full_Name: Matthieu Guegan >>>>> Version: current >>>>> OS: SmartOS >>>>> URL: >>>>> Submission from: (NULL) (109.190.148.77) >>>>> >>>>> >>>>> Hi, >>>>> >>>>> In order to compile knot[1] on SmartOS, I have done a series of little patches. >>>>> One of them is linked to the LMDB project, which redirects me here. >>>>> >>>>> The idea is to tell SmartOS (and I think the SunOS family) to make use of >>>>> posix_madvise(3), instead of madvise(3) as implementation on this OS[2] is >>>>> different than, for example, the Linux[3] one. >>>>> >>>>> So, I kindly ask you to take a look at the merge request of the knot project[4], >>>>> in particular the `src/contrib/lmdb/mdb.c` patch (that's just a ifdef >>>>> modification) : would it be possible to apply this patch directly in this >>>>> repository ? >>>> >>>> I see no functional difference between your references [2] and [3]. >>>> What is the benefit of this patch? >>> >>> Well, the Solaris/SunOS's madvise(3) is defined with an old `caddr_t` >>> type whereas Linux (and *BSD OSes too) use a more modern approach with >>> `void *`. >>> On the `knot` project, which embed the lmdb part of openldap, >>> compilation failed when using madvise(3) on SmartOS, so I did a >>> replacement with posix_madvise(3) which uses the `void *` argument >>> too. >>> I don't know what's the best way to handle this case, that's why I ask here. >> >> A caddr_t is just a typedef of "char *". It should always be legal to >> pass a void * to such a parameter. What is your compile error message? >> > > ``` > contrib/lmdb/mdb.c: In function 'mdb_env_map': > contrib/lmdb/mdb.c:3998:3: error: implicit declaration of function > 'madvise'; did you mean 'raise'? > [-Werror=implicit-function-declaration] > madvise(env->me_map, env->me_mapsize, MADV_RANDOM); > ^~~~~~~ > raise This has absolutely nothing to do with whether the parameter is a caddr_t or a void *. Why would you even mention something totally irrelevant to the actual error? Strange that you have MADV_RANDOM defined but no declaration of madvise(). Sounds like your system's header files are playing stupid games with definitions. If you can find out what flags must be used to make the header files properly declare the madvise function, a patch for that may be acceptable. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8976) LMDB > use posix_madvise(3) instead of madvise(3) for SmartOS
by matthieu.guegan＠smile-suisse.com 16 Feb '19

16 Feb '19

On Fri, Feb 15, 2019 at 4:33 PM Howard Chu <hyc(a)symas.com> wrote: > > Matthieu GUEGAN wrote: > > On Fri, Feb 15, 2019 at 3:56 PM Howard Chu <hyc(a)symas.com> wrote: > >> > >> mguegan(a)virtua.ch wrote: > >>> Full_Name: Matthieu Guegan > >>> Version: current > >>> OS: SmartOS > >>> URL: > >>> Submission from: (NULL) (109.190.148.77) > >>> > >>> > >>> Hi, > >>> > >>> In order to compile knot[1] on SmartOS, I have done a series of little patches. > >>> One of them is linked to the LMDB project, which redirects me here. > >>> > >>> The idea is to tell SmartOS (and I think the SunOS family) to make use of > >>> posix_madvise(3), instead of madvise(3) as implementation on this OS[2] is > >>> different than, for example, the Linux[3] one. > >>> > >>> So, I kindly ask you to take a look at the merge request of the knot project[4], > >>> in particular the `src/contrib/lmdb/mdb.c` patch (that's just a ifdef > >>> modification) : would it be possible to apply this patch directly in this > >>> repository ? > >> > >> I see no functional difference between your references [2] and [3]. > >> What is the benefit of this patch? > > > > Well, the Solaris/SunOS's madvise(3) is defined with an old `caddr_t` > > type whereas Linux (and *BSD OSes too) use a more modern approach with > > `void *`. > > On the `knot` project, which embed the lmdb part of openldap, > > compilation failed when using madvise(3) on SmartOS, so I did a > > replacement with posix_madvise(3) which uses the `void *` argument > > too. > > I don't know what's the best way to handle this case, that's why I ask here. > > A caddr_t is just a typedef of "char *". It should always be legal to > pass a void * to such a parameter. What is your compile error message? > ``` contrib/lmdb/mdb.c: In function 'mdb_env_map': contrib/lmdb/mdb.c:3998:3: error: implicit declaration of function 'madvise'; did you mean 'raise'? [-Werror=implicit-function-declaration] madvise(env->me_map, env->me_mapsize, MADV_RANDOM); ^~~~~~~ raise ```

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs February 2019