openldap-devel

openldap-devel@openldap.org

1416 discussions

slapd memory alloc/free functions
by Hallvard B Furuseth 06 Jul '07

06 Jul '07

#define LDAP_MEMORY_DEBUG and SLAP_NO_SL_MALLOC breaks slapd in a number of places. Some of the fixes are straightforward, but others require more sweeping changes - or leaving LDAP_MEMORY_DEBUG useless, which it has been for some time anyway. However the LDAP_MEMORY_DEBUG troubles match a Windows bug: According to ITS#4901, Windows gets heap corruption unless mallocs done in one DLL are balanced by frees done from the same DLL. (I'll summarize memory management at the end, so the uninitiated can have some clue what I'm talking about.) The main problem is memory allocated with malloc(), either in slapd or a library - in particular librewrite. proto-slap.h does #define free ch_free unless CH_FREE is #defined. ch_free(malloc(x)) ends up calling ber_memfree, and LDAP_MEMORY_DEBUG breaks ber_memfree(malloc(x)). With Windows, the freeing DLL will be liblber instead of librewrite or whatever. The simplest fix looks like a global search-and-replace with malloc & co -> ber_memalloc & co except leaving free (-> ch_free) alone in slapd. (Or use LDAP_MALLOC, LBER_MALLOC and SLAP_MALLOC and friends instead. They all expand to ber_memalloc & co.) Is there a reason why we use both malloc() and ber_memalloc()? Functionality aside, it also makes the code harder to read (and to search for bugs). I think it should be done in librewrite at least, since it is used to insert data into ber_memalloced data structures in slapd's. I suppose such a change could break some Windows module which by somehow explicitly uses _current_ free vs ber_memfree vs ber_memfree_x correctly - in particular if it uses librewrite as a library outside of slapd. Should we worry? Maybe librewrite should have its own memory handler functions which can be overridden like liblber's. Regarding the '#define free ch_free' in slapd, I'm not convinced that is a good idea. I'm sure it has saved some code which wrote free where it should have written ch_free, but it also makes it even harder to see what is going on - and apparently it can break slapd modules that use malloc() - free() on Windows. Maybe we should begin to phase it out. (First stage: /free/ch_free/ in slapd, #define CH_FREE in all slapd .c files, and make the #define free ch_free in slap.h cause some warning. Then wait indefinitely for modules to follow suit.) Summary of OpenLDAP memory management, as I understand it -- Some memory is allocated with plain malloc, but most via ber_mem*() functions in liblber. Of those, the ones ending with "_x" take an extra "context" argument. If you pass a non-NULL context and you have set malloc handler functions in liblber, ber_*_x() call these with the context. Otherwise they call plain malloc() co. slapd also has thread-local malloc heaps, used by slap_sl_malloc() & co. A thread can have a context, which has a reference to the thread's heap. Threads must not free/alloc in other threads' heaps. This avoids locks or mutexes which malloc() needs to ensure that two concurrent malloc()s do not corrupt malloc()'s heap. Slapd can use them for memory which gets allocated and freed while handling a single request (which gets handled by a single thread), I think. slapd sets slap_sl_free() co or wrappers around them as memory handlers, and a special ch_free() for the free() handler. Since it can be a bit hard to keep track of which memory comes from where, ch_free() fetches the current thread's context if passed a NULL context. Then it calls slap_sl_free if there is a context, ber_memfree otherwise. slap_sl_free in turn checks if the memory to be freed came from the context's heap, otherwise it passes it on to ber_memfree() without a context. With the '#define free ch_free', a free/ch_free/slap_sl_free/ber_memfree_x() ends up in the right place. Except if you try to free another thread's memory, or use ber_memfree[_x]() without a context to free a thread's memory, or try to free something from a plain malloc() on Windows. Oh, and finally there are wrapper functions and macros. LBER_MALLOC, LDAP_MALLOC, SLAP_MALLOC and friends, which expand to ber_memalloc & co. ch_malloc() and friends in slapd, which check that the malloc was successful and exit otherwise. (Naughty, we really ought to phase out that someday.) -- Regards, Hallvard

1 1

Alphabetised defines.sh?
by Gavin Henry 06 Jul '07

06 Jul '07

Dear all, I've been trying to follow defines.sh to find a few variables. Is there any reason the individual sections aren't alphabetised? Just wondering ;-) -- Kind Regards, Gavin Henry. Managing Director. T +44 (0) 1224 279484 M +44 (0) 7930 323266 F +44 (0) 1224 824887 E ghenry(a)suretecsystems.com Open Source. Open Solutions(tm). http://www.suretecsystems.com/

3 4

loglevel index intention?
by Quanah Gibson-Mount 02 Jul '07

02 Jul '07

I noticed that loglevel index is unused by anything currently, and I was wondering what the intention was initially behind that loglevel. There are a number of back-bdb/hdb error messages that get generated around indices that currently come out at the "any" level. I thought perhaps index might be better for them, but wasn't sure of the intent. --Quanah

2 2

Re: commit: ldap/servers/slapd/back-perl SampleLDAP.pm
by Gavin Henry 01 Jul '07

01 Jul '07

<quote who="hallvard(a)OpenLDAP.org"> > Update of /repo/OpenLDAP/pkg/ldap/servers/slapd/back-perl > > Modified Files: > SampleLDAP.pm 1.12 -> 1.13 > I've run Perl::Critic over this and fixed/updated to latest "Perl Best Practices". We might as well have an up to date sample. Do you mind if I commit my changes? cpan Perl::Critic perlcritic -1 SampleLDAP.pm Then fixed accordingly. Thanks, Gavin.

5 16

filtered logging
by Kambiz Behrooz 29 Jun '07

29 Jun '07

I am new on openldap (and ldap and this maillist). My company wants to have more control on the logging and I have to make changes on openldap and probably make changes so the can be on next version openldap. Does openldap use only syslog for all logging ? where is easiest to begin ? How good need the code be to have chance to be add to openldap ? Thanks for any information ____________________________________________________________________________________ Bored stiff? Loosen up... Download and play hundreds of games for free on Yahoo! Games. http://games.yahoo.com/games/front

2 1

Recommended backend for Replication Config examples?
by Gavin Henry 27 Jun '07

27 Jun '07

Dear All, For the backend type for all the replication examples I'm doing just now, should we use bdb or hdb? I notice that in slapd.conf.default we are using bdb. Thoughts? Gavin. -- Kind Regards, Gavin Henry. Managing Director. T +44 (0) 1224 279484 M +44 (0) 7930 323266 F +44 (0) 1224 824887 E ghenry(a)suretecsystems.com Open Source. Open Solutions(tm). http://www.suretecsystems.com/

2 2

test043 failure
by Luca Scamoni 21 Jun '07

21 Jun '07

I'm seeing failure with current HEAD. Anyone else? Ing. Luca Scamoni Responsabile Ricerca e Sviluppo SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it ----------------------------------- Office: +39 0382 573859 (137) Mobile: +39 347 1014425 Email: luca.scamoni(a)sys-net.it -----------------------------------

2 1

ITS#4933
by Howard Chu 19 Jun '07

19 Jun '07

So it looks like changing libldap to use the tm_api timeout might have been a bad idea. All the existing apps assume that passing a NULL timeout gives an unlimited timeout. Now this is only true if there is no default timeout set in ldap.conf/ldaprc. If there is such a setting, then an app can't actually get an unlimited timeout unless it explicitly calls ldap_set_option() to override any value read from a config file. Any thoughts? Without that change, the tm_api value is a no-op and totally useless. Perhaps it's an option that should not be valid in any config file, and only valid from an app? -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 1

Necessity of new control ...
by Elier Delgado 18 Jun '07

18 Jun '07

Hi guys, I'm using OpenLdap and I want not desist .. but I need get random entries from a search result or a list .. I was reading and this could be implemented by a control, what you think ? Like paged control result, it most efficient that this functionality reside on the server, but maybe you think that it not necessary .. or inefficient If you agree I can develop it, with your help because I'm not familiarized neither a little ... The problem it's that I'm very hurried, this is a critical issue that I need solve to continues with the system that I'm designing ..otherwise I will need replicate some data to SQL ... Will be complicated develop this control ? Best regards, Elier

2 1

Status of ldbm in HEAD?
by Gavin Henry 15 Jun '07

15 Jun '07

Dear all, What's the status with ldbm in 2.4? The only mention of it in HEAD is in: doc/man/man5/slapd-ldbm.5 (obviously ;-) ) doc/man/man5/slapd.conf.5 doc/man/man5/slapo-syncprov.5 build/openldap.m4 servers/slapd/back-bdb/dn2id.c and loads of #ldbm#index in tests/data/* It looks like it's completely dropped to me, so I was going to remove mention of it from the man pages later, but wasn't sure about core code. Gavin. -- Kind Regards, Gavin Henry. Managing Director. T +44 (0) 1224 279484 M +44 (0) 7930 323266 F +44 (0) 1224 824887 E ghenry(a)suretecsystems.com Open Source. Open Solutions(tm). http://www.suretecsystems.com/

3 4

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-devel