openldap-bugs August 2009

openldap-bugs@openldap.org

35 participants
225 discussions

Re: (ITS#6234) CONFIGURABLE TCP BUFFERS IN SLAPD FEATURE REQUEST
by masarati＠aero.polimi.it 04 Aug '09

04 Aug '09

Your message was unreadable, you should use text form for messages sent to the ITS. In any case, according to many sources, the default values for TCP buffers can be pretty small. Usually, they can be configured system-wide at the OS level (see tcp(7)), or even auto-tuned. Personally, I have no objections in optionally allowing to hard-set the buffer sizes to values one considers appropriate for the intended use of slapd. Since the optimal size roughly depends on latency and bandwidth, this may probably need to be configured on a per-listener basis, actually. However, in deployments where this problem becomes critical, usually hardware dedicated to slapd is used, so system-wide tuning (and auto-tuning) would be the preferred option anyway. Manual setting should be used only when system-wide or auto-tuning is not available or accessible for some reason. p.

1 0

Re: (ITS#6234) CONFIGURABLE TCP BUFFERS IN SLAPD FEATURE REQUEST
by evaristojosec＠yahoo.es 04 Aug '09

04 Aug '09

Hi,=0A=0AProbably "heavy LDAP" load was a very generic simplified definitio= n.=0AThe traffic model we are testing contains search and modify operations= . The=0Asearch response contains usually a single entry about 3KB in LDIF f= ormat. There=0Ais only a reduce number of LDAP clients (usually one or two)= , that are using=0Aasync LDAP API and sending a lot of operations per secon= d (> 1000 TPS per=0Aclient).=0A=0AUnder these conditions we have tested tha= t increasing the sending buffer of the=0Aserver has improved the performanc= e. That's the reason I am proposing this=0Afeature request.=0A=0ARegards,= =0A=0AEvaristo=0A=0A----- Mensaje original ----=0ADe: Howard Chu <hyc@symas= .com>=0APara: evaristojosec(a)yahoo.es=0ACC: openldap-its(a)openldap.org=0AEnvi= ado: martes, 4 de agosto, 2009 0:43:36=0AAsunto: Re: (ITS#6234) CONFIGURABL= E TCP BUFFERS IN SLAPD FEATURE REQUEST=0A=0Aevaristojosec(a)yahoo.es wrote:= =0A> Full_Name: Evaristo Camarero=0A> Version: 2.4.x=0A> OS: Linux=0A> URL:= ftp://ftp.openldap.org/incoming/=0A> Submission from: (NULL) (194.237.142.= 20)=0A> =0A> =0A> =0A> Hi,=0A> =0A> Working with slapd under heavy load it = could be interesting to have TCP buffers=0A> size configurable at slapd app= lication level instead of getting default OS=0A> values.=0A=0AI don't see t= his having any positive impact on slapd performance. "Heavy LDAP load" gene= rally implies lots of small queries with sets of small result objects; very= rarely do queries return large objects. When a system is under such heavy = load, requesting larger buffers per socket will make the system load worse,= as the kernel will have more memory consumed in these socket buffers and l= ess available for other demands.=0A=0AWhat workload have you tested this wi= th, and what performance impact did you measure as a result?=0A=0A> Some fa= st modifications in daemon.c file to achieve a hardcoded value for the=0A> = sender buffer are:=0A> =0A> int buf_size =3D 1572864; /* 1.5 Mb */= =0A> rc =3Dsetsockopt ( s, SOL_SOCKET, SO_SNDBUF, (int*)&buf_size,= =0A> sizeof(buf_size));=0A> if(rc=3D=3D-1) {=0A> Debug(= LDAP_DEBUG_ANY, "slapd(%ld): "=0A> "setsockopt(SO_SNDBUF) = failed errno=3D%d (%s)\n",=0A> (long) l.sl_sd, err, sock_er= rstr(err) );=0A> } else {=0A> Debug( LDAP_DEBUG_ANY, "s= lapd(%ld): setsockopt(SO_SNDBUF) to %d\n",=0A> (long) l.sl_= sd, buf_size, 0);=0A> }=0A=0A-- -- Howard Chu=0A CTO, Symas Corp= . http://www.symas.com=0A Director, Highland Sun http://highla= ndsun.com/hyc/=0A Chief Architect, OpenLDAP http://www.openldap.org/proje= ct/=0A=0A=0A=0A

1 0

Re: (ITS#6240) cn=config replication failing on repeat olcDbConfig
by hyc＠symas.com 03 Aug '09

03 Aug '09

matt.edlefsen(a)gmail.com wrote: > Full_Name: Matt Edlefsen > Version: 2.4.16 > OS: FreeBSD > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (98.213.15.129) > > > When trying to setup multi-master replication of a cn=config database I > encountered the following error: > Aug 3 20:21:38 gabbro slapd[81578]: syncrepl_message_to_entry: > rid=001 mods check (olcDbConfig: value #14 provided more than once) Thanks for the report, this is fixed in HEAD. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#6240) cn=config replication failing on repeat olcDbConfig
by matt.edlefsen＠gmail.com 03 Aug '09

03 Aug '09

Full_Name: Matt Edlefsen Version: 2.4.16 OS: FreeBSD URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (98.213.15.129) When trying to setup multi-master replication of a cn=config database I encountered the following error: Aug 3 20:21:38 gabbro slapd[81578]: syncrepl_message_to_entry: rid=001 mods check (olcDbConfig: value #14 provided more than once) Process: The cn=config used was copied to both servers (using ldapsearch -LLL | slapadd -n 0) from a third server, which in turn was originally converted from a basic slapd.conf using slapd -f -F. To turn on replication I added the following entries/attributes to the cn=config tree of both servers following the steps outlined in the Users Guide (only the relevant attributes are listed): dn: cn=config olcServerID: 1 ldap://rock.earlham.edu/ olcServerID: 2 ldap://gabbro.earlham.edu/ dn: olcDatabase={0}config,cn=config olcSyncrepl: {0}rid=001 provider=dap://rock.earlham.edu/ binddn="<snip>" bindmethod=simple credentials=<snip> searchbase="cn=config" type=refreshAndPersist retry="5 5 300 5" timeout=1 olcSyncrepl: {1}rid=002 provider=ldap://gabbro.earlham.edu/ binddn="<snip>" bindmethod=simple credentials=<snip> searchbase="cn=config" type=refreshAndPersist retry="5 5 300 5" timeout=1 olcMirrorMode: TRUE olcOverlay={0}syncprov,olcDatabase={0}config,cn=config objectClass: olcOverlayConfig objectClass: olcSyncProvConfig olcOverlay: {0}syncprov When I added the above to the second server it started replicating against the first and when it reached the olcDbConfig attributes of the bdb database config it failed with the error message given above. The olcDbConfig attributes are: olcDbConfig: {0}# $OpenLDAP: pkg/ldap/servers/slapd/DB_CONFIG,v 1.3.2.4 2007/12/18 11:53:27 ghenry Exp $ olcDbConfig: {1}# Example DB_CONFIG file for use with slapd(8) BDB/HDB databases. olcDbConfig: {2}# olcDbConfig: {3}# See the Oracle Berkeley DB documentation olcDbConfig: {4}# <http://www.oracle.com/technology/documentation/berkeley-db/db/ref/env/db_co…> olcDbConfig: {5}# for detail description of DB_CONFIG syntax and semantics. olcDbConfig: {6}# olcDbConfig: {7}# Hints can also be found in the OpenLDAP Software FAQ olcDbConfig: {8}# <http://www.openldap.org/faq/index.cgi?file=2> olcDbConfig: {9}# in particular: olcDbConfig: {10}# <http://www.openldap.org/faq/index.cgi?file=1075> olcDbConfig: {11} olcDbConfig: {12}# Note: most DB_CONFIG settings will take effect only upon rebuilding olcDbConfig: {13}# the DB environment. olcDbConfig: {14} olcDbConfig: {15}# one 1 GB cache olcDbConfig: {16}set_cachesize 0 1073741824 1 olcDbConfig: {17} olcDbConfig: {18}# Data Directory olcDbConfig: {19}#set_data_dir db olcDbConfig: {20} olcDbConfig: {21}# Transaction Log settings olcDbConfig: {22}set_lg_regionmax 262144 olcDbConfig: {23}set_lg_bsize 104857600 olcDbConfig: {24}set_lk_max_locks 100000 olcDbConfig: {25}#set_lg_dir logs olcDbConfig: {26} olcDbConfig: {27}# Note: special DB_CONFIG flags are no longer needed for "quick" olcDbConfig: {28}# slapadd(8) or slapindex(8) access (see their -q option). Fix: Removing all the duplicate entries (not counting the {#} prefixes) fixed the problem.

1 0

(ITS#6163)
by karavelov＠spnet.net 03 Aug '09

03 Aug '09

I have tested with 2.4.17 with servers/slapd/back-sql/search.c from CVS HEAD and the issue is resolved. Thanks Luben Karavelov

1 0

Re: (ITS#6234) CONFIGURABLE TCP BUFFERS IN SLAPD FEATURE REQUEST
by hyc＠symas.com 03 Aug '09

03 Aug '09

evaristojosec(a)yahoo.es wrote: > Full_Name: Evaristo Camarero > Version: 2.4.x > OS: Linux > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (194.237.142.20) > > > > Hi, > > Working with slapd under heavy load it could be interesting to have TCP buffers > size configurable at slapd application level instead of getting default OS > values. I don't see this having any positive impact on slapd performance. "Heavy LDAP load" generally implies lots of small queries with sets of small result objects; very rarely do queries return large objects. When a system is under such heavy load, requesting larger buffers per socket will make the system load worse, as the kernel will have more memory consumed in these socket buffers and less available for other demands. What workload have you tested this with, and what performance impact did you measure as a result? > Some fast modifications in daemon.c file to achieve a hardcoded value for the > sender buffer are: > > int buf_size = 1572864; /* 1.5 Mb */ > rc =setsockopt ( s, SOL_SOCKET, SO_SNDBUF, (int*)&buf_size, > sizeof(buf_size)); > if(rc==-1) { > Debug( LDAP_DEBUG_ANY, "slapd(%ld): " > "setsockopt(SO_SNDBUF) failed errno=%d (%s)\n", > (long) l.sl_sd, err, sock_errstr(err) ); > } else { > Debug( LDAP_DEBUG_ANY, "slapd(%ld): setsockopt(SO_SNDBUF) to %d\n", > (long) l.sl_sd, buf_size, 0); > } -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#6217) proxycache not returning cached data
by masarati＠aero.polimi.it 03 Aug '09

03 Aug '09

>> But he said that reversing the order of the filters (i.e. no >> objectClass=someclass first) cures the problem. > > Sorry, if he told you that it must have been in a private email. There is > no > such statement recorded in the ITS. He said it would be simple to reverse > the > filters, but that it would not accomplish anything since he wants to cache > searches from Samba. He didn't say that he actually tried reversing the > filters or that it worked. At any rate, the issue in 5756 was confirmed > fixed > so again, I do not believe it's responsible for the current situation. No private message. I probably misunderstood the message in followup 2. My understanding of that message is that he tried swapping the simple filters, but I admit the sentence sound ambiguous to me. p.

1 0

Re: (ITS#6217) proxycache not returning cached data
by hyc＠symas.com 03 Aug '09

03 Aug '09

masarati(a)aero.polimi.it wrote: >> Jim van Keulen wrote: >>> hyc(a)symas.com wrote: >>> > The filter behavior you're describing was noted in ITS#5756, fixed >>> in >>> January. >>> >>> If it was fixed in January, why does openldap-2.4.16 (20090411) still >>> exhibit the described behaviour? >> >> I don't believe what you're seeing now is due to the situation Ando >> described. > > But he said that reversing the order of the filters (i.e. no > objectClass=someclass first) cures the problem. Sorry, if he told you that it must have been in a private email. There is no such statement recorded in the ITS. He said it would be simple to reverse the filters, but that it would not accomplish anything since he wants to cache searches from Samba. He didn't say that he actually tried reversing the filters or that it worked. At any rate, the issue in 5756 was confirmed fixed so again, I do not believe it's responsible for the current situation. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#6227) autogroup overlay breaks all searches
by hyc＠symas.com 03 Aug '09

03 Aug '09

mbuschman(a)cashnetusa.com wrote: > Confirmed working. > Also experienced (possibly unrelated?) segfault on slapadd. We can't tell if it's related or not unless you post the stack trace from the segfault. > > hyc(a)symas.com wrote: >> Fixed now in CVS HEAD. >> >> mbuschman(a)cashnetusa.com wrote: >> >>> Full_Name: Marshall Buschman >>> Version: 2.4.17 >>> OS: Debian Etch >>> URL: ftp://ftp.openldap.org/incoming/ >>> Submission from: (NULL) (38.98.177.132) >>> >>> >> >> >>> I'm using OpenLDAP 2.4.17 with bdb 4.7 Debian Lenny. I've enabled the >>> autogroup overlay, and I get the following error when I do an ldapsearch: >>> >>> $ ldapsearch -x -H "ldap://ldapserver" -b dc=cashnetusa,dc=com >>> # extended LDIF >>> ... >>> # search result >>> search: 2 >>> result: 53 Server is unwilling to perform >>> text: operation not supported within namingContext >>> >>> Relevant config: >>> ---------------------------------------------- >>> moduleload autogroup >>> >>> database bdb >>> suffix "dc=cashnetusa,dc=com" >>> >>> overlay autogroup >>> autogroup-attrset groupOfURLs memberURL member >>> ---------------------------------------------- >>> >>> OpenLDAP does start normally without the overlay. >>> >>> >> >> >> > > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#6217) proxycache not returning cached data
by masarati＠aero.polimi.it 03 Aug '09

03 Aug '09

> Jim van Keulen wrote: >> hyc(a)symas.com wrote: >> > The filter behavior you're describing was noted in ITS#5756, fixed >> in >> January. >> >> If it was fixed in January, why does openldap-2.4.16 (20090411) still >> exhibit the described behaviour? > > I don't believe what you're seeing now is due to the situation Ando > described. But he said that reversing the order of the filters (i.e. no objectClass=someclass first) cures the problem. p.

1 0

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs August 2009