openldap-bugs

openldap-bugs@openldap.org

20879 discussions

(ITS#4717) Efficient handling of response lists in C-API
by ando＠sys-net.it 22 Oct '06

22 Oct '06

Full_Name: Pierangelo Masarati Version: HEAD OS: irrelevant URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (81.72.89.40) Submitted by: ando In proxies, we occasionally need to mark some response messages as "don't care". This is slightly different than "abandon", in the sense that, for performance and band occupation, an abandon would likely get too late to the remote server, but if the response is not removed from the list that would grow and waste resources. In realistic cases we profiled proxies that spend 50-60% of the CPU time running the list of messages to check if any of them has been abandoned. The proposed fix consists in: - keeping abandoned msgid's sorted, so that a bisection algorithm can be used to check if a msgid has been abandoned - add a ldap_int_abandon() call that marks a message as "abandoned" without actually sending an "abandon" operation. This call is, by now, private (ldap_int_) since its use in normal clients would make little sense. However, whenever it does make sense, there's no objection in making the call public (sort of ldap_abandon_mark()). With this change, the overall CPU used to check response list in the very same test cases dropped to less than 1%. p.

1 0

Re: (ITS#4716) syncrepl - consumer missed some delete
by richton＠nbcs.rutgers.edu 20 Oct '06

20 Oct '06

Are you patched for ITS#4708 and ITS#4695? Those are my current "don't try to syncrepl without it" for RE23... Is this refreshAndPersist, or refreshOnly? delta-syncrepl? Do you use a session log? Maybe just post the relevant config entirely...

1 0

Re: (ITS#4715) proxy retries anonymously
by ando＠sys-net.it 20 Oct '06

20 Oct '06

ashish(a)ratboy.net wrote: > I tried the rebind-as-user approach and that works :) > Will try out the HEAD soon. > Good. I'll rework the documentation accordingly. Cheers, p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.n.c. Via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it ------------------------------------------ Office: +39.02.23998309 Mobile: +39.333.4963172 Email: pierangelo.masarati(a)sys-net.it ------------------------------------------

1 0

Re: (ITS#4715) proxy retries anonymously
by ashish＠ratboy.net 20 Oct '06

20 Oct '06

> I see that the connection is retried anonymously, which is incorrect. > This issue is known, and it has been fixed in HEAD code by using identity > assertion to retry non-anonymous connections. Another option would be to > set "rebind-as-user", so that user credentials are saved and used to retry > non-anonymous connections. I tried the rebind-as-user approach and that works :) Will try out the HEAD soon. Thank you. Ashish Gawarikar

1 0

Re: (ITS#4715) proxy retries anonymously
by ando＠sys-net.it 20 Oct '06

20 Oct '06

> Either you provide an all-OpenLDAP setup, consisting of proxy, remote > server and operation sequence that clearly shows the issue, so that we can > reproduce and track it, or you should rather investigate what's happening > between the proxy and the remote server, e.g. by providing a tcpdump of > the communications resulting in the error you reported. Sorry, I missed your intermediate posting with a trace of the problem (either I didn't get that message or I simply overlooked it; I've found it right now on the ITS). I see that the connection is retried anonymously, which is incorrect. This issue is known, and it has been fixed in HEAD code by using identity assertion to retry non-anonymous connections. Another option would be to set "rebind-as-user", so that user credentials are saved and used to retry non-anonymous connections. Personally, I'd prefer the idassert approach, but "rebind-as-user" could be useful in case the remote server does not support proxyAuthz, or in case your applications need to use proxyAuthz themselves. Can you try either (or both) approaches? The former requires you to build HEAD (or re24, since last night it was sync'ed with HEAD) code. p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.n.c. Via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it ------------------------------------------ Office: +39.02.23998309 Mobile: +39.333.4963172 Email: pierangelo.masarati(a)sys-net.it ------------------------------------------

1 0

Re: (ITS#4715) proxy retries anonymously
by ando＠sys-net.it 20 Oct '06

20 Oct '06

> Attaching the relevant slapd.conf I don't think this is by any means relevant. I've run some basic checks, as I said, and nothing at the proxy side seems to be able to cause the error you report. Either you provide an all-OpenLDAP setup, consisting of proxy, remote server and operation sequence that clearly shows the issue, so that we can reproduce and track it, or you should rather investigate what's happening between the proxy and the remote server, e.g. by providing a tcpdump of the communications resulting in the error you reported. p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.n.c. Via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it ------------------------------------------ Office: +39.02.23998309 Mobile: +39.333.4963172 Email: pierangelo.masarati(a)sys-net.it ------------------------------------------

1 0

Re: (ITS#4702) out-of-memory on huge DB ?
by Paolo.Rossi.con＠h3g.it 19 Oct '06

19 Oct '06

Hi, today I'm finally succeed to export the huge DB with the 64bit build, solaris8/sparc, shminfo_shmmax=536870912 DB_CONFIG whit set_cachesize 0 384000000 1 and using slapcat -f slapd.conf -l producer_dump.ldif FYI, here the final memory allocation and output ldif file: PID RSS VSZ %MEM TIME STIME COMMAND 10294 7491744 7497840 93.1 02:22:27 10:35:37 slapcat ls -la -rw-r--r-- 1 test other 10486259712 Oct 19 13:16 producer_dump.ldif therefore, about 7,5 GB of RAM for ca. 10 GB of ldif Any suggestion, cue? Paolo

1 0

Re: (ITS#4716) syncrepl - consumer missed some delete
by Paolo.Rossi.con＠h3g.it 19 Oct '06

19 Oct '06

I've do some other test: I've shutdown consumer (desynched), and I've do about 10000 random modify on producer, then I've set loglevel -1 on producer (due to find something wrong) and started consumer. After a night, I've restarted the cluster again (producer and consumer) setting a relaxed loglevel on producer, but this time consumer was resynched! Also the deleted DN ignored previously, are now in sych. very strange. Paolo

1 0

Re: (ITS#4715) proxy retries anonymously
by ashish＠ratboy.net 18 Oct '06

18 Oct '06

This is a multi-part message in MIME format. --------------040702090906080801050004 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Attaching the relevant slapd.conf --------------040702090906080801050004 Content-Type: text/plain; name="slapd.conf" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="slapd.conf" pidfile /var/run/ldap/slapd.pid argsfile /var/run/ldap/slapd.args # Logging is disabled by default. Enable logging when necessary. # (May significantly affect performance, depending on the level of # server activity.) # To enable logging of access, read, modify, and deletions, # set the loglevel to 256 loglevel none # disallow bind_anon gentlehup on # The sizelimit restriction controls the potential to which the directory # contents may be exploited. Directory queries can only return *this* # many responses. # #sizelimit unlimited sizelimit 200 # Limit the amount of time the server will spend performing # a single query # #timelimit unlimited timelimit 3600 allow bind_v2 ####################################################################### # SASL: ####################################################################### # # SMD is configured with SASL support for most environments. # See the provided slapd.conf man page for additional information. # ## sasl-secprops: # Used to specify Cyrus SASL security properties. The "none" value # by itself causes the default "noanonymous,noplain" to be cleared. sasl-secprops none # ## sasl-host: # Used to specify the fully qualified domain name # used for SASL processing. #sasl-host <fqdn> # ## sasl-realm: # Specify the SASL realm. Default is empty. #sasl-realm <realm> ####################################################################### # schema information ####################################################################### include /usr/local/example/smd-4.0/schema/syntax.defs include /usr/local/example/smd-4.0/schema/core.schema include /usr/local/example/smd-4.0/schema/cosine.schema include /usr/local/example/smd-4.0/schema/inetorgperson.schema include /usr/local/example/smd-4.0/schema/openldap.schema include /usr/local/example/smd-4.0/schema/nis.schema include /usr/local/example/smd-4.0/schema/messageRecipient.schema include /usr/local/example/smd-4.0/schema/smi.schema include /usr/local/example/smd-4.0/schema/sieve.schema include /usr/local/example/smd-4.0/schema/sendmail.schema # All user-customized schema additions must be made in the files # stored under /etc/mail/openldap/schema. include /etc/mail/openldap/schema/custom.schema ####################################################################### # ldap database definitions ####################################################################### database ldap lastmod off suffix "" # The proxy cache function requires that the 'rootdn' parameter is set. # Note that with the password configured below, it cannot be used to bind rootdn "cn=Manager" rootpw {SHA}example-proxycache ###################################################################### # Back-LDAP connection settings ###################################################################### # # Two settings are necessary: # A) Optional TLS settings, used with all ldap:// connections. # This setting must appear before the LDAP URI. # B) LDAP URI list, with hosts separated by spaces # # To require TLS on ldap:// connections, use "tls start" # To try TLS (but not require) on ldap:// connections, use "tls try-start" # # tls start # tls try-start # # This URI example has two LDAP replica servers to try, using ldaps and ldap #uri "ldaps://rep1.domain.com ldaps://rep2.domain.com ldap://rep1.domain.com" uri ldap://ldap.smi.example.com ###################################################################### # Proxycache settings ###################################################################### overlay pcache ########## <database> <max_entries> <numattrsets> <entry_limit> <cc_period> proxycache bdb 10000 3 1 600 proxycachequeries 10000 # Important proxycache notes: # - You may have multiple queries per cache set # - The same query can appear in multiple cache sets # - Only one proxyattrset definition may appear for any cache set # - A given attribute may only appear in one proxyattrset. # - Taking all four conditions above, if multiple queries need access # to the same attribute in their result, the queries must appear # within the same cache set. Also, the attrset for that cache set # must contain a list of all attributes returned from any of the # queries in that set. # - All attributes used in search queries must be defined in the schema. # Sendmail has pre-defined schema including the vendor-specific attributes. # - Additional application and target directory server notes appear # below the cache definitions # - An entry will be cached once for each different search query that # is being used to find it. This means that the actual number of # LDAP entries which may be cached is not the value set in MAX_ENTRIES # above, but instead is this value divided by the number of distinct # LDAP queries being used. # Cache set 0 is used for Flow Control, Authentication, LDAP routing, Proxy lookups proxyattrset 0 DN mail mailRoutingAddress mailHost imapHost popHost objectClass smiAuthDisabled # # Cache set 1 is used for distribution list expansion proxyattrset 1 mgrpRFC822MailMember objectClass # # Cache set 2 is used for Sieve lookups proxyattrset 2 messageStoreUserFilter objectClass # Set_# ttl neg-ttl # # SMD: proxytemplate (|(mailLocalAddress=)(objectClass=)) 0 900 120 proxytemplate (|(mailRoutingAddress=)(objectClass=)) 0 900 120 proxytemplate (&(objectClass=)(mailRoutingAddress=)) 1 900 120 proxytemplate (mailRoutingAddress=) 2 900 120 # # AD: proxytemplate (|(mail=)(proxyAddresses=)(userPrincipalName=)(objectClass=)) 0 900 120 # # Domino: proxytemplate (|(mail=)(uid=)(&(uid=)(mailDomain=))) 0 900 120 # # Novell: proxytemplate (mail=) 0 900 120 # # Netscape/ iPlanet / SunOne / Fedora proxytemplate (|(mail=)(mailAlternateAddress=)(objectClass=)) 0 900 120 proxytemplate (&(objectClass=)(|(mail=)(mailAlternateAddress=))) 1 900 120 proxytemplate (|(mail=)(mailAlternateAddress=)(objectClass=)) 2 900 120 ###################################################################### # BDB Settings # Proxycache uses BDB to store its local information ###################################################################### directory /var/example/ldap/smd-proxycache dbconfig set_cachesize 0 8388608 0 dbconfig set_lg_max 10485760 dbconfig set_flags db_log_autoremove dbconfig set_flags DB_TXN_NOSYNC dbconfig set_lg_bsize 2097152 cachesize 10000 idlcachesize 10100 cachefree 20 dbnosync index queryid,objectClass,mail,mailLocalAddress,mailRoutingAddress eq index uid,mailDomain,userPrincipalName,proxyAddresses,mailAlternateAddress eq ###################################################################### # TLS information, required to enable TLS and LDAPS connections ###################################################################### TLSCipherSuite ALL:!EXP:!LOW:!ADH:@STRENGTH TLSCertificateFile /etc/mail/openldap/ssl/certs/ashish.smi.example.com/default.crt TLSCertificateKeyFile /etc/mail/openldap/ssl/certs/ashish.smi.example.com/default.key --------------040702090906080801050004--

1 0

Re: (ITS#4707) patch: option to bind client socket to an address
by rtsai＠ironport.com 18 Oct '06

18 Oct '06

On Wed, Oct 18, 2006 at 12:23:03PM -0700, Kurt D. Zeilenga wrote: > At 12:12 PM 10/18/2006, hyc(a)symas.com wrote: > >rtsai(a)ironport.com wrote: > >> Full_Name: Robert Tsai > >> Version: 2.3.27 > >> OS: FreeBSD 6.1-RELEASE > >> URL: ftp://ftp.openldap.org/incoming/openldap-2.3.27-bindaddr.patch.txt > >> Submission from: (NULL) (63.251.108.100) > >> > >> This is a patch that provides a mechanism to bind the LDAP client > >> connection to a desired address via ldap_set_option(..., > >> LDAP_OPT_BINDADDR). This call saves some state which is then used > >> by ldap_int_prepare_socket to bind the socket before opening a > >> connection to the LDAP server. > > > >I can't think of any good reason to need such a feature. Can you > >give some background on why anyone would use it? > > I can see a few cases where a client might want to bind the local > address, for instance, to ensure use of a particular network > interface. Yes, this was the need addressed by the patch. The client host has two interfaces to two separate networks. We could have simply configured a host route to the LDAP server, but that would have been too coarse-grained (all traffic, instead of just the LDAP traffic). > However, my concern with patch is one of the approach taken to > accommodate this binding. I'm thinking it might be better to > provide an alternative to ldap_initialize(3) which takes a connected > descriptor instead of a URL. Then calling program can do whatever > it pleases before its used by slapd(8). That would probably work for me. Contributing from the "outside", I was hesitant to propose adding another interface into the library, so I just crammed it into ldap_set_option :). -- Robert Tsai | IronPort Systems | http://www.ironport.com/ | 650-989-2063

1 0

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs