openldap-bugs

openldap-bugs@openldap.org

1 participants
20965 discussions

Re: (ITS#5409) paged results invalid sizelimit error
by pshao＠zimbra.com 16 Mar '08

16 Mar '08

------=_Part_3887_1760018525.1205731478084 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit I did more testing and found there is more to the bug, page size is not the only factor. The number of results in the next search also matters. However, in all cases, if JNDI connection pooling is disabled, then it always works. Test description: create 10000 account: user-1 to user-10000 search size limit = 5000 test 1: step 1. search with filter (uid=*user*): expect SizeLimitExceededException step 2. search with filter (uid=*user-9999*): expect 1 entry test 2: step 1. search with filter (uid=*user*): expect SizeLimitExceededException step 2. search with filter (uid=*user-10*): expect 112 entries With Connection Pooling ======================= (A) page size = factor of 5000, e.g. 1000, 2500 test 1: fails intermittently, step 2 got SizeLimitExceededException while 1 entry is expected test 2: fails intermittently, step 2 got SizeLimitExceededException while 112 entry is expected (B) page size = not factor of 5000, e.g. 999, 2000 test 1: always works test 2: fails intermittently, step 2 got SizeLimitExceededException while 112 entry is expected Without Connection Pooling ========================== test 1, test 2 always work, regardless of page size. Apparently the page size and result size of test 1 in (B) somehow mask the real bug, which has to do with connection reuse. Updated test java program attached. Thanks, phoebe ------=_Part_3887_1760018525.1205731478084 Content-Type: application/octet_stream; name=TestPagedControl.java Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=TestPagedControl.java package com.zimbra.sandbox; import java.io.IOException; import java.util.ArrayList; import java.util.Hashtable; import java.util.List; import javax.naming.Context; import javax.naming.NameNotFoundException; import javax.naming.NamingEnumeration; import javax.naming.NamingException; import javax.naming.SizeLimitExceededException; import javax.naming.directory.Attribute; import javax.naming.directory.Attributes; import javax.naming.directory.DirContext; import javax.naming.directory.InvalidSearchFilterException; import javax.naming.directory.SearchControls; import javax.naming.directory.SearchResult; import javax.naming.ldap.Control; import javax.naming.ldap.InitialLdapContext; import javax.naming.ldap.LdapContext; import javax.naming.ldap.PagedResultsControl; import javax.naming.ldap.PagedResultsResponseControl; /* README: Intermittently if the max entries requested is exact multiple of page size, after a paged search hits SizeLimitExceededException, the next search also always throws a SizeLimitExceededException, even when it should not. e.g. if max requested is 5000, then page size of: 1000 => bad 2500 => bad 999 => good 2000 => good - create 10100 user entries in OpenLDAP: with uid user-0 to user-10100 - Perform the following steps in a loop 1. search (uid=*user*) with limit 5000 and paged control, expecting SizeLimitExceededException 2. search (uid=*user-10001*) with limit 5000 and paged control, expecting 1 entry returned - observed that if the limit is multiple of page size (e.g. 1000, 2500), the test failed at the second iteration when it performs 2 - it throws SizeLimitExceededException when it should find one entry. OpenLDAP returns err=4: conn=262 op=8 SRCH base="" scope=2 deref=3 filter="(uid=*user-10001*)" conn=262 op=8 SRCH attr=uid conn=262 op=8 SEARCH RESULT tag=101 err=4 nentries=0 text= otherwise (e.g. when page size is 999, or 123, or 2000) the test passes. Example test runs: --------------------------------------------------- >java TestPagedControl 1000 Testing page size 1000 iter 0 iter 1 javax.naming.SizeLimitExceededException: [LDAP: error code 4 - Sizelimit Exceeded]; remaining name '' at com.sun.jndi.ldap.LdapCtx.mapErrorCode(LdapCtx.java:3037) at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2931) at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2737) at com.sun.jndi.ldap.LdapCtx.searchAux(LdapCtx.java:1808) at com.sun.jndi.ldap.LdapCtx.c_search(LdapCtx.java:1731) at com.sun.jndi.toolkit.ctx.ComponentDirContext.p_search(ComponentDirContext.java:368) at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:338) at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:321) at javax.naming.directory.InitialDirContext.search(InitialDirContext.java:248) at TestPagedControl.testPagedControl(TestPagedControl.java:81) at TestPagedControl.doTest(TestPagedControl.java:127) at TestPagedControl.main(TestPagedControl.java:140) --------------------------------------------------- >java TestPagedControl 999 Testing page size 999 iter 0 iter 1 iter 2 iter 3 iter 4 iter 5 iter 6 iter 7 iter 8 iter 9 pass */ public class TestPagedControl { static byte[] getCookie(LdapContext lctxt) throws NamingException { Control[] controls = lctxt.getResponseControls(); if (controls != null) { for (int i = 0; i < controls.length; i++) { if (controls[i] instanceof PagedResultsResponseControl) { PagedResultsResponseControl prrc = (PagedResultsResponseControl)controls[i]; return prrc.getCookie(); } } } return null; } static List<String> testPagedControl(String query, int maxResults, int pageSize, boolean useConnPool) throws Exception { List<String> results = new ArrayList<String>(); // int total = 0; DirContext ctxt = null; try { Hashtable<String, String> env = new Hashtable<String, String>(); env.put(Context.INITIAL_CONTEXT_FACTORY, "com.sun.jndi.ldap.LdapCtxFactory"); env.put(Context.PROVIDER_URL, "ldap://localhost:389"); env.put(Context.SECURITY_AUTHENTICATION, "simple"); env.put(Context.SECURITY_PRINCIPAL, "uid=zimbra,cn=admins,cn=zimbra"); env.put(Context.SECURITY_CREDENTIALS, "zimbra"); env.put(Context.REFERRAL, "follow"); env.put("com.sun.jndi.ldap.connect.timeout", "30000"); env.put("com.sun.jndi.ldap.read.timeout", "30000"); if (useConnPool) env.put("com.sun.jndi.ldap.connect.pool", "true"); else env.put("com.sun.jndi.ldap.connect.pool", "false"); ctxt = new InitialLdapContext(env, null); String base = ""; String[] returnAttrs = {"uid"}; SearchControls searchControls = new SearchControls(SearchControls.SUBTREE_SCOPE, maxResults, 0, returnAttrs, false, false); byte[] cookie = null; LdapContext lctxt = (LdapContext)ctxt; NamingEnumeration ne = null; try { do { lctxt.setRequestControls(new Control[]{new PagedResultsControl(pageSize, cookie, Control.CRITICAL)}); ne = ctxt.search(base, query, searchControls); while (ne != null && ne.hasMore()) { // total++; SearchResult sr = (SearchResult) ne.nextElement(); String dn = sr.getNameInNamespace(); results.add(dn); } cookie = getCookie(lctxt); } while (cookie != null); } finally { if (ne != null) ne.close(); } } catch (InvalidSearchFilterException e) { throw e; } catch (NameNotFoundException e) { throw e; } catch (SizeLimitExceededException e) { throw e; } catch (NamingException e) { throw e; } catch (IOException e) { throw e; } finally { ctxt.close(); } return results; } static void test1(int pageSize, boolean useConnPool) throws Exception { /* * first search: expecting SizeLimitExceededException */ boolean good = false; try { testPagedControl("(uid=*user*)", 5000, pageSize, useConnPool); } catch (SizeLimitExceededException e) { good = true; } if (!good) throw new Exception("expecting SizeLimitExceededException"); List<String> results; /* * second search: expecting one result */ results = testPagedControl("(uid=*user-9999*)", 5000, pageSize, useConnPool); // for (String s : results) System.out.println(s); if (results.size() != 1) throw new Exception("expecting 1 result, got " + results.size()); } static void test2(int pageSize, boolean useConnPool) throws Exception { /* * first search: expecting SizeLimitExceededException */ boolean good = false; try { testPagedControl("(uid=*user*)", 5000, pageSize, useConnPool); } catch (SizeLimitExceededException e) { good = true; } if (!good) throw new Exception("expecting SizeLimitExceededException"); List<String> results; /* * second search: expecting 112 results */ results = testPagedControl("(uid=*user-10*)", 5000, pageSize, useConnPool); // for (String s : results) System.out.println(s); if (results.size() != 112) throw new Exception("expecting 112 result, got " + results.size()); } private static void usage() { System.out.println("usage: TestPagedControl <page size> <whcih test 1|2> <use conn pool: true|false>"); } public static void main(String[] args) { if (args.length != 3) { usage(); System.exit(1); } int pageSize = Integer.parseInt(args[0]); int test = Integer.parseInt(args[1]); boolean useConnPool = Boolean.parseBoolean(args[2]); System.out.println("Testing page size " + pageSize); try { for (int i=0; i<10; i++) { System.out.println(" iter " + i); if (test == 1) test1(pageSize, useConnPool); else if (test == 2) test2(pageSize, useConnPool); else { usage(); System.exit(1); } } System.out.println("pass"); } catch (Exception e) { e.printStackTrace(); } } } ------=_Part_3887_1760018525.1205731478084--

1 0

Re: (ITS#5412) Add operation should not require naming attrs
by h.b.furuseth＠usit.uio.no 16 Mar '08

16 Mar '08

ando(a)sys-net.it writes: >> slapadd: tools.c:494: bdb_tool_entry_put: Assertion `text->bv_val[0] == >> '\0'' failed. > > I only tested with ldapadd, but at a glance I don't see what could be the > difference. The protocol Add operation doesn't run into that assert. Have only looked briefly at the code, but I guess entry_schema_check()'s params textbuf[0] (and textlen?) must be zeroed if they were filled with a message but entry_schema_check() returns success anyway. Don't know where (in which function) that should be done though. -- Hallvard

1 0

Re: (ITS#5412) Add operation should not require naming attrs
by ando＠sys-net.it 16 Mar '08

16 Mar '08

> slapadd: tools.c:494: bdb_tool_entry_put: Assertion `text->bv_val[0] == > '\0'' failed. I only tested with ldapadd, but at a glance I don't see what could be the difference. Unfortunately I won't be able to work at it/connect for a while (a week or so); I ran the whole test suite and this fix shouldn't break any existing test, in the meanwhile. p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati(a)sys-net.it ---------------------------------------

1 0

Re: (ITS#5412) Add operation should not require naming attrs
by h.b.furuseth＠usit.uio.no 16 Mar '08

16 Mar '08

Pierangelo Masarati writes: > Patched in HEAD; please test. p. test.conf: include servers/slapd/schema/core.schema database bdb suffix o=foo directory /tmp/db test.ldif: dn: o=foo objectClass: organization $ servers/slapd/slapadd -f test.conf < test.ldif ... slapadd: tools.c:494: bdb_tool_entry_put: Assertion `text->bv_val[0] == '\0'' failed. -- Hallvard

1 0

Re: (ITS#5412) Add operation should not require naming attrs
by ando＠sys-net.it 15 Mar '08

15 Mar '08

h.b.furuseth(a)usit.uio.no wrote: > slapd and slapadd should generate them if they are missing. Patched in HEAD; please test. p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati(a)sys-net.it ---------------------------------------

1 0

Re: (ITS#5412) Add operation should not require naming attrs
by ando＠sys-net.it 15 Mar '08

15 Mar '08

h.b.furuseth(a)usit.uio.no wrote: > A somewhat belated ITS: ldapadd of > dn: o=foo > objectClass: organization > complains that "naming attribute 'o' is not present in entry". > RFC 2251 required this, but RFC 4511 changed it to the opposite: > The RDN attrs need not be present in AddRequest.attributes. > > slapd and slapadd should generate them if they are missing. I'm patching HEAD code in this sense. For this purpose, I decided to hijack the add_soc parameter of entry_schema_check() that controlled adding the structural objectClass. In fact, in the current code so far I couldn't find any call to that function where the structural objectClass had to be added and the naming attributes hadn't. If this distinction becomes necessary the parameter can be used as a mask instead of a boolean. > I don't know if the change belongs in RE23. re23 claims to comply with rfc 2251-2256, and knows nothing of rfc 4510-4519, so IMHO it shouldn't be affected. p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati(a)sys-net.it ---------------------------------------

1 0

(ITS#5428) Enhancement: allow slapo-refint(5) to implement global referential integrity
by ando＠sys-net.it 15 Mar '08

15 Mar '08

Full_Name: Pierangelo Masarati Version: HEAD OS: irrelevant URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (82.63.140.131) Submitted by: ando See <http://www.openldap.org/lists/openldap-technical/200803/msg00068.html> for discussion. p.

1 0

(ITS#5427) slapo-refint(5) erroneously uses smr_str field for dnSubtreeMatch
by ando＠sys-net.it 15 Mar '08

15 Mar '08

Full_Name: Pierangelo Masarati Version: HEAD/re24 OS: irrelevant URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (82.63.140.131) Submitted by: ando The right field is smr_bvoid (or smr_names[0]). p.

1 0

(ITS#5426) syncrepl and syncprov clash on contextCSN updates
by hyc＠OpenLDAP.org 14 Mar '08

14 Mar '08

Full_Name: Howard Chu Version: 2.4/HEAD OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (76.91.220.157) Submitted by: hyc If syncprov-checkpoint is configured on a database with a syncrepl consumer (either for cascaded syncrepl or for mirrormode), the syncrepl consumer may fail to update the contextCSN because syncprov updates it first. In 2.4 the syncrepl consumer tries to delete its old value and add the new value, instead of just doing a modify/Replace as it used to in 2.3. The 2.4 consumer was changed in this respect because multiple consumers are now allowed on a database, thus using multiple values in the contextCSN attribute. Replacing the entire attribute when only a single value needed to change seemed like the wrong thing to do. The syncprov overlay still just uses a modify/Replace, so it doesn't have this problem. Since both syncrepl and syncprov now currently track all of the contextCSNs in use, probably syncrepl should just be switched back to using modify/Replace, with the full list of known contextCSNs.

1 0

Re: Adding new Attribute in OpenLdap (ITS#5423)
by ando＠sys-net.it 14 Mar '08

14 Mar '08

vijayaragavanp(a)zenithsoft.com wrote: > i was't able to find the manual ,can you please send me the link Please send software usage requests to the OpenLDAP-software mailing list. The ITS is to track software bug. p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati(a)sys-net.it ---------------------------------------

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs