September 2007 - openldap-bugs

Re: (ITS#5114) pcache cache results for searches that hit size/timelimit

by ando＠sys-net.it

ando(a)sys-net.it wrote: Ralf, This patch should now work as expected. <http://www.sys-net.it/~ando/Download/pcache_sizelimit.patch> Does it harmonize with your improvements? Or, if you prefer, first commit your changes, and I'll harmonize the rest. p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati(a)sys-net.it ---------------------------------------

15 years, 9 months

1
0
0 / 0

Re: (ITS#5114) pcache cache results for searches that hit size/timelimit

by ando＠sys-net.it

Howard Chu wrote: > If the client provides a sizelimit, save that away. Forward the request > with no sizelimit, so the cache can see everything. > > If the forwarded request hits a sizelimit, I think we can still use the > result. While there's no guarantee that repeated attempts to search the > remote server would return the exact same set of entries, there's also > no harm done if the cache does so. > > But if the result exceeds the cache's sizelimit, the result set must be > uncached, same as now. Well, I don't quite agree about this. In fact, if we know in advance that the cache can only cache up to <entry_limit>, removing the client requested size limit might lead to waste of resources, because a search could then potentially return much many entries than <entry_limit>, which wouldn't be cached nor returned to the client, thus defeating the purpose of caching. So if the client requests a sizelimit <client_sl>, we should: if <client_sl> > <entry_limit>, leave it in place. If less than <entry_limit> entries result, fine; if less than <client_sl> entries result, fine but don't cache; otherwise, return LDAP_SIZELIMIT_EXCEEDED. if <client_sl> < <entry_limit>, set size limit to <entry_limit>. If less than <client_sl> entries result, fine; if less than <entry_limit> entries result, return LDAP_SIZE_LIMIT, but keep caching; otherwise, don't cache at all. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati(a)sys-net.it ---------------------------------------

15 years, 9 months

1
0
0 / 0

Re: (ITS#5114) pcache cache results for searches that hit size/timelimit

by ando＠sys-net.it

hyc(a)symas.com wrote: > Can you spell out again what behavior you're aiming for? > > I think what makes sense so far is: > If the client provides a sizelimit, save that away. Forward the request with no > sizelimit, so the cache can see everything. OK. > If the forwarded request hits a sizelimit, I think we can still use the result. In principle, yes. But not with current query containment, because if the sizelimit is hit, say, with mail=*(a)example.com, then current query containment would indicate that mail=foo(a)example.com is contained in the sizelimit-hit query, but if mail=foo(a)example.com exists but is not contained in the subset of mail=*(a)example.com that is cached, nothing would be returned. > While there's no guarantee that repeated attempts to search the remote server > would return the exact same set of entries, there's also no harm done if the > cache does so. > > But if the result exceeds the cache's sizelimit, the result set must be > uncached, same as now. Yes, but the query would be cached (as failing), and this would make query containment behave as above. I think I'm missing where query containment can be modified to not check whether a query is contained in that failing because sizelimit. p. p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati(a)sys-net.it ---------------------------------------

15 years, 9 months

1
0
0 / 0

Re: (ITS#5114) pcache cache results for searches that hit size/timelimit

by hyc＠symas.com

ando(a)sys-net.it wrote: > ando(a)sys-net.it wrote: > >> Please disregard; there's still a couple of issues that I wasn't aware >> and that need to be dealt with. Actually, caching those results may >> make further requests that look compatible with this to erroneously use >> that dataset. > > The issue here is that a search exceeding sizelimit, if not cached, > would destroy cacheability of all searches contained in it. Since a > search that could exceed sizelimit is likely to include substrings or > so, things like (mail=*domain.com) exceeding sizelimit would make all > searches for (mail=foo(a)domain.com) non cacheable. I fear there's little > to do about this, unless we want to heavily rework query containment > stuff (and I don't feel like, as I'm having hard times in understanding > what that code does). So I have now a bottomline fix that handles > sizelimit correctly, but makes all cacheable queries contained in the > offending one no longer answerable. Can you spell out again what behavior you're aiming for? I think what makes sense so far is: If the client provides a sizelimit, save that away. Forward the request with no sizelimit, so the cache can see everything. If the forwarded request hits a sizelimit, I think we can still use the result. While there's no guarantee that repeated attempts to search the remote server would return the exact same set of entries, there's also no harm done if the cache does so. But if the result exceeds the cache's sizelimit, the result set must be uncached, same as now. -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

15 years, 9 months

1
0
0 / 0

Re: (ITS#5114) pcache cache results for searches that hit size/timelimit

by ando＠sys-net.it

ando(a)sys-net.it wrote: > Please disregard; there's still a couple of issues that I wasn't aware > and that need to be dealt with. Actually, caching those results may > make further requests that look compatible with this to erroneously use > that dataset. The issue here is that a search exceeding sizelimit, if not cached, would destroy cacheability of all searches contained in it. Since a search that could exceed sizelimit is likely to include substrings or so, things like (mail=*domain.com) exceeding sizelimit would make all searches for (mail=foo(a)domain.com) non cacheable. I fear there's little to do about this, unless we want to heavily rework query containment stuff (and I don't feel like, as I'm having hard times in understanding what that code does). So I have now a bottomline fix that handles sizelimit correctly, but makes all cacheable queries contained in the offending one no longer answerable. p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati(a)sys-net.it ---------------------------------------

15 years, 9 months

1
0
0 / 0

Re: (ITS#5114) pcache cache results for searches that hit size/timelimit

by ando＠sys-net.it

ando(a)sys-net.it wrote: > here's a patch that implements the above. > > <http://www.sys-net.it/~ando/Download/pcache_sizelimit.patch> Please disregard; there's still a couple of issues that I wasn't aware and that need to be dealt with. Actually, caching those results may make further requests that look compatible with this to erroneously use that dataset. p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati(a)sys-net.it ---------------------------------------

15 years, 9 months

1
0
0 / 0

Re: (ITS#5114) pcache cache results for searches that hit size/timelimit

by ando＠sys-net.it

ando(a)sys-net.it wrote: > I'm getting inclined towards handling this case like a regular search, > only replacing LDAP_SUCCESS with LDAP_SIZELIMIT_EXCEEDED. In fact, it > would be more appropriate to make this request not cacheable, because > there's no guarantee the server would return the same entries for > repeated searches, but for caching purposes this shouldn't really matter. Ralf, here's a patch that implements the above. <http://www.sys-net.it/~ando/Download/pcache_sizelimit.patch> Does it harmonize with your improvements? Cheers, p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati(a)sys-net.it ---------------------------------------

15 years, 9 months

1
0
0 / 0

Re: (ITS#5114) pcache cache results for searches that hit size/timelimit

by ando＠sys-net.it

Howard Chu wrote: >> Right. Well, I understand if no limit was hit the request would become >> cacheable. In this case, the approach you suggest would be fine. > > Of course the result size has to also fit within the cache's configured > size limit. We may want to add another type of negative caching record > here for this case "we already know the result set is too large, ignore > the cache DB"... I'm getting inclined towards handling this case like a regular search, only replacing LDAP_SUCCESS with LDAP_SIZELIMIT_EXCEEDED. In fact, it would be more appropriate to make this request not cacheable, because there's no guarantee the server would return the same entries for repeated searches, but for caching purposes this shouldn't really matter. p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati(a)sys-net.it ---------------------------------------

15 years, 9 months

1
0
0 / 0

Re: (ITS#5102) back-perl build issues

by ando＠sys-net.it

hyc(a)symas.com wrote: > quanah(a)zimbra.com wrote: >> In the name of getting 2.4.5 released, I've updated to Kurt's suggestion >> until such a time as this can be acted upon otherwise. > > That's fine. > > But I think renaming config.h to slapconf.h is a good idea anyway. > > Another thing that has nagged me ever since we unified the slap tools into > slapd - should have renamed slapadd.c -> tooladd.c etc... Personally, not. Only, slap* makes them quite personal, while tool* sounds quite generic. It's like calling them db*: could be any type of db (no pun with Berkeley intended :). p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati(a)sys-net.it ---------------------------------------

15 years, 9 months

1
0
0 / 0

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs September 2007