openldap-devel November 2010

openldap-devel@openldap.org

8 participants
7 discussions

OpenLDAP git repo?
by Volker Lendecke 29 Mar '11

29 Mar '11

Hi! Out of several rumors I've heard that OpenLDAP converted it source repo to git. Is that true? www.openldap.org still speaks about cvs. If you changed to git, is the repo publically available? Thanks, Volker

5 12

librewrite & co vs. logging & ldap-int.h
by Hallvard B Furuseth 13 Jan '11

13 Jan '11

The ITS#6625 patch causes preprocessor warnings about conflicts with ldap-int.h, so I'm looking again at old ITS#5421 to kill off #include "...libldap/ldap-int.h" These occur in librewrite, rwm, back-ldap and back-meta. The #include is for two things: Expose struct ldapmsg & LDAP_FREE, the need of which I've just removed in HEAD, and set up libldap logging. I expect the slapd components just as well can drop the #include and use slapd logging? back-*/search.c then need lber-int.h instead. librewrite still needs the #define ldap_debug in ldap-int.h. Instead we could either: (1) use liblber logging in librewrite, (2) or throw the code LDAP_INT_IN_KERNEL .. #endif /* LDAP_DEBUG */ out of ldap-int.h to a separate file, either include/ldap_log.h or something like libldap/ldap-int-log.h. This file would not expose structs with different definitions depending on LDAP_R_COMPILE & co. Since the file when not LDAP_INT_IN_KERNEL does need to refer ldap_debug to ldap_int_global_options.ldo_debug, that can be done with: /* This struct starts with the same members as ldapoptions */ struct ldapoptions_prefix { short ldo_valid; int ldo_debug; }; #define ldap_debug \ (*(int *) ((char *) &ldap_int_global_options \ + offsetof(struct ldapoptions_prefix, ldo_debug))) struct ldapoptions; LDAP_V ( struct ldapoptions ) ldap_int_global_options; Opinions? Put it in ldap-int.h, or ldap_log.h? Maybe the latter only if #defined LDAP_LIBRARY_LOGGING (defined by the user)? -- Hallvard

2 1

Re: (ITS#6710) Mods already refreshed on a forwarding server is lost by its consumer
by Rein Tollevik 22 Nov '10

22 Nov '10

[I have switched this thread from -its to -devel] On 11/22/10 1:31 , Howard Chu wrote: > rein(a)OpenLDAP.org wrote: >> On 11/15/10 16:39 , rein(a)OpenLDAP.org wrote: >>> New syncprov consumers connecting to a forwarding server and >>> presenting an >>> apperently up-to-date cookie will loose any mods that have already >>> taken place >>> on the forwarding server if it itself is refreshing from its >>> provider. This >>> should not be a problem if the forwarding server have a sufficiently >>> large >>> syncprov log, but a fix for servers without is coming. >> The currently committed fix for this its leaves one problem open. If a >> forwarding server restarts in the middle of the refresh phase, after >> making some changes but before updating the csn, new consumers >> connecting with an apparently up to date csn set after the server comes >> up again will not know that the context is dirty and will loose these >> changes. The same problem arise if the server restarts between the time >> a locally initiated delete operation is performed in the database and >> the accompanying csn set is saved. >> >> A fix could be to always assume the context is dirty after start up, and >> thereby forcing all clients to undergo the refresh phase even if they >> are in sync until some operation that updates the csn set is performed. >> An unnecessary refresh is probably better than loosing changes.. > > I haven't had the opportunity to review these patches yet, but the bug > description sounds a little flaky to me. This ITS as well as the previous should probably have mentioned that they are related to refreshAndPersist replication. > The original design is this: when a consumer requests a refresh from the > provider, the provider uses a snapshot of its current contextCSN. All > changes from the consumer's cookie to this snapshot (inclusive) are sent > to the consumer. Any changes currently in progress that have not yet > been committed will be skipped until the next refresh. Nothing is lost, > it's simply delayed, and that's in accordance with syncrepl's "eventual > convergence" model. This is true for the refresh only configuration, I guess these problems don't show up there as they will be fixed during the next refresh. But if the persist phase is entered without a proper refresh first then the consumer have forever lost the missing changes. Or well, until the admin forces a full refresh that is. > Likewise, the provider only updates its contextCSN when a change is > fully committed. syncprov should NOT need to defer any consumer while it > has outstanding mods. There is no reason that would be needed by the > sync protocol. This ITS does not delay the consumer, it forces a refresh even if the consumer appear to be in sync with the provider (from looking at the CSNs of the provider and consumer). ITS#6709 introduce a delay, this is not dictated by the protocol but by the implementation. syncprov_matchops() decide which consumers the change should be sent to before the change is actually made to the db. This leaves a window where a consumer may connect and apparently be in sync, enter its persistent phase but not receive the changes that was active when it connected. Waiting for this active change to complete before deciding whether the consumer is in sync or not closes this window. > Servers only update their contextCSN after an entire refresh has > completed. If a downstream consumer connects while a forwarder is still > refreshing, the consumer should receive nothing. (Or, it should only > receive the changes between its cookie and the server's committed > contextCSN, if any.) The consumer connects and appear to be in sync with the provider, therefore the provider initiates the persistent phase immediately without any refresh first. Without the fix in this ITS syncprov had no idea that there was uncommitted changes (i.e changes not accompanied by a CSN update) within its context, changes that requires the refresh phase to be performed even when an consumer that appear to be in sync connects. And an restart of the provider at the inappropriate time can make it loose this knowledge again as I outlined in my first reply. Yes, in refresh only mode the server sends the changes between the cookie the consumer presented and the servers committed csn. But in the refresh phase of refreshAndPersistent mode it sends all changes newer than the consumers csn. Which opens the windows where the ITS#6709 and ITS#6710 cases can loose modifications. > You may very well have found bugs in the implementation, but it sounds > to me like you've changed the overall behavior to something outside of > the original design. No, I don't think I have changed the design, if it was changed then it was the persistent mode that did it.. Rein

1 0

tab-width 4 and line length < 80, please
by Hallvard B Furuseth 22 Nov '10

22 Nov '10

A reminder: The OpenLDAP source code uses tab-width 4 and generally lines < 80 chars. Please commit code written the same way, except in files that are already written with tab-width 8. Some people have been committing with tab-width 8, so the code now contains a mixture of tab-width 4 and 8, but it still mostly looks better with the original setting. Maybe we'll reformat the entire source tree someday, but that's not happened yet. To take a simple example, look at #defines and structs in ldap.h. At the top, someone has been using tab-width 8, but further down we get: [Displayed with tab-width 8] typedef struct ldapcontrol { char * ldctl_oid; /* numericoid of control */ struct berval ldctl_value; /* encoded value of control */ char ldctl_iscritical; /* criticality */ } LDAPControl; [Displayed with tab-width 4] typedef struct ldapcontrol { char * ldctl_oid; /* numericoid of control */ struct berval ldctl_value; /* encoded value of control */ char ldctl_iscritical; /* criticality */ } LDAPControl; One can write code which displays OK both ways, if anyone prefers that: * Do not use tabs after non-whitespace. - Put comments above instead of to the right of code. - Do not line up variables in declarations, use a single space. * Or: Align with with spaces instead of tabs. But in that case, only align on lines that have the same indentation. I.e. not if (foo) /* hi there */ bar(); /* ho there */ -- Hallvard

1 0

test061
by Rein Tollevik 21 Nov '10

21 Nov '10

I have finally found some time to look at openldap stuff again, and have been implementing the changes required to fix the final failures in test058. But to my annoyance the race tests continued to fail :-( It turns out that in addition to exercise the race condition outlined in test058 it managed to fall into a stream of other race conditions. These races are not related to the test058 configuration but to the initiation of the syncrepl refresh phase. The new test061 runs an extended version of the race test from test058, but in a simplified configuration. So far I have found and (hopefully) fixed 4 race conditions, and seen the glimpse of 2 or 3 more problems. ITSes and fixes are coming, but as of now you should expect test061 to fail occasionally. Rein

3 2

use of alloca()
by Howard Chu 05 Nov '10

05 Nov '10

A lot of what sl_malloc() is used for was really intended for a function that behaves like alloca(). What problems are we signing up for by allowing use of alloca()? -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

3 3

Info about statically compiled overlays and backends
by Ralf Haferkamp 01 Nov '10

01 Nov '10

Hi, I'd like slapd to be able to report the overlays and backends that are statically compiled into the binary. I am currently thinking about adding that to the -V option, e.g. if slapd is called with -VVV (-VV is already taken it seem) it lists the statically included overlays and backends additionally to the Version Information. Does that make sense or should be introduce another argument for that? Ralf

3 3

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-devel November 2010