OpenLDAP git repo?
by Volker Lendecke
Hi!
Out of several rumors I've heard that OpenLDAP converted it
source repo to git. Is that true? www.openldap.org still
speaks about cvs. If you changed to git, is the repo
publically available?
Thanks,
Volker
12 years, 5 months
librewrite & co vs. logging & ldap-int.h
by Hallvard B Furuseth
The ITS#6625 patch causes preprocessor warnings about conflicts
with ldap-int.h, so I'm looking again at old ITS#5421 to kill off
#include "...libldap/ldap-int.h"
These occur in librewrite, rwm, back-ldap and back-meta.
The #include is for two things: Expose struct ldapmsg & LDAP_FREE, the
need of which I've just removed in HEAD, and set up libldap logging.
I expect the slapd components just as well can drop the #include and
use slapd logging? back-*/search.c then need lber-int.h instead.
librewrite still needs the #define ldap_debug in ldap-int.h.
Instead we could either:
(1) use liblber logging in librewrite,
(2) or throw the code LDAP_INT_IN_KERNEL .. #endif /* LDAP_DEBUG */
out of ldap-int.h to a separate file, either include/ldap_log.h or
something like libldap/ldap-int-log.h. This file would not expose
structs with different definitions depending on LDAP_R_COMPILE & co.
Since the file when not LDAP_INT_IN_KERNEL does need to refer
ldap_debug to ldap_int_global_options.ldo_debug, that can be done with:
/* This struct starts with the same members as ldapoptions */
struct ldapoptions_prefix {
short ldo_valid;
int ldo_debug;
};
#define ldap_debug \
(*(int *) ((char *) &ldap_int_global_options \
+ offsetof(struct ldapoptions_prefix, ldo_debug)))
struct ldapoptions;
LDAP_V ( struct ldapoptions ) ldap_int_global_options;
Opinions? Put it in ldap-int.h, or ldap_log.h? Maybe the latter
only if #defined LDAP_LIBRARY_LOGGING (defined by the user)?
--
Hallvard
12 years, 8 months
Re: (ITS#6710) Mods already refreshed on a forwarding server is lost by its consumer
by Rein Tollevik
[I have switched this thread from -its to -devel]
On 11/22/10 1:31 , Howard Chu wrote:
> rein(a)OpenLDAP.org wrote:
>> On 11/15/10 16:39 , rein(a)OpenLDAP.org wrote:
>>> New syncprov consumers connecting to a forwarding server and
>>> presenting an
>>> apperently up-to-date cookie will loose any mods that have already
>>> taken place
>>> on the forwarding server if it itself is refreshing from its
>>> provider. This
>>> should not be a problem if the forwarding server have a sufficiently
>>> large
>>> syncprov log, but a fix for servers without is coming.
>> The currently committed fix for this its leaves one problem open. If a
>> forwarding server restarts in the middle of the refresh phase, after
>> making some changes but before updating the csn, new consumers
>> connecting with an apparently up to date csn set after the server comes
>> up again will not know that the context is dirty and will loose these
>> changes. The same problem arise if the server restarts between the time
>> a locally initiated delete operation is performed in the database and
>> the accompanying csn set is saved.
>>
>> A fix could be to always assume the context is dirty after start up, and
>> thereby forcing all clients to undergo the refresh phase even if they
>> are in sync until some operation that updates the csn set is performed.
>> An unnecessary refresh is probably better than loosing changes..
>
> I haven't had the opportunity to review these patches yet, but the bug
> description sounds a little flaky to me.
This ITS as well as the previous should probably have mentioned that
they are related to refreshAndPersist replication.
> The original design is this: when a consumer requests a refresh from the
> provider, the provider uses a snapshot of its current contextCSN. All
> changes from the consumer's cookie to this snapshot (inclusive) are sent
> to the consumer. Any changes currently in progress that have not yet
> been committed will be skipped until the next refresh. Nothing is lost,
> it's simply delayed, and that's in accordance with syncrepl's "eventual
> convergence" model.
This is true for the refresh only configuration, I guess these problems
don't show up there as they will be fixed during the next refresh. But
if the persist phase is entered without a proper refresh first then the
consumer have forever lost the missing changes. Or well, until the
admin forces a full refresh that is.
> Likewise, the provider only updates its contextCSN when a change is
> fully committed. syncprov should NOT need to defer any consumer while it
> has outstanding mods. There is no reason that would be needed by the
> sync protocol.
This ITS does not delay the consumer, it forces a refresh even if the
consumer appear to be in sync with the provider (from looking at the
CSNs of the provider and consumer).
ITS#6709 introduce a delay, this is not dictated by the protocol but by
the implementation. syncprov_matchops() decide which consumers the
change should be sent to before the change is actually made to the db.
This leaves a window where a consumer may connect and apparently be in
sync, enter its persistent phase but not receive the changes that was
active when it connected. Waiting for this active change to complete
before deciding whether the consumer is in sync or not closes this window.
> Servers only update their contextCSN after an entire refresh has
> completed. If a downstream consumer connects while a forwarder is still
> refreshing, the consumer should receive nothing. (Or, it should only
> receive the changes between its cookie and the server's committed
> contextCSN, if any.)
The consumer connects and appear to be in sync with the provider,
therefore the provider initiates the persistent phase immediately
without any refresh first. Without the fix in this ITS syncprov had no
idea that there was uncommitted changes (i.e changes not accompanied by
a CSN update) within its context, changes that requires the refresh
phase to be performed even when an consumer that appear to be in sync
connects. And an restart of the provider at the inappropriate time can
make it loose this knowledge again as I outlined in my first reply.
Yes, in refresh only mode the server sends the changes between the
cookie the consumer presented and the servers committed csn. But in the
refresh phase of refreshAndPersistent mode it sends all changes newer
than the consumers csn. Which opens the windows where the ITS#6709 and
ITS#6710 cases can loose modifications.
> You may very well have found bugs in the implementation, but it sounds
> to me like you've changed the overall behavior to something outside of
> the original design.
No, I don't think I have changed the design, if it was changed then it
was the persistent mode that did it..
Rein
12 years, 10 months
tab-width 4 and line length < 80, please
by Hallvard B Furuseth
A reminder:
The OpenLDAP source code uses tab-width 4 and generally lines < 80
chars. Please commit code written the same way, except in files
that are already written with tab-width 8.
Some people have been committing with tab-width 8, so the code now
contains a mixture of tab-width 4 and 8, but it still mostly looks
better with the original setting. Maybe we'll reformat the entire
source tree someday, but that's not happened yet.
To take a simple example, look at #defines and structs in ldap.h. At
the top, someone has been using tab-width 8, but further down we get:
[Displayed with tab-width 8]
typedef struct ldapcontrol {
char * ldctl_oid; /* numericoid of control */
struct berval ldctl_value; /* encoded value of control */
char ldctl_iscritical; /* criticality */
} LDAPControl;
[Displayed with tab-width 4]
typedef struct ldapcontrol {
char * ldctl_oid; /* numericoid of control */
struct berval ldctl_value; /* encoded value of control */
char ldctl_iscritical; /* criticality */
} LDAPControl;
One can write code which displays OK both ways, if anyone prefers that:
* Do not use tabs after non-whitespace.
- Put comments above instead of to the right of code.
- Do not line up variables in declarations, use a single space.
* Or: Align with with spaces instead of tabs. But in that case,
only align on lines that have the same indentation. I.e. not
if (foo) /* hi there */
bar(); /* ho there */
--
Hallvard
12 years, 10 months
test061
by Rein Tollevik
I have finally found some time to look at openldap stuff again, and have
been implementing the changes required to fix the final failures in
test058. But to my annoyance the race tests continued to fail :-( It
turns out that in addition to exercise the race condition outlined in
test058 it managed to fall into a stream of other race conditions.
These races are not related to the test058 configuration but to the
initiation of the syncrepl refresh phase.
The new test061 runs an extended version of the race test from test058,
but in a simplified configuration. So far I have found and (hopefully)
fixed 4 race conditions, and seen the glimpse of 2 or 3 more problems.
ITSes and fixes are coming, but as of now you should expect test061 to
fail occasionally.
Rein
12 years, 10 months
Info about statically compiled overlays and backends
by Ralf Haferkamp
Hi,
I'd like slapd to be able to report the overlays and backends that are
statically compiled into the binary. I am currently thinking about adding
that to the -V option, e.g. if slapd is called with -VVV (-VV is already
taken it seem) it lists the statically included overlays and backends
additionally to the Version Information. Does that make sense or should
be introduce another argument for that?
Ralf
12 years, 10 months
