Re: (ITS#7182) back-ldap monitoring improvements
by Ondrej Kuznik
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 02/28/2012 04:26 PM, ondrej.kuznik(a)acision.com wrote:
> I have prepared some patches to back-ldap (and one to back-monitor) that expose
> operation and connection monitoring information from a running back-ldap
> database and I would like to see this or similar functionality included in the
> OpenLDAP codebase.
>
> The url above contains a patchset against HEAD for review.
>
> The following things are yet to be resolved:
> - there is no monitoring of completed operations yet, only operations initiated
> against the remote database(s).
> - the binds performed by the ldap_back_default_rebind function are not counted
> - slapo-chain has not been tested yet
After playing with slapo-chain, looks like the monitoring support has
been a noop and properly enabling it might take a more intrusive patch
than the one included above.
> - test suite integration: back-ldap looks excluded from the test suite
Looking further at the test suite, enabling back-ldap testing might also
be a little more effort, maybe out of scope of such an ITS.
> - better connection handling (connections should have stable identifiers)
So far, I have had no revelation on how to proceed on this one.
Pierangelo and others, could some of you take a look at the proposed
changes and comment on what else should be improved or fixed?
- --
Ondrej Kuznik
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk9XjowACgkQ9GWxeeH+cXvwgwCgo4SGGBhVCYLcx6wcRI3kkXDW
uDkAnROsOqTPeEDVL/tDXnva1sX9yM1Y
=zSS9
-----END PGP SIGNATURE-----
This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you for understanding.
11 years, 8 months
Open Position
by kurt＠OpenLDAP.org
I would like to take this time to welcome you to our hiring process
and give you a brief synopsis of the position's benefits and requirements.
If you are taking a career break, are on a maternity leave,
recently retired or simply looking for some part-time job, this position is for you.
Occupation: Flexible schedule 2 to 8 hours per day. We can guarantee a minimum 20 hrs/week occupation
Salary: Starting salary is 2000 EUR per month plus commission, paid every month.
Business hours: 9:00 AM to 5:00 PM, MON-FRI, 9:00 AM to 1:00 PM SAT or part time (Europe time).
Region: Europe.
Please note that there are no startup fees or deposits to start working for us.
To request an application form, schedule your interview and receive more information about this position
please reply to Terrell(a)jobdayseu.com,with your personal identification number for this position IDNO: 7830
11 years, 8 months
slapmodify and back-config
by Ondrej Kuznik
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi,
for slapmodify to be usable on a database, that database needs to
implement the bi_tool_dn2id_get and bi_tool_entry_modify functions,
which is not the case of the back-config. I've put together a minimal
(and maybe too naive) implementation of these for back-config and
back-ldif here:
ftp://ftp.openldap.org/incoming/Ondrej-Kuznik-20120319-back-config-slapmo...
Currently, there is no validation whether the modification passes
back-config checks since config_modify_internal takes a modify
operation, while bi_tool_entry_modify receives only the new version of
the entry to be modified.
Also, for slapmodify to be really useful, it would have to allow entry
deletion, for which there is no tool mode callback. To implement
deletes, do you think it is better that deletion of an entry with
children fail or delete the entire subtree? While the former seems both
easier to implement and more sensible to me, I would like your know opinion.
The attached file is derived from OpenLDAP Software. All of the
modifications to OpenLDAP Software represented in the following
patch(es) were developed by Acision. Acision has not assigned rights
and/or interest in this work to any party. I, Ondrej Kuznik am
authorized by Acision, my employer, to release this work under the
following terms.
The attached modifications to OpenLDAP Software are subject to the
following notice:
Copyright 2012 Acision
Redistribution and use in source and binary forms, with or without
modification, are permitted only as authorized by the OpenLDAP Public
License.
- --
Ondrej Kuznik
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk9m6z8ACgkQ9GWxeeH+cXuHIgCfdeRlNgd+o3+hcO8a23zPBPIz
D4kAniBVWmDDdD/fubrYKHpmIfUFZ9T/
=9U3w
-----END PGP SIGNATURE-----
This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you for understanding.
11 years, 8 months
MDB, SQLite
by Howard Chu
I'll be presenting some more details at the UKUUG LISA conference in Edinburgh
in 2 days. Just a quick note here...
SQLite 3.7.7.1 (unmodified) took ~22.43 seconds to insert 1000
randomly-ordered records on my laptop (with Samsung SSD). The SQLite modified
to use MDB took only 1.06 seconds.
This 20:1 improvement in write performance is most significant in terms of
absolute efficiency. Given how pervasively SQLite is used in Android and apps
like Firefox, on smartphones, tablets, and other battery-powered devices,
there's a potential to significantly increase the runtimes of these devices by
switching to SQLite+MDB.
Read efficiency is somewhat of a lost cause, >95% of the CPU time in reads is
spent in the SQL parser. Even though MDB is much more efficient at reads than
stock SQLite, the difference is lost in the noise under the parser overhead.
I believe, for the vast majority of situations where SQLite is embedded in
applications, significant efficiency gains would be obtained by using a pure
binary data interface. While a simple key-value interface may be too primitive
for app convenience, an inline implementation of the X.500 data model (with
indexing) may be more useful. I.e., slapd back-mdb turned into an application
library.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
11 years, 8 months
slapd: attr.c:481: attr_merge: Assertion
by Michael Ströder
HI!
What does it mean when slapd aborts with this assertion?
------------------------------------ snip ------------------------------------
slapd: attr.c:481: attr_merge: Assertion `( nvals == ((void *)0) &&
(*a)->a_nvals == (*a)->a_vals ) || ( nvals != ((void *)0) && ( ( (*a)->a_vals
== ((void *)0) && (*a)->a_nvals == ((void *)0) ) || ( (*a)->a_nvals !=
(*a)->a_vals ) ) )' failed.
Aborted
------------------------------------ snip ------------------------------------
In the source code there is a FIXME remark. Does that remark mean the
assertion is still not as strict as required or that the assertion might be
too strict?
Ciao, Michael.
11 years, 8 months
slapcat of selected attributes
by Hallvard Breien Furuseth
I'd like slapcat to accept an attribute list to output:
slapcat -H "ldap:///dc=uio,dc=no?telephoneNumber,cn"
slapcat URLs won't mean the same as in the LDAP spec, though: An
empty attribute list = all attrs, both user user and operational
ones: "*,+". Elsewhere in LDAP it means just user attrs: "*".
Do anyone have a problem with that, or see a more sensible way to
express it? It's not problem currently, when slapcat does not
support an attribute list anyway.
Come to think of it, ldapsearch ought to support full URLs too - but
until it does, at least there's no risk of confusion with ldapsearch
attribute lists.
--
Hallvard
11 years, 9 months
ITS candidates for 2.4.31
by Quanah Gibson-Mount
Looking at open ITSes, I would note the following are probably worthwhile
to fix for 2.4.31:
6825, 6942, 7042, 7088, 7100, 7102, 7109, 7149, 7168, 7197, 7202
I know Hallvard is also looking at 6883.
--Quanah
--
Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
11 years, 9 months
Ancient ITSes
by Quanah Gibson-Mount
We have a lot of ITSes going back well over 5 years in the ITS system that
are "open". Would it be possible for someone with some idea on them to go
through and close them out as WONT FIX or otherwise prioritize them? I
think a lot of them no longer really apply.
--Quanah
--
Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
11 years, 9 months
ITS#7052, syncrepl, deletes, and MMR
by Howard Chu
I'm still seeing cases where deleted entries are getting resurrected when a
number of concurrent Add/Delete sequences are occurring, with multiple MMR
servers (4 minimum to show the error).
The problem begins because multiple writes are outstanding, and they are
replicated in persist mode without a CSN in their syncrepl cookie. This is a
normal occurrence when the current op does not correspond to the last
committed CSN.
Because there is no CSN, the consumer doesn't update its cookie state while
performing a particular op.
As a result, if a client does Add/Delete/Add/Delete of the same DN, it's
possible for the Adds to propagate several times (more than the client
actually executed).
Adds and Modifies can usually be rejected if they're too old, because they
carry an entryCSN attribute which can be compared against the existing entry,
even if the consumer cookie state has not been updated. But Deletes don't
carry any attributes, and Deleted entries can't be checked.
So, given a MMR setup like so:
1 -- 2
| |
3 -- 4
A sequence of Add/Del/Add/Del performed at server 1 will be replicated to both
2 and 3 immediately. They will then cascade it to server 4. If many other
writes were occurring at the same time, causing these writes to be propagated
without a cookie CSN, then server 4 will propagate them back to 3 and 2
respectively, and 3 and 2 will re-add the deleted entries because they have
nothing to check that says the Adds are old. This cycle only gets broken if
server 1 eventually sends an op with accompanying cookie update, so that all
the downstream servers can see that the ops are old.
...
OK, upon further digging, this appears to be caused by ITS#6024. rein's patch
prevents the consumer and provider from informing each other of their SIDs
when no CSN is present; this prevents syncprov's propagation loop detection
from working. Sigh. Reverting ITS#6024 patch...
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
11 years, 9 months
