I've been wondering why I couldn't get the wonders of refint and memberOf to play together, and even went to the extent of writing a testsuite to prove that they could work (indeed, they do).
However, I eventually found my problem - while memberOf works very nicely as a global overlay, the refint module does not (yet :-).
With it loaded per-backend (for now), it seems to work. Any chance someone could knock me up a patch for that, much like was done so well for memberOf?
Thanks,
Andrew Bartlett
Andrew Bartlett wrote:
I've been wondering why I couldn't get the wonders of refint and memberOf to play together, and even went to the extent of writing a testsuite to prove that they could work (indeed, they do).
However, I eventually found my problem - while memberOf works very nicely as a global overlay, the refint module does not (yet :-).
With it loaded per-backend (for now), it seems to work. Any chance someone could knock me up a patch for that, much like was done so well for memberOf?
Well, it doesn't seem to be impossible, but it'd require some redesign. Basically, it should work if the suffix of the database is the empty DN (""), or if, in case of multiple databases, there is an empty DN database that glues the whole thing together.
The expected behavior should probably consist in:
- letting slapo-refint understand it's instantiated as global
- make it linearly operate on all the existing databases instead of on just the one it's instantiated for in refint_response().
The fix looks relatively simple, although I don't know whether there are any side effects, as I'm not too much familiar with slapo-refint's internals.
p.
Ing. Pierangelo Masarati OpenLDAP Core Team
SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati@sys-net.it ---------------------------------------
Andrew Bartlett wrote:
With it loaded per-backend (for now), it seems to work. Any chance someone could knock me up a patch for that, much like was done so well for memberOf?
The fix is not as trivial as I thought, but it should definitely be possible. However, let me add that loading a slapo-refint(5) per-database is not going to fix the problem, as far as I understand it, since each instance would take care of referential integrity only within the database, but inconsistencies across databases would not be handled. I think the latter case is what you'd like to see addressed, right?
p.
Ing. Pierangelo Masarati OpenLDAP Core Team
SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati@sys-net.it ---------------------------------------
Pierangelo Masarati wrote:
... each instance would take care of referential integrity only within the database, but inconsistencies across databases would not be handled. I think the latter case is what you'd like to see addressed, right?
In any case it's now implemented in HEAD (ITS#5428); please test.
p.
Ing. Pierangelo Masarati OpenLDAP Core Team
SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati@sys-net.it ---------------------------------------
On Sat, 2008-03-15 at 18:28 +0100, Pierangelo Masarati wrote:
Pierangelo Masarati wrote:
... each instance would take care of referential integrity only within the database, but inconsistencies across databases would not be handled. I think the latter case is what you'd like to see addressed, right?
In any case it's now implemented in HEAD (ITS#5428); please test.
p.
Is the refint module synchronous or transaction based?
I'm seeing some *very* odd effects in my test script, that could be explained if the overlay did it's work asynchronously after returning success to the client (for the original operation).
Things I'm seeing are: When loaded as a per-database overlay, the search for:
member=member=CN=ldaptestuser4,CN=ldaptestcontainer2,DC=samba,DC=example,DC=com
fails in my test script, but succeeds if I restart slapd and search with ldbsearch. Just earlier in the script, cn=ldaptestcontainer is renamed to cn=ldaptestcontainer2.
If I load the overlay globally, this entry (cn=ldaptestuser4) actually disappears completely, including the member reference. (It shouldn't be deleted, but perhaps one of the earlier operations triggers it).
This seems very odd - if you don't have any clues, I'll look into running this over TCP (rather than ldapi) and get a trace.
Andrew Bartlett
Andrew Bartlett wrote:
Is the refint module synchronous or transaction based?
Asynchronous, but no transaction. It does its best to preserve integrity, but in case of failure errors are just logged and do not alter the result of the add operation.
I'm seeing some *very* odd effects in my test script, that could be explained if the overlay did it's work asynchronously after returning success to the client (for the original operation).
Things I'm seeing are: When loaded as a per-database overlay, the search for:
member=member=CN=ldaptestuser4,CN=ldaptestcontainer2,DC=samba,DC=example,DC=com
Do you mean '(member=CN=ldaptestuser4,CN=ldaptestcontainer2,DC=samba,DC=example,DC=com)' ?
fails in my test script, but succeeds if I restart slapd and search with ldbsearch. Just earlier in the script, cn=ldaptestcontainer is renamed to cn=ldaptestcontainer2.
If I load the overlay globally, this entry (cn=ldaptestuser4) actually disappears completely, including the member reference. (It shouldn't be deleted, but perhaps one of the earlier operations triggers it).
This seems very odd - if you don't have any clues, I'll look into running this over TCP (rather than ldapi) and get a trace.
OK. p.
Ing. Pierangelo Masarati OpenLDAP Core Team
SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati@sys-net.it ---------------------------------------
On Mon, 2008-03-17 at 07:08 +0100, Pierangelo Masarati wrote:
Andrew Bartlett wrote:
Is the refint module synchronous or transaction based?
Asynchronous, but no transaction. It does its best to preserve integrity, but in case of failure errors are just logged and do not alter the result of the add operation.
OK. I really need it to be synchronous, within a transaction. If the linked DN is invalid, then I need the whole operation to have never happened. I test certain invalid operations in this test script, which is probably why we end up here.
I'm beginning to suspect this is in direct conflict with my first requirement, to operate globally...
I'm seeing some *very* odd effects in my test script, that could be explained if the overlay did it's work asynchronously after returning success to the client (for the original operation).
Things I'm seeing are: When loaded as a per-database overlay, the search for:
member=member=CN=ldaptestuser4,CN=ldaptestcontainer2,DC=samba,DC=example,DC=com
Do you mean '(member=CN=ldaptestuser4,CN=ldaptestcontainer2,DC=samba,DC=example,DC=com)' ?
Yes, sorry, cut&paste typo...
Andrew Bartlett
Andrew Bartlett wrote:
On Mon, 2008-03-17 at 07:08 +0100, Pierangelo Masarati wrote:
Andrew Bartlett wrote:
Is the refint module synchronous or transaction based?
Asynchronous, but no transaction. It does its best to preserve integrity, but in case of failure errors are just logged and do not alter the result of the add operation.
OK. I really need it to be synchronous, within a transaction. If the linked DN is invalid, then I need the whole operation to have never happened. I test certain invalid operations in this test script, which is probably why we end up here.
I'm beginning to suspect this is in direct conflict with my first requirement, to operate globally...
Making it (optionally) synchronous shouldn't be a big deal; making it transactional is totally another business (at least until transaction support is available, at least for internal operations).
p.
Ing. Pierangelo Masarati OpenLDAP Core Team
SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati@sys-net.it ---------------------------------------
On Mon, 2008-03-17 at 07:21 +0100, Pierangelo Masarati wrote:
Andrew Bartlett wrote:
On Mon, 2008-03-17 at 07:08 +0100, Pierangelo Masarati wrote:
Andrew Bartlett wrote:
Is the refint module synchronous or transaction based?
Asynchronous, but no transaction. It does its best to preserve integrity, but in case of failure errors are just logged and do not alter the result of the add operation.
OK. I really need it to be synchronous, within a transaction. If the linked DN is invalid, then I need the whole operation to have never happened. I test certain invalid operations in this test script, which is probably why we end up here.
I'm beginning to suspect this is in direct conflict with my first requirement, to operate globally...
Making it (optionally) synchronous shouldn't be a big deal;
Any chance of that including pre-op validation and post-op implementation? That would get us most of the way.
making it transactional is totally another business (at least until transaction support is available, at least for internal operations).
Any idea if/when this will be available?
Andrew Bartlett
Andrew Bartlett wrote:
On Sat, 2008-03-15 at 18:28 +0100, Pierangelo Masarati wrote:
Pierangelo Masarati wrote:
... each instance would take care of referential integrity only within the database, but inconsistencies across databases would not be handled. I think the latter case is what you'd like to see addressed, right?
In any case it's now implemented in HEAD (ITS#5428); please test.
p.
Is the refint module synchronous or transaction based?
It is necessarily asynchronous. It will cause deadlocks in combination with other overlays if it's made synchronous. (Been there, done that, ITS's already dealt with...)
I'm seeing some *very* odd effects in my test script, that could be explained if the overlay did it's work asynchronously after returning success to the client (for the original operation).
Things I'm seeing are: When loaded as a per-database overlay, the search for:
member=member=CN=ldaptestuser4,CN=ldaptestcontainer2,DC=samba,DC=example,DC=com
fails in my test script, but succeeds if I restart slapd and search with ldbsearch. Just earlier in the script, cn=ldaptestcontainer is renamed to cn=ldaptestcontainer2.
If I load the overlay globally, this entry (cn=ldaptestuser4) actually disappears completely, including the member reference. (It shouldn't be deleted, but perhaps one of the earlier operations triggers it).
This seems very odd - if you don't have any clues, I'll look into running this over TCP (rather than ldapi) and get a trace.
Just use -d7 to get packet traces, no need to change your connection method.
On Mon, 2008-03-17 at 01:20 -0700, Howard Chu wrote:
Andrew Bartlett wrote:
On Sat, 2008-03-15 at 18:28 +0100, Pierangelo Masarati wrote:
Pierangelo Masarati wrote:
... each instance would take care of referential integrity only within the database, but inconsistencies across databases would not be handled. I think the latter case is what you'd like to see addressed, right?
In any case it's now implemented in HEAD (ITS#5428); please test.
p.
Is the refint module synchronous or transaction based?
It is necessarily asynchronous. It will cause deadlocks in combination with other overlays if it's made synchronous. (Been there, done that, ITS's already dealt with...)
Ouch. So, is there any way I can ensure that linked attributes, possibly generated with memberOf, are always strictly consistent?
Andrew Bartlett
openldap-technical@openldap.org
-
Andrew Bartlett -
Howard Chu -
Pierangelo Masarati