I'm trying to write a perl script using PersistentSearch from Net::LDAP::Control::PersistentSearch against a freshly compiled OpenLDAP 2.4.23 server. I've configured/compiled the server with --enable-syncprov=yes and I have N-way multimaster setup and working.
In the directory that I'm querying, I have syncrepl and mirrormode configured for the suffix dc=coas,dc=oregonstate,dc=edu.
But, when I run the perl script using PersistentSearch I'm getting the following in my slapd log file when I search for something in the base ou=People,dc=coas,dc=oregonstate,dc=edu: slap_global_control: unrecognized control: 2.16.840.1.113730.3.4.3
I'm under the impression that PersistentSearch is enabled by the syncprov overlay so why would I be getting the unknown control message?
Any help would be appreciated, I'm hitting dead ends everywhere I look. Tom Leach leach@coas.oregonstate.edu
--On Tuesday, August 24, 2010 5:26 PM -0700 Tom Leach leach@coas.oregonstate.edu wrote:
I'm under the impression that PersistentSearch is enabled by the syncprov overlay so why would I be getting the unknown control message?
Did you load and enable the overlay for your primary database?
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration
Here is the primary database definition, and it's the tree that I'm trying to run a persistent search against. I have the syncprov overlay loaded towards the end after the syncrepl and accesslog stuff. The "overlay syncprov" three lines are the only lines in slapd.conf that reference syncprov. Thanks! Tom
database bdb suffix "dc=coas,dc=oregonstate,dc=edu" rootdn "XXX" rootpw secret directory /usr/local/var/openldap-data index objectClass eq index ou,cn,mail,surname,givenname eq,pres,sub index uid,memberUid eq,pres,sub index uniqueMember eq,pres index entryCSN,entryUUID eq
syncrepl rid=123 provider=ldap://XXX starttls=critical type=refreshAndPersist interval=00:00:05:00 retry="5 20 300 +" searchbase="dc=coas,dc=oregonstate,dc=edu" filter="(objectClass=*)" attrs="*,+" scope=sub schemachecking=off bindmethod=simple binddn="XXX" credentials=XXX
mirrormode TRUE
overlay accesslog logdb cn=accesslog logops add delete modify modrdn logold (objectclass=person) logsuccess TRUE logpurge 07+00:00 01+00:00
overlay syncprov syncprov-checkpoint 100 10 syncprov-sessionlog 100
On 08/24/2010 05:39 PM, Quanah Gibson-Mount wrote:
--On Tuesday, August 24, 2010 5:26 PM -0700 Tom Leach leach@coas.oregonstate.edu wrote:
I'm under the impression that PersistentSearch is enabled by the syncprov overlay so why would I be getting the unknown control message?
Did you load and enable the overlay for your primary database?
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc
Zimbra :: the leader in open source messaging and collaboration
Tom Leach leach@coas.oregonstate.edu writes:
Here is the primary database definition, and it's the tree that I'm trying to run a persistent search against. I have the syncprov overlay loaded towards the end after the syncrepl and accesslog stuff. The "overlay syncprov" three lines are the only lines in slapd.conf that reference syncprov. Thanks! Tom
database bdb suffix "dc=coas,dc=oregonstate,dc=edu" rootdn "XXX" rootpw secret directory /usr/local/var/openldap-data index objectClass eq index ou,cn,mail,surname,givenname eq,pres,sub index uid,memberUid eq,pres,sub index uniqueMember eq,pres index entryCSN,entryUUID eq
syncrepl rid=123 provider=ldap://XXX starttls=critical type=refreshAndPersist interval=00:00:05:00 retry="5 20 300 +" searchbase="dc=coas,dc=oregonstate,dc=edu" filter="(objectClass=*)" attrs="*,+" scope=sub schemachecking=off bindmethod=simple binddn="XXX" credentials=XXX
[...]
This is a syncrepl synchronisation, persistantSearch is not supported by OpenLDAP. If you want to receive modified entries, you should use Net::LDAP::Control::SyncRequest Unfortunately, this perl modul doesn't connect properly in refresh and persist mode (according to the author) but only in refresh only mode.
-Dieter
Dieter, what I'm trying to do is have a separate process (this being the perl script) connect with a persistent search to one of a pair of syncrepl'd multimasters. Are syncrepl and syncprov mutually exclusive? If that's the case, then I'll have to take another tack since I require the syncrepl multimasters and the persistent search via syncprov was a "nice thing to have" but not required. Thanks! Tom
On 08/25/2010 11:04 AM, Dieter Kluenter wrote:
Tom Leach leach@coas.oregonstate.edu writes:
Here is the primary database definition, and it's the tree that I'm trying to run a persistent search against. I have the syncprov overlay loaded towards the end after the syncrepl and accesslog stuff. The "overlay syncprov" three lines are the only lines in slapd.conf that reference syncprov. Thanks! Tom
database bdb suffix "dc=coas,dc=oregonstate,dc=edu" rootdn "XXX" rootpw secret directory /usr/local/var/openldap-data index objectClass eq index ou,cn,mail,surname,givenname eq,pres,sub index uid,memberUid eq,pres,sub index uniqueMember eq,pres index entryCSN,entryUUID eq
syncrepl rid=123 provider=ldap://XXX starttls=critical type=refreshAndPersist interval=00:00:05:00 retry="5 20 300 +" searchbase="dc=coas,dc=oregonstate,dc=edu" filter="(objectClass=*)" attrs="*,+" scope=sub schemachecking=off bindmethod=simple binddn="XXX" credentials=XXX
[...]
This is a syncrepl synchronisation, persistantSearch is not supported by OpenLDAP. If you want to receive modified entries, you should use Net::LDAP::Control::SyncRequest Unfortunately, this perl modul doesn't connect properly in refresh and persist mode (according to the author) but only in refresh only mode.
-Dieter
Tom Leach leach@coas.oregonstate.edu writes:
Dieter, what I'm trying to do is have a separate process (this being the perl script) connect with a persistent search to one of a pair of syncrepl'd multimasters. Are syncrepl and syncprov mutually exclusive? If that's the case, then I'll have to take another tack since I require the syncrepl multimasters and the persistent search via syncprov was a "nice thing to have" but not required. Thanks! Tom
[...] I ldap speach persistant search and content synchronization are different operations with different control oid, although both operations have quite similar intentions. With regard to OpenLDAP, the consumer operation syncrepl and the provider operation syncprov are different operations but not mutually exclusive, as an example see test 019 syncreplication-cascade of the test suite. OpenLDAP does not support draft-ietf-ldapext-psearch, thus control oid 2.16.840.1.113730.3.4.3 is unknown to the DSA. What you can do is to attach a syncrepl consumer, i.e. a Net::LDAP::Control::SyncRequest object, in order to receive notification of added or modified entries.
-Dieter
Tom Leach wrote:
Dieter, what I'm trying to do is have a separate process (this being the perl script) connect with a persistent search to one of a pair of syncrepl'd multimasters. Are syncrepl and syncprov mutually exclusive? If that's the case, then I'll have to take another tack since I require the syncrepl multimasters and the persistent search via syncprov was a "nice thing to have" but not required.
Tom,
did you really read my answer? http://www.openldap.org/lists/openldap-technical/201008/msg00226.html
Did you read Dieter's answer? Dieter even pointed you to Net::LDAP::Control::SyncRequest.
It's the client-side programming.
Ciao, Michael.
Michael, yes, I read you answer. I missed the 'latter' and thought that you meant the persistentSearch was supplied by syncprov and not syncrepl. I guess I read latter but was thinking former.
I also read Dieter's answer pointing to SyncRequest but here's my issue. He states that the module (according to the author) needs refreshOnly and won't connect with refreshAndPersist. I need refreshAndPersist to make sure that my multi-master work correctly and a change on either server is quickly replicated to the other.
I've also read both the RFC and the (expired) draft for persistentSearch. It appears that persistentSearch has died and there is no server-side push type replacement. That's unfortunate since I will now need to poll on a frequent basis to catch a change that happens on a very infrequent basis.
Anyway, it looks like PersistentSearch isn't supported. Other approaches that I'm investigating are the accesslog with a process that queries it on a reqular basis looking for modifies, or create another syncrepl instance with a refreshOnly aimed at my client process.
Thanks! Tom
On 08/26/2010 01:09 AM, Michael Ströder wrote:
Tom Leach wrote:
Dieter, what I'm trying to do is have a separate process (this being the perl script) connect with a persistent search to one of a pair of syncrepl'd multimasters. Are syncrepl and syncprov mutually exclusive? If that's the case, then I'll have to take another tack since I require the syncrepl multimasters and the persistent search via syncprov was a "nice thing to have" but not required.
Tom,
did you really read my answer? http://www.openldap.org/lists/openldap-technical/201008/msg00226.html
Did you read Dieter's answer? Dieter even pointed you to Net::LDAP::Control::SyncRequest.
It's the client-side programming.
Ciao, Michael.
Tom Leach leach@coas.oregonstate.edu writes:
Michael, yes, I read you answer. I missed the 'latter' and thought that you meant the persistentSearch was supplied by syncprov and not syncrepl. I guess I read latter but was thinking former.
I also read Dieter's answer pointing to SyncRequest but here's my issue. He states that the module (according to the author) needs refreshOnly and won't connect with refreshAndPersist. I need refreshAndPersist to make sure that my multi-master work correctly and a change on either server is quickly replicated to the other.
You may build a little shell script around ldapsearch:
- get the actual contextCSN from provider - construct a cookie of rid, contextCSN and searchlimits - run ldapsearch with extended operation sync in persistant mode. something like this function snipped:
search_sync(){ CSN="$(awk -F': ' '/contextCSN/{print $2}' "$FILE")" COOKIE="rid=099,csn=${CSN}"
$SEARCH -LLL\ -H $HOST -Y DIGEST-MD5\ -U $BINDDN -w $BINDPW\ -b $BASE -s sub -E!sync=rp/$COOKIE/0 "*" + }
-Dieter
--On August 26, 2010 9:43:33 AM -0700 Tom Leach leach@coas.oregonstate.edu wrote:
Michael, yes, I read you answer. I missed the 'latter' and thought that you meant the persistentSearch was supplied by syncprov and not syncrepl. I guess I read latter but was thinking former.
I also read Dieter's answer pointing to SyncRequest but here's my issue. He states that the module (according to the author) needs refreshOnly and won't connect with refreshAndPersist. I need refreshAndPersist to make sure that my multi-master work correctly and a change on either server is quickly replicated to the other.
I've also read both the RFC and the (expired) draft for persistentSearch. It appears that persistentSearch has died and there is no server-side push type replacement. That's unfortunate since I will now need to poll on a frequent basis to catch a change that happens on a very infrequent basis.
Anyway, it looks like PersistentSearch isn't supported. Other approaches that I'm investigating are the accesslog with a process that queries it on a reqular basis looking for modifies, or create another syncrepl instance with a refreshOnly aimed at my client process.
Net::LDAPapi supports refresh & persist.
--Quanah
Tom Leach wrote:
I'm trying to write a perl script using PersistentSearch from Net::LDAP::Control::PersistentSearch against a freshly compiled OpenLDAP 2.4.23 server. I've configured/compiled the server with --enable-syncprov=yes and I have N-way multimaster setup and working.
In the directory that I'm querying, I have syncrepl and mirrormode configured for the suffix dc=coas,dc=oregonstate,dc=edu.
But, when I run the perl script using PersistentSearch I'm getting the following in my slapd log file when I search for something in the base ou=People,dc=coas,dc=oregonstate,dc=edu: slap_global_control: unrecognized control: 2.16.840.1.113730.3.4.3
Persistent search is not syncrepl with the latter being implemented by syncprov overlay.
Compare these docs:
http://tools.ietf.org/html/draft-ietf-ldapext-psearch
http://www.ietf.org/rfc/rfc4533.txt
Ciao, Michael.
Sorry, forgot to include that I have "overlay syncprov" in the config as well. I'm reviewing the two RFCs now. Thanks! Tom Leach
On 08/24/2010 08:31 PM, Michael Ströder wrote:
Tom Leach wrote:
I'm trying to write a perl script using PersistentSearch from Net::LDAP::Control::PersistentSearch against a freshly compiled OpenLDAP 2.4.23 server. I've configured/compiled the server with --enable-syncprov=yes and I have N-way multimaster setup and working.
In the directory that I'm querying, I have syncrepl and mirrormode configured for the suffix dc=coas,dc=oregonstate,dc=edu.
But, when I run the perl script using PersistentSearch I'm getting the following in my slapd log file when I search for something in the base ou=People,dc=coas,dc=oregonstate,dc=edu: slap_global_control: unrecognized control: 2.16.840.1.113730.3.4.3
Persistent search is not syncrepl with the latter being implemented by syncprov overlay.
Compare these docs:
http://tools.ietf.org/html/draft-ietf-ldapext-psearch
http://www.ietf.org/rfc/rfc4533.txt
Ciao, Michael.
openldap-technical@openldap.org
-
Dieter Kluenter -
Michael Ströder -
Quanah Gibson-Mount -
Tom Leach