Delta-Syncrepl has started failing to actually replicate the consumer starting with an empty database, it fails with code 0x50 because of the auditContext attribute that is present in the suffix entry on the master server due to the accesslog overlay being used.
I can make it work if I load the accesslog overlay in the replica's configuration (without actually configuring it). This appears to be new behavior since 2.4.42.
Is this expected, and now required with 2.4.44, or should I open an ITS?
--On Monday, June 20, 2016 12:31 PM -0400 Frank Swasey Frank.Swasey@uvm.edu wrote:
Delta-Syncrepl has started failing to actually replicate the consumer starting with an empty database, it fails with code 0x50 because of the auditContext attribute that is present in the suffix entry on the master server due to the accesslog overlay being used.
I can make it work if I load the accesslog overlay in the replica's configuration (without actually configuring it). This appears to be new behavior since 2.4.42.
Is this expected, and now required with 2.4.44, or should I open an ITS?
Not quite sure what you mean. I have no issues with empty consumers replicating from their master servers.
--Quanah
--
Quanah Gibson-Mount Platform Architect Manager, Systems Team Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration A division of Synacor, Inc
Today at 11:45am, Quanah Gibson-Mount wrote:
--On Monday, June 20, 2016 12:31 PM -0400 Frank Swasey Frank.Swasey@uvm.edu wrote:
Delta-Syncrepl has started failing to actually replicate the consumer starting with an empty database, it fails with code 0x50 because of the auditContext attribute that is present in the suffix entry on the master server due to the accesslog overlay being used.
I can make it work if I load the accesslog overlay in the replica's configuration (without actually configuring it). This appears to be new behavior since 2.4.42.
Is this expected, and now required with 2.4.44, or should I open an ITS?
Not quite sure what you mean. I have no issues with empty consumers replicating from their master servers.
I mean that an empty consumer fails to replicate from its master servers unless I load the accesslog module. It fails with the following:
Jun 20 12:29:29 ldap7r slapd[21153]: syncrepl_message_to_entry: rid=100 DN: dc=uvm,dc=edu, UUID: b745df88-a640-1027-9374-fb1dd3799b92 Jun 20 12:29:29 ldap7r slapd[21153]: UNKNOWN attributeDescription "AUDITCONTEXT" inserted. Jun 20 12:29:29 ldap7r slapd[21153]: syncrepl_entry: rid=100 LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_ADD) Jun 20 12:29:29 ldap7r slapd[21153]: syncrepl_message_to_entry: rid=100 DN: dc=uvm,dc=edu, UUID: b745df88-a640-1027-9374-fb1dd3799b92 Jun 20 12:29:29 ldap7r slapd[21153]: UNKNOWN attributeDescription "AUDITCONTEXT" inserted. Jun 20 12:29:29 ldap7r slapd[21153]: syncrepl_entry: rid=100 LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_ADD) Jun 20 12:29:29 ldap7r slapd[21153]: syncrepl_entry: rid=100 inserted UUID b745df88-a640-1027-9374-fb1dd3799b92 Jun 20 12:29:29 ldap7r slapd[21153]: syncrepl_entry: rid=100 be_search (32) Jun 20 12:29:29 ldap7r slapd[21153]: syncrepl_entry: rid=100 inserted UUID b745df88-a640-1027-9374-fb1dd3799b92 Jun 20 12:29:29 ldap7r slapd[21153]: syncrepl_entry: rid=100 be_search (32) Jun 20 12:29:29 ldap7r slapd[21153]: syncrepl_entry: rid=100 dc=uvm,dc=edu Jun 20 12:29:29 ldap7r slapd[21153]: syncrepl_entry: rid=100 dc=uvm,dc=edu Jun 20 12:29:29 ldap7r slapd[21153]: syncrepl_entry: rid=100 be_add dc=uvm,dc=edu (80) Jun 20 12:29:29 ldap7r slapd[21153]: syncrepl_entry: rid=100 be_add dc=uvm,dc=edu (80) Jun 20 12:29:29 ldap7r slapd[21153]: syncrepl_entry: rid=100 be_add dc=uvm,dc=edu failed (80) Jun 20 12:29:29 ldap7r slapd[21153]: syncrepl_entry: rid=100 be_add dc=uvm,dc=edu failed (80) Jun 20 12:29:29 ldap7r slapd[21153]: do_syncrepl: rid=100 rc 80 retrying Jun 20 12:29:29 ldap7r slapd[21153]: do_syncrepl: rid=100 rc 80 retrying
That tells me that the masters are sending the auditContext attribute but my replica has no clue what to do with it so fails to add the base dn and nothing works after that.
Frank Swasey wrote:
Today at 11:45am, Quanah Gibson-Mount wrote:
--On Monday, June 20, 2016 12:31 PM -0400 Frank Swasey Frank.Swasey@uvm.edu wrote:
Delta-Syncrepl has started failing to actually replicate the consumer starting with an empty database, it fails with code 0x50 because of the auditContext attribute that is present in the suffix entry on the master server due to the accesslog overlay being used.
I can make it work if I load the accesslog overlay in the replica's configuration (without actually configuring it). This appears to be new behavior since 2.4.42.
Is this expected, and now required with 2.4.44, or should I open an ITS?
Not quite sure what you mean. I have no issues with empty consumers replicating from their master servers.
I mean that an empty consumer fails to replicate from its master servers unless I load the accesslog module. It fails with the following:
Jun 20 12:29:29 ldap7r slapd[21153]: syncrepl_message_to_entry: rid=100 DN: dc=uvm,dc=edu, UUID: b745df88-a640-1027-9374-fb1dd3799b92 Jun 20 12:29:29 ldap7r slapd[21153]: UNKNOWN attributeDescription "AUDITCONTEXT" inserted. Jun 20 12:29:29 ldap7r slapd[21153]: syncrepl_entry: rid=100 LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_ADD) Jun 20 12:29:29 ldap7r slapd[21153]: syncrepl_message_to_entry: rid=100 DN: dc=uvm,dc=edu, UUID: b745df88-a640-1027-9374-fb1dd3799b92 Jun 20 12:29:29 ldap7r slapd[21153]: UNKNOWN attributeDescription "AUDITCONTEXT" inserted. Jun 20 12:29:29 ldap7r slapd[21153]: syncrepl_entry: rid=100 LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_ADD) Jun 20 12:29:29 ldap7r slapd[21153]: syncrepl_entry: rid=100 inserted UUID b745df88-a640-1027-9374-fb1dd3799b92 Jun 20 12:29:29 ldap7r slapd[21153]: syncrepl_entry: rid=100 be_search (32) Jun 20 12:29:29 ldap7r slapd[21153]: syncrepl_entry: rid=100 inserted UUID b745df88-a640-1027-9374-fb1dd3799b92 Jun 20 12:29:29 ldap7r slapd[21153]: syncrepl_entry: rid=100 be_search (32) Jun 20 12:29:29 ldap7r slapd[21153]: syncrepl_entry: rid=100 dc=uvm,dc=edu Jun 20 12:29:29 ldap7r slapd[21153]: syncrepl_entry: rid=100 dc=uvm,dc=edu Jun 20 12:29:29 ldap7r slapd[21153]: syncrepl_entry: rid=100 be_add dc=uvm,dc=edu (80) Jun 20 12:29:29 ldap7r slapd[21153]: syncrepl_entry: rid=100 be_add dc=uvm,dc=edu (80) Jun 20 12:29:29 ldap7r slapd[21153]: syncrepl_entry: rid=100 be_add dc=uvm,dc=edu failed (80) Jun 20 12:29:29 ldap7r slapd[21153]: syncrepl_entry: rid=100 be_add dc=uvm,dc=edu failed (80) Jun 20 12:29:29 ldap7r slapd[21153]: do_syncrepl: rid=100 rc 80 retrying Jun 20 12:29:29 ldap7r slapd[21153]: do_syncrepl: rid=100 rc 80 retrying
That tells me that the masters are sending the auditContext attribute but my replica has no clue what to do with it so fails to add the base dn and nothing works after that.
That's definitely odd. auditContext is a dsaOperational attribute, so it is not supposed to propagate in replication. slap_send_search_entry in slapd/result.c explicitly strips these out. The only thing I can think of is that someone has changed your auditContext schema definition so it's no longer a dsaOperation attribute.
--On Monday, June 20, 2016 1:33 PM -0400 Frank Swasey Frank.Swasey@uvm.edu wrote:
Not quite sure what you mean. I have no issues with empty consumers replicating from their master servers.
I mean that an empty consumer fails to replicate from its master servers unless I load the accesslog module. It fails with the following:
I don't have the accesslog module loaded on my replicas, and they do not have this issue. ;) Are you doing cn=config replication as well as replicating the main db?
--Quanah
--
Quanah Gibson-Mount Platform Architect Manager, Systems Team Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration A division of Synacor, Inc
Today at 1:06pm, Quanah Gibson-Mount wrote:
--On Monday, June 20, 2016 1:33 PM -0400 Frank Swasey Frank.Swasey@uvm.edu wrote:
Not quite sure what you mean. I have no issues with empty consumers replicating from their master servers.
I mean that an empty consumer fails to replicate from its master servers unless I load the accesslog module. It fails with the following:
I don't have the accesslog module loaded on my replicas, and they do not have this issue. ;) Are you doing cn=config replication as well as replicating the main db?
No, I am not replicating cn=config (and this consumer is not a member of the MMR set)
Today at 12:46pm, Howard Chu wrote:
That tells me that the masters are sending the auditContext attribute but my replica has no clue what to do with it so fails to add the base dn and nothing works after that.
That's definitely odd. auditContext is a dsaOperational attribute, so it is not supposed to propagate in replication. slap_send_search_entry in slapd/result.c explicitly strips these out. The only thing I can think of is that someone has changed your auditContext schema definition so it's no longer a dsaOperation attribute.
I do make some modifications to the codebase, but have never done that. I'll double check the patch decks to make sure something did not match and mess that attribute up unintentionally.
On Mon, 20 Jun 2016 at 3:36pm, Frank Swasey wrote:
I do make some modifications to the codebase, but have never [changed auditContext from being DSA]. I'll double check the patch decks to make sure something did not match and mess that attribute up unintentionally.
I have confirmed that my code modifications have not changed the auditContext attribute's definition, it is still "Usage directoryOperation".
To further complicate the matter, I have pointed my delta-syncrepl slave consumer at a different MMR configured master and it replicates from empty just fine. That second MMR configured master is actually a VMware clone of the one of the two test MMR systems and is identical in all respects except the slapd configuration (it is paired with my current non-MMR production master). This leads me to believe it is either something I've managed to mess up in the configuration of the testing MMR pair, or it is because the base dn in the testing MMR pair has multiple contextCSN values (for serverid's 0, 1, and 2) and (so far) the production setup still has a single contextCSN value.
So, let me back up a few feet and make sure I am not setting myself up for ultimate failure when I complete the migration of my production environment to a full MMR master pair with delta-syncrepl slaves.
My plan is that I will have two systems which are configured as MMR, so that should a datacenter fail, I can (manually) fail-over the service ip address to the surviving datacenter and continue operation with LDAP updates. I also will have six read-only delta-syncrepl slaves to carry the workload of approximately 20 million queries per day.
I did not think that using MMR for the master server would preclude having delta-syncrepl slaves, and I find no discussion in my googling to indicate either way.
Have I made a bad design?
--On Tuesday, June 21, 2016 8:09 AM -0400 Frank Swasey Frank.Swasey@uvm.edu wrote:
On Mon, 20 Jun 2016 at 3:36pm, Frank Swasey wrote:
I do make some modifications to the codebase, but have never [changed auditContext from being DSA]. I'll double check the patch decks to make sure something did not match and mess that attribute up unintentionally.
I have confirmed that my code modifications have not changed the auditContext attribute's definition, it is still "Usage directoryOperation".
To further complicate the matter, I have pointed my delta-syncrepl slave consumer at a different MMR configured master and it replicates from empty just fine. That second MMR configured master is actually a VMware clone of the one of the two test MMR systems and is identical in all respects except the slapd configuration (it is paired with my current non-MMR production master). This leads me to believe it is either something I've managed to mess up in the configuration of the testing MMR pair, or it is because the base dn in the testing MMR pair has multiple contextCSN values (for serverid's 0, 1, and 2) and (so far) the production setup still has a single contextCSN value.
So, let me back up a few feet and make sure I am not setting myself up for ultimate failure when I complete the migration of my production environment to a full MMR master pair with delta-syncrepl slaves.
My plan is that I will have two systems which are configured as MMR, so that should a datacenter fail, I can (manually) fail-over the service ip address to the surviving datacenter and continue operation with LDAP updates. I also will have six read-only delta-syncrepl slaves to carry the workload of approximately 20 million queries per day.
I did not think that using MMR for the master server would preclude having delta-syncrepl slaves, and I find no discussion in my googling to indicate either way.
Have I made a bad design?
I only use delta-syncrepl slaves, and I have multiple contextCSNs served out by masters, and do not have this issue at all. In my case, it is CSNS 0, 1, 2, 3, and 4. In fact, I recently updated it so that my replicas can pull from N number of masters. ;)
--Quanah
--
Quanah Gibson-Mount Platform Architect Manager, Systems Team Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration A division of Synacor, Inc
Today, I am not able to make it fail (I swear, all I did was change the master the slave pointed to and then change it back).
Frank Swasey wrote:
On Mon, 20 Jun 2016 at 3:36pm, Frank Swasey wrote:
I do make some modifications to the codebase, but have never [changed auditContext from being DSA]. I'll double check the patch decks to make sure something did not match and mess that attribute up unintentionally.
I have confirmed that my code modifications have not changed the auditContext attribute's definition, it is still "Usage directoryOperation".
This is exactly wrong. It must be "usage dsaOperation". You have indeed broken this in your patching.
Today at 5:22pm, Howard Chu wrote:
Frank Swasey wrote:
On Mon, 20 Jun 2016 at 3:36pm, Frank Swasey wrote:
I do make some modifications to the codebase, but have never [changed auditContext from being DSA]. I'll double check the patch decks to make sure something did not match and mess that attribute up unintentionally.
I have confirmed that my code modifications have not changed the auditContext attribute's definition, it is still "Usage directoryOperation".
This is exactly wrong. It must be "usage dsaOperation". You have indeed broken this in your patching.
Howard, before you accuse someone of such a thing, you should really go look at the source... I have done nothing to the definition in the servers/slapd/overlays/accesslog.c file, the file is exactly as delivered in the 2.4.44 tar ball, as well as the existing code here:
http://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=blob_plain;f=serve...
Today at 7:49pm, Frank Swasey wrote:
Today at 5:22pm, Howard Chu wrote:
Frank Swasey wrote:
On Mon, 20 Jun 2016 at 3:36pm, Frank Swasey wrote:
I do make some modifications to the codebase, but have never [changed auditContext from being DSA]. I'll double check the patch decks to make sure something did not match and mess that attribute up unintentionally.
I have confirmed that my code modifications have not changed the auditContext attribute's definition, it is still "Usage directoryOperation".
This is exactly wrong. It must be "usage dsaOperation". You have indeed broken this in your patching.
Howard, before you accuse someone of such a thing, you should really go look at the source... I have done nothing to the definition in the servers/slapd/overlays/accesslog.c file, the file is exactly as delivered in the 2.4.44 tar ball, as well as the existing code here:
http://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=blob_plain;f=serve...
I see... that "Usage directoryOperation" is inside a comment block. The real code is 5 lines further down and does have "USAGE dSAOperation" in it.
/sigh
Frank Swasey wrote:
Today at 5:22pm, Howard Chu wrote:
Frank Swasey wrote:
On Mon, 20 Jun 2016 at 3:36pm, Frank Swasey wrote:
I do make some modifications to the codebase, but have never [changed auditContext from being DSA]. I'll double check the patch decks to make sure something did not match and mess that attribute up unintentionally.
I have confirmed that my code modifications have not changed the auditContext attribute's definition, it is still "Usage directoryOperation".
This is exactly wrong. It must be "usage dsaOperation". You have indeed broken this in your patching.
Howard, before you accuse someone of such a thing, you should really go look at the source... I have done nothing to the definition in the servers/slapd/overlays/accesslog.c file, the file is exactly as delivered in the 2.4.44 tar ball, as well as the existing code here:
http://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=blob_plain;f=serve...
The above referenced code clearly has dSAOperation on line 399, not directoryOperation.
Today at 2:35am, Michael Ströder wrote:
Frank Swasey wrote:
I see... that "Usage directoryOperation" is inside a comment block. The real code is 5 lines further down and does have "USAGE dSAOperation" in it.
Why not simply query the subschema subentry and have a look.
Because it was Howard asking, and he always says to read the source...
olcAttributeTypes: ( 1.3.6.1.4.1.4203.666.11.5.1.30 NAME 'auditContext' DESC 'DN of auditContainer' SYNTAX 1.3.6.1.4.1.1466.115.121.1.12 SINGLE-VALUE NO-USER-MODIFICATION USAGE dSAOperation )
openldap-technical@openldap.org
-
Frank Swasey -
Howard Chu -
Michael Ströder -
Quanah Gibson-Mount