Hi All -
I'm having an odd issue where on a rare occasion (a couple of times a week), a new LDAP user entry is being deleted shortly after it is created. Sometimes it happens within a few minutes, sometimes it happens within an hour or so.
I have a 4 way multi-master setup, with all writes being directed at a single server with a load balancer. I have the auditlog enabled (from failed attempts at delta sync) and I see auditDelete entries in the auditdb, but its being executed from the internal admin user, not a "real" user. I do not see anything suspect in my system logs running at the normal loglevel.
I'm running 2.4.39 on Redhat 6, x64 with mdb. Below is a snippet of my configuration from the specific database in question. Does anyone know why this might be occurring? Any idea on how to further troubleshoot this issue?
Thanks in advance -
Al
dn: olcDatabase={1}mdb,cn=config objectClass: olcDatabaseConfig objectClass: olcMdbConfig olcDatabase: {1}mdb olcDbDirectory: /PATH/TO/OPENLDAP/var/openldap-data olcSuffix: dc=company,dc=com olcAddContentAcl: FALSE olcLastMod: TRUE olcLimits: {0}dn.base="XXXXXXX" size.soft=unlimited size.hard=unlimited time.soft=unlimited time.hard=unlimited olcMaxDerefDepth: 15 olcReadOnly: FALSE olcRootDN: cn=Manager,dc=company,dc=com olcRootPW:: XXXXXXXXX olcSyncUseSubentry: FALSE olcMirrorMode: TRUE olcMonitoring: TRUE olcDbCheckpoint: 512 5 olcDbNoSync: TRUE olcDbIndex: objectClass eq olcDbIndex: entryUUID eq olcDbIndex: entryCSN eq olcDbIndex: cn pres,eq,sub olcDbIndex: uid eq olcDbIndex: uidNumber eq olcDbIndex: gidNumber eq olcDbIndex: uniqueMember eq olcDbIndex: nisNetgroupTriple eq olcDbIndex: sudoUser eq,sub olcDbIndex: mail eq olcDbIndex: pwmToken eq,sub olcDbIndex: memberOf eq olcDbMaxSize: 25000000000 olcDbMode: 0600 structuralObjectClass: olcMdbConfig entryUUID: xxxx-xxxx-xxxxx-xxxxx creatorsName: cn=config createTimestamp: 20111014131247Z olcSyncrepl: {0}rid=011 provider=ldap://server1:21389/ bind method=simple timeout=0 network-timeout=0 binddn="XXXXXXX" credentials="XXXX" keepalive=0:0:0 startt ls=critical filter="(objectclass=*)" searchbase="dc=company,dc=com" scope=sub schemachecking=off type=refreshOnly retry="30 +" interval=00:00:00:30 olcSyncrepl: {1}rid=012 provider=ldap://server2:21389/ bind method=simple timeout=0 network-timeout=0 binddn="XXXXXXX" credentials="XXXX" keepalive=0:0:0 startt ls=critical filter="(objectclass=*)" searchbase="dc=company,dc=com" scope=sub schemachecking=off type=refreshOnly retry="30 +" interval=00:00:00:30 olcSyncrepl: {2}rid=013 provider=ldap://server3:21389/ bind method=simple timeout=0 network-timeout=0 binddn="XXXXXXX" credentials="XXXX" keepalive=0:0:0 startt ls=critical filter="(objectclass=*)" searchbase="dc=company,dc=com" scope=sub schemachecking=off type=refreshOnly retry="30 +" interval=00:00:00:30 olcSyncrepl: {3}rid=014 provider=ldap://server4:21389/ bind method=simple timeout=0 network-timeout=0 binddn="XXXXXXX" credentials="XXXX" keepalive=0:0:0 startt ls=critical filter="(objectclass=*)" searchbase="dc=company,dc=com" scope=sub schemachecking=off type=refreshOnly retry="30 +" interval=00:00:00:30 entryCSN: 20140924095732.634049Z#000000#001#000000 modifiersName: cn=Manager,cn=config modifyTimestamp: 20140924095732Z
This is the bug, which reproduced stably in our production environment. I plan fix it in our version (fork of OpenLDAP), immediatly after other crashes. See https://github.com/ReOpen/ReOpenLDAP/issues/3
Regards, Leonid.
2015-01-20 17:38 GMT+03:00 Al afrunning@gmail.com:
Hi All -
I'm having an odd issue where on a rare occasion (a couple of times a week), a new LDAP user entry is being deleted shortly after it is created. Sometimes it happens within a few minutes, sometimes it happens within an hour or so.
I have a 4 way multi-master setup, with all writes being directed at a single server with a load balancer. I have the auditlog enabled (from failed attempts at delta sync) and I see auditDelete entries in the auditdb, but its being executed from the internal admin user, not a "real" user. I do not see anything suspect in my system logs running at the normal loglevel.
I'm running 2.4.39 on Redhat 6, x64 with mdb. Below is a snippet of my configuration from the specific database in question. Does anyone know why this might be occurring? Any idea on how to further troubleshoot this issue?
Thanks in advance -
Al
dn: olcDatabase={1}mdb,cn=config objectClass: olcDatabaseConfig objectClass: olcMdbConfig olcDatabase: {1}mdb olcDbDirectory: /PATH/TO/OPENLDAP/var/openldap-data olcSuffix: dc=company,dc=com olcAddContentAcl: FALSE olcLastMod: TRUE olcLimits: {0}dn.base="XXXXXXX" size.soft=unlimited size.hard=unlimited time.soft=unlimited time.hard=unlimited olcMaxDerefDepth: 15 olcReadOnly: FALSE olcRootDN: cn=Manager,dc=company,dc=com olcRootPW:: XXXXXXXXX olcSyncUseSubentry: FALSE olcMirrorMode: TRUE olcMonitoring: TRUE olcDbCheckpoint: 512 5 olcDbNoSync: TRUE olcDbIndex: objectClass eq olcDbIndex: entryUUID eq olcDbIndex: entryCSN eq olcDbIndex: cn pres,eq,sub olcDbIndex: uid eq olcDbIndex: uidNumber eq olcDbIndex: gidNumber eq olcDbIndex: uniqueMember eq olcDbIndex: nisNetgroupTriple eq olcDbIndex: sudoUser eq,sub olcDbIndex: mail eq olcDbIndex: pwmToken eq,sub olcDbIndex: memberOf eq olcDbMaxSize: 25000000000 olcDbMode: 0600 structuralObjectClass: olcMdbConfig entryUUID: xxxx-xxxx-xxxxx-xxxxx creatorsName: cn=config createTimestamp: 20111014131247Z olcSyncrepl: {0}rid=011 provider=ldap://server1:21389/ bind method=simple timeout=0 network-timeout=0 binddn="XXXXXXX" credentials="XXXX" keepalive=0:0:0 startt ls=critical filter="(objectclass=*)" searchbase="dc=company,dc=com" scope=sub schemachecking=off type=refreshOnly retry="30 +" interval=00:00:00:30 olcSyncrepl: {1}rid=012 provider=ldap://server2:21389/ bind method=simple timeout=0 network-timeout=0 binddn="XXXXXXX" credentials="XXXX" keepalive=0:0:0 startt ls=critical filter="(objectclass=*)" searchbase="dc=company,dc=com" scope=sub schemachecking=off type=refreshOnly retry="30 +" interval=00:00:00:30 olcSyncrepl: {2}rid=013 provider=ldap://server3:21389/ bind method=simple timeout=0 network-timeout=0 binddn="XXXXXXX" credentials="XXXX" keepalive=0:0:0 startt ls=critical filter="(objectclass=*)" searchbase="dc=company,dc=com" scope=sub schemachecking=off type=refreshOnly retry="30 +" interval=00:00:00:30 olcSyncrepl: {3}rid=014 provider=ldap://server4:21389/ bind method=simple timeout=0 network-timeout=0 binddn="XXXXXXX" credentials="XXXX" keepalive=0:0:0 startt ls=critical filter="(objectclass=*)" searchbase="dc=company,dc=com" scope=sub schemachecking=off type=refreshOnly retry="30 +" interval=00:00:00:30 entryCSN: 20140924095732.634049Z#000000#001#000000 modifiersName: cn=Manager,cn=config modifyTimestamp: 20140924095732Z
Леонид Юрьев schrieb (20.01.2015 16:22 Uhr):
This is the bug, which reproduced stably in our production environment. I plan fix it in our version (fork of OpenLDAP), immediatly after other crashes. See https://github.com/ReOpen/ReOpenLDAP/issues/3
how does patching your fork help anyone here? Do you submit patches here too?
Why did you fork in the first place? "The fork of OpenLDAP with few new features (mostly for LMDB), additional bug fixing and code quality improvement."
Marc
Marc Patermann wrote:
Леонид Юрьев schrieb (20.01.2015 16:22 Uhr):
This is the bug, which reproduced stably in our production environment. I plan fix it in our version (fork of OpenLDAP), immediatly after other crashes. See https://github.com/ReOpen/ReOpenLDAP/issues/3
how does patching your fork help anyone here? Do you submit patches here too?
Marc, you should look into OpenLDAP's ITS to see what's going on.
Ciao, Michael.
Currently it is no OpenLDAP's ITS for this bug, unfortunatelly
2015-01-22 17:39 GMT+03:00 Michael Ströder michael@stroeder.com:
Marc Patermann wrote:
Леонид Юрьев schrieb (20.01.2015 16:22 Uhr):
This is the bug, which reproduced stably in our production environment. I plan fix it in our version (fork of OpenLDAP), immediatly after other crashes. See https://github.com/ReOpen/ReOpenLDAP/issues/3
how does patching your fork help anyone here? Do you submit patches here too?
Marc, you should look into OpenLDAP's ITS to see what's going on.
Ciao, Michael.
Леонид Юрьев wrote:
Currently it is no OpenLDAP's ITS for this bug, unfortunatelly
Well, I understood Marcs question as being more about your github repo in general.
BTW: Many thanks for digging into OpenLDAP's code and contributing your patches.
Ciao, Michael.
2015-01-22 17:39 GMT+03:00 Michael Ströder michael@stroeder.com:
Marc Patermann wrote:
Леонид Юрьев schrieb (20.01.2015 16:22 Uhr):
This is the bug, which reproduced stably in our production environment. I plan fix it in our version (fork of OpenLDAP), immediatly after other crashes. See https://github.com/ReOpen/ReOpenLDAP/issues/3
how does patching your fork help anyone here? Do you submit patches here too?
Marc, you should look into OpenLDAP's ITS to see what's going on.
Ciao, Michael.
Al afrunning@gmail.com schrieb am 20.01.2015 um 15:38 in Nachricht
CAAVuYqGpZ1JLeTsNghXCrZCAZV6+XJH1Uhj=53QwWkhfV+WpxA@mail.gmail.com:
Hi All -
I'm having an odd issue where on a rare occasion (a couple of times a week), a new LDAP user entry is being deleted shortly after it is created. Sometimes it happens within a few minutes, sometimes it happens within an hour or so.
I have a 4 way multi-master setup, with all writes being directed at a single server with a load balancer. I have the auditlog enabled (from
I suspect your MM configuration has a problem. Maybe try to circumvent the load balancer and add a user to each of the four servers to find out whether the effect occurs on every server. Did you also check system clocks for current time?
failed attempts at delta sync) and I see auditDelete entries in the auditdb, but its being executed from the internal admin user, not a "real" user. I do not see anything suspect in my system logs running at the normal loglevel.
I have experience with auditlog, but with accesslog you could see the "modifier" of a change (that is not triggered by openLDAP itself)...
I'm running 2.4.39 on Redhat 6, x64 with mdb. Below is a snippet of my configuration from the specific database in question. Does anyone know why this might be occurring? Any idea on how to further troubleshoot this issue?
Thanks in advance -
Al
dn: olcDatabase={1}mdb,cn=config objectClass: olcDatabaseConfig objectClass: olcMdbConfig olcDatabase: {1}mdb olcDbDirectory: /PATH/TO/OPENLDAP/var/openldap-data olcSuffix: dc=company,dc=com olcAddContentAcl: FALSE olcLastMod: TRUE olcLimits: {0}dn.base="XXXXXXX" size.soft=unlimited size.hard=unlimited time.soft=unlimited time.hard=unlimited olcMaxDerefDepth: 15 olcReadOnly: FALSE olcRootDN: cn=Manager,dc=company,dc=com olcRootPW:: XXXXXXXXX olcSyncUseSubentry: FALSE olcMirrorMode: TRUE olcMonitoring: TRUE olcDbCheckpoint: 512 5 olcDbNoSync: TRUE olcDbIndex: objectClass eq olcDbIndex: entryUUID eq olcDbIndex: entryCSN eq olcDbIndex: cn pres,eq,sub olcDbIndex: uid eq olcDbIndex: uidNumber eq olcDbIndex: gidNumber eq olcDbIndex: uniqueMember eq olcDbIndex: nisNetgroupTriple eq olcDbIndex: sudoUser eq,sub olcDbIndex: mail eq olcDbIndex: pwmToken eq,sub olcDbIndex: memberOf eq olcDbMaxSize: 25000000000 olcDbMode: 0600 structuralObjectClass: olcMdbConfig entryUUID: xxxx-xxxx-xxxxx-xxxxx creatorsName: cn=config createTimestamp: 20111014131247Z olcSyncrepl: {0}rid=011 provider=ldap://server1:21389/ bind method=simple timeout=0 network-timeout=0 binddn="XXXXXXX" credentials="XXXX" keepalive=0:0:0 startt ls=critical filter="(objectclass=*)" searchbase="dc=company,dc=com" scope=sub schemachecking=off type=refreshOnly retry="30 +" interval=00:00:00:30 olcSyncrepl: {1}rid=012 provider=ldap://server2:21389/ bind method=simple timeout=0 network-timeout=0 binddn="XXXXXXX" credentials="XXXX" keepalive=0:0:0 startt ls=critical filter="(objectclass=*)" searchbase="dc=company,dc=com" scope=sub schemachecking=off type=refreshOnly retry="30 +" interval=00:00:00:30 olcSyncrepl: {2}rid=013 provider=ldap://server3:21389/ bind method=simple timeout=0 network-timeout=0 binddn="XXXXXXX" credentials="XXXX" keepalive=0:0:0 startt ls=critical filter="(objectclass=*)" searchbase="dc=company,dc=com" scope=sub schemachecking=off type=refreshOnly retry="30 +" interval=00:00:00:30 olcSyncrepl: {3}rid=014 provider=ldap://server4:21389/ bind method=simple timeout=0 network-timeout=0 binddn="XXXXXXX" credentials="XXXX" keepalive=0:0:0 startt ls=critical filter="(objectclass=*)" searchbase="dc=company,dc=com" scope=sub schemachecking=off type=refreshOnly retry="30 +" interval=00:00:00:30 entryCSN: 20140924095732.634049Z#000000#001#000000 modifiersName: cn=Manager,cn=config modifyTimestamp: 20140924095732Z
openldap-technical@openldap.org
-
Al -
Marc Patermann -
Michael Ströder -
Ulrich Windl -
Леонид Юрьев