1:38 a.m.
Hello list,
I have an OpenLDAP provider/consumer installation on two SLES10 systems.
One is set up as a provider LDAP, the second is a consumer LDAP using
"refreshOnly" synrepl synchronization. The LDAP provides user
information for a Samba installation.
The initial synchronization of the consumer works as expected. All LDAP
entries are copied to the consumer directory. But after some time,
usually when users log in into the Samba running with the provider LDAP,
nearly 50% of all LDAP entries on the consumer are deleted. This happens
without any change on the provider LDAP!
Checking the logs I found, that delete-messages can be found in the
consumers system log.
I do not understand the source of the problem. No entries in the
provider LDAP are deleted, so no entries should be deleted in the consumer.
To check the installation, I set up a second consumer in a VMWare
environment. And also in this system, which had been set up from scratch
and only holds the OpenLDAP-consumer, the entries are deleted at the
same time when they are deleted in the "real" OpenLDAP consumer system.
When I change an attribute of an entry in the provider LDAP which has
been deleted from the consumer by this process, like adding a
description, this change is forwarded to th consumer and the entry
"reappears" in the LDAP of the consumer.
Any help or suggestion is appreciated.
Thanks in advance
Joachim Hergeth
The providers slapd.conf is:
#
# See slapd.conf(5) for details on configuration options.
# This file should NOT be world readable.
#
include /etc/openldap/schema/core.schema
include /etc/openldap/schema/cosine.schema
include /etc/openldap/schema/inetorgperson.schema
#include /etc/openldap/schema/rfc2307bis.schema
include /etc/openldap/schema/nis.schema
include /etc/openldap/schema/yast.schema
include /etc/openldap/schema/samba3.schema
# Define global ACLs to disable default read access.
# Do not enable referrals until AFTER you have a working directory
# service AND an understanding of referrals.
#referral ldap://root.openldap.org
pidfile /var/run/slapd/slapd.pid
argsfile /var/run/slapd/slapd.args
# Load dynamic backend modules:
modulepath /usr/lib/openldap/modules
# moduleload back_ldap.la
# moduleload back_meta.la
# moduleload back_monitor.la
# moduleload back_perl.la
# Sample security restrictions
# Require integrity protection (prevent hijacking)
# Require 112-bit (3DES or better) encryption for updates
# Require 63-bit encryption for simple bind
# security ssf=1 update_ssf=112 simple_bind=64
# Sample access control policy:
# Root DSE: allow anyone to read it
# Subschema (sub)entry DSE: allow anyone to read it
# Other DSEs:
# Allow self write access to user password
# Allow anonymous users to authenticate
# Allow read access to everything else
# Directives needed to implement policy:
## Yast2 samba hack ACL
## allow the "ldap admin dn" access, but deny everyone else
access to attrs=SambaLMPassword,SambaNTPassword
by dn="cn=Administrator,dc=master-ldap,dc=gts-systems.de" write
by dn="cn=repladmin,dc=master-ldap,dc=gts-systems.de" write
by * none
## Yast2 samba hack ACL done
access to dn.base=""
by dn="cn=repladmin,dc=master-ldap,dc=gts-systems.de" write
by self write
by * auth
access to dn.base="cn=Subschema"
by dn="cn=repladmin,dc=master-ldap,dc=gts-systems.de" write
by * read
access to attrs=userPassword,userPKCS12
by dn="cn=repladmin,dc=master-ldap,dc=gts-systems.de" write
by self write
by * auth
access to attrs=shadowLastChange
by self write
by dn="cn=repladmin,dc=master-ldap,dc=gts-systems.de" write
by * read
access to *
by dn="cn=repladmin,dc=master-ldap,dc=gts-systems.de" write
by * read
by anonymous auth
# if no access controls are present, the default policy
# allows anyone and everyone to read anything but restricts
# updates to rootdn. (e.g., "access to * by * read")
#
# rootdn can always read and write EVERYTHING!
#######################################################################
# BDB database definitions
#######################################################################
schemacheck on
idletimeout 30
loglevel 131
backend bdb
database bdb
suffix "dc=master-ldap,dc=gts-systems.de"
rootdn "cn=Administrator,dc=master-ldap,dc=gts-systems.de"
rootpw <some working password>
directory /var/lib/ldap/
checkpoint 1024 5
cachesize 10000
index objectClass,entryCSN,entryUUID,uidNumber,gidNumber eq
index member,mail eq,pres
index cn,displayname,uid,sn,givenname sub,eq,pres
index memberUID eq
index sambaSID eq
index sambaPrimaryGroupSID eq
index sambaDomainName eq
index default sub
overlay syncprov
syncprov-checkpoint 100 1
syncprov-sessionlog 1
---------------------------------------------------------------------------
the consumers slapd.conf is:
#
# See slapd.conf(5) for details on configuration options.
# This file should NOT be world readable.
#
include /etc/openldap/schema/core.schema
include /etc/openldap/schema/cosine.schema
include /etc/openldap/schema/inetorgperson.schema
#include /etc/openldap/schema/rfc2307bis.schema
include /etc/openldap/schema/nis.schema
include /etc/openldap/schema/samba3.schema
include /etc/openldap/schema/yast.schema
# Define global ACLs to disable default read access.
# Do not enable referrals until AFTER you have a working directory
# service AND an understanding of referrals.
#referral ldap://root.openldap.org
pidfile /var/run/slapd/slapd.pid
argsfile /var/run/slapd/slapd.args
# Load dynamic backend modules:
modulepath /usr/lib/openldap/modules
# moduleload back_ldap.la
# moduleload back_meta.la
# moduleload back_monitor.la
# moduleload back_perl.la
# Sample security restrictions
# Require integrity protection (prevent hijacking)
# Require 112-bit (3DES or better) encryption for updates
# Require 63-bit encryption for simple bind
# security ssf=1 update_ssf=112 simple_bind=64
# Sample access control policy:
# Root DSE: allow anyone to read it
# Subschema (sub)entry DSE: allow anyone to read it
# Other DSEs:
# Allow self write access to user password
# Allow anonymous users to authenticate
# Allow read access to everything else
# Directives needed to implement policy:
access to dn.base=""
by dn="cn=repladmin,dc=master-ldap,dc=gts-systems.de" write
by self write
by * auth
access to dn.base="cn=Subschema"
by dn="cn=repladmin,dc=master-ldap,dc=gts-systems.de" write
by * read
access to attrs=userPassword,userPKCS12
by dn="cn=repladmin,dc=master-ldap,dc=gts-systems.de" write
by self write
by * auth
access to attrs=shadowLastChange
by dn="cn=repladmin,dc=master-ldap,dc=gts-systems.de" write
by self write
by * read
access to *
by dn="cn=repladmin,dc=master-ldap,dc=gts-systems.de" write
# by anonymous auth
by * read
# if no access controls are present, the default policy
# allows anyone and everyone to read anything but restricts
# updates to rootdn. (e.g., "access to * by * read")
#
# rootdn can always read and write EVERYTHING!
#######################################################################
# BDB database definitions
#######################################################################
schemackeck on
idletimeout 30
loglevel 15288
# TLSCertificateFile /etc/ssl/servercerts/servercert.pem
# TLSCACertificatePath /etc/ssl/certs/
# TLSCertificateKeyFile /etc/ssl/servercerts/serverkey.pem
database bdb
suffix "dc=master-ldap,dc=gts-systems.de"
rootdn "cn=Administrator,dc=master-ldap,dc=gts-systems.de"
rootpw "<some working password>"
directory /var/lib/ldap
checkpoint 1024 5
cachesize 10000
index objectClass,uidNumber,gidNumber,entryCSN,entryUUID eq
index member,mail eq,pres
index cn,displayname,uid,sn,givenname sub,eq,pres
index memberUID eq
index sambaSID eq
index sambaPrimaryGroupSID eq
index sambaDomainName eq
index default sub
syncrepl rid=321
provider=ldap://192.168.1.210:389
type=refreshOnly
interval=00:00:00:30
searchbase="dc=master-ldap,dc=gts-systems.de"
scope=sub
schemachecking=off
bindmethod=simple
binddn="cn=repladmin,dc=master-ldap,dc=gts-systems.de"
credentials=<another nice password>
---------------------------------------------------------------------------
1:48 a.m.
On Thursday 12 April 2007 10:38, Joachim Hergeth wrote:
Hello list,
I have an OpenLDAP provider/consumer installation on two SLES10 systems.
One is set up as a provider LDAP, the second is a consumer LDAP using
"refreshOnly" synrepl synchronization. The LDAP provides user
information for a Samba installation.
The initial synchronization of the consumer works as expected. All LDAP
entries are copied to the consumer directory. But after some time,
usually when users log in into the Samba running with the provider LDAP,
nearly 50% of all LDAP entries on the consumer are deleted. This happens
without any change on the provider LDAP!
I guess that you are testing this with the
OpenLDAP Version that shipped with
SLES10? Would you mind try a newer Version, e.g. the RPMs from
http://software.opensuse.org/download/OpenLDAP/SLE_10/ and check if the
problem is there as well? Note, that we have an update in the queue for
SLES10 to bring it to a more recent version.
Some general comments regarding you configuration (I guess you special problem
is not related to those):
- The provider config has a line "schemacheck on" this is not a valid
slapd.conf statement (IIRC it is from OpenLDAP 2.0.X times or even older)
- To debug syncrepl Problems it is most helpful to have the loglevel "sync"
enabled. You can to that by just adding "sync" to you "loglevel"
line.
- The "backend bdb" statement is superfluous
- The "syncprov-sessionlog 1" begins with whitespaces, might be a copy 'n
paste error in the mail. If it also begins with whitespaces in your
slapd.conf you should remove the whitespace. Otherwise it would be treated
as the continuation of the previous line (which is wrong). Additionally a
sessionlog of "1" operation doesn't make much sense IMO. I suggest you
either remove that option or set it to a more reasonable value.
- I don't know how large you database is (how many entries) but I should make
sure that the syncrepl consumer does not hit the sizelimit of your provider.
As you have not configure any sizelimit the default is used which is 500.
You can adjust the sizelimit with the "sizelimit" or the "limits"
directive
in slapd.conf (see slapd.conf man-page for details).
Checking the logs I found, that delete-messages can be found in the
consumers system log.
I do not understand the source of the problem. No entries in the
provider LDAP are deleted, so no entries should be deleted in the consumer.
Checking if it works with a more recent Version and logfiles with syncrepl
logging enabled might help to clear up the issue.
To check the installation, I set up a second consumer in a VMWare
environment. And also in this system, which had been set up from scratch
and only holds the OpenLDAP-consumer, the entries are deleted at the
same time when they are deleted in the "real" OpenLDAP consumer system.
When I change an attribute of an entry in the provider LDAP which has
been deleted from the consumer by this process, like adding a
description, this change is forwarded to th consumer and the entry
"reappears" in the LDAP of the consumer.
--
Ralf
3:38 a.m.
I would like to make a (late) conclusion to my initial question and the
answers I got.
Ralf Haferkamp suggested an LDAP update but also pointed to some flaws
in the slapd.conf file. We did not check the update (internal
reasons...) but I experimented with the problems in slapd.conf.
And after some trials it seems, that the erroneous entry
"syncprov-sessionlog 1" starting a continuation line was the reason for
the problems......
After fixing this, all entries in the consumer LDAP remained stable.
Thanks for your help
J.Hergeth
Ralf Haferkamp schrieb:
On Thursday 12 April 2007 10:38, Joachim Hergeth wrote:
> Hello list,
>
> I have an OpenLDAP provider/consumer installation on two SLES10 systems.
> One is set up as a provider LDAP, the second is a consumer LDAP using
> "refreshOnly" synrepl synchronization. The LDAP provides user
> information for a Samba installation.
>
> The initial synchronization of the consumer works as expected. All LDAP
> entries are copied to the consumer directory. But after some time,
> usually when users log in into the Samba running with the provider LDAP,
> nearly 50% of all LDAP entries on the consumer are deleted. This happens
> without any change on the provider LDAP!
I guess that you are testing this with the OpenLDAP Version that shipped with
SLES10? Would you mind try a newer Version, e.g. the RPMs from
http://software.opensuse.org/download/OpenLDAP/SLE_10/ and check if the
problem is there as well? Note, that we have an update in the queue for
SLES10 to bring it to a more recent version.
Some general comments regarding you configuration (I guess you special problem
is not related to those):
- The provider config has a line "schemacheck on" this is not a valid
slapd.conf statement (IIRC it is from OpenLDAP 2.0.X times or even older)
- To debug syncrepl Problems it is most helpful to have the loglevel "sync"
enabled. You can to that by just adding "sync" to you "loglevel"
line.
- The "backend bdb" statement is superfluous
- The "syncprov-sessionlog 1" begins with whitespaces, might be a copy 'n
paste error in the mail. If it also begins with whitespaces in your
slapd.conf you should remove the whitespace. Otherwise it would be treated
as the continuation of the previous line (which is wrong). Additionally a
sessionlog of "1" operation doesn't make much sense IMO. I suggest you
either remove that option or set it to a more reasonable value.
- I don't know how large you database is (how many entries) but I should make
sure that the syncrepl consumer does not hit the sizelimit of your provider.
As you have not configure any sizelimit the default is used which is 500.
You can adjust the sizelimit with the "sizelimit" or the "limits"
directive
in slapd.conf (see slapd.conf man-page for details).
> Checking the logs I found, that delete-messages can be found in the
> consumers system log.
>
> I do not understand the source of the problem. No entries in the
> provider LDAP are deleted, so no entries should be deleted in the consumer.
Checking if it works with a more recent Version and logfiles with syncrepl
logging enabled might help to clear up the issue.
> To check the installation, I set up a second consumer in a VMWare
> environment. And also in this system, which had been set up from scratch
> and only holds the OpenLDAP-consumer, the entries are deleted at the
> same time when they are deleted in the "real" OpenLDAP consumer system.
>
> When I change an attribute of an entry in the provider LDAP which has
> been deleted from the consumer by this process, like adding a
> description, this change is forwarded to th consumer and the entry
> "reappears" in the LDAP of the consumer.
1:35 a.m.
Joachim Hergeth wrote:
The initial synchronization of the consumer works as expected. All
LDAP
entries are copied to the consumer directory. But after some time,
usually when users log in into the Samba running with the provider LDAP,
nearly 50% of all LDAP entries on the consumer are deleted. This happens
without any change on the provider LDAP!
I am experiencing a similar problem, database entries being deleted when
they shouldn't be, on a number of replicas. This is causing disruption
to users and we'd really like to get it cleared up.
Here's the environment:
OpenLDAP 2.3.32 running on Debian 3.1 (Sarge)
compiled with sync logging patch discussed about 4 months ago
loglevel config sync on all servers
BDB 4.2 backend
Syncrepl replication all round
A "master" server (com)
- holds the master copy of the database
A number of servers that replicate directly from com
An "intermediate" server (wwsv04) that
- is on the same LAN and subnet as com
- replicates from com
- acts as provider for all other servers
88 servers/replicas in total
Approx 9000 records
All replicas are supposed to be complete copies
Nothing particularly fancy or clever going on
I have spent today dissecting the logs from two incidents this week in
which entries were erroneously deleted. Although the circumstances of
the two incidents are quite different, from examining the logs I believe
it is the same thing happening in each case.
Incident 1:
A test server ran out of space in /var. After cleaning up, rebooting and
doing db_recover, 244 entries were erroneously deleted. This server is
on the same building network as com and wwsv04 (different subnet), and
replicates from wwsv04.
Incident 2:
A production server on the other side of the world lost contact with its
provider, and after reconnecting, erroneously deleted 249 entries. This
server is connected to the main network via an OpenVPN tunnel across the
internet, and replicates from com. (The really odd thing about this is
that the network link came back up over an hour 1hr before the first
timestamp of this incident, but that is most likely a separate issue.)
I have attached three files (I hope they're not too big):
- log_analysis.ods - an OpenOffice spreadsheet containing
correlated log entries and my comments for both incidents
- incident_1.tgz - syslogs relating to incident 1
- and a CSV version of my analysis
of incident 1
- incident_2.tgz - likewise for incident 2
Replication configs are as follows:
---------------------------------
com
---
# Syncrepl provider
overlay syncprov
syncprov-checkpoint 10 5
syncprov-sessionlog 100
syncprov-reloadhint TRUE
-------------------------------------
wwsv04
------
# Syncrepl provider
overlay syncprov
syncprov-checkpoint 10 5
syncprov-sessionlog 100
syncprov-reloadhint TRUE
# syncrepl consumer
syncrepl rid=123
provider=ldap://com.example.co.nz
type=refreshAndPersist
searchbase="dc=example,dc=co,dc=nz"
scope=sub
schemachecking=off
bindmethod=simple
binddn="cn=root,dc=example,dc=co,dc=nz"
credentials=secret
retry=5,5,30,5,60,5,300,+
-------------------------------------
zzsv01
------
syncrepl rid=123
provider=ldap://wwsv04.example.co.nz
type=refreshAndPersist
searchbase="dc=example,dc=co,dc=nz"
scope=sub
schemachecking=off
bindmethod=simple
binddn="cn=root,dc=example,dc=co,dc=nz"
credentials=secret
retry=5,5,30,5,60,5,300,+
-------------------------------------
nosv01
------
syncrepl rid=123
provider=ldap://com.example.co.nz
type=refreshAndPersist
searchbase="dc=example,dc=co,dc=nz"
scope=sub
schemachecking=off
bindmethod=simple
binddn="cn=root,dc=example,dc=co,dc=nz"
credentials=secret
retry=5,5,30,5,60,5,300,+
-------------------------------------
--
Lesley Walker
Linux Systems Administrator
Opus International Consultants Ltd
Email lesley.walker(a)opus.co.nz
Tel +64 4 471 7002, Fax +64 4 473 3017
http://www.opus.co.nz
Level 9 Majestic Centre, 100 Willis Street, PO Box 12 343
Wellington, New Zealand
8:22 a.m.
On Wed, 18 Apr 2007, Lesley Walker wrote:
OpenLDAP 2.3.32 running on Debian 3.1 (Sarge)
[...]
A production server on the other side of the world lost contact with
its
provider, and after reconnecting, erroneously deleted 249 entries. This
syncprov with a session log drops entries. Makes you wonder if there could
be something wrong with the presence list like the numerous other ITS on
point...
syncprov.c 1.167 Thu Jan 25 01:06:33 2007 UTC; 2 months, 3 weeks ago by hyc
ITS#4813 don't limit the presence list to the current snapshot. add
some debug msgs for the sessionlog list.
So...try again with 2.3.35, and let us know.
3:53 p.m.
Aaron Richton wrote:
syncprov.c 1.167 Thu Jan 25 01:06:33 2007 UTC; 2 months, 3 weeks ago
by hyc
ITS#4813 don't limit the presence list to the current snapshot. add
some debug msgs for the sessionlog list.
This would seem to match at least part of the problem.
So...try again with 2.3.35, and let us know.
I can put it on a test server, but I'm not sure how I'll know if it's
fixed, since it doesn't happen all the time and I haven't yet been able
to identify any common triggers.
As discussed previously, I won't be putting into production any version
that hasn't been declared stable.
--
Lesley Walker
Linux Systems Administrator
Opus International Consultants Ltd
Email lesley.walker(a)opus.co.nz
Tel +64 4 471 7002, Fax +64 4 473 3017
http://www.opus.co.nz
Level 9 Majestic Centre, 100 Willis Street, PO Box 12 343
Wellington, New Zealand
5871
days inactive
5890
days old
openldap-software@openldap.org
5 comments
4 participants
participants (4)
-
Aaron Richton -
Joachim Hergeth -
Lesley Walker -
Ralf Haferkamp