MDB data replication issues
by Gurjot Kaur
Hi,
I am using a OpenLDAP 2.4.44 Multi master configuration with two slapd servers, master and replica using MDB backend. I got a problem in replicating when the data is added using slapadd.
I have two slapd with ports 2016 and 2017. slapd.conf file for both the servers are attached.
Scenario 1:
When an LDIF entry is added using ldapadd or deleted using ldapdelete, it gets replicated in the replica server correctly.
Below is the ldapsearch result om Master server:
GURKES254 linus> ldapsearch -h xx.xx.xx.xx -p 2016 -b "dc=my-domain,dc=com" "ou=Test9"
# extended LDIF
#
# LDAPv3
# base <dc=my-domain,dc=com> with scope subtree
# filter: ou=Test9
# requesting: ALL
#
# Test9, people, my-domain.com
dn: ou=Test9,ou=people,dc=my-domain,dc=com
ou: Test9
objectClass: organizationalUnit
companyName: Test9Grp
# search result
search: 2
result: 0 Success
# numResponses: 2
# numEntries: 1
Below is the ldapsearch result om replica server:
GURKES254 linus> ldapsearch -h xx.xx.xx.xx -p 2017 -b "dc=my-domain,dc=com" "ou=Test9"
# extended LDIF
#
# LDAPv3
# base <dc=my-domain,dc=com> with scope subtree
# filter: ou=Test9
# requesting: ALL
#
# Test9, people, my-domain.com
dn: ou=Test9,ou=people,dc=my-domain,dc=com
ou: Test9
objectClass: organizationalUnit
companyName: Test9Grp
# search result
search: 2
result: 0 Success
# numResponses: 2
# numEntries: 1
Scenario 2:
When an LDIF entry is imported using slapadd, it doesn't get replicated in the replica server at all.
Below is the ldapsearch result om Master server:
GURKES254 linus> ldapsearch -h xx.xx.xx.xx -p 2016 -b "dc=my-domain,dc=com" "ou=Test9"
# extended LDIF
#
# LDAPv3
# base <dc=my-domain,dc=com> with scope subtree
# filter: ou=Test9
# requesting: ALL
#
# Test9, people, my-domain.com
dn: ou=Test9,ou=people,dc=my-domain,dc=com
ou: Test9
objectClass: organizationalUnit
companyName: Test9Grp
# search result
search: 2
result: 0 Success
# numResponses: 2
# numEntries: 1
Below is the ldapsearch result om replica server:
GURKES254 linus> ldapsearch -h xx.xx.xx.xx -p 2017 -b "dc=my-domain,dc=com" "ou=Test9"
# extended LDIF
#
# LDAPv3
# base <dc=my-domain,dc=com> with scope subtree
# filter: ou=Test9
# requesting: ALL
#
# search result
search: 2
result: 0 Success
# numResponses: 1
Please let me know in case any other information is required.
Br
Gurjot Kaur
"DISCLAIMER: This message is proprietary to Aricent and is intended solely for the use of the individual to whom it is addressed. It may contain privileged or confidential information and should not be circulated or used for any purpose other than for what it is intended. If you have received this message in error, please notify the originator immediately. If you are not the intended recipient, you are notified that you are strictly prohibited from using, copying, altering, or disclosing the contents of this message. Aricent accepts no responsibility for loss or damage arising from the use of the information transmitted by this email including damage from virus."
7 years, 5 months
unclear documentation about openldap ACL definitions
by Florian Best
Hello,
studying the slapd.access man page left me with an open question regarding the
control of object creation:
* How to allow the creation of objects with a specific objectclass only?
For example, I want to prevent that an object with a object class other
than 'foobar' is created.
Assumming the following LDIF should be valid for an "add" operation:
> dn: uid=anton1,cn=settings,dc=ldap,dc=base
> objectClass: foobar
> uid: anton1
And this one to be invalid:
> dn: uid=anton1,cn=settings,dc=ldap,dc=base
> objectClass: CourierMailAccount
> objectClass: univentionAdminUserSettings
> uid: anton1
> uidNumber: 0
> gidNumber: 0
All of the following examples aren't doing their job when *creating* an
entry. Most of them (and even more simple ones) work fine when *modifying* an entry:
## 1. a attribute blacklist ##
access to dn="cn=settings,dc=ldap,dc=base" attrs="entry,children"
by users write
by * +0 break
access to dn.regex="^uid=([^,]+),cn=settings,dc=ldap,dc=base$"
filter="objectClass=foobar" attrs=!foobar
by dn.regex="^uid=$1,.*dc=ldap,dc=base$$" none
by * +0 break
access to dn.regex="^uid=([^,]+),cn=settings,dc=ldap,dc=base$"
filter="objectClass=foobar" attrs=entry,@foobar
by dn.regex="^uid=$1,.*dc=ldap,dc=base$$" write
by * none
##################
→ does probably not work because "!foobar" also disallows "objectClass"?
## 2. a negative regex blacklist ##
access to dn="cn=settings,dc=ldap,dc=base" attrs="entry,children"
by users write
by * +0 break
access to dn.regex="^uid=([^,]+),cn=settings,dc=ldap,dc=base$"
attrs=objectClass
val.regex="^([^f]|f[^o]|fo[^o]|foo[^b]|foob[^a]|fooba[^r])$"
by dn.regex="^uid=$1,.*dc=ldap,dc=base$$" none
by * +0 break
access to dn.regex="^uid=([^,]+),cn=settings,dc=ldap,dc=base$"
filter="objectClass=foobar" attrs=entry,@foobar
by dn.regex="^uid=$1,.*dc=ldap,dc=base$$" write
by * none
##################
## 3. a whitelist ##
access to dn="cn=settings,dc=ldap,dc=base" attrs="entry,children"
by users write
by * +0 break
access to dn.regex="^uid=([^,]+),cn=settings,dc=ldap,dc=base$"
attrs=objectClass value="foobar"
by dn.regex="^uid=$1,.*dc=ldap,dc=base$$" write
by * +0 break
access to dn.regex="^uid=([^,]+),cn=settings,dc=ldap,dc=base$"
attrs=objectClass
by dn.regex="^uid=$1,.*dc=ldap,dc=base$$" none
by * +0 break
access to dn.regex="^uid=([^,]+),cn=settings,dc=ldap,dc=base$"
filter="objectClass=foobar" attrs=entry,@foobar
by dn.regex="^uid=$1,.*dc=ldap,dc=base$$" write
by * none
##################
* Is the filter="…" in the "to" clause of an ACL evaluated when "adding"
an entry?
* Is the filter="…" when modifying an entry also applied against the
modified result or only against the current state of the object?
* the manpage says the syntax is "attrs=<attrlist>[
val[/matchingRule][.<attrstyle>]=<attrval>]" → The hardcoded "val" can
also be "value"
* no multiple value-lists can be used in a single defintion (e.g. access
to attrs="objectClass val=foo" attrs="someAttribute val="bar" or
attrs="objectClass val=foo,someAttribute val=bar")
* "matchingRule" is nowhere defined in the manpage
Some further suggestions for the development are:
* It would reduce a lot of redundancy if multiple "to" statements could
be used in one ACL definition (so that the different by clauses doesn't
always need to be copied).
* If the "by" clause would also have a filter="" one wouldn't need to
use "set"s anymore - sets are slower and doesn't even work with all
things (e.g. if you have special characters in the DN). There is no way
to escape "]" / "[" and urlencode things which are e.g. used in a LDAP
URI filter. This can even lead to security issues.
* ACL rules can't be bound to the ldap operation (search, auth, add,
modify, delete, ...), you can only remove e.g. some of the permission
bits (e.g. access to if-operation="search" ...)
* Using backreferences of the DN in the filter="" or attrs="" would also
be very handy (how to restrict e.g. the "uid" value to be only what's in
the DN of the target/operating user?)
Best regards
Florian
--
Florian Best
Open Source Software Engineer
Univention GmbH
be open
Mary-Somerville-Str.1
28359 Bremen
Tel.: +49 421 22232-0
Fax : +49 421 22232-99
best(a)univention.de
http://www.univention.de
Geschäftsführer: Peter H. Ganten
HRB 20755 Amtsgericht Bremen
Steuer-Nr.: 71-597-02876
7 years, 5 months
Problem in replicated entries
by Gurjot Kaur
Hi,
I have noticed a problem in OpenLDAP multimaster 2.4.44 with MDB backend.
I did the following steps:
i) Add four entries into Master 1 (using slapadd with option -w).
ii) Configure Master 1 and Master 2 in Multi Master mode. Syncrepl information is given below:
##############
Master1 Starts port (2016)
##############
syncRepl rid=100
provider=ldap://xx.xx.xx.xx:2017
type=refreshAndPersist
retry="5 + 5 +"
searchbase="dc=my-domain,dc=com"
attrs=*
interval=00:00:00:09
schemachecking=off
bindmethod=simple
binddn="cn=Manager, dc=my-domain,dc=com"
credentials=secret
mirrormode on
overlay syncprov
# contextCSN saved to database every 100 updates or ten minutes
syncprov-checkpoint 100 10
syncprov-sessionlog 100
##############
Master1 Ends
##############
##############
Master2 Starts port (2017)
##############
syncRepl rid=100
provider=ldap://xx.xx.xx.xx:2016
type=refreshAndPersist
retry="5 + 5 +"
searchbase="dc=my-domain,dc=com"
attrs=*
interval=00:00:00:09
schemachecking=off
bindmethod=simple
binddn="cn=Manager, dc=my-domain,dc=com"
credentials=secret
mirrormode on
overlay syncprov
# contextCSN saved to database every 100 updates or ten minutes
syncprov-checkpoint 100 10
syncprov-sessionlog 100
##############
Master2 Ends
##############
iii) These four entries get replicated on the Master 2 correctly and are visible from ldap browser accurately.
iv) Add three more entries to Master 1. (using slapadd with option -w)
v) After restarting both Master 1 and Master 2, on Ldap Browser the new added entries are visible in Master 1 correctly. But in Master 2, all the previous entries got missing. Ldap browser is giving error "No entries returned" ( It seems just like Master 2 DB got empty)
vi) But then I execute ldapsearch command on Master 2 as given below:
###################################################
ldapsearch -h xx.xx.xx.xx -p 2017 -b "dc=my-domain,dc=com"
###################################################
The above command returns all the entries exactly same to the Master 1 with the below search response:
# search result
search: 2
result: 0 Success
# numResponses: 10
# numEntries: 9
Now I am unable to understand how it this possible that the entries exist in the DB but not showing through Ldap browser. ( I have checked Master 2 entries in different ldap browsers too)
But it is just showing empty DB. Master 1 entries are showing correctly in Ldap browser.
Context CSN for both the servers is same as given below:
##################################
ldapsearch -H ldap://xx.xx.xx.xx:2016 -LLL -x -s base -b "dc=my-domain,dc=com" contextCSN
dn: "dc=my-domain,dc=com"
contextCSN: 20160705065521.205150Z#000000#000#000000
ldapsearch -H ldap://xx.xx.xx.xx:2017 -LLL -x -s base -b "dc=my-domain,dc=com" contextCSN
dn: "dc=my-domain,dc=com"
contextCSN: 20160705065521.205150Z#000000#000#000000
##################################
Can you please let me know how it can happen and how to solve this problem.
Best Regards,
Gurjot Kaur
"DISCLAIMER: This message is proprietary to Aricent and is intended solely for the use of the individual to whom it is addressed. It may contain privileged or confidential information and should not be circulated or used for any purpose other than for what it is intended. If you have received this message in error, please notify the originator immediately. If you are not the intended recipient, you are notified that you are strictly prohibited from using, copying, altering, or disclosing the contents of this message. Aricent accepts no responsibility for loss or damage arising from the use of the information transmitted by this email including damage from virus."
7 years, 5 months
syncrepl constantly desyncing
by Eugene M. Zheganin
Hi,
I'm using a configuration with two slapd servers, master and replica.
The problem is that replica is constantly desyncing. I'm monitoring it's
number of users and groups and the cound of members of one particular
group. Users and group entities are always in sync, but entities like
groupOfUniqueNames are desyncing - they are not receiveing deltas from
master, keeping their members count constant.
My config (done according to the documentation):
===Cut===
overlay syncprov
syncprov-checkpoint 100 10
syncprov-sessionlog 100
syncrepl rid=123
provider=ldap://xx.xx.xx.xx:389
type=refreshAndPersist
interval=00:00:10:00
retry="60 10 300 +"
filter="(objectClass=*)"
searchbase="dc=my,dc=domain"
attrs="*,+"
schemachecking=off
bindmethod=simple
binddn="uid=proxy,ou=accounts,ou=internal,dc=my,dc=domain"
credentials=XXXXXXXXXXXXXX
===Cut===
I've also tried the refreshOnly method, which gave me same result. In
order to resync replica I have to flush the directory contents each time
and restart the slapd. I'm also suspecting that this desyncing happens
because for some reason replica slapd isn't refreshing attribute values,
only entities themselves: today I found a user, which userPassword
attributes were out of sync on the replica. As far as I understand the
documentation, syncrepl should sync the attributes.
And the last question - is there any simple way to enable logging of
syncrepl warnings and errors ? My experience with openldap logging tells
me there's to mode of logging - "none" and "generate 10Gb of logs per
day", but may me it's just me.
Thanks.
Eugene.
7 years, 5 months
Caching nested groups (member:1.2.840.113556.1.4.1941)
by Ryan Hammond
We have an Active Directory domain with >6000 users and several hundred
groups. The group hierarchy is usually nested 4-5 levels deep. Queries for
the full set of nested groups for a user take > 10 seconds each. We have
an app (Nexus OSS) that asks for a user's nested groups on every
interaction (e.g. for each dependency needed to perform a maven build) and
does not perform local caching.
I am trying to implement a slapd + meta + pcache solution that will proxy
the LDAP query, cache the results, and significantly speed up user
interactions with this app. I have tried many configurations. I can
always proxy the request, but I haven't yet been able to fully cache the
results. I have had the most success with the passthru module from
https://github.com/cbueche/openldap-passthru and rwm. In this
configuration, slapd proxies inbound filters like "(member=CN=my
name,ou=users,dc=some,dc=structure)" and passes that on to AD as
"member:1.2.840.113556.1.4.1941:=CN=my
name,ou=userss,dc=some,dc=structure)". AD returns the group DNs but slapd
partially caches the result. The cached results are the direct group
memberships, not the complete set of nested groups.
Is what I want to do possible? If so, can someone please point me in the
right direction?
Here is my slapd.conf file:
-----------------------------
### Logging
###################################################################
loglevel -1
logfile /var/log/openldap.log
### Schema includes
###########################################################
include /etc/openldap/schema/microsoftattributetype.schema
include /etc/openldap/schema/microsoftattributetypestd.schema
include /etc/openldap/schema/corba.schema
include /etc/openldap/schema/core.schema
include /etc/openldap/schema/cosine.schema
include /etc/openldap/schema/duaconf.schema
include /etc/openldap/schema/dyngroup.schema
include /etc/openldap/schema/inetorgperson.schema
include /etc/openldap/schema/java.schema
include /etc/openldap/schema/misc.schema
include /etc/openldap/schema/nis.schema
include /etc/openldap/schema/openldap.schema
include /etc/openldap/schema/ppolicy.schema
include /etc/openldap/schema/collective.schema
include /etc/openldap/schema/microsoftobjectclass.schema
## Module paths
##############################################################
modulepath /usr/local/libexec/openldap
moduleload back_meta
moduleload back_ldap
moduleload back_bdb
moduleload mr_passthru
moduleload pcache
moduleload rwm
# Main settings
###############################################################
pidfile /var/run/openldap/slapd.pid
argsfile /var/run/openldap/slapd.args
### Database definition (Proxy to AD)
#########################################
database meta
chase-referrals yes
suffix "dc=some,dc=structure"
uri "ldap://a.b.c.d:3268/dc=some,dc=structure"
readonly yes
protocol-version 3
rebind-as-user yes
rootdn "CN=zNexus,OU=Service Accounts,OU=myorg,DC=some,DC=structure"
rootpw "--obfuscated--"
idassert-bind binddn="CN=zNexus,OU=Service
Accounts,OU=myorg,DC=some,DC=structure"
credentials="--obfuscated--"
bindmethod=simple
mode=none
idassert-authzFrom "*"
rewriteEngine on
#rewrite searches
rewriteContext searchFilter
rewriteRule "member=(.*)" "(member:1.2.840.113556.1.4.1941:=%1" ":"
rewriteRule "memberOf=(.*)" "(memberOf:1.2.840.113556.1.4.1941:=%1" ":"
overlay pcache
pcache bdb 100000 2 10000 600
directory /var/tmp/openldap-cache
cachesize 20000
pcacheMaxQueries 10000
index objectClass eq,pres
index userAccountControl eq
index member eq
index memberOf eq
index ou,cn,mail,surname,givenname eq,pres,sub
index objectCategory,sAMAccountName eq,pres,sub
pcacheAttrset 0 cn
pcacheTemplate (&(objectClass=)(&(cn=)(member=))) 0 600 60
pcacheTemplate (member=) 0 600 60
pcacheTemplate (memberOf=) 0 600 60
pcacheTemplate (&(cn=*)(member=)) 0 600 60
pcacheTemplate
(&(objectClass=)(sAMAccountName=)(&(objectCategory=)(userAccountControl=)))
0 600 60
pcacheAttrset 1 sAMAccountName cn mail labeledUri memberOf member
objectClass
pcacheTemplate
(&(objectClass=)(sAMAccountName=)(&(objectCategory=)(userAccountControl=)))
1 600 60
pcacheTemplate
(&(objectClass=)(sAMAccountName=*)(&(objectCategory=)(userAccountControl=)))
1 600 60
-----------------------------
Here is what the client sees the first and second time the query is run for
a user that is directly in one group but in many groups via nesting. (The
group DNs have been obfuscated but in reality they are all unique.) On the
first run, ldapsearch correctly returns 158 total (nested and direct)
groups. On the second run, ldapsearch only returns the one direct group.
### RUN 1 ###
ldap_initialize( ldap://127.0.0.1 )
filter: (member=cn=Some User,ou=Users,ou=External,dc=some,dc=structure)
requesting: cn
dn: cn=TT-RndGroup-Read,ou=...,ou=External,dc=some,dc=structure
cn: TT-RndGroup-Read
dn: cn=IO-RndGroup-Read,ou=...,ou=External,dc=some,dc=structure
cn: IO-RndGroup-Read
dn: cn=Vi-RndGroup-Read,ou=...,ou=External,dc=some,dc=structure
cn: Vi-RndGroup-Read
dn: cn=SE-RndGroup-Read,ou=...,ou=External,dc=some,dc=structure
cn: SE-RndGroup-Read
<snip 154 more CNs....>
real 0m14.167s
user 0m0.003s
sys 0m0.025s
### RUN 2 ###
ldap_initialize( ldap://127.0.0.1 )
filter: (member=cn=Some User,ou=Users,ou=External,dc=some,dc=structure)
requesting: cn
dn: cn=Vi-RndGroup-Read,ou=...,ou=External,dc=some,dc=structure
cn: Vi-RndGroup-Read
real 0m4.310s
user 0m0.000s
sys 0m0.004s
-----------------------------
Thanks,
Ryan
7 years, 5 months
RHEL7 issues maybe???
by Frank Swasey
Folks,
I am working on moving my OpenLDAP 2.4.44 environment from RHEL6 to
RHEL7. I want to double check an assumption I have made before I open a
report with Red Hat about the quality of their product.
I am assuming that with "loglevel sync" and an empty consumer, that
there should be one sincrepl_entry line that has "inserted UUID" in the
syslog output for every entry created. Is that a valid assumption?
Thanks,
--
Frank Swasey | http://www.uvm.edu/~fcs
Sr Systems Administrator | Always remember: You are UNIQUE,
University of Vermont | just like everyone else.
"I am not young enough to know everything." - Oscar Wilde (1854-1900)
7 years, 5 months
