RE: OPENLDAP SYNCREPL
by Quanah Gibson-Mount
--On Tuesday, March 13, 2012 2:36 PM -0400 "Borresen, John - 0442 - MITLL"
<john.borresen(a)ll.mit.edu> wrote:
> Thanks, Howard;
You are clearly missing the syncprov overlay on the accesslog DB, as I
already stated. If you fix that, you'll get a lot further.
--Quanah
--
Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
11 years, 6 months
RE: OPENLDAP SYNCREPL
by Quanah Gibson-Mount
--On Tuesday, March 13, 2012 1:12 PM -0400 "Borresen, John - 0442 - MITLL"
<john.borresen(a)ll.mit.edu> wrote:
> Thanks, Quanah;
>
> As requested:
>
> cn=config]# ll
> total 140
> -rw------- 1 ldap ldap 398 Jan 24 11:28 cn=module{0}.ldif
> -rw------- 1 ldap ldap 398 Feb 8 14:02 cn=module{1}.ldif
> -rw------- 1 ldap ldap 397 Feb 21 09:06 cn=module{2}.ldif
> -rw------- 1 ldap ldap 399 Feb 24 12:32 cn=module{3}.ldif
You really don't need 4 separate files. ;) Use my cn=module{0}.ldif as a
template for fixing yours. ;)
I would also note that you're using delta-syncrepl, not syncrepl, and you
failed to provide any of the configuration for the accesslog DB on the
provider. My guess is that is the actual source of the error you are
encountering. I.e., my guess is that you never loaded the syncprov overlay
on the master's accesslog DB, which is also required.
Here's an example configuration from my master's cn=accesslog configuration
for the syncprov overlay there. Note that they *do* differ between your
primary DB and your accesslog DB.
root@zre-ldap002:/opt/zimbra/data/ldap/config/cn=config# ls
olcDatabase\=\{2\}mdb
olcOverlay={0}syncprov.ldif
root@zre-ldap002:/opt/zimbra/data/ldap/config/cn=config# cat
olcDatabase\=\{2\}mdb/olcOverlay\=\{0\}syncprov.ldif
# AUTO-GENERATED FILE - DO NOT EDIT!! Use ldapmodify.
# CRC32 fd177258
dn: olcOverlay={0}syncprov
objectClass: olcOverlayConfig
objectClass: olcSyncProvConfig
olcOverlay: {0}syncprov
olcSpNoPresent: TRUE
olcSpReloadHint: TRUE
structuralObjectClass: olcSyncProvConfig
entryUUID: e97228d0-edf0-1030-98ed-dbd1519e60e2
creatorsName: cn=config
createTimestamp: 20120217202252Z
entryCSN: 20120217202252.161808Z#000000#000#000000
modifiersName: cn=config
modifyTimestamp: 20120217202252Z
--Quanah
--
Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
11 years, 6 months
Re-binding to a failed connection segfaults
by Jan Synacek
Hi list,
I've created a small reproducer, that calls ldap_sasl_interactive_bind_s after
it has been called once and failed, which causes a segfault.
I've traced this bug with gdb:
$ gdb ./reproducer
GNU gdb (GDB) Fedora (7.3.50.20110722-10.fc16)
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from
/home/jsynacek/work/bz784989-openldap-rebinding/reproducer...done.
(gdb) r
Starting program: /home/jsynacek/work/bz784989-openldap-rebinding/reproducer
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
ldap_sasl_interactive_bind: user selected: GSSAPI
ldap_int_sasl_bind: GSSAPI
ldap_new_connection 1 1 0
ldap_int_open_connection
ldap_connect_to_host: TCP localhost:636
ldap_new_socket: 7
ldap_prepare_socket: 7
ldap_connect_to_host: Trying ::1 636
ldap_pvt_connect: fd: 7 tm: -1 async: 0
TLS: error: tlsm_PR_Recv returned 0 - error 21:Is a directory
TLS: error: connect - force handshake failure: errno 21 - moznss error -5938
TLS: can't connect: TLS error -5938:Encountered end of file.
ldap_msgfree
ldap_err2string
bind failed: Can't contact LDAP server, retrying for fun and profit!
ldap_sasl_interactive_bind: user selected: GSSAPI
ldap_int_sasl_bind: GSSAPI
Program received signal SIGSEGV, Segmentation fault.
ldap_int_sasl_bind (ld=0x603130, dn=0x0, mechs=0x401a30 "GSSAPI", sctrls=0x0,
cctrls=0x0, flags=1,
interact=0x401660 <lutil_sasl_interact>, defaults=0x60cae0, result=0x0,
rmech=0x7fffffffd878,
msgid=0x7fffffffd88c) at ../../../libraries/libldap/cyrus.c:444
444 oldctx = ld->ld_defconn->lconn_sasl_authctx;
(gdb) p ld->ldc->ldc_defconn
$1 = (LDAPConn *) 0x0
If you set slapd to use TLS certs (uncomment the 'TLS*' lines in the config),
there is no segfault.
Reproducer and cn=config.ldif attached.
Original bugreport: https://bugzilla.redhat.com/show_bug.cgi?id=784989
Regards,
--
Jan Synacek
BaseOS team Brno
11 years, 6 months
olcTLSVerifyClient: demand not taking effect
by Peter Wood
Hi,
I setup openldap-2.4.23 server on centos6.2 using cn=config. The server
itself doesn't authenticate through ldap. Network clients are able to
authenticate but the network traffic is unencrypted.
Next step is to configure SSL/TLS. I was reading multiple sources of
documentation, trying to understand what I'm doing not just follow
instructions. I created a CA, generated certificate for the server and
setup these in cn=config:
olcTLSCACertificateFile: /etc/openldap/cacerts/CA.crt
olcTLSCertificateFile: /etc/pki/tls/certs/ldapserver.crt
olcTLSCertificateKeyFile: /etc/pki/tls/private/ldapserver-nopass.key
olcTLSVerifyClient: demand
I was expecting that after slapd restart clients will not being able to
authenticate but they still can and the traffic is still unencrypted. On
the server if I run tcpdump I can clearly see usernames and passwords.
Service slapd is running as user ldap and I made sure the user has read
access to all cert files and private keys. I enabled logging level
"olcLogLevel: stats" but I don't see any errors in the log file.
Shouldn't "olcTLSVerifyClient: demand" drop the connection if the client
doesn't provide valid certificate?
Thank you for any pointers.
--
Peter
11 years, 6 months
Dynamic to Static LDAP
by Gaurav Gugnani
Hello All,
We are using openldap 2.4 in our systems with berkley DB as a backend.
Now, in current scenario we are using static LDAP i.e.
/etc/openldap/slapd.conf and whenever we want to change any configuration
parameter then we need to re-start LDAP.
So, we decided to switch to dynamic LDAP i.e. using cn=config.
So, *my Q's is:*
I made my system running on dynamic LDAP like from past 2 months and in
this span of time i implemented several changes. Now, suppose today i plan
to again move back to use static configuration - Will i able to save all my
changes what i did in all this time? OR All changes made are lost?
If Lost - Is there any way to retrieve the changes which i made?
Please suggest.
Thanks and Regards,
Gaurav Gugnani
11 years, 6 months
How to Modify OpenLDAP Entry for Replication
by Scott McLane Gardner
OpenLDAP newbie here, hopefully this is the right list to send this sort
of thing to. I have successfully set up two OpenLDAP servers one as the
master and the other replicating the masters database using the
instructions in the ubuntu documentation, here:
https://help.ubuntu.com/11.10/serverguide/C/openldap-server.html#openldap-s
erver-replication
What I'd like to do now is change the admin password on the master, but
that will break the replication. How can I change the consumer
configuration (further down the page linked above) to update it with the
new password? I'm sure that using ldapmodify is probably what I want, I
just don't know enough to do it correctly.
11 years, 6 months
Large delay before transmitting search result
by Rene B. Elgaard
Hello,
OpenLDAP 2.4.26 + BerkeleyDB 5.2.28, also seen on OpenLDAP 2.4.22 + BerkeleyDB 4.6.21
I am trying to investigate a response time issue.
Basically I am doing a search with ldapsearch that results in the transfer of an 8.5 Mb binary entry. This takes around 800 ms measured on the server itself from the request has been received until the response has been delivered..
Looking at what is going on on the network I see a large delay between slapd receiving the search request and the actual start of the transfer. This delay accounts for around 600ms.
So what is going on during this time ?
I am on Solaris 10, so I use dtrace to look at slapd, and it turns out that it calls memmove(<ADDR>,<ADDR> + 1, ~8.5 Mb).
That is it shifts the result 1 byte, and it does this 4 times. Each memmove takes around 150 ms, which accounts almost completely for the delay before doing the transfer over network.
I have written a small C program which does the same memmove and verified that on my setup, it does indeed take 150ms to shift 8.5 Mb 1 byte.
The stacktrace up until each memmove is
0 42451 memmove:entry *** Moving from 213650401 to 213650402 ***
libc.so.1`memmove
slapd`0x82461c9
slapd`ber_printf+0x4ff
slapd`slap_send_search_entry+0x1a94
slapd`bdb_search+0x1ce0
slapd`fe_op_search+0x574
slapd`do_search+0xc82
slapd`0x80927a9
slapd`0x8092d21
slapd`0x8210034
The first line shows the source and destination address for memmove. The numbers themselves are not interesting, but the difference (1) is.
Why is it shifting the buffer 1 byte ?
If this could be optimized somehow, it could lead to a huge improvement in response time.
Thanks,
Rene
11 years, 6 months
Re: slow or inconsistent syncrepl
by Amol Kulkarni
I see that I'm outdated in my tuning as per http://wiki.zimbra.com/wiki/OpenLDAP_Performance_Tuning.
I'll do these changes and observe the servers.
In the mean time a new observation regarding syncrepl is that if I do ldapmodify for 5 - 10 entries in one go the changes get replicated but if I change about 300 entries in one go, then some entries do not get replicated on some servers. These changes dont get replicated until I change those entries again.
Does this indicate provider resource problem or consumer resource problem ( or normal behaviour ) ?
----- Original Message -----
From: Quanah Gibson-Mount
Sent: 03/11/12 02:05 AM
To: Amol Kulkarni, Howard Chu
Subject: Re: slow or inconsistent syncrepl
--On Saturday, March 10, 2012 2:34 PM +0100 Amol Kulkarni <amolkulkarni(a)gmx.com> wrote: > > >>> That depends entirely on the speed of your server and network. > > Taking the server part first - I'm also doubt that my provider server is > underpowered. > > Following are the ldap specific parameters on our provider server : > > entries : 0.5 million > avg entry size : 1k to 4k > cachesize : 0.5 million > dncachesize : 0.5 million > database type : bdb > bdb cachesize : 3 gb > ldap threads : 16 > > Foll is hardware configuration of the provider server : > ram : 8gb > swap : 8gb > cpu : Virtual machine with 4 cpus. (vmware vsphere) > architecture : 64 bit > > Also we have some administrative services/tasks running on the provider > server other than openldap. But the sar output seems normal i.e iowait > below 10 and idle time above 50. > > Does the hardware configuration seem ok for the given ldap size and > configuration ? > > If not, can u suggest some changes? You don't provide any useful information for suggesting changes. I would advise you read over <http://wiki.zimbra.com/wiki/OpenLDAP_Performance_Tuning> -- Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
11 years, 6 months
Re: slow or inconsistent syncrepl
by Amol Kulkarni
>>That depends entirely on the speed of your server and network. Taking the server part first - I'm also doubt that my provider server is underpowered. Following are the ldap specific parameters on our provider server : entries : 0.5 million avg entry size : 1k to 4k cachesize : 0.5 million dncachesize : 0.5 million database type : bdb bdb cachesize : 3 gb ldap threads : 16 Foll is hardware configuration of the provider server : ram : 8gb swap : 8gb cpu : Virtual machine with 4 cpus. (vmware vsphere) architecture : 64 bit Also we have some administrative services/tasks running on the provider server other than openldap. But the sar output seems normal i.e iowait below 10 and idle time above 50. Does the hardware configuration seem ok for the given ldap size and configuration ? If not, can u suggest some changes?
Thanks and Regards,
Amol Kulkarni.
----- Original Message -----
From: Howard Chu
Sent: 03/10/12 01:29 PM
To: Amol Kulkarni
Subject: Re: slow or inconsistent syncrepl
Amol Kulkarni wrote: > Dear Quanah, > > Thanks a lot for these 2 pointers. I'll check out the 2.4.30 version. > We had used delta syncrepl earlier but our accesslog size used to grow > suddenly sometimes and the disk used to get full crashing/hanging the ldap > service itself on the provider. But at that time we had kept the max age for > the accesslog to be 7 days. I'll reduce it and give it a try again. > > Also it would be helpful if you can throw some light on : > > 2. On a really busy ldap server, can replication slow down drastically? i.e > does the read operations affect the replication in any way? Syncrepl executes as an LDAP Search operation, so of course it competes for server resources with other Search activity on the server. > 4. We are currently having about 60 consumers - is this too much ? What can be > the max numbers of consumers ? That depends entirely on the speed of your server and network. If you're seeing that replication speed is inconsistent, you probably should reduce the load on the provider. A simple approach in your scenario would be to just point the 10 consumers on the fast LAN at the provider, and configure these consumers to act as providers for your WAN nodes (i.e., cascaded replication). > 5. Sometimes we urgently need some particular node to be present on the > consumer - for which we cannot wait - in that case we get ldif of that node > from provider and do ldapadd on the consumer ( mirrormode is ON on the > consumers ). Is this safe and correct or could it cause some side effects ? Is > there a better way to handle it? If you have configured distinct serverIDs for each consumer, this might work. Otherwise, no, not safe, not correct. Fix your configuration layout, or fix your apps. If your apps can't wait then they are mis-designed; there is no such thing as instantaneous propagation of information in the real world. > Thanks and Regards, > Amol Kulkarni. > >> ----- Original Message ----- >> >> From: Quanah Gibson-Mount >> >> Sent: 03/09/12 11:57 PM >> >> To: Amol Kulkarni, openldap-technical(a)openldap.org >> >> Subject: Re: slow or inconsistent syncrepl >> >> >> >> --On Friday, March 09, 2012 2:20 PM +0100 Amol Kulkarni >> <amolkulkarni(a)gmx.com> wrote: >> >> > I have a following openldap setup with syncrepl : >> > - openldap version 2.4.23 >> >> This is your #1 issue. >> >> > - 1 provider and about 10 consumers in lan and 50 consumers on wan >> >> This is your #2 issue. >> >> Upgrade to a stable release. Use delta-syncrepl, which uses significantly >> less bandwidth than syncrepl. >> >> --Quanah >> >> -- >> >> Quanah Gibson-Mount >> Sr. Member of Technical Staff >> Zimbra, Inc >> A Division of VMware, Inc. >> -------------------- >> Zimbra :: the leader in open source messaging and collaboration -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
11 years, 6 months
OpenLDAP 2.4 : replication doesn't work when customer is stopped
by PROST Frédéric
Hello,
I configured MirrorMode replication between 2 openldap 2.4 node installed on Debian (from apt).
Everything is working fine when the two nodes are online but if I stop the second node, and add new datas to the first node, then restart the second node, the new data are not synced.
However, if I then add new datas on node 1, they are replicated to node2 without problem.
Here is a scenario of this problem :
1/ node1 and node 2 are online : I add user1 to node 1 => user1 appears on node2 => ok
2/ node1 is online and node2 is off : I add user2 on node1 => nothing happens on node2 as it is off => ok
3/ I restart node2 => user2 is not replicated to node2 => not ok
4/ node1 and node 2 are online : I add user3 to node 1 => user3 appears on node2 => ok
At the end of this scenario, node1 contains user1, user2 and user3 and node2 contains only user1 and user3 (but not user2).
How can I slove this problem ?
Thank you for your help,
Best regards,
Fred
Here is my config :
version: 1
dn: cn=config
objectClass: olcGlobal
cn: config
olcAllows: bind_v2
olcArgsFile: /var/run/slapd/slapd.args
olcLogLevel: any
olcPidFile: /var/run/slapd/slapd.pid
olcServerID: 1 ldap://192.168.1.103
olcServerID: 2 ldap://192.168.1.104
olcSizeLimit: 1000000
olcToolThreads: 1
dn: cn=module{0},cn=config
objectClass: olcModuleList
cn: module{0}
olcModuleLoad: {0}back_hdb
olcModuleLoad: {1}syncprov
olcModulePath: /usr/lib/ldap
dn: olcBackend={0}hdb,cn=config
objectClass: olcBackendConfig
olcBackend: {0}hdb
dn: olcDatabase={-1}frontend,cn=config
objectClass: olcDatabaseConfig
objectClass: olcFrontendConfig
olcDatabase: {-1}frontend
olcAccess: {0}to * by dn.exact=gidNumber=0+uidNumber=0,cn=peercred,cn=extern
al,cn=auth manage by * break
olcAccess: {1}to dn.exact="" by * read
olcAccess: {2}to dn.base="cn=Subschema" by * read
olcSizeLimit: 500
dn: olcDatabase={0}config,cn=config
objectClass: olcDatabaseConfig
olcDatabase: {0}config
olcAccess: {0}to * by dn.exact="uid=syncrepl,dc=tracteur91,dc=local" read by
* break
olcAccess: {1}to * by dn.exact=gidNumber=0+uidNumber=0,cn=peercred,cn=extern
al,cn=auth manage by * break
olcLimits: {0}dn.exact="uid=syncrepl,dc=tracteur91,dc=local" size=unlimited
olcMirrorMode: TRUE
olcRootDN: cn=admin,cn=config
olcRootPW: {MD5}BkY718PMIcgBNjpfXmGpOA==
olcSyncrepl: {0}rid=001 provider="ldap://192.168.1.103" searchbase="cn=confi
g" type=refreshAndPersist bindmethod=simple binddn="uid=syncrepl,dc=tracteu
r91,dc=local" credentials="Tr@cteur91" retry="30 +" network-timeout=5 timeo
ut=30
olcSyncrepl: {1}rid=002 provider="ldap://192.168.1.104" searchbase="cn=confi
g" type=refreshAndPersist bindmethod=simple binddn="uid=syncrepl,dc=tracteu
r91,dc=local" credentials="Tr@cteur91" retry="30 +" network-timeout=5 timeo
ut=30
dn: olcOverlay={0}syncprov,olcDatabase={0}config,cn=config
objectClass: olcOverlayConfig
objectClass: olcSyncProvConfig
olcOverlay: {0}syncprov
olcSpCheckpoint: 100 5
dn: olcDatabase={1}hdb,cn=config
objectClass: olcDatabaseConfig
objectClass: olcHdbConfig
olcDatabase: {1}hdb
olcDbDirectory: /var/lib/ldap
olcAccess: {0}to * by dn.exact="uid=syncrepl,dc=tracteur91,dc=local" read by
* break
olcAccess: {1}to attrs=userPassword,shadowLastChange by self write by anonym
ous auth by dn="cn=admin,dc=tracteur91,dc=local" write by * none
olcAccess: {2}to dn.base="" by * read
olcAccess: {3}to * by self write by dn="cn=admin,dc=tracteur91,dc=local" wri
te by * read
olcDbCheckpoint: 512 30
olcDbConfig: {0}set_cachesize 0 2097152 0
olcDbConfig: {1}set_lk_max_objects 1500
olcDbConfig: {2}set_lk_max_locks 1500
olcDbConfig: {3}set_lk_max_lockers 1500
olcDbIndex: objectClass eq
olcDbIndex: uid eq
olcDbIndex: cn eq
olcDbIndex: ou eq
olcDbIndex: dc eq
olcDbIndex: entryCSN eq
olcDbIndex: entryUUID eq
olcLastMod: TRUE
olcLimits: {0}dn.exact="uid=syncrepl,dc=tracteur91,dc=local" size=unlimited
olcMirrorMode: TRUE
olcRootDN: cn=admin,dc=tracteur91,dc=local
olcRootPW: {SSHA}ZtvvlHUQYloI17cv2/cjPFmx51+Ut/+5
olcSuffix: dc=tracteur91,dc=local
olcSyncrepl: {0}rid=003 provider="ldap://192.168.1.103" searchbase="dc=tract
eur91,dc=local" type=refreshAndPersist bindmethod=simple binddn="uid=syncre
pl,dc=tracteur91,dc=local" credentials="Tr@cteur91" retry="30 +" network-ti
meout=5 timeout=30
olcSyncrepl: {1}rid=004 provider="ldap://192.168.1.104" searchbase="dc=tract
eur91,dc=local" type=refreshAndPersist bindmethod=simple binddn="uid=syncre
pl,dc=tracteur91,dc=local" credentials="Tr@cteur91" retry="30 +" network-ti
meout=5 timeout=30
dn: olcOverlay={0}syncprov,olcDatabase={1}hdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcSyncProvConfig
olcOverlay: {0}syncprov
olcSpCheckpoint: 100 5
11 years, 6 months
