search-filter processing within overlay? (struct Filter)
by Daniel
Hi Folks,
I'm currently making my first steps into overlay development using
OpenLDAP 2.4.15. Digging into some already existing overlays was very
helpful. Thanks to the nice work!
My sample overlay is just for personal testing to get some kind of
feeling how to handle the slapd(-overlay) API. All I want to do with my
testing-overlay is to get some kind of datastructure-list of the
attribute names used in the current search-filter.
For example processing the following filter
(&(objectClass=*)(ou=test)(hasSubordinates=TRUE)) my overlay's
_search-callback() should print out "objectClass, ou, hasSubordinates".
Of course I could (regexp) string-compare the filterstr contained in
struct Filter but I thought there has to be a more slapd-API'ic way
available. After browsing through the mailinglist and various
openldap-source files my perhabs silly question is:
Is there some kind of Attribute-list (Attribute *) datastructure
available that can be traversed (in a more linear way) getting the
attributes (and might be the corresponding search-patterns used in a
search-filter statement? By linaer I mean some kind of (pseudo-code):
Attribute attr;
for(attr=filter->attrs;attr;attr=attrs->next;) {
attr->a_desc...
...
}
All I've seen so far is struct Filter with it's "filterstr" (char *) and
its various pointers to f_desc, f_av_desc, f_and, f_or, f_list and so
on. I've also had a look into function "test_filter()" and into
translucent-overlay search-callback, both use switch-case constructions
to strip (complex) filter-strings (struct Filter) into atoms. I think
that's nearly what I would need - again. ;-)
Because the filter atomization has already been done during
filter-processing my actual intention is to avoid coding (90%
copy&paste) the nearly same filter-processing mechanism into my own
overlay's _search()-callback in favor to use a might be already
existing datastructure.
In the moment I'm in doubt splitting the search filter twice could not
be very elegant nor very efficient regarding the resulting
performance... Probably I've not understood the overlay-processing and
datastructures correct, thus I'm looking forward to your advice, hints
and help.
Thanks a lot!
Cheers
Daniel
14 years, 8 months
can't delete bdb backend from cn=config
by Michal Rejda
Hello,
Im trying to delete olcDatabase={1}bdb,cn=config using ldapmodify:
dn: olcDatabase={1}bdb,cn=config
changetype: delete
But the server answer is:
deleting entry "olcDatabase={1}bdb,cn=config"
ldap_delete: Server is unwilling to perform (53)
I'm performing this operation as rootdn why can I not delete the entry?
My OpenLDAP version is 2.4.11
Thanks,
Michal
__________ Informace od ESET NOD32 Antivirus, verze databaze 3968 (20090327) __________
Tuto zpravu proveril ESET NOD32 Antivirus.
http://www.eset.cz
14 years, 8 months
dynlist and memberOf Overlay Combo
by Pete Giesin
I am attempting to combine the dynlist overlay with the memberOf overlay.
I have the following configuration:
overlay dynlist
dynlist-attrset groupOfURLs memberURL member
overlay memberof
memberof-group-oc groupOfURLs
memberof-member-ad member
memberof-memberof-ad memberOf
I then have inserted the following entries:
dn: uid=test1,ou=People,dc=hcm,dc=com
objectClass: top
objectClass: person
objectClass: inetOrgPerson
uid: test1
cn: test1
sn: account
dn: cn=WebSelfServiceUser,ou=group,dc=hcm,dc=com
cn: WebSelfServiceUser
objectClass: groupOfURLs
memberURL: ldap:///ou=people,dc=hcm,dc=com??one?(objectClass=inetOrgPerson)
If I query the WebSelfService group it indicates that test1 is a member of
the group. However, the memberOf attribute never gets set on the user
entry. I think the issue is that the dynamic group never gets updated and
hence doesn't force the memberOf updates. Is it possible to combine these
two overlays?
Thanks,
Pete
14 years, 8 months
Problem with searches
by Angel L. Mateo
Hello,
I have a problem with my ldap servers. We have a farm of 4 openldap
servers that synchronizes its information from a fifth. These servers
are used as user backend for our mail system.
The problem we have is that sometimes mail systems launchs searches and
get 0 entries, but there are entries matching the query. In fact, if
they launch the query after a few seconds, it gets the results.
My ldap servers are debian servers (etch version) with openldap
2.3.30-5 (debian version) and 1GB of memmory running in a xen VM.
These are the logs I get (these are from a script that joins the logs
of the ldap server by operation). The two first gets 0 entries, but the
third (launched from the same client and the same ldap user) gets 1
entry. Between the queries there were no modification operation (add,
mod or del) with this entry:
Mar 24 11:01:59 canis2 conn=202855 op=43 (IP=155.54.212.199) SRCH
(base=ou=Usuarios,dc=Telematica, scope=2,
filter=(&(objectClass=posixAccount)(uid=lsc63903)(irisUserStatus=urn:mace:rediris.es:um.es:userstatus:correo:estado:activo)), attr=uid userPassword uidNumber gidNumber cn homeDirectory loginShell gecos description objectClass ) -> tag=101, err=0, text= , nentries=0
Mar 24 11:01:59 canis2 conn=202855 op=44 (IP=155.54.212.199) SRCH
(base=ou=Usuarios,dc=Telematica, scope=2,
filter=(&(objectClass=posixAccount)(uid=lsc63903)(irisUserStatus=urn:mace:rediris.es:um.es:userstatus:correo:estado:activo)), attr=uid userPassword uidNumber gidNumber cn homeDirectory loginShell gecos description objectClass ) -> tag=101, err=0, text= , nentries=0
Mar 24 11:32:10 canis2 conn=696 op=101481 (IP=) SRCH
(base=ou=Usuarios,dc=Telematica, scope=2,
filter=(&(objectClass=posixAccount)(uid=lsc63903)(irisUserStatus=urn:mace:rediris.es:um.es:userstatus:correo:estado:activo)), attr=uid userPassword uidNumber gidNumber cn homeDirectory loginShell gecos description objectClass ) -> tag=101, err=0, text= , nentries=1
Any idea?
--
Angel L. Mateo Martínez
Sección de Telemática
Área de Tecnologías de la Información _o)
y las Comunicaciones Aplicadas (ATICA) / \\
http://www.um.es/atica _(___V
Tfo: 968367590
Fax: 968398337
14 years, 8 months
sladp crash during stress test
by Pete Giesin
I was performing some stress tests and the slapd server crashed. The only
log information I can find as to why the server crashed is the following:
Mar 25 12:30:53 PAS209 kernel: slapd[29502] general protection
rip:3004a71650 rsp:4136db70 error:0
Mar 26 01:09:24 PAS209 kernel: slapd[1514]: segfault at 0000000000000000 rip
00002aab11868901 rsp 0000000042f7ae68 error 6
I know this is not much to go on but I was hoping someone might be able to
help explain why the server crashed. I am running Symas OpenLDAP 2.4.12.1 on
Red Hat Enterprise Linux 5.2 x86_64. I was running a test with a
read-to-write ratio of 999-1. The test was using 10 clients with 30 threads
per client. Two distinct users were being modified and all of the users were
being searched randomly.
Thanks,
Pete
14 years, 8 months
Installaing openldap 2.4 in ubuntu 8.04 whith libdb4.6 (or higher)
by fida aljounaidi
Hi
I upgrade my server Os (from feisty to hardy) to install the last ubuntu
packaged version of openldap (slapd 2.4.9).
I want to use also the db4.6 utils with this version to do a better tuning
of DB_CONFIG.
The problem is that this version is packaged with libdb4.2 dependency. With
this version of Berkley DB, I've observed an important amount of CPU
consumed when synchronisation between master and slaves is executed. This
led sometimes in loss of synchronisation state.
So, I have 2 questions:
is it any know bug in libdb4.2 that can lead to this beahaviour ?
is it better ton install openldap from package Source so that i can
configure it with higher version of libdb ?
Thank you
P.S: Sorry for my english
14 years, 8 months
Mirrormode and cn=config
by Peter Mogensen
Hi,
I'm doing a mirrormode setup from what I can understand of the admin
guide A.2.5 it's possible to mirror cn=config so you only have to make
changes on one server.
I have two problems though.
First:
I've looked at test49 and it doesn't seem to use mirrormode.
Also - server 1 in test49 has it self as provider ???
I would guess that there would be some troubles with mirroring cn=config
since, server 1 should have server 2 as provider and vice-versa. (and
that must not be replicated)
Is this possible at all or am I chasing a dead end?
Second:
I have problem with SASL/EXTERNAL and TLS. The server can't seem to find
the client certificate. I'm using slapd from Debian Lenny and Ubuntu
Hardy, and it's probably due to GnuTLS problems.
I get error from slapd like:
"TLS: can't accept: A TLS packet with unexpected length was received.."
"unable to get TLS client DN, error=-4 id=0"
Are GnuTLS just completely broken on Debian Lenny or can this be made to
work?
/Peter
14 years, 8 months
Re: slapd syncrepl consumer having permanent high CPU load
by Rodrigo Costa
Howard and all,
I made more tests and looks like problem persists. I saw some changes but only in the memory consumption in "consumer(slave)" syncrepl.
Let me try to explain better. I have a pair of provider/consumer machines where one machine will always receive all read/writes and the other is just for High Availability(HA) purposes, so it is better have the more close as possible the DBs.
I start the provider(master) and then just after start the consumer(slave). The configuration doesn't appear to have problems since I have in my configuration 2 DBs, CONTENT and INDEX, and I see consumer doing 2 searches in these DBs when started(this is ok).
After this both consumer and provider CPU usage increases so as memory allocation by slapd process. After the HEAD changes the memory consumption in consumer increases in a much more fast rate, something like 10:1. In this way to reproduce the issue I needed to reduce the dncachesize directive in consumer to 1/10 of the provider value, or from 4,000,000 to 400,000. This avoid the process to consume all memory before the issue arises.
Let me try to summarize :
1) Start provider(mater) slapd process;
2) Start consumer(slave) slapd process;
3) Monitor memory and CPU usage in both provider and consumer;
4) Make sometimes a monitor check to see the cache information;
5) Before cache is full in provider(master) I made a gdb debug to check the consumer(slave) process threads;
6) Wait until the consumer(slave) process starts to use around 200% CPU and then collect again a gdb debug;
7) Wait a little more until the provider(master) CPU usage becomes 0% and then see that consumer(slave) CPU stay stable in 200%. Collect a gdb debug.
8) Wait some more time just for more gdb debug to see if something changed.
I re-compile the HEAD with GDB symbols for debugging. In this way I created the file attached where more than once I collect the debug information from the consumer slapd(includes the syncrepl thread). Please see file attached for details.
The item 7) is the issue I think is happening. The synchronization never ends, the responsiveness from consumer(slave) to queries is very slow, CPU usage becomes fixed in 200%, and then the logic appears never be working as expected, or in the end never synchronizing.
In the end appears that syncrepl still with some issue to synchronize the DBs.
Regards,
Rodrigo.
--- On Thu, 3/19/09, Howard Chu <hyc(a)symas.com> wrote:
> From: Howard Chu <hyc(a)symas.com>
> Subject: Re: slapd syncrepl consumer having permanent high CPU load
> To: rlvcosta(a)yahoo.com
> Cc: openldap-software(a)openldap.org, "John Morrissey" <jwm(a)horde.net>
> Date: Thursday, March 19, 2009, 2:04 PM
> Rodrigo Costa wrote:
> >
> > Folks,
> >
> > I was preparing openLDAP with GDB symbols but looks
> like the issue was
> identified and solved in HEAD. Just to identify this issue;
> was created any
> sort of ITS for verification in a new load?
>
> No, the further work was just associated with ITS#5860.
>
> > Sorry my late response but my baby daughter just born
> last week and I was
> having some work at home.
>
> Congratulations!
>
> > I will give a try in the HEAD load.
>
> Try RE24 now, that's the current release candidate.
> >
> > Best Regards,
> >
> > Rodrigo.
> >
> > PS-> Just some link from my daughter
> > http://sites.google.com/site/lauramenina/laura_english
> >
> > --- On Wed, 3/18/09, Howard Chu<hyc(a)symas.com>
> wrote:
> >
> >> From: Howard Chu<hyc(a)symas.com>
> >> Subject: Re: slapd syncrepl consumer having
> permanent high CPU load
> >> To: "John Morrissey"<jwm(a)horde.net>
> >> Cc: openldap-software(a)openldap.org
> >> Date: Wednesday, March 18, 2009, 5:21 AM
> >> John Morrissey wrote:
> >>> After ~16h uptime, slapd with this BDB had
> increased
> >> its DN cache to ~250k
> >>> entries after it previously appeared stable at
> the
> >> configured 20k entries,
> >>> and its entry cache had ballooned to ~480k
> entries.
> >> Its RSS was about 3.6GB
> >>> at this point, with a BDB cache size of 2GB.
> >>
> >> I was finally able to reproduce this (took several
> hours of
> >> searches. Fortunately I was at a St. Pat's party
> so I didn't
> >> have to wait around, just got home in time to see
> it start
> >> going bad...). A fix is now in HEAD.
> >>
> >> (And now we'll see if Guinness is Good For Your
> Code... ;)
> >> -- -- Howard Chu
> >> CTO, Symas Corp.
> >> http://www.symas.com
> >> Director, Highland Sun
> http://highlandsun.com/hyc/
> >> Chief Architect, OpenLDAP http://www.openldap.org/project/
> >>
> >
> >
> >
> >
> >
>
>
> --
> -- Howard Chu
> CTO, Symas Corp.
> http://www.symas.com
> Director, Highland Sun
> http://highlandsun.com/hyc/
> Chief Architect, OpenLDAP http://www.openldap.org/project/
>
14 years, 8 months
Re: Mirrormode and cn=config
by Mathieu MILLET
On Wed, 25 Mar 2009 15:09:09 +0100, Peter Mogensen <apm(a)mutex.dk> wrote:
> Hi,
>
[snip]
>
> Second:
> I have problem with SASL/EXTERNAL and TLS. The server can't seem to find
> the client certificate. I'm using slapd from Debian Lenny and Ubuntu
> Hardy, and it's probably due to GnuTLS problems.
> I get error from slapd like:
> "TLS: can't accept: A TLS packet with unexpected length was received.."
> "unable to get TLS client DN, error=-4 id=0"
>
> Are GnuTLS just completely broken on Debian Lenny or can this be made to
> work?
Which version of OpenLDAP are you using ?
If using 2.4.15, the ldap "client" libs have broken SASL/EXTERNAL
implementation. These libs are also used for consumer to connect to
provider.
Patch already submitted and to be available with 2.4.16.
> /Peter
Hope it can help,
Sincerely yours, Mathieu.
14 years, 8 months
Xenserver 5 + Centos 5.2 x64 DomU + DB 4.7.25 + Openldap 2.4.13 or 2.4.15 do not work
by Steven Truong
Dear, all. I googled and found some people mentioned that openldap
does not work under Xen with db database backends and the information
were from 2006 http://www.openldap.org/lists/openldap-software/200603/msg00199.html.
I actually encountered this myself and I am not sure if this is still
the same problem.
Openldap and DB were compiled and installed from sources and
everything was configure manually. My /etc/hosts include entry such
as:
192.168.10.119 alfresco.mynetwork.com alfresco.
[root@alfresco ~]# hostname
alfresco.mynetwork.com
[root@alfresco ~]# hostname -f
alfresco.mynetwork.com
Could some experts please look into this problem and suggest ways to fix it?
Thank you.
Attached is the output for strace for this problem and below is the
part where the problem started.
2499 open("/etc/hosts", O_RDONLY) = 4
2499 fcntl(4, F_GETFD) = 0
2499 fcntl(4, F_SETFD, FD_CLOEXEC) = 0
2499 fstat(4, {st_mode=S_IFREG|0644, st_size=294, ...}) = 0
2499 mmap(NULL, 4096, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b32b87fd000
2499 read(4, "# Do not remove the following li"..., 4096) = 294
2499 close(4) = 0
2499 munmap(0x2b32b87fd000, 4096) = 0
2499 getrlimit(RLIMIT_NOFILE, {rlim_cur=1024, rlim_max=1024}) = 0
2499 pipe([4, 5]) = 0
2499 epoll_create(1024) = 6
2499 open("/etc/gai.conf", O_RDONLY) = -1 ENOENT (No such file or directory)
2499 futex(0x3f837509a8, FUTEX_WAKE, 2147483647) = 0
2499 socket(PF_NETLINK, SOCK_RAW, 0) = 7
2499 bind(7, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0
2499 getsockname(7, {sa_family=AF_NETLINK, pid=2499,
groups=00000000}, [17449945317507072012]) = 0
2499 time(NULL) = 1237846847
2499 sendto(7, "\24\0\0\0\26\0\1\3?\v\310I\0\0\0\0\0\0\0\0", 20, 0,
{sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20
2499 connect(7, {sa_family=AF_INET6, sin6_port=htons(389),
inet_pton(AF_INET6, "::", &sin6_addr), sin6_flowinfo=0,
sin6_scope_id=0}, 28) = 0
2499 getsockname(7, {sa_family=AF_INET6, sin6_port=htons(42391),
inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=0,
sin6_scope_id=0}, [8589934620]) = 0
2499 close(7) = 0
2499 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 7
2499 connect(7, {sa_family=AF_INET, sin_port=htons(389),
sin_addr=inet_addr("0.0.0.0")}, 16) = 0
2499 getsockname(7, {sa_family=AF_INET, sin_port=htons(52357),
sin_addr=inet_addr("127.0.0.1")}, [8589934608]) = 0
2499 close(7) = 0
2499 socket(PF_INET6, SOCK_STREAM, IPPROTO_IP) = 7
2499 setsockopt(7, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
2499 setsockopt(7, SOL_IPV6, IPV6_V6ONLY, [1], 4) = 0
2499 bind(7, {sa_family=AF_INET6, sin6_port=htons(389),
inet_pton(AF_INET6, "::", &sin6_addr), sin6_flowinfo=0,
sin6_scope_id=0}, 28) = 0
2499 socket(PF_INET, SOCK_STREAM, IPPROTO_IP) = 8
2499 setsockopt(8, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
2499 bind(8, {sa_family=AF_INET, sin_port=htons(389),
sin_addr=inet_addr("0.0.0.0")}, 16) = 0
2499 time([1237846847]) = 1237846847
2499 stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2819, ...}) = 0
2499 stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2819, ...}) = 0
2499 stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2819, ...}) = 0
2499 sendto(3, "<167>Mar 23 15:20:47 slapd[2499]"..., 72,
MSG_NOSIGNAL, NULL, 0) = 72
2499 time([1237846847]) = 1237846847
2499 stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2819, ...}) = 0
2499 stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2819, ...}) = 0
2499 stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2819, ...}) = 0
2499 sendto(3, "<167>Mar 23 15:20:47 slapd[2499]"..., 49,
MSG_NOSIGNAL, NULL, 0) = 49
2499 close(3) = 0
2499 time([1237846847]) = 1237846847
2499 stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2819, ...}) = 0
2499 stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2819, ...}) = 0
2499 stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2819, ...}) = 0
2499 socket(PF_FILE, SOCK_DGRAM, 0) = 3
2499 fcntl(3, F_SETFD, FD_CLOEXEC) = 0
2499 connect(3, {sa_family=AF_FILE, path="/dev/log"}, 110) = 0
2499 sendto(3, "<167>Mar 23 15:20:47 slapd[2499]"..., 75,
MSG_NOSIGNAL, NULL, 0) = 75
2499 shutdown(5, 2 /* send and receive */) = -1 ENOTSOCK (Socket
operation on non-socket)
2499 close(5) = 0
2499 shutdown(4, 2 /* send and receive */) = -1 ENOTSOCK (Socket
operation on non-socket)
2499 close(4) = 0
2499 close(6) = 0
2499 exit_group(1) = ?
2498 <... wait4 resumed> [{WIFEXITED(s) && WEXITSTATUS(s) == 1}], 0,
NULL) = 2499
2498 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
2498 --- SIGCHLD (Child exited) @ 0 (0) ---
2498 wait4(-1, 0x7fffedefdfc4, WNOHANG, NULL) = -1 ECHILD (No child processes)
2498 rt_sigreturn(0xffffffffffffffff) = 0
2498 rt_sigaction(SIGINT, {SIG_IGN}, {SIG_IGN}, 8) = 0
2498 exit_group(1) = ?
2470 <... wait4 resumed> [{WIFEXITED(s) && WEXITSTATUS(s) == 1}], 0,
NULL) = 2498
2470 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
2470 --- SIGCHLD (Child exited) @ 0 (0) ---
2470 wait4(-1, 0x7fffec338194, WNOHANG, NULL) = -1 ECHILD (No child processes)
2470 rt_sigreturn(0xffffffffffffffff) = 0
2470 rt_sigaction(SIGINT, {SIG_IGN}, {SIG_IGN}, 8) = 0
2470 rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
2470 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
2470 rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
2470 rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
2470 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
2470 rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
2470 rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
2470 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
2470 write(1, "\33[60G", 5) = 5
2470 write(1, "[", 1) = 1
2470 rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
2470 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
2470 write(1, "\33[0;31m", 7) = 7
2470 write(1, "FAILED", 6) = 6
2470 rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
2470 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
2470 write(1, "\33[0;39m", 7) = 7
2470 write(1, "]", 1) = 1
2470 write(1, "\r", 1) = 1
2470 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
2470 rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
2470 stat("/usr/bin/rhgb-client", {st_mode=S_IFREG|0755,
st_size=13704, ...}) = 0
2470 geteuid() = 0
2470 getegid() = 0
2470 getuid() = 0
2470 getgid() = 0
2470 access("/usr/bin/rhgb-client", X_OK) = 0
2470 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
2470 rt_sigprocmask(SIG_BLOCK, [INT CHLD], [], 8) = 0
2470 clone(child_stack=0,
flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
child_tidptr=0x2b1cbe77fe50) = 2500
2500 close(255 <unfinished ...>
2470 rt_sigprocmask(SIG_SETMASK, [], <unfinished ...>
2500 <... close resumed> ) = 0
14 years, 8 months
