openldap-devel

openldap-devel@openldap.org

1 participants
1421 discussions

Re: back-llog
by Howard Chu 10 Aug '16

10 Aug '16

Michael Ströder wrote: > Howard Chu wrote: >> Michael Ströder wrote: >>> Howard Chu wrote: >>>> Was just thinking we could do a quite simple backend for use with the accesslog >>>> overlay and delta-syncrepl. It would write into flat files and do typical >>>> logfile rotation on its own. The backing store would have a minimum of two files >>>> - one for the suffix entry, one for the current chunk of logs. There would be a >>>> configurable number of logfiles, with the oldest simply being deleted when it's >>>> time to purge. >>> >>> Two things I'd consider helpful for long-time archiving/auditing: >>> >>> 1. filenames to allow of "merging" different log flat files into one big file >>> store on a different slapd instance. >> >> Not sure what you mean by this. As sequential logs you could just cat them if >> you want to combine files into one, what does the filename have to do with it? > > Sorry for unclear wording. > I'd like to move archived files of all MMR replicas into another single > filesystem directory and let a special auditing slapd with back-llog search in > them without having to muck with file names. Then you would have to merge them entry-by-entry to preserve chronological order. >> If we just use datestamps for filenames YYYYMMDD it should be no problem for you >> to cat them in proper order. > > There could be conflicting timestamps from different MMR replicas. Maybe > appending the serverID could be a solution? Just keep them all in their own separate filesystem directories. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

2 1

Re: back-llog
by Howard Chu 10 Aug '16

10 Aug '16

Michael Ströder wrote: > Howard Chu wrote: >> Was just thinking we could do a quite simple backend for use with the accesslog >> overlay and delta-syncrepl. It would write into flat files and do typical >> logfile rotation on its own. The backing store would have a minimum of two files >> - one for the suffix entry, one for the current chunk of logs. There would be a >> configurable number of logfiles, with the oldest simply being deleted when it's >> time to purge. > > Two things I'd consider helpful for long-time archiving/auditing: > > 1. filenames to allow of "merging" different log flat files into one big file > store on a different slapd instance. Not sure what you mean by this. As sequential logs you could just cat them if you want to combine files into one, what does the filename have to do with it? If we just use datestamps for filenames YYYYMMDD it should be no problem for you to cat them in proper order. > 2. instead of deleting moving files to an archive directory. Sounds fine as an option. > I know it's a different use-case but quite handy. > > Ciao, Michael. > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

2 1

back-llog
by Howard Chu 09 Aug '16

09 Aug '16

Was just thinking we could do a quite simple backend for use with the accesslog overlay and delta-syncrepl. It would write into flat files and do typical logfile rotation on its own. The backing store would have a minimum of two files - one for the suffix entry, one for the current chunk of logs. There would be a configurable number of logfiles, with the oldest simply being deleted when it's time to purge. It would only support Add/Search in general, and Modify on the suffix entry. A rolling append-only format like this would allow much higher throughput for accesslog recording. Using a binary entry format would of course also help. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

2 1

Any update on LMDB support for OpenBSD?
by Tinker 13 Jun '16

13 Jun '16

Hi Howard, Any plan on making LMDB work on OpenBSD? OpenBSD should deliver LMDB in the MDB_WRITEMAP mode. (http://lmdb.tech/doc/group__mdb.html) (At least some effort has been done e.g. https://gitlab.com/mdb/sqlightning/issues/6 ) Thanks, Tinker

2 2

When must slap_callback.sc_writewait be set?
by Hallvard Breien Furuseth 09 Jun '16

09 Jun '16

When is it safe to leave sc_writewait uninitialized? The field seems to have emerged in the middle of RE24, and I don't see any flags which say it should/should not be used. Since not setting it crashed back-relay in ITS#8428, does that mean everything which creates a slap_callback should set it? Seems unfortunate for third-party modules... Also, I think this should be documented. -- Hallvard

3 12

Debugging left over in test008?
by Quanah Gibson-Mount 30 Apr '16

30 Apr '16

After hitting test008 failure on a new system because "time" was not installed, I looked at test008, and I think the problem is someone was debugging something years ago, and checked the script in still in that state: echo "Using tester for concurrent server access..." time $SLAPDTESTER -P "$PROGDIR" -d "$TESTDIR" -h $LOCALHOST -p $PORT1 -D "$MANAGERDN" -w $PASSWD -l $TESTLOOPS #$SLAPDTESTER -P "$PROGDIR" -d "$TESTDIR" -h $LOCALHOST -p $PORT1 -D "$MANAGERDN" -w $PASSWD -l $TESTLOOPS RC=$? Note the commented out line. The checkin for this change seems to confirm it was accidental as well, and that all that was intended was the "sleep 1": quanah@b4c92f65-2064-44a6-9707-a97562610945:~/git/openldap/openldap-head$ git show 572ca2db -- tests/scripts/test008-concurrency commit 572ca2db94b0e7af95e7ac16a91cb1ea3b3d136a Author: Howard Chu <hyc(a)openldap.org> Date: Fri Sep 30 07:32:49 2005 +0000 Tweak slapd startup delays diff --git a/tests/scripts/test008-concurrency b/tests/scripts/test008-concurrency index 3841c33..6ea541f 100755 --- a/tests/scripts/test008-concurrency +++ b/tests/scripts/test008-concurrency @@ -36,6 +36,8 @@ if test $WAIT != 0 ; then fi KILLPIDS="$PID" +sleep 1 + echo "Using ldapsearch to check that slapd is running..." for i in 0 1 2 3 4 5; do $LDAPSEARCH -s base -b "$MONITOR" -h $LOCALHOST -p $PORT1 \ @@ -49,7 +51,8 @@ for i in 0 1 2 3 4 5; do done echo "Using tester for concurrent server access..." -$SLAPDTESTER -P "$PROGDIR" -d "$DATADIR" -h $LOCALHOST -p $PORT1 -D "$MANAGERDN" -w $PASSWD -l 50 +time $SLAPDTESTER -P "$PROGDIR" -d "$DATADIR" -h $LOCALHOST -p $PORT1 -D "$MANAGERDN" -w $PASSWD -l 50 +#$SLAPDTESTER -P "$PROGDIR" -d "$DATADIR" -h $LOCALHOST -p $PORT1 -D "$MANAGERDN" -w $PASSWD -l 50 RC=$? if test $RC != 0 ; then Are there any objections to me switching these around so that the time line is commented out instead? ;) --Quanah -- Quanah Gibson-Mount Platform Architect Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration A division of Synacor, Inc

2 3

LMDB: mdb_cursor_put() and the rest of page in case MDB_RESERVE
by Леонид Юрьев 31 Mar '16

31 Mar '16

Hi, Howard. Why mdb_cursor_put() doesn't copy the rest of page in case MDB_RESERVE? It is here (current mdb.master, mdb.c line 7479) = http://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=blob;f=libraries/… Regards, Leonid.

3 3

Performance degredation between 2.4.39 and 2.4.44
by Quanah Gibson-Mount 15 Mar '16

15 Mar '16

I've noticed a bit of a performance degradation between the 2.4.39 release and 2.4.44, when doing searches that require ACL processing (anonymous or authenticated user). To test, I set up my server with 2.4.39, freshly load the DB, reboot, and then do a query of each type to ensure its data set is fully loaded in memory, and then get timing on the subsequent searches. The server is idle other than my searches. Overall, there's about a 30 second consistent degradation in search results, regardless of whether or not ACLs are involved. It works out to about a 15% degredation for authenticated queries, and a 32% degredation for rootdn searches. Results are as follows: 2.4.39 anonymous search (only returns some data, subject to ACL processing): zimbra@zre-ldap001:~$ time ldapsearch -x -h zre-ldap001.eng.zimbra.com >/dev/null real 3m25.871s user 1m43.872s sys 0m45.254s zimbra@zre-ldap001:~$ time ldapsearch -x -h zre-ldap001.eng.zimbra.com >/dev/null real 3m22.730s user 1m39.575s sys 0m26.400s zimbra@zre-ldap001:~$ time ldapsearch -x -h zre-ldap001.eng.zimbra.com >/dev/null real 3m23.506s user 1m41.919s sys 0m25.968s 2.4.44 anonymous search (only returns some data, subject to ACL processing): zimbra@zre-ldap001:~$ time ldapsearch -x -h zre-ldap001.eng.zimbra.com >/dev/null real 3m54.841s user 1m36.754s sys 0m44.509s zimbra@zre-ldap001:~$ time ldapsearch -x -h zre-ldap001.eng.zimbra.com >/dev/null real 3m56.609s user 1m37.875s sys 0m48.151s zimbra@zre-ldap001:~$ time ldapsearch -x -h zre-ldap001.eng.zimbra.com >/dev/null real 3m53.275s user 1m36.293s sys 0m45.131s 2.4.39 authenticated search (returns all data, subject to ACL processing): zimbra@zre-ldap001:~$ time ldapsearch -x -h zre-ldap001.eng.zimbra.com -D uid=zimbra,cn=admins,cn=zimbra -w zimbra >/dev/null real 3m23.057s user 1m59.358s sys 0m35.809s zimbra@zre-ldap001:~$ time ldapsearch -x -h zre-ldap001.eng.zimbra.com -D uid=zimbra,cn=admins,cn=zimbra -w zimbra >/dev/null real 3m23.478s user 2m0.236s sys 0m26.972s zimbra@zre-ldap001:~$ time ldapsearch -x -h zre-ldap001.eng.zimbra.com -D uid=zimbra,cn=admins,cn=zimbra -w zimbra >/dev/null real 3m22.965s user 1m58.245s sys 0m28.030s 2.4.44 authenticated search (returns all data, subject to ACL processing): zimbra@zre-ldap001:~$ time ldapsearch -x -h zre-ldap001.eng.zimbra.com -D uid=zimbra,cn=admins,cn=zimbra -w zimbra >/dev/null real 3m57.967s user 1m56.335s sys 0m49.483s zimbra@zre-ldap001:~$ time ldapsearch -x -h zre-ldap001.eng.zimbra.com -D uid=zimbra,cn=admins,cn=zimbra -w zimbra >/dev/null real 3m57.405s user 1m55.415s sys 0m50.772s zimbra@zre-ldap001:~$ time ldapsearch -x -h zre-ldap001.eng.zimbra.com -D uid=zimbra,cn=admins,cn=zimbra -w zimbra >/dev/null real 3m57.717s user 1m57.427s sys 0m41.401s 2.4.39 rootdn search (no ACL processing): zimbra@zre-ldap001:~$ time ldapsearch -x -h zre-ldap001.eng.zimbra.com -D cn=config -w zimbra >/dev/null real 1m20.718s user 1m14.501s sys 0m6.164s zimbra@zre-ldap001:~$ time ldapsearch -x -h zre-ldap001.eng.zimbra.com -D cn=config -w zimbra >/dev/null real 1m19.569s user 1m13.557s sys 0m5.935s zimbra@zre-ldap001:~$ time ldapsearch -x -h zre-ldap001.eng.zimbra.com -D cn=config -w zimbra >/dev/null real 1m20.488s user 1m14.514s sys 0m5.917s 2.4.44 rootdn search (no ACL processing): zimbra@zre-ldap001:~$ time ldapsearch -x -h zre-ldap001.eng.zimbra.com -D cn=config -w zimbra >/dev/null real 1m57.199s user 1m10.855s sys 0m14.161s zimbra@zre-ldap001:~$ time ldapsearch -x -h zre-ldap001.eng.zimbra.com -D cn=config -w zimbra >/dev/null real 1m58.030s user 1m10.797s sys 0m14.970s zimbra@zre-ldap001:~$ time ldapsearch -x -h zre-ldap001.eng.zimbra.com -D cn=config -w zimbra >/dev/null real 1m57.754s user 1m10.069s sys 0m15.336s --Quanah -- Quanah Gibson-Mount Platform Architect, Manager Release Engineering Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration A division of Synacor, Inc

2 6

Supported backends/overlays in 2.5
by Quanah Gibson-Mount 04 Mar '16

04 Mar '16

For 2.5, it would be good to take a look at existing backends and overlays we ship as supported, and see if there are any we want to retire. I think it would be nice as well to make it so people could configure in various contrib modules via configure flags, rather than the current manual make process. I'll start with my thoughts: backends: I would like to see us retire slapd-bdb and slapd-hdb. We first documented they were deprecated in March of 2013 and updated the man pages to officially note that in May of 2014. Nearly 2 years have passed since then, and it'll likely be at least another year before 2.5 is production ready. That's some 3 years. In addtion, both slapd-bdb and slapd-hdb depend on BDB versions that are not supported by Oracle, and the source to which Oracle is doing its best to remove and make inaccessible. Another possibility is back-perl. back-sock is a better solution. OTOH, We have gotten contributions fixing issues in it as recently as April 2015, so there may be folks using it who would like to see it continue. overlays: I'd like to see the passwd/sha2 overlay be considered for promotion. Having the ability to set strong hashes for passwords is crucial in todays world. --Quanah -- Quanah Gibson-Mount Platform Architect Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration A division of Synacor, Inc

6 9

LTB Project off the air?
by Brian Reichert 03 Mar '16

03 Mar '16

I know this isn't the forum for this, but I can't reach the LTB Project's website or mail server; traceroutes fall off somewhere in Europe. I've tried from a few different networks. Does anyone here any have insight as to what's going on? -- Brian Reichert <reichert(a)numachi.com> BSD admin/developer at large

2 2

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-devel