separate loglevels for different databases?
by Meike Stone
Hello,
I've configured two different databases (one ldap, one bdb) in openLDAP.
Is it possible, to configure separate loglevels for each database?
Thanks Meike
8 years, 7 months
LMDB space amplification
by Shu, Xinxin
Hi list,
Recently I have integrated lmdb into ceph , got intial results , I dump all records of lmdb, there are several different size key-value pairs,
1. key size : 45 byte, value size = 124 byte
2. key size : 85 byte, value size = 1187 byte
3. key size : 57 byte, value size = 135 byte
4. key size : 64 byte, value size = 182 byte
5. key size : 33 byte, value size = 4096 byte (used for real data, about 80 %)
this db contains about ~ 10000000 records, about 80 percent of records are the last key-value pairs (key size 33 byte, value size 4096 byte), I calculate size of all key-value pairs, about 33GB , but on-disk size is about 63GB, so space amplification is ~ 2, is this amplification reasonable ? In which situation can lmdb get such large space amplification? How can I reduce this amplification? If you need any other details , please email me , any tips will be appreciate.
Cheers,
xinxin
8 years, 7 months
[WIP] WiredTiger Backend
by HAMANO Tsukasa
Hi,
I'm writing OpenLDAP WiredTiger backend.
It will get high write throughput and high concurrency performance.
Now this backend module is incomplete but basic feature is working, So
I publish the code.
https://www.osstech.co.jp/download/hamano/openldap/0001-OpenLDAP-WiredTig...
Some note:
* slapadd, slapcat, slapindex will works.
* LDAP BIND, ADD, DELETE, SEARCH will works.
* MODIFY, MODRDN does not implement yet.
* deref search does not implement yet.
* WiredTiger is not support multiprocess access yet.
It mean that we can't to do slatcat while running slapd.
It will be supported in the future.
* This backend module may have a issue about licensing when
distributing whole binary under OpenLDAP Public Licence.
Now WiredTiger, Inc are working cooperatively toward a resolve the
issue.
I'm glad if you get to try it and give me advice.
Thank you.
--
Open Source Solution Technology Corporation
HAMANO Tsukasa <hamano(a)osstech.co.jp>
fingerprint = 2285 2111 6D34 3816 3C2E A5B9 16BE D101 6069 BE55
8 years, 7 months
modifying cn=config with ldapmodify
by Robert Munn
I have been trying to replace the SSL cert settings on my OpenLDAP instance running on Ubuntu using ldapmodify.
I followed directions on the Ubuntu wiki:
https://help.ubuntu.com/lts/serverguide/openldap-server.html#openldap-tls
using a modified ldif file for the replace:
dn: cn=config
changetype: modify
replace: olcTLSCACertificateFile
olcTLSCACertificateFile: /etc/ssl/certs/cacert.pem
-
replace: olcTLSCertificateFile
olcTLSCertificateFile: /etc/ssl/certs/ldap01_slapd_cert.pem
-
replace: olcTLSCertificateKeyFile
olcTLSCertificateKeyFile: /etc/ssl/private/ldap01_slapd_key.pem
When it didn’t work on my existing instance I built a new instance in a new Ubuntu VM (14.04) and tried the original directions from Ubuntu. That did not work either.
The ldapmodify command executes correctly but it seems that the change is not registered by the server. This is the case in both the new instance and the old instance of OpenLDAP.
I ended up replacing the values (or adding them in the new instance) in the /etc/ldap/slapd.d/cn=config.ldif file manually. Making the changes manually and restarting slapd works, but my understanding was that changes to cn=config should be made through ldapmodify.
I also found a tech note at CentOS:
https://www.centos.org/docs/5/html/CDS/cli/8.0/Configuration_Command_File...
in section 2.2.2.2 that indicates changes to cn=config will be ignored:
"If an attribute is added to cn=config, the server ignores it."
So am I mistaken? Do I need to do something different? I would prefer to manage the config with ldapmodify, but since I don’t change cn=config that often, I can change it manually.
Robert
8 years, 7 months
Re: Antw: LMDB usage with long or variable length keys
by Alex V.
On 4/24/2015 10:45 AM, Ulrich Windl wrote:
>>>> "Alex V."<alex.v.odesk(a)gmail.com> schrieb am 24.04.2015 um 08:54 in Nachricht
> <5539E88C.3080006(a)gmail.com>:
>> On 4/24/2015 9:02 AM, Ulrich Windl wrote:
>>> I think this depends very much on your access patterns (which you didn't
>> explain).
>>> Regards,
>>> Ulrich
>> What required is `get(key)` and `set(key, val)`. And, optional but not
>> unwanted, range lookup.
> Still unclear: What is the expected ratio between get and set for a specific key? Do you ever delete keys? If so, how many accesses do you expect before deleting a key?
>
> You could hash your keys using MD5 or SHA, and use the first 16 bytes as key. As your access pattern does not include enumerating the original keys, that would work, saving space vs. consuming some extra CPU cycles. I can think of other algorithms, but you were not specific with your access pattern.
>
> Regards,
> Ulrich
>
I haven't figured all the details yet, I am just looking for different
possibilities. number of `get`s is probably many times higher than
number of `set`s or `delete`s. One possible usage case is database that
stores source files metadata(symbols) for autocompletion/navigation,
something like "source navigator ng"(it uses bdb). Another use case is
almost the same, with urls as keys and word index as metadata. Looks
like basename/url parts hierarchy will do, I am going to test it some day.
8 years, 7 months
Re: Different contextCSN on multi-master replication servers
by NICOLAS KOVAC NEUMANN
Thank you both Howard and Leonid.
Yes, you're right, it happened the other way around; the modification was
made on the second server and propagated back to the first one. However, I
don't know why the change was in turn returned to the second server - SIDs
are ok as far as I know.
However, as Leonid mentioned, both servers weren't synchronized correctly
anyway. Turns out that yesterday we upgraded to 2.4.39-6 (as I stated in my
first mail). Previously, we were using 2.4.39-3 and it seemed to work fine.
We also noted that 2.4.39-6 produced some additional issues (like client
and syncrepl sockets dying without any apparent reason), so today we
downgraded back to 2.4.39-3 and everything seems to work just fine again.
We had a look at the changelog from 2.4.39-3 to 2.4.39-6 and no change
seems to be explicitly syncrepl related, but rather related to LDAPS
(strange, as we use the LDAP protocol for syncrepl instead of LDAPS).
Anyway, we'll keep version 2.4.39-3 as far as it works well.
Thanks.
Regards.
2015-04-21 23:53 GMT+01:00 Леонид Юрьев <leo(a)yuriev.ru>:
> Hi Nicolás,
>
> 1) If contextCSN(s) are differs on servers, then are still not
> syncronized (or has a glitches).
> http://www.openldap.org/lists/openldap-technical/201108/threads.html#00001
>
> 2) Replication takes a some time. Therefore contextCSN(s) may be equals
> only when some time was no any changes.
>
> 3) Make sure that the time is synchronized on servers (e.g. by using
> ntpdate).
>
> 4) Unfortunatelly, all current releases (include 2.4.39 and 2.4.40) have
> enough bugs in replication code.
> For example, by ITS#8081 (
> http://www.openldap.org/its/index.cgi/Software%20Bugs?id=8081) you could
> get segfault, but also lost (like undo) some changes by a replication.
>
> 5) We made a fork of OpenLDAP project for our usecase (highload
> TELCO-aware multi-master), it called ReOpenLDAP.
> If you decide to build slapd from sources, I recommend use our ReOpenLDAP
> ;)
>
> New features yet not documented in english man-pages, by you can translate
> by Google:
> https://github.com/ReOpen/ReOpenLDAP/releases/tag/ReOpenLDAP-2.4.41-rc
>
> https://github.com/ReOpen/ReOpenLDAP/commit/4fc4bc18dd4bd80909aa80700c5c1...
>
> https://github.com/ReOpen/ReOpenLDAP/commit/95808b156ee36a886523b7096a75d...
>
> https://github.com/ReOpen/ReOpenLDAP/commit/1c94bc17ec285388e8a8299399ed5...
>
> Leonid.
>
> 2015-04-21 16:01 GMT+03:00 Nicolás Kovac Neumann <nkovacne(a)ull.edu.es>:
>
>> Hi,
>>
>> We're currently using N-way multimaster replication on two servers for
>> our LDAP directory, both for the config and the hdb databases. It's working
>> fine mostly, but we've run into a possible issue with the syncrepl engine
>> which we would like to cast light on. We're using CentOS 7 with
>> openldap-servers version 2.4.39-6.
>>
>> We made an update on one of the entries (server1, in this case), so
>> server2 replicated that change (as shown below in the log):
>>
>> ==> server1/ldap.log <==
>> Apr 21 13:38:55 server1 slapd[1835]: do_syncrep2: rid=002
>> cookie=rid=002,sid=002,csn=20150421123855.643239Z#000000#002#000000
>> Apr 21 13:38:55 server1 slapd[1835]: syncrepl_message_to_entry:
>> rid=002 DN: uid=user1,cn=subtree,dc=example,dc=org, UUID:
>> 18a2436c-73ce-1030-95dd-b52dc05ced13
>> Apr 21 13:38:55 server1 slapd[1835]: syncrepl_entry: rid=002
>> LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_MODIFY)
>> Apr 21 13:38:55 server1 slapd[1835]: syncrepl_entry: rid=002
>> be_search (0)
>> Apr 21 13:38:55 server1 slapd[1835]: syncrepl_entry: rid=002
>> uid=user1,cn=subtree,dc=example,dc=org
>> Apr 21 13:38:55 server1 slapd[1835]: slap_queue_csn: queing
>> 0x7ff8f42789f0 20150421123855.643239Z#000000#002#000000
>> Apr 21 13:38:55 server1 slapd[1835]: slap_graduate_commit_csn:
>> removing 0x7ff8f435e770 20150421123855.643239Z#000000#002#000000
>> Apr 21 13:38:55 server1 slapd[1835]: syncrepl_entry: rid=002
>> be_modify uid=user1,cn=subtree,dc=example,dc=org (0)
>> Apr 21 13:38:55 server1 slapd[1835]: syncprov_sendresp:
>> cookie=rid=001,sid=001,csn=20150421123855.643239Z#000000#002#000000
>> Apr 21 13:38:55 server1 slapd[1835]: slap_queue_csn: queing
>> 0x7ff8f42789f0 20150421123855.643239Z#000000#002#000000
>> Apr 21 13:38:55 server1 slapd[1835]: slap_graduate_commit_csn:
>> removing 0x7ff8f41b7b90 20150421123855.643239Z#000000#002#000000
>>
>> ==> server2/ldap.log <==
>> Apr 21 13:38:55 server2 slapd[1948]: slap_queue_csn: queing
>> 0x7f897affb220 20150421123855.643239Z#000000#002#000000
>> Apr 21 13:38:55 server2 slapd[1948]: syncprov_sendresp: to=001,
>> cookie=rid=002,sid=002,csn=20150421123855.643239Z#000000#002#000000
>> Apr 21 13:38:55 server2 slapd[1948]: slap_graduate_commit_csn:
>> removing 0x7f89307f42a0 20150421123855.643239Z#000000#002#000000
>>
>> Nothing strange up to now, however, if we query the contextCSN, it
>> differs on both servers.
>>
>> For server1, we have:
>>
>> contextCSN: 20150421123523.281736Z#000000#001#000000
>> contextCSN: 20150421123417.889477Z#000000#002#000000
>>
>> For server2, the value for server ID 001 differs:
>>
>> contextCSN: 20150421115324.003502Z#000000#001#000000
>> contextCSN: 20150421123417.889477Z#000000#002#000000
>>
>> However, the entry seems to replicate the entryCSN correctly on both
>> servers:
>>
>> entryCSN: 20150421123417.889477Z#000000#002#000000
>>
>> Is this the expected behavior? Shouldn't both contextCSN values match on
>> both servers?
>>
>> Thanks!
>>
>> Regards,
>>
>> Nicolás
>>
>>
>
8 years, 7 months
LMDB usage with long or variable length keys
by Alex V.
I am new to key-value stores. I would like to use it like big persistent
associative array and I want to be able to use long length keys. For
example file path or URL. What technique should I use to implement such
dictionary on top of LMDB? Probably some kind of hashing and collision
resolution? Or just recompile with MDB_MAXKEYSIZE=2047 or something like
this? Or LMDB is a wrong tool for this job?
8 years, 7 months
ldap configuration question
by Ross, Daniel B.
I have looked a couple options, and read the openldap documentation, but I really dont know how to accomplish what I need to do.
Here is what I am trying to do.
I have a greater organization that is stuck on using Microsoft products namely Microsoft LDS. To make matters worse they present the data to my linux servers in a completely non-standard way. Its driving my solaris and linux box nuts and they simply dont want to work with it.
What i need to do is continue to use the campus usernames and passwords but present the Data in a format that my linux/unix hosts can use. Is this possible?
i.e. userid would still be samwise but instead of a bizzarre OU=monkeypeople,dc=example,dc=com I want it to present as people,dc=example,dc=com. The mismatches are far more and I need to add things like nfs mounts and it would be nice to allow users on only certain hosts, none of which their LDS can do, or even if it can they are not willing to do.
I looked at referral and aliasing but it does not seem to be doing what I am trying to do. Passthrough authentication looks close but I cant find sufficient documentation to actually configure a system to use it. So what I want to do is use campus for the username and password authentication, but the local ldap for everything else, groups, dns, etc, etc.
Thanks
Daniel
8 years, 7 months
Re: Antw: Re: Different contextCSN on multi-master replication servers
by Nicolás Kovac Neumann
Hi Ulrich,
El 24/04/15 a las 07:07, Ulrich Windl escribió:
> [...]
> After haveing had such problems, I recommand exporting the databases of both
> servers and compare the contents: If the mismatch, I'd import one database from
> the other server's dump...
>
> Regards,
> Ulrich
That's the approach we've been using years ago but as this is a patchy
workaround we rather preferred get syncrepl working correctly. After
downgrading the package to 2.4.39-3 again it seems to work again, so I
think we'll apply the old and good advise "If it works, don't fix it".
At least not until a certain time.
Thanks.
Regards,
Nicolás.
8 years, 7 months
