openldap-technical

openldap-technical@openldap.org

13 participants
9794 discussions

LMDB on Ramdisk (tmpfs)
by Luca Foppiano 23 Mar '18

23 Mar '18

Dear everybody, I'm using one of your component (LMDB) via a java JNDI bindings implementation (https://github.com/deephacks/lmdbjni <https://github.com/deephacks/lmdbjni>) and I'm having an issue when I deploy my LMDB file on a tempfs filesystem in RAM. The issue do not occur when the LMDB files are stored on a "normal" filesystem. When the data is in the tempfs ramdisk all the allocated memory ends up being in the Dirty area (it has not been written back to the Filesytem). Here an example using the ramdisk: 7ce320000000-7cfc20000000 r--s 00000000 00:26 2459 /ramfs/nerd/data/db/db-en/entityEmbeddings/data.mdb Size: 104857600 kB Rss: 1255680 kB Pss: 1255680 kB Shared_Clean: 0 kB Shared_Dirty: 0 kB Private_Clean: 0 kB Private_Dirty: 1255680 kB <--- Referenced: 1255680 kB Anonymous: 0 kB AnonHugePages: 0 kB Shared_Hugetlb: 0 kB Private_Hugetlb: 0 kB Swap: 0 kB SwapPss: 0 kB KernelPageSize: 4 kB MMUPageSize: 4 kB Locked: 0 kB VmFlags: rd sh mr mw me ms sd and here an example without: 7ca4fc000000-7cbdfc000000 r--s 00000000 fd:00 11154951 /data/workspace/shared/nerd-data/db/db-en/entityEmbeddings/data.mdb Size: 104857600 kB Rss: 838124 kB Pss: 838124 kB Shared_Clean: 0 kB Shared_Dirty: 0 kB Private_Clean: 838124 kB <---- Private_Dirty: 0 kB Referenced: 764872 kB Anonymous: 0 kB AnonHugePages: 0 kB ShmemPmdMapped: 0 kB Shared_Hugetlb: 0 kB Private_Hugetlb: 0 kB Swap: 0 kB SwapPss: 0 kB KernelPageSize: 4 kB MMUPageSize: 4 kB Locked: 0 kB VmFlags: rd sh mr mw me ms sd According to my understanding the memory is dirty when 1)there are open transactions, 2) the data has not been written back to the filesystem What I don't understand is why there is a difference between filesystem and ramdisk? Is there any reason? The application (listed above) is not writing on the lmdb, but just reading (using reading transaction). Thank you Luca

3 5

Re: Migrate from hdb/bdb to mdb backend.
by Quanah Gibson-Mount 22 Mar '18

22 Mar '18

--On Thursday, March 22, 2018 3:59 PM -0700 rammohan ganapavarapu <rammohanganap(a)gmail.com> wrote: > > Thank you! As an aside, I would note that OpenLDAP 2.4.46 was released today, and it contains multiple fixes for replication. ;) Fixed slapd syncrepl deadlock when updating cookie (ITS#8752) Fixed slapd syncrepl callback to always be last in the stack (ITS#8752) Fixed slapd CSN queue processing (ITS#8801) Fixed slapo-accesslog cleanup to only occur on failed operations (ITS#8752) (delta-sync specific) Fixed slapo-syncprov memory leak with delete operations (ITS#8690) Fixed slapo-syncprov to not clear pending operation when checkpointing (ITS#8444) Fixed slapo-syncprov to correctly record contextCSN values in the accesslog (ITS#8100) (delta-sync specific) Fixed slapo-syncprov not to log checkpoints to accesslog db (ITS#8607) (delta-sync specific) Fixed slapo-syncprov to process changes from this SID on REFRESH (ITS#8800) Fixed slapo-syncprov session log parsing to not block other operations (ITS#8486) --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Re: Migrate from hdb/bdb to mdb backend.
by Quanah Gibson-Mount 22 Mar '18

22 Mar '18

--On Thursday, March 22, 2018 3:50 PM -0700 rammohan ganapavarapu <rammohanganap(a)gmail.com> wrote: > > Quanah, > > > Rest of the config looks OK right? also i can replicate from hdb/bdb to > mdb as i described in previous email right? I am using > openldap-servers-2.4.44-5.el7.x86_64 Replication is a function of protocol, so yes, you can replicate from hdb/bdb to mdb. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Re: Migrate from hdb/bdb to mdb backend.
by Quanah Gibson-Mount 22 Mar '18

22 Mar '18

--On Thursday, March 22, 2018 3:12 PM -0700 rammohan ganapavarapu <rammohanganap(a)gmail.com> wrote: > olcSyncrepl: {0}rid=1 provider=ldap://ldapservertest01:10389/ > binddn="cn=manager,dc=example,dc=com" bindmethod=simple > credentials=dontshare searchbase="dc=example,dc=com" attrs="*,+" > type=refreshAndPersist retry="5 +" timeout=0 keepalive=5:5:5 > network-timeout=0 starttls=no filter="(objectclass=*)" scope=sub > schemachecking=off > olcSyncrepl: {1}rid=2 provider=ldap://ldapservertest02:10389/ > binddn="cn=manager,dc=example,dc=com" bindmethod=simple > credentials=dontshare searchbase="dc=example,dc=com" attrs="*,+" > type=refreshAndPersist retry="5 +" timeout=0 keepalive=5:5:5 > network-timeout=0 starttls=no filter="(objectclass=*)" scope=sub > schemachecking=off > olcSyncrepl: {2}rid=3 provider=ldap://ldapservertest03:10389/ > binddn="cn=manager,dc=example,dc=com" bindmethod=simple > credentials=dontshare searchbase="dc=example,dc=com" attrs="*,+" > type=refreshAndPersist retry="5 +" timeout=0 keepalive=5:5:5 > network-timeout=0 starttls=no filter="(objectclass=*)" scope=sub > schemachecking=off > olcSyncrepl: {3}rid=4 provider=ldap://ldapservertest04:10389/ > binddn="cn=manager,dc=example,dc=com" bindmethod=simple > credentials=dontshare searchbase="dc=example,dc=com" attrs="*,+" > type=refreshAndPersist retry="5 +" timeout=0 keepalive=5:5:5 > network-timeout=0 starttls=no filter="(objectclass=*)" scope=sub > schemachecking=off Remove the pointer back to itself on each server, so they stop trying to replicate from themselves, or use a URL with the serverID. Also, what version of OpenLDAP are you using? --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Re: Migrate from hdb/bdb to mdb backend.
by Quanah Gibson-Mount 22 Mar '18

22 Mar '18

--On Thursday, March 22, 2018 2:25 PM -0700 rammohan ganapavarapu <rammohanganap(a)gmail.com> wrote: > serverID 300 ldap://ldapserver300 > serverID 301 ldap://ldapserver301 >From the slapd.conf man page for serverID: "The fully qualified hostname of each server should be used in the supplied URLs." > And each of them have the same olcSyncrepl You fail to note the olcServerID setting for the MDB cluster. You fail to note the syncprov settings for any of the databases. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Re: Migrate from hdb/bdb to mdb backend.
by Quanah Gibson-Mount 22 Mar '18

22 Mar '18

--On Wednesday, March 21, 2018 11:48 AM -0700 rammohan ganapavarapu <rammohanganap(a)gmail.com> wrote: > It seems data is getting replicate but i am seeing this message in ldap > logs. > > > 5ab27b63 dn_callback : entries have identical CSN (lot of them), i did > find some post regarding this but couldn't find any conclusion. It seems > like this message is because of node is trying to self replicate? can i > have same node information in "olcSyncrepl" config when i am only > replicating user database not config databse? You've not provided any information on your configurations, so one cannot really surmise anything about the validity of said configurations. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

OpenLDAP: ACLs using sockname and DN?
by Norman Gray 22 Mar '18

22 Mar '18

Greetings. In OpenLDAP (2.4.45, on FreeBSD), I'm trying to constrain the access of a DN to an attribute, by giving a DN access only when the connection is made via a socket; but without success. I may just be looking for an example of correct use. What I'm trying is olcAccess: to attrs=userPassword by dn.base="uid=pwreset,ou=service,dc=example,dc=edu" sockname.exact="/var/run/openldap/ldapi" write (the idea is that the pwreset DN can be used by an automatic password-reset script, but that that DN will have that access only when the script is running on the same machine as the LDAP server). This `by` phrase appears to match the production in Sect. 8.3 of the OpenLDAP access control documentation, and the remark in slapd.access(5) that the items in the <who> field ‘may be specified in combination’. And indeed there are no syntax warnings generated. I'm presuming that the combination implies an AND rather than an OR – this isn't made explicit in the documentation. I can find no examples covering this in either OpenLDAP documentation or on the web. This stanza works when the sockname element is absent, suggesting that the configuration is otherwise working as I expect. When I try to write the userPassword attribute using this DN, I get an ldap_modify: Insufficient access (50) error. The OpenLDAP documentation doesn't (somewhat surprisingly) explicitly state what the effect of this sockname element is, and the slapd.access(5) page says, rather obliquely, that: > The statements peername=<peername>, sockname=<sockname>, > domain=<domain>, and sockurl=<sockurl> mean that the contacting host > IP (in the form IP=<ip>:<port> for IPv4, or IP=[<ipv6>]:<port> for > IPv6) or the contacting host named pipe file name (in the form > PATH=<path> if connecting through a named pipe) for peername, the > named pipe file name for sockname, the contacting host name for > domain, and the contacting URL for sockurl are compared against > pattern to determine access. Saying 'determine access' doesn't actually say very much. Have I completely misunderstood the point of this access specification, or is there another way to do this? Best wishes, Norman -- Norman Gray : https://nxg.me.uk

2 2

slapd ACL defintion for access to specific object class
by Space One 22 Mar '18

22 Mar '18

Hello, I have a problem configuring correct ACL's: If you want to grant access to a specific attribute and allow adding the necessary object class for it, we could define: Assuming objectClass is "O" and Attribute name is "A": access to attrs=@O by self write by * +0 break This works but it allows also access to any value in the "objectClass" attribute and is therefor a massive security hole. An alternative would be, which the manpage seem to describe (https://linux.die.net/man/5/slapd.access): access to attrs=objectClass value="O" by self write by * +0 break access to attrs=A by self write by * +0 break But when I apply this, and want to add the object class, I simply get the INSUFFICIENT_ACCESS error code. Maybe one can help? If it's not possible I think the manpage should be adjusted and mention this more explicit. Maye there is a exception for "objectClass"? Or it's a bug in the implementation? Best regards spaceone

1 0

Re: LMDB on Ramdisk (tmpfs)
by Luca Foppiano 22 Mar '18

22 Mar '18

> On 22 Mar 2018, at 11:31, Леонид Юрьев <leo(a)yuriev.ru> wrote: > > Hi. Hi Leonid, > Pages from tmpfs will be in "clean" state if ones swapped-out from RAM. > But while stay in RAM such pages always "dirty”. Thanks for the clarification. Luca

1 0

Re: learning olcAccess
by Lists Nethead 20 Mar '18

20 Mar '18

Quoting Sven Mäder <maeder(a)phys.ethz.ch>: Thank you very much for this! In our case we need access to dn.subtree I believe because admins are supposed to have full control, ie. add and delete accounts. What stopped me was the lack of documentation for the "control" keyword but after a bit of sleep and reading your answer I found a some deep down in the Faq-O-Matic bit it does not seem to cover all of it. So, as I gather, "by * none" is the default processing stops, "by * continue" makes it possible to add rules to the same object and "by * break" resets the processong? And finally, there seems to be additional switches available for the control processing but I cannot find them documented *anywhere*. Thanks again, //per > Hi > > You could insert something like this: > > to dn.exact="ou=somedomain,dc=example,dc=com" > attrs=children > by group="cn=admingroup,ou=ldap,dc=example,dc=com" write stop > by * break > to dn.children="ou=somedomain,dc=example,dc=com" > attrs=@inetOrgPerson,@organizationalUnit,@posixAccount, > @shadowAccount,entry > by group="cn=admingroup,ou=ldap,dc=example,dc=com" write stop > by * break > > This will grant administrative rights by the specified group to: > > - the "children" attr of the entry: ou=somedomain,dc=example,dc=com > - all attrs of children of: ou=somedomain,dc=example,dc=com in the > specified objectClasses plus the entries itself > - all other users/groups will continue with ACL evaluation (by * break) > > This will prevent modifying the entry: ou=somedomain,dc=example,dc=com > If you do not need that, replace the two ACLs with: > > to dn.subtree=ou=somedomain,dc=example,dc=com" > ... > > But you have to add these ACLs before the rule with: > to attrs=userPassword ... > otherwise if you add it at the end, the "by * none" will prevent all > access. > > ou=ldap is just an ou that contains my slapd admin groups, for example: > > dn: ou=ldap,dc=example,dc=com > objectClass: organizationalUnit > ou: ldap > > dn: cn=admingroup,ou=ldap,dc=example,dc=com > objectClass: groupOfNames > member: uid=someuser,ou=somedomain,dc=example,dc=com > cn: admingroup > > Hope that helps a bit with the bleeding :) > And carefully test your ACLs, I had to do a lot of try and error to > get it right. > > Kind regards > Sven > > On 03/18/2018 04:54 PM, Lists Nethead wrote: >> >> Hi all, >> >> First post here, and it is asking for advice on an access rule. Setup is, >> >> +-dc=example,dc=com >> +--ou=somedomain >> +---uid=someuser,ou=somedomain,dc=example,dc=com >> +--ou=someotherdomain >> +---uid=otheruser,ou=someotherdomain,dc=example,dc=com >> +--ou=yetanotherdomain >> >> The ruleset so far that seems to work is >> >> to dn.base="" by * read >> to dn.base="cn=subschema" by * read >> to attrs=userPassword by dn.base="cn=admin,dc=example,dc=com" >>   write by >> dn.base="uid=otheradmin,ou=System,dc=example,dc=com" write by self >>   write by anonymous auth  by * none >> to *  by dn.base="uid=someadmin,ou=System,dc=example,dc=com" w >>  rite by self read  by >> peername.ip=<address>%255.255.255.0 read by peern >>  ame.ip=<address>%255.255.255.0 read by peername.ip=<address>%255.255.2 >>  55.0 read by users tls_ssf=256 read  by * none >> >> What I want next is that >> "uid=someuser,ou=somedomain,dc=example,dc=com" should be able to >> administer accounts in "ou=somedomain" and likewise for other "ou=". >> >> My guess is that a group with admin accounts is the way to go but >> right now my eyes are bleeding after reading the whole day about >> access rules... >> >> Thanks, >> >> //per >> >>

2 1

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical