Re: ldapdelete recursive (-r) with syncrepl
by Frank Crow
I did see that and do know how to do slapadd/slapcat backups. What I'm
not clear on is how that works with N-Way MMR. Wouldn't I have to go and
delete the the /var/lib/ldap on every master replica machine prior to
loading the backup with slapadd? Or is that not necessary?
In any case, I did update my syncrepl to use cn=accesslog and I no longer
have any issues with bulk operations being propagated to the other master
replicas.
Thanks,
Frank
On Wed, Apr 13, 2016 at 2:15 AM, Michael Ströder <michael(a)stroeder.com>
wrote:
> Please stay on the list so others can answer and learn as well.
>
>
> Frank Crow wrote:
> > OK, if I do a backup with slapcat, I still would want to wipe the
> existing
> > contents of the DIT first, right?
>
> That's why I wrote...
>
> > On Tue, Apr 12, 2016 at 5:47 PM, Michael Ströder <michael(a)stroeder.com>
> > wrote:
> >
> >> Frank Crow wrote:
> >>> I'm trying to create backup and restore scripts using LDAP command line
> >>> tools.
> >>
> >> For various reasons backup and restore should be done with command-line
> >> tools
> >> slapcat and slapadd which operate directly on the database files.
> >>
> >> And yes, with recent backend modules like back-mdb and back-hdb you can
> do
> >> hot
> >> backup while slapd is running.
> >>
> >> Of course, before a restore you have to stop slapd and remove the DB
> files.
>
> ^^^^^^^^^^^^^^^^^^^
> ...to remove the database files.
>
> >> After using slapadd you should check whether ownership/permissions are
> >> still
> >> correct.
>
> Please read the responses carefully.
>
> Ciao, Michael.
>
>
--
Frank
7 years, 8 months
Re: ldapdelete recursive (-r) with syncrepl
by Frank Crow
Well, I can strongly suggest that we upgrade past that but I know that I'll
get push back on it. I'll change the syncrepl to use refreshAndPersist.
Thanks,
Frank
On Tue, Apr 12, 2016 at 7:04 PM, Quanah Gibson-Mount <quanah(a)zimbra.com>
wrote:
> --On Tuesday, April 12, 2016 7:33 PM -0400 Frank Crow <
> fjcrow2008(a)gmail.com> wrote:
>
>
>> OpenLDAP 2.4.40
>>
>
> Upgrade. There are serious MMR issues in that release.
>
>
> Syncrepl configuration:
>>
>> olcSyncUseSubentry: FALSE
>> olcSyncrepl: {0}rid=101 provider=ldap://server1
>> searchbase="o=xxx,dc=yyy,
>> dc=zzz" type=refreshOnly bindmethod=sasl saslmech=EXTERNAL
>>
>
> I strongly advise against using refreshOnly. There's virtually no
> instance where that is the correct option.
>
>
> --Quanah
>
>
> --
>
> Quanah Gibson-Mount
> Platform Architect
> Zimbra, Inc.
> --------------------
> Zimbra :: the leader in open source messaging and collaboration
> A division of Synacor, Inc
>
--
Frank
7 years, 8 months
Re: ldapdelete recursive (-r) with syncrepl
by Frank Crow
OpenLDAP 2.4.40
Syncrepl configuration:
olcSyncUseSubentry: FALSE
> olcSyncrepl: {0}rid=101 provider=ldap://server1 searchbase="o=xxx,dc=yyy,
> dc=zzz" type=refreshOnly bindmethod=sasl saslmech=EXTERNAL
> tls_cert=/etc/openldap/certs/xxxxx.crt
> tls_key=/etc/openldap/certs/xxxxx.key
> tls_cacert=/etc/openldap/certs/cacert.pem interval=00:00:00:10
> retry="5 10 10 10 30 +" timeout=1 starttls=critical
> olcSyncrepl: {1}rid=102 provider=ldap://server2
> searchbase="o=xxx,dc=yyyy,
> dc=zzz" type=refreshOnly bindmethod=sasl saslmech=EXTERNAL
> tls_cert=/etc/openldap/certs/ldapadmin.crt
> tls_key=/etc/openldap/certs/xxxxx.key
> tls_cacert=/etc/openldap/certs/cacert.pem interval=00:00:00:10
> retry="5 10 10 10 30 +" timeout=1 starttls=critical
> olcMirrorMode: TRUE
BTW, I just tried addinging:
dn: olcOverly={3}syncprov,olcDatabase={2},cn=config
> changetype: modify
> replace: olcSpCheckpoint
> olcSpCheckpoint: 1024
> -
> add: olcSpSessionlog
> olcSpSessionlog: 1024
> -
> add: olcSpReloadhint
> olcSpReloadhint: TRUE
And that seemed to fix it! Maybe it was just the checkpoint being "1 1"
that was messing it up? Or maybe I needed the session log. I realize
that this is the deprecated approach. I probably put in cn=changelog
instead if there's a good reason to do so.
-Frank
On Tue, Apr 12, 2016 at 6:26 PM, Frank Crow <fjcrow2008(a)gmail.com> wrote:
> OK, if I do a backup with slapcat, I still would want to wipe the existing
> contents of the DIT first, right?
>
> Also, I just tried doing a list of deleted uid entries using "ldapdelete
> -ZZ -f /file.ldif" and although the command did not complain, not all of
> the entries in the file.ldif were deleted from all replicas. I really
> think there is something wrong with my configuration! I suppose that I'll
> try cn=changelog next.
>
> Thanks,
> Frank
>
>
> On Tue, Apr 12, 2016 at 5:47 PM, Michael Ströder <michael(a)stroeder.com>
> wrote:
>
>> Frank Crow wrote:
>> > I'm trying to create backup and restore scripts using LDAP command line
>> > tools.
>>
>> For various reasons backup and restore should be done with command-line
>> tools
>> slapcat and slapadd which operate directly on the database files.
>>
>> And yes, with recent backend modules like back-mdb and back-hdb you can
>> do hot
>> backup while slapd is running.
>>
>> Of course, before a restore you have to stop slapd and remove the DB
>> files.
>> After using slapadd you should check whether ownership/permissions are
>> still
>> correct.
>>
>> Ciao, Michael.
>>
>>
>
>
> --
> Frank
>
--
Frank
7 years, 8 months
Re: ldapdelete recursive (-r) with syncrepl
by Frank Crow
Oops... sorry... had inadvertently used "reply" rather than "reply all"!
Seems that if I used "ldapdelete -ZZ <dn>" on a single entry e.g.,
"uid=someuser,ou=Users,o=xxx,dc=yyy,dc=zzz" that propagates fine. But if
I delete the parent in bulk "ldapdelete -ZZ ou=Users,o=xxx,dc=yyy,dc=zzz"
then none of the "delete entry" actions get propagated to the other
masters. They still have the entire tree.
Is that the expected behavior? Should I only ever delete children nodes
and not containers?
Thanks,
Frank
On Tue, Apr 12, 2016 at 5:33 PM, Frank Crow <fjcrow2008(a)gmail.com> wrote:
> The interesting thing to me, is that a single "ldapdelete -ZZ <dn>" works
> and propagates no problem. That's really why I'm wondering if there is
> something non-obvious about "ldapdelete -ZZ *-r* <dn>" (recursive). Or
> is it that there is some issue in my configuration?
>
> Well, anyway, my test is going to do "higher up the tree" bulk deletes and
> see if I have any different results.
>
> Thanks,
> Frank
>
>
> On Tue, Apr 12, 2016 at 4:49 PM, Frank Crow <fjcrow2008(a)gmail.com> wrote:
>
>> I'm trying to use ldapsearch and ldapadd for backup and restore with
>> N-Way MMR on the backend. Maybe it's just my MMR config that's incorrect??
>>
>> And I still need to have "delete entry" propagate to all masters. We
>> have formal tests that will be exercising that capability with our users,
>> roles, perms, etc. We have GUIs that add/change/delete entries for
>> administration, etc. I can even expect "bulk" deletes from those GUIs.
>>
>> I'll be doing various experiments to change the replication parameters
>> and probably add a session log, etc. I was just hoping that someone could
>> point me to how/where I can better understand this. (I have and am
>> reading the Admin Guide, man pages, and even "Mastering LDAP", etc., just
>> BTW.) I must have missed something somewhere.
>>
>> Thanks,
>> Frank
>>
>>
>>
>> On Tue, Apr 12, 2016 at 4:11 PM, Quanah Gibson-Mount <quanah(a)zimbra.com>
>> wrote:
>>
>>> --On Tuesday, April 12, 2016 4:49 PM -0400 Frank Crow <
>>> fjcrow2008(a)gmail.com> wrote:
>>>
>>>
>>>> I'm trying to create backup and restore scripts using LDAP command line
>>>> tools. Creating a suitable backup and loading it are no problem.
>>>> However, if I attempt to:
>>>>
>>>
>>> Generally one does backup and restore using slapcat & slapadd. What are
>>> you trying to accomplish?
>>>
>>> --Quanah
>>>
>>>
>>> --
>>>
>>> Quanah Gibson-Mount
>>> Platform Architect
>>> Zimbra, Inc.
>>> --------------------
>>> Zimbra :: the leader in open source messaging and collaboration
>>> A division of Synacor, Inc
>>>
>>
>>
>>
>> --
>> Frank
>>
>
>
>
> --
> Frank
>
--
Frank
7 years, 8 months
ldapdelete recursive (-r) with syncrepl
by Frank Crow
I'm trying to create backup and restore scripts using LDAP command line
tools. Creating a suitable backup and loading it are no problem.
However, if I attempt to:
# ldapdelete -ZZ -r o=xxx,dc=yyyy,dc=zzz
It appears to work fine on the master replica where the command was run,
but the other (MMR) master still maintains the entire DIT fragment. It
doesn't ever seem to delete any of the entries.
Is there anything unique about the (-r) recursive ldapdelete? Is this
behavior unique to "refreshOnly"? Should I use "refreshAndPersist"? Or
do I need to implement a session log?
Thanks for any help or advice,
--
Frank
7 years, 8 months
Re: slapd ACL - limit bind to employeeType=<various>
by Tim Watts
On 12/04/16 18:32, Michael Ströder wrote:
> Tim Watts wrote:
>> "Delete" - they vanish from libnss and other places, but we still hold their
>> LDAP record for easy resurrection (this happens a lot - we have a class of
>> nomadic users - they work on a project, go away, then come back on another
>> project 2 years later).
>
> That state I'd call "inactive" or similar. But that's cosmetic.
>
> In my systems there's always exactly one status "active" for which I allow
> "auth" on "userPassword".
>
> E.g. in Æ-DIR the attribute 'aeStatus' can have this Integer values:
>
> -1: requested
> 0: active
> 1: deactivated
> 2: archived
>
> I suspect you're overloading the semantics of 'employeeType' by putting two
> meaning into one attribute.
>
> Ciao, Michael.
>
Yes - I think I am. It was partly convenience (I'm using the LAM web
front end to manage, with ldapvi where LAM cannot cope (odd problems
with empty mail attributes). But also I did not think of your idea -
it's pretty cool.
I'd have to admit to being more of an LDAP n00b (we populated
/etc/passwd directly from postgresql in my last place).
--
Tim Watts Tel (VOIP): +44 (0)1580 848360
Systems Manager Kings Digital Lab (KDL), King's College London
Systems Messages and Notifications: https://systemsblog.cch.kcl.ac.uk/
7 years, 8 months
Re: slapd ACL - limit bind to employeeType=<various>
by Tim Watts
Sir, you are a genius :)
On 11/04/16 07:31, Michael Ströder wrote:
> # some entries matching filter
> access to
> attrs=userPassword
> filter=(!(employeeType=Archive)(employeeType=Delete))
> by ..some who clauses for setting password
> by * auth
>
> # all other entries
> access to
> attrs=userPassword
> by * none
Very slight tweak to the syntax (with huge thanks - I would not have
guessed this was the required technique - I was concentrating on finding
an "auth" ACL when I was googling.)
This is the full ACL list for context, should anyone find this useful.
The nice thing is that I can add new employeeTypes (eg "BA" for
undergrads) without breaking the core logic.
Also, this should be fundamentally more secure as we can now disable
accounts in one place.
In case anyone is wondering the difference between "Archive" and
"Delete", they are as follows:
"Archive" - keep the user in the libnss lists (eg getent passwd still
returns them, so that file UIDs resolve). User cannot log in.
"Delete" - they vanish from libnss and other places, but we still hold
their LDAP record for easy resurrection (this happens a lot - we have a
class of nomadic users - they work on a project, go away, then come back
on another project 2 years later).
===========================================
access to attrs=userPassword
filter=(&(!(employeeType=Archive))(!(employeeType=Delete)))
by peername.path="/var/run/slapd/ldapi" manage
by set="user/uid &
[cn=sysadmin,ou=groups,dc=dighum,dc=kcl,dc=ac,dc=uk
]/memberUid" manage
by self write
by * auth
access to attrs=userPassword
by * none
# Certain attributes that should not be publicly readable
access to
attrs=bindTimestamp,modifyTimestamp,modifiersName,creatorsName,creat
eTimestamp
by peername.path="/var/run/slapd/ldapi" manage
by set="user/uid &
[cn=sysadmin,ou=groups,dc=dighum,dc=kcl,dc=ac,dc=uk
]/memberUid" manage
by self read
by * none
# The admin dn has full write access, everyone else
# can read everything. Local unix domain socket (root only)
# Can do everything
access to *
by peername.path="/var/run/slapd/ldapi" manage
by set="user/uid &
[cn=sysadmin,ou=groups,dc=dighum,dc=kcl,dc=ac,dc=uk
]/memberUid" manage
by * read
===========================================
I'll do some more testing and have a look at that "set=" inefficiency too.
Thanks very much for your help :)
Tim
--
Tim Watts Tel (VOIP): +44 (0)1580 848360
Systems Manager Kings Digital Lab (KDL), King's College London
Systems Messages and Notifications: https://systemsblog.cch.kcl.ac.uk/
7 years, 8 months
Re: slapd ACL - limit bind to employeeType=<various>
by Tim Watts
On 12/04/16 10:54, Michael Ströder wrote:
> Writing OpenLDAP ACLs is a bit like functional
> programming - at least to what I vaguely remember
> from my time at University many years ago.
>
> I'd recommend to look into the OpenLDAP FAQ to
> find some more not so obvious examples.
:)
It's something that takes time to "get" - bit like exim configuration.
Very powerful, but a little bit of an art that's not quite like anything
else!
--
Tim Watts Tel (VOIP): +44 (0)1580 848360
Systems Manager Kings Digital Lab (KDL), King's College London
Systems Messages and Notifications: https://systemsblog.cch.kcl.ac.uk/
7 years, 8 months
Help needed to migrate htpasswd APR1 hash to OpenLdap
by Michel Jouvin
Hi,
We recently upgraded one of our Apache server from an htpasswd-based
authentication to an OpenLdap-based one. As part of this, we had to move
users identified by a username+password and faced the problem of
converting the APR1 (Apache salted-MD5) hashes to OpenLDAP. We managed
to migrate crypt and MD5 ($1$) hashes successfully as they are
successfully managed by OpenLdap standard componenents.
This is not the case for APR1 hashes, AFAIK. As documented on OpenLdap
site, we built, installed and loaded pw-apr1.so from contrib/ in
sources. With Perl Digest::MD5 + MIME::Base64::encode_base64, I have
been able to generate a MD5 hash ({MD5} prefix in userPassword) that
works with {MD5} so I'm pretty confident that pw-apr1.so works properly.
But I'm stucked with APR1 password as I don't understand what should be
done to transform a $apr1$... has into the {APR1}... format. According
to an example given in the issue
http://www.openldap.org/its/index.cgi/Contrib?id=7869;page=9;statetype=-1 (related
to a former bug in apr1.c), $apr1$pepperon$VBD3GaYfoFXuIcZrTw/Y// should
transform into {APR1}FK7q3pAf80k0Wh9BYZJ0FHBlcHBlcm9u but I have no clue
how to get such a transformation done (does not seem to be a trivial
encoding to base64).
Any help would be very much appreciated! Cheers,
Michel
7 years, 8 months
Re: slapd ACL - limit bind to employeeType=<various>
by Tim Watts
Hi Michael,
On 11/04/16 09:11, Michael Ströder wrote:
>>
>> OK - I'm going to have to get my head around that :) On a test
>> platform... Am I right in thinking the job of the 2nd ACL is because
>> if employeeType is Archive|Delete, the first ACL will simple fall
>> through - so the second ACL is semantically a "Deny All"?
>
> Yepp.
Thanks! That's clearer now.
>> One other thing - I did not mention, which is retrospect might be
>> important:
>>
>> I don't let slapd store password hashes - it passes through to
>> Kerberos via saslauthd. So the attribute is of this form:
>>
>> userPassword: {SASL}someuser(a)MY.KERB.REALM
>>
>> I presume that blocking access to userPassword will still cause
>> authentication to fail in this case as it won't be able to do that
>> lookup?
>
> Yes, I think so. But I never used saslauthd myself.
I'll set up a test and confirm to this list (to make the archive of this
thread more useful to someone else).
>> I thought you'd say that :) I'm OK with limiting access to the parent
>> directory (in this case to the slapd user and root). For me, it feels
>> simpler. You may disagree, but I just wanted to say it wasn't an
>> oversight.
>
> Your server, your attack vectors...
:)
Cheers!
Tim
--
Tim Watts Tel (VOIP): +44 (0)1580 848360
Systems Manager Kings Digital Lab (KDL), King's College London
Systems Messages and Notifications: https://systemsblog.cch.kcl.ac.uk/
7 years, 8 months