On Apr 20, 2010, at 1:50 PM, Sergiy Stepanenko wrote:
On 04/20/2010 10:49 AM, Ariel wrote:
On Apr 20, 2010, at 12:20 PM, Sergiy Stepanenko wrote:
On 04/20/2010 06:12 AM, Ariel wrote:
I have inherited an openldap server (2.4.9) and have set about to making it a bit more fault tolerant. So I have added a syncrepl slave and everything seems to work fine. It pulls down the whole ldap tree and stays in sync in real time with type=refreshAndPersist. I can use command line tools to very that it has all the information from the original master server using a command like this:
ldapsearch -xLL -H ldaps://ldap2.domain.com/ -b "dc=domain,dc=com" -D "cn=admin,dc=domain,dc=com" -W
My problem however is that when I try to authenticate users against the slave server, it does not work. All attempts fail. Also when I use ApacheDS (graphical LDAP browser) to view its contents, it only shows the Root DSE and none of the child objects like cn=config or any of the users or any of that. I can use ApacheDS fine to view and modify everything on the master server though.
The slapd.conf config files between the two are exactly the same (except one is declared as sync master and one as slave), the password hashes are successfully replicated to the slave as I can see with ldapsearch, but I have no idea how to debug why it won't authenticate users. For reference, here is my syncrepl config section (in slapd.conf) on the slave:
syncrepl rid=123 provider=ldaps://ldap1.domain.com:636 type=refreshAndPersist searchbase="dc=domain,dc=com" filter="(objectClass=*)" scope=sub schemachecking=off bindmethod=simple binddn="uid=syncuser,ou=People,dc=domain,dc=com" credentials=syncpassword
and on the master server:
moduleload syncprov.la overlay syncprov syncprov-checkpoint 100 10 syncprov-sessionlog 100
As a failover/backup server seems extremely prudent especially on the ancient hardware we have running these things, I really want to get this to work properly. Perhaps even later doing a round-robin style load balancing between the two or what have you.
I have no idea how to debug this, any help would be greatly appreciated!
-a
Hi
If you can show ACL on master and slave it will help to understand why you cannot auth users against slave. From what you described, it looks like slave has either access restriction to userPassword and base DN for users or access fails. I also never noticed any TLS certificates configuration mentioned.
-- Sergiy Stepanenko Systems Administrator Information Technology Services University of Saskatchewan
phone: (306) 966-2762 email:sergiy.stepanenko@usask.ca
Hey, thanks for the reply! I can only see the ACL on the slave using ldapsearch from CLI (every user I have tested has seen exactly only the information they are allowed), and all privileges seem to be in working order.
When I use ApacheDS (or any other graphical client) using the same users, I only get the Root DSE which describes what Schema are loaded. Also, we do not use TLS, just ldaps protocol with simple authentication. All clients have `TLS_REQCERT allow` in their /etc/ldap/ldap.conf so a self signed ssl certificate on the server will work.
Make sure to use port 663 for SSL connection in ApacheDS, also check what bas DN is set up in Apache (Browser Options)
Connecting and authenticating from ldapsearch works, but no other mechanisms will work.
I'm probably missing something so simple it would make me cry. Perhaps I should set the loglevel to something ridiculously high and see what it says?
Try loglevel acl conns for starters and look at Apache search log. If it has been an error it would be there.
-a
Read log ldap or syslog where ever you point slapd output. I suspect that ldap appliances connect->bind->authenticate->access as self, but GUI clients fail to authenticate and resort in anonymous access level which normally looks like access to DSE entry only.
-- Sergiy Stepanenko Systems Administrator Information Technology Services University of Saskatchewan
phone: (306) 966-2762 email:sergiy.stepanenko@usask.ca
Putting loglevel to -1 (everything) and logging in with ApacheDS as cn=admin,dc=domain,dc=com (which is supposed to supersede any ACL rules and have read/write to everything I believe) I find a whole lot of "access granted" lines and then towards the end":
=> access_allowed: search access to "cn=config" "entry" requested => slap_access_allowed: backend default search access denied to "cn=admin,dc=domain,dc=com" => access_allowed: no more rules send_ldap_result: conn=0 op=8 p=3 send_ldap_result: err=32 matched="" text=""
Error 32 means object doesn't exist (I think). Which would be true, our LDAP tree has no cn=config. We get the same error on the primary server, so I suppose it is ApacheDS trying to look for what would be in the Apache LDAP implementation. But that's the only error I can find, everything else is miles and miles of "search access granted".
I tried to get it to list DN="dc=domain,dc=com" by hand from ApacheDS, and it would not return anything (it says "No base DN returned from server.") although in the logs it shows:
conn=6 op=3 SRCH base="dc=domain,dc=com" scope=0 deref=3 filter="(objectClass=*)" conn=6 op=3 SRCH attr=hasSubordinates objectClass => hdb_search bdb_dn2entry("dc=domain,dc=com") => access_allowed: search access to "dc=domain,dc=com" "entry" requested <= root access granted access_allowed: search access granted by manage(=mwrscxd) base_candidates: base: "dc=domains,dc=com" (0x00000001) send_ldap_result: conn=6 op=3 p=3 send_ldap_result: err=0 matched="" text="" send_ldap_response: msgid=4 tag=101 err=0
But when I run the ldapsearch command (as any user) from other computers on the network it returns the DN's information... So I am thoroughly confused... I am pretty sure it is not logging in as anonymous, but I have no idea why only the ldapsearch command is the only thing that can authenticate and retrieve information. It is the same version of openldap as the primary server, it has the same exact config, it has all the same schema loaded, it has the exact full ldap tree. I'm going to explode!@$#@
-a
I guess your backend is bdb or hdb. Run
ldapsearch -LL -H <your_ldap_host> -x -D <rootdn> -w <rootpw> -b "cn=config" * +
and see what it produces. But before you do that open slapd.conf if you use flat file configuration and check what exactly configured for rootdn and rootpw because if "backend default search access denied to "cn=admin,dc=domain,dc=com" then you have a problem in config.
It is much easier to talk if I know what is configured actually, because snippets of log and config file dont tell whole story.
Cheers
Putting loglevel to -1 (everything) and logging in with ApacheDS as cn=admin,dc=domain,dc=com (which is supposed to supersede any ACL rules and have read/write to everything I believe) I find a whole lot of "access granted" lines and then towards the end":
=> access_allowed: search access to "cn=config" "entry" requested => slap_access_allowed: backend default search access denied to "cn=admin,dc=domain,dc=com" => access_allowed: no more rules send_ldap_result: conn=0 op=8 p=3 send_ldap_result: err=32 matched="" text=""
Error 32 means object doesn't exist (I think). Which would be true, our LDAP tree has no cn=config. We get the same error on the primary server, so I suppose it is ApacheDS trying to look for what would be in the Apache LDAP implementation. But that's the only error I can find, everything else is miles and miles of "search access granted".
I tried to get it to list DN="dc=domain,dc=com" by hand from ApacheDS, and it would not return anything (it says "No base DN returned from server.") although in the logs it shows:
conn=6 op=3 SRCH base="dc=domain,dc=com" scope=0 deref=3 filter="(objectClass=*)" conn=6 op=3 SRCH attr=hasSubordinates objectClass => hdb_search bdb_dn2entry("dc=domain,dc=com") => access_allowed: search access to "dc=domain,dc=com" "entry" requested <= root access granted access_allowed: search access granted by manage(=mwrscxd) base_candidates: base: "dc=domains,dc=com" (0x00000001) send_ldap_result: conn=6 op=3 p=3 send_ldap_result: err=0 matched="" text="" send_ldap_response: msgid=4 tag=101 err=0
But when I run the ldapsearch command (as any user) from other computers on the network it returns the DN's information... So I am thoroughly confused... I am pretty sure it is not logging in as anonymous, but I have no idea why only the ldapsearch command is the only thing that can authenticate and retrieve information. It is the same version of openldap as the primary server, it has the same exact config, it has all the same schema loaded, it has the exact full ldap tree. I'm going to explode!@$#@
-a
My backend is hdb. Also, I'm very sure the rootdn and rootpw are correct, as I have been using it for most of my testing. As for being denied read for cn=config, I'm pretty sure that is because it doesn't exist.
After running your command this is the result:
No such object (32)
It does not exist on either the master or slave server. I am using the slapd.conf configuration method, is there anything else I could show you?
-a
On Apr 20, 2010, at 4:29 PM, Sergiy Stepanenko wrote:
I guess your backend is bdb or hdb. Run
ldapsearch -LL -H <your_ldap_host> -x -D <rootdn> -w <rootpw> -b "cn=config" * +
and see what it produces. But before you do that open slapd.conf if you use flat file configuration and check what exactly configured for rootdn and rootpw because if "backend default search access denied to "cn=admin,dc=domain,dc=com" then you have a problem in config.
It is much easier to talk if I know what is configured actually, because snippets of log and config file dont tell whole story.
Cheers
Putting loglevel to -1 (everything) and logging in with ApacheDS as cn=admin,dc=domain,dc=com (which is supposed to supersede any ACL rules and have read/write to everything I believe) I find a whole lot of "access granted" lines and then towards the end":
=> access_allowed: search access to "cn=config" "entry" requested => slap_access_allowed: backend default search access denied to "cn=admin,dc=domain,dc=com" => access_allowed: no more rules send_ldap_result: conn=0 op=8 p=3 send_ldap_result: err=32 matched="" text=""
Error 32 means object doesn't exist (I think). Which would be true, our LDAP tree has no cn=config. We get the same error on the primary server, so I suppose it is ApacheDS trying to look for what would be in the Apache LDAP implementation. But that's the only error I can find, everything else is miles and miles of "search access granted".
I tried to get it to list DN="dc=domain,dc=com" by hand from ApacheDS, and it would not return anything (it says "No base DN returned from server.") although in the logs it shows:
conn=6 op=3 SRCH base="dc=domain,dc=com" scope=0 deref=3 filter="(objectClass=*)" conn=6 op=3 SRCH attr=hasSubordinates objectClass => hdb_search bdb_dn2entry("dc=domain,dc=com") => access_allowed: search access to "dc=domain,dc=com" "entry" requested <= root access granted access_allowed: search access granted by manage(=mwrscxd) base_candidates: base: "dc=domains,dc=com" (0x00000001) send_ldap_result: conn=6 op=3 p=3 send_ldap_result: err=0 matched="" text="" send_ldap_response: msgid=4 tag=101 err=0
But when I run the ldapsearch command (as any user) from other computers on the network it returns the DN's information... So I am thoroughly confused... I am pretty sure it is not logging in as anonymous, but I have no idea why only the ldapsearch command is the only thing that can authenticate and retrieve information. It is the same version of openldap as the primary server, it has the same exact config, it has all the same schema loaded, it has the exact full ldap tree. I'm going to explode!@$#@
-a
-- Sergiy Stepanenko Systems Administrator Information Technology Services University of Saskatchewan
phone: (306) 966-2762 email:sergiy.stepanenko@usask.ca
Ok, since I should be updating this system to using the slapd.d/ configuration method eventually anyway, I did a `slaptest -f slapd.conf -F slapd.d` and got it to where I believe it works fine from the cn=config. This time, after running your command, this is the output I get:
version: 1
dn: cn=config structuralObjectClass: olcGlobal entryUUID: 2226bfc8-e1a6-102e-80d2-ab1f64ee6c67 creatorsName: createTimestamp: 20100421152725Z entryCSN: 20100421152725.747572Z#000000#000#000000 modifiersName: modifyTimestamp: 20100421152725Z entryDN: cn=config subschemaSubentry: cn=Subschema
dn: cn=module{0},cn=config structuralObjectClass: olcModuleList entryUUID: 2226bfc8-e1a6-102e-80d3-ab1f64ee6c67 creatorsName: createTimestamp: 20100421152725Z entryCSN: 20100421152725.747572Z#000001#000#000000 modifiersName: modifyTimestamp: 20100421152725Z entryDN: cn=module{0},cn=config subschemaSubentry: cn=Subschema
dn: cn=schema,cn=config structuralObjectClass: olcSchemaConfig entryUUID: 2226bfc8-e1a6-102e-80d4-ab1f64ee6c67 creatorsName: createTimestamp: 20100421152725Z entryCSN: 20100421152725.747572Z#000002#000#000000 modifiersName: modifyTimestamp: 20100421152725Z entryDN: cn=schema,cn=config subschemaSubentry: cn=Subschema
dn: cn={0}core,cn=schema,cn=config structuralObjectClass: olcSchemaConfig entryUUID: 2226bfc8-e1a6-102e-80d5-ab1f64ee6c67 creatorsName: createTimestamp: 20100421152725Z entryCSN: 20100421152725.747572Z#000003#000#000000 modifiersName: modifyTimestamp: 20100421152725Z entryDN: cn={0}core,cn=schema,cn=config subschemaSubentry: cn=Subschema
dn: cn={1}cosine,cn=schema,cn=config structuralObjectClass: olcSchemaConfig entryUUID: 2226bfc8-e1a6-102e-80d6-ab1f64ee6c67 creatorsName: createTimestamp: 20100421152725Z entryCSN: 20100421152725.747572Z#000004#000#000000 modifiersName: modifyTimestamp: 20100421152725Z entryDN: cn={1}cosine,cn=schema,cn=config subschemaSubentry: cn=Subschema
dn: cn={2}nis,cn=schema,cn=config structuralObjectClass: olcSchemaConfig entryUUID: 2226bfc8-e1a6-102e-80d7-ab1f64ee6c67 creatorsName: createTimestamp: 20100421152725Z entryCSN: 20100421152725.747572Z#000005#000#000000 modifiersName: modifyTimestamp: 20100421152725Z entryDN: cn={2}nis,cn=schema,cn=config subschemaSubentry: cn=Subschema
dn: cn={3}inetorgperson,cn=schema,cn=config structuralObjectClass: olcSchemaConfig entryUUID: 2226bfc8-e1a6-102e-80d8-ab1f64ee6c67 creatorsName: createTimestamp: 20100421152725Z entryCSN: 20100421152725.747572Z#000006#000#000000 modifiersName: modifyTimestamp: 20100421152725Z entryDN: cn={3}inetorgperson,cn=schema,cn=config subschemaSubentry: cn=Subschema
dn: cn={4}ppolicy,cn=schema,cn=config structuralObjectClass: olcSchemaConfig entryUUID: 2226bfc8-e1a6-102e-80d9-ab1f64ee6c67 creatorsName: createTimestamp: 20100421152725Z entryCSN: 20100421152725.747572Z#000007#000#000000 modifiersName: modifyTimestamp: 20100421152725Z entryDN: cn={4}ppolicy,cn=schema,cn=config subschemaSubentry: cn=Subschema
dn: olcDatabase={-1}frontend,cn=config structuralObjectClass: olcDatabaseConfig entryUUID: 2226bfc8-e1a6-102e-80da-ab1f64ee6c67 creatorsName: createTimestamp: 20100421152725Z entryCSN: 20100421152725.747572Z#000008#000#000000 modifiersName: modifyTimestamp: 20100421152725Z entryDN: olcDatabase={-1}frontend,cn=config subschemaSubentry: cn=Subschema
dn: olcDatabase={0}config,cn=config structuralObjectClass: olcDatabaseConfig entryUUID: 2226bfc8-e1a6-102e-80db-ab1f64ee6c67 creatorsName: createTimestamp: 20100421152725Z entryCSN: 20100421152725.747572Z#000009#000#000000 modifiersName: modifyTimestamp: 20100421152725Z entryDN: olcDatabase={0}config,cn=config subschemaSubentry: cn=Subschema
dn: olcDatabase={1}hdb,cn=config structuralObjectClass: olcHdbConfig entryUUID: 2226bfc8-e1a6-102e-80dc-ab1f64ee6c67 creatorsName: createTimestamp: 20100421152725Z entryCSN: 20100421152725.747572Z#00000a#000#000000 modifiersName: modifyTimestamp: 20100421152725Z entryDN: olcDatabase={1}hdb,cn=config subschemaSubentry: cn=Subschema
dn: olcOverlay={0}ppolicy,olcDatabase={1}hdb,cn=config structuralObjectClass: olcPPolicyConfig entryUUID: 2226bfc8-e1a6-102e-80dd-ab1f64ee6c67 creatorsName: createTimestamp: 20100421152725Z entryCSN: 20100421152725.747572Z#00000b#000#000000 modifiersName: modifyTimestamp: 20100421152725Z entryDN: olcOverlay={0}ppolicy,olcDatabase={1}hdb,cn=config subschemaSubentry: cn=Subschema
I hope this is what you were looking for and helps. -a
On Apr 20, 2010, at 4:29 PM, Sergiy Stepanenko wrote:
I guess your backend is bdb or hdb. Run
ldapsearch -LL -H <your_ldap_host> -x -D <rootdn> -w <rootpw> -b "cn=config" * +
and see what it produces. But before you do that open slapd.conf if you use flat file configuration and check what exactly configured for rootdn and rootpw because if "backend default search access denied to "cn=admin,dc=domain,dc=com" then you have a problem in config.
It is much easier to talk if I know what is configured actually, because snippets of log and config file dont tell whole story.
Cheers
Putting loglevel to -1 (everything) and logging in with ApacheDS as cn=admin,dc=domain,dc=com (which is supposed to supersede any ACL rules and have read/write to everything I believe) I find a whole lot of "access granted" lines and then towards the end":
=> access_allowed: search access to "cn=config" "entry" requested => slap_access_allowed: backend default search access denied to "cn=admin,dc=domain,dc=com" => access_allowed: no more rules send_ldap_result: conn=0 op=8 p=3 send_ldap_result: err=32 matched="" text=""
Error 32 means object doesn't exist (I think). Which would be true, our LDAP tree has no cn=config. We get the same error on the primary server, so I suppose it is ApacheDS trying to look for what would be in the Apache LDAP implementation. But that's the only error I can find, everything else is miles and miles of "search access granted".
I tried to get it to list DN="dc=domain,dc=com" by hand from ApacheDS, and it would not return anything (it says "No base DN returned from server.") although in the logs it shows:
conn=6 op=3 SRCH base="dc=domain,dc=com" scope=0 deref=3 filter="(objectClass=*)" conn=6 op=3 SRCH attr=hasSubordinates objectClass => hdb_search bdb_dn2entry("dc=domain,dc=com") => access_allowed: search access to "dc=domain,dc=com" "entry" requested <= root access granted access_allowed: search access granted by manage(=mwrscxd) base_candidates: base: "dc=domains,dc=com" (0x00000001) send_ldap_result: conn=6 op=3 p=3 send_ldap_result: err=0 matched="" text="" send_ldap_response: msgid=4 tag=101 err=0
But when I run the ldapsearch command (as any user) from other computers on the network it returns the DN's information... So I am thoroughly confused... I am pretty sure it is not logging in as anonymous, but I have no idea why only the ldapsearch command is the only thing that can authenticate and retrieve information. It is the same version of openldap as the primary server, it has the same exact config, it has all the same schema loaded, it has the exact full ldap tree. I'm going to explode!@$#@
-a
-- Sergiy Stepanenko Systems Administrator Information Technology Services University of Saskatchewan
phone: (306) 966-2762 email:sergiy.stepanenko@usask.ca
I simulated conditions close to what you described and in my case it was caused by database config not being defined in slapd.conf. Ldapwhoami returns proper response, ldap search on anything but cn=config returns proper response, cn=config search returns err 32 no such object and Apache is unable to connect at all. Sounds about right... Make sure that you do have database config defined either way (slapd.conf or slap.d via ldif) with proper access definition to it by either rootdn or access to.
Hope it is all you need to do.
Best regards
On Apr 21, 2010, at 5:53 PM, Sergiy Stepanenko wrote:
I simulated conditions close to what you described and in my case it was caused by database config not being defined in slapd.conf. Ldapwhoami returns proper response, ldap search on anything but cn=config returns proper response, cn=config search returns err 32 no such object and Apache is unable to connect at all. Sounds about right... Make sure that you do have database config defined either way (slapd.conf or slap.d via ldif) with proper access definition to it by either rootdn or access to.
Hope it is all you need to do.
Best regards
-- Sergiy Stepanenko Systems Administrator Information Technology Services University of Saskatchewan
phone: (306) 966-2762 email:sergiy.stepanenko@usask.ca
I ended up taking down the primary, doing a `slapcat -l backup.ldif` and bringing it back up. Then went to the slave, took down slapd then `rm -rf /var/lib/ldap/*; slapadd -l backup.ldif; chown -r openldap:openldap /var/lib/ldap/*` and then bringing slapd back up. Checked to make sure replication was working. And hurray, I can authenticate against the slave server now!
That was very strange, replication to go out of sync but keep updating the slave with no errors or warnings. I have backups from both servers and I cannot see any difference in their ldif's. Very confusing because everything is the same, the flavor and version of linux, the version of openldap, the config files (except for the syncrepl parts). I hope this doesn't happen again.
Without your ideas and support I would not have kept going to fixing this. Thank you!
-a
openldap-technical@openldap.org
-
Ariel -
Sergiy Stepanenko