Hummel, Wolfgang wrote:

Hello OpenLDAP community,

we are currently planning for a largescale directory with

...

1 Bio. entries in a single namespace.

The idea is to divide the DB in 10 equal sized DBs and let them access by slapd-meta proxies.

Example: dn: ou=rsp1,c=de,o=mno entries with uid=79101234567890, 79101234567891 etc. 791 is always fix

In our scenario each server would have 100 Mio. entries using the last digit of uid as a naturally even balanced distribution mechanism.

Here are the questions:

• is slapd-meta a feasible approach for this scenario ?
• how could the slapd.conf for the proxy look like ?

Here is a (non working) example with 2 backend servers. What is wrong about it ?

Try using the correct DN to authenticate with. Your config uses "cn=admin" and your ldapsearch used "uid=admin" so naturally the request failed.

Other folks have been looking to do sharding this way as well. I think back-meta is a good starting point but it isn't the most convenient for it in its current form.

... moduleload back_meta moduleload back_ldap ... ####################################################################### # Meta database ####################################################################### database meta suffix " ou=rsp1,c=de,o=mno" dncache-ttl forever lastmod off rootdn "cn=admin,ou=rsp1,c=de,o=mno" rootpw secret network-timeout 1 uri "ldap://10.11.12.170/ ou=rsp1,c=de,o=mno" rewriteEngine on #rewriteContext searchFilterAttrDN rewriteContext searchFilter rewriteRule '^uid=[0-9]{11}1,.*' 'ldap://10.11.12.170/%0' ':@' uri "ldap://10.11.12.180/ ou=rsp1,c=de,o=mno" rewriteEngine on #rewriteContext searchFilterAttrDN rewriteContext searchFilter rewriteRule '^uid=[0-9]{11}2,.*' 'ldap://10.11.12.180/%0' ':@' ...

logfile snippet for # ldapsearch -LLL -xD uid=admin,ou=rsp1,c=de,o=mno -w secret -b ou=rsp1,c=de,o=mno uid=791720001981 ldap_bind: Invalid credentials (49)

Apr 23 08:44:13 slapd[26200]: >>> dnPrettyNormal: <uid=admin,ou=rsp1,c=de,o=mno> Apr 23 08:44:13 slapd[26200]: <<< dnPrettyNormal: <uid=admin,ou=rsp1,c=de,o=mno>, <uid=admin,ou=rsp1,c=de,o=mno> Apr 23 08:44:13 slapd[26200]: conn=1015 op=0 BIND dn="uid=admin,ou=rsp1,c=de,o=mno" method=128 Apr 23 08:44:13 slapd[26200]: do_bind: version=3 dn="uid=admin,ou=rsp1,c=de,o=mno" method=128 Apr 23 08:44:13 slapd[26200]: conn=1015 op=0 meta_back_bind: dn="uid=admin,ou=rsp1,c=de,o=mno". Apr 23 08:44:13 slapd[26200]: conn=1015 op=0: meta_back_getconn[0] Apr 23 08:44:13 slapd[26200]: conn=1015 op=0: meta_back_getconn[1] Apr 23 08:44:13 slapd[26200]: conn=1015 op=0 meta_back_getconn: candidates=2 conn=ANON fetched Apr 23 08:44:13 slapd[26200]: conn=1015 op=0 >>> meta_back_search_start[0] Apr 23 08:44:13 slapd[26200]: conn=1015 op=0 >>> meta_search_dobind_init[0] Apr 23 08:44:13 slapd[26200]: conn=1015 op=0 <<< meta_search_dobind_init[0]=1 Apr 23 08:44:13 slapd[26200]: ==> rewrite_context_apply [depth=1] string='uid=admin,ou=rsp1,c=de,o=mno' Apr 23 08:44:13 slapd[26200]: ==> rewrite_context_apply [depth=1] res={0,'NULL'} Apr 23 08:44:13 slapd[26200]: [rw] searchBase: "uid=admin,ou=rsp1,c=de,o=mno" -> "uid=admin,ou=rsp1,c=de,o=mno" Apr 23 08:44:13 slapd[26200]: ==> rewrite_context_apply [depth=1] string='(objectClass=*)' Apr 23 08:44:13 slapd[26200]: ==> rewrite_rule_apply rule=''^uid=[0-9]{11}1,.*'' string='(objectClass=*)' [1 pass(es)] Apr 23 08:44:13 slapd[26200]: ==> rewrite_context_apply [depth=1] res={0,'(objectClass=*)'} Apr 23 08:44:13 slapd[26200]: [rw] searchFilter: "(objectClass=*)" -> "(objectClass=*)" Apr 23 08:44:13 slapd[26200]: conn=1015 op=0 <<< meta_back_search_start[0]=1 Apr 23 08:44:13 slapd[26200]: conn=1015 op=0 >>> meta_back_search_start[1] Apr 23 08:44:13 slapd[26200]: conn=1015 op=0 >>> meta_search_dobind_init[1] Apr 23 08:44:13 slapd[26200]: conn=1015 op=0 <<< meta_search_dobind_init[1]=1 Apr 23 08:44:13 slapd[26200]: ==> rewrite_context_apply [depth=1] string='uid=admin,ou=rsp1,c=de,o=mno' Apr 23 08:44:13 slapd[26200]: ==> rewrite_context_apply [depth=1] res={0,'NULL'} Apr 23 08:44:13 slapd[26200]: [rw] searchBase: "uid=admin,ou=rsp1,c=de,o=mno" -> "uid=admin,ou=rsp1,c=de,o=mno" Apr 23 08:44:13 slapd[26200]: ==> rewrite_context_apply [depth=1] string='(objectClass=*)' Apr 23 08:44:13 slapd[26200]: ==> rewrite_rule_apply rule=''^uid=[0-9]{11}2,.*'' string='(objectClass=*)' [1 pass(es)] Apr 23 08:44:13 slapd[26200]: ==> rewrite_context_apply [depth=1] res={0,'(objectClass=*)'} Apr 23 08:44:13 slapd[26200]: [rw] searchFilter: "(objectClass=*)" -> "(objectClass=*)" Apr 23 08:44:13 slapd[26200]: conn=1015 op=0 <<< meta_back_search_start[1]=1 Apr 23 08:44:13 slapd[26200]: conn=1015 op=0 meta_back_search: ncandidates=2 cnd="**" Apr 23 08:44:13 slapd[26200]: daemon: activity on 1 descriptor Apr 23 08:44:13 slapd[26200]: daemon: activity on: Apr 23 08:44:13 slapd[26200]: Apr 23 08:44:13 slapd[26200]: daemon: epoll: listen=8 active_threads=0 tvp=zero Apr 23 08:44:13 slapd[26200]: daemon: epoll: listen=9 active_threads=0 tvp=zero Apr 23 08:44:13 slapd[26200]: conn=1015 op=0 meta_back_search[0] match="" err=32 (No such object). Apr 23 08:44:13 slapd[26200]: conn=1015 op=0 meta_back_search[1] match="" err=32 (No such object). Apr 23 08:44:13 slapd[26200]: send_ldap_result: conn=1015 op=0 p=3 Apr 23 08:44:13 slapd[26200]: send_ldap_result: err=32 matched="ou=rsp1,c=de,o=mno" text="" Apr 23 08:44:13 slapd[26200]: conn=1015 op=0 meta_back_bind: no target for dn "uid=admin,ou=rsp1,c=de,o=mno" (32).