Hi,
We are on a process of merging datas from a remote database to a local database. The two databases have the same base dn. To ease this process, I thought for a way to make a union of the remote database and the local database until remote datas are merged to local database. From my reading I found this thread http://thread.gmane.org/gmane.network.openldap.technical/11893 that is something that correspond I think to what I want.
The practical part is done on a debian jessie 8.2 with openldap 2.4.40+dfsg-1+deb8u1 version. The local database definition is like this.
# Entry 1: olcDatabase={2}mdb,cn=config dn: olcDatabase={2}mdb,cn=config objectclass: olcDatabaseConfig objectclass: olcMdbConfig olcaccess: ... olcdatabase: {2}mdb olcdbdirectory: /var/lib/ldap/base_dn olcdbindex: ... olcdbmaxsize: 104857600 olclimits: ... olcrootdn: cn=admin,dc=base,dc=dn olcrootpw: {SSHA}....... olcsuffix: dc=base,dc=dn olcsyncrepl: ... olcupdateref: ldap://master.ldap.server/
To this database definition I have added this part to make translucent work.
# ldapadd -Y EXTERNAL -H ldapi:/// << EOF dn: olcOverlay=translucent,olcDatabase={2}mdb,cn=config objectClass: olcConfig objectClass: olcOverlayConfig objectClass: olcTranslucentConfig objectClass: top olcOverlay: translucent EOF
# ldapadd -Y EXTERNAL -H ldapi:/// << EOF dn: olcDatabase=ldap,olcOverlay={3}translucent,olcDatabase={2}mdb,cn=config objectClass: olcConfig objectClass: olcLDAPConfig objectClass: olcTranslucentDatabase objectClass: olcDatabaseConfig olcDbURI: ldap://remote-ldap.server olcDbIDAssertBind: bindmethod="simple" binddn="cn=binddn,dc=base,dc=dn" credentials="onepassword" mode="self" EOF
With this configuration, I can see on the local server, the entries that are available on the remote server only, an ldapsearch does not return entries available on the local server. Is this the normal behavior ?
Another problem is that when I restart slapd, I have an error like this
slapd[3440]: @(#) $OpenLDAP: slapd (Sep 11 2015 15:11:55) $#012#011buildd@babin:/build/openldap-nFTO9j/openldap-2.4.40+dfsg/debian/build/servers/slapd slapd[3441]: syncprov_db_open: invalid config, lastmod must be enabled slapd[3441]: backend_startup_one (type=mdb, suffix="dc=linkeo,dc=com"): bi_db_open failed! (-1) slapd[3441]: DIGEST-MD5 common mech free slapd[3441]: slapd stopped.
I have to reload config without dn: olcOverlay=translucent,olcDatabase={2}mdb,cn=config and dn: olcDatabase=ldap,olcOverlay={3}translucent,olcDatabase={2}mdb,cn=config entries to make slapd start properly.
Can somebody tell me what I have done wrong ?
Thanks,
I used suffix massage to combine customer LDAP with my local LDAP server; this allows us to have internal users. Documentation on doing this is very sparse.
Client side; sssd points at dc=local.
# BDB database definitions #######################################################################
#local database b database bdb idlcachesize 50000 suffix "dc=b,dc=com" rootdn "cn=adm,dc=b,dc=com" rootpw {SSHA}xx cachesize 50000 dirtyread dbnosync checkpoint 128 15 idlcachesize 50000 index objectClass eq
#database meta - COMBINES the LDAP DATABASES database meta suffix "dc=local" rootdn "cn=adm,dc=local" rootpw {SSHA}xx
#internal LDAP uri "ldap://127.0.0.1/ou=internal,dc=local" lastmod off suffixmassage "ou=internal,dc=local" "dc=b,dc=com"
#external - customer LDAP #uncomment lines and only change vars inside [] to match env # #uri "ldap://[myldap]/ou=external,dc=local" #lastmod off #suffixmassage "ou=external,dc=local" "[dc=a,dc=a,dc=com]" #
*Jason K Cafarelli* Desk: (508) 637-5705 (primary) Mobile: (508) 215-9712 jason.cae@gmail.com
On Fri, Jan 8, 2016 at 5:48 AM, M. P. kisscoolandthegangbang@hotmail.fr wrote:
Hi,
We are on a process of merging datas from a remote database to a local database. The two databases have the same base dn. To ease this process, I thought for a way to make a union of the remote database and the local database until remote datas are merged to local database. From my reading I found this thread http://thread.gmane.org/gmane.network.openldap.technical/11893 that is something that correspond I think to what I want.
The practical part is done on a debian jessie 8.2 with openldap 2.4.40+dfsg-1+deb8u1 version. The local database definition is like this.
# Entry 1: olcDatabase={2}mdb,cn=config dn: olcDatabase={2}mdb,cn=config objectclass: olcDatabaseConfig objectclass: olcMdbConfig olcaccess: ... olcdatabase: {2}mdb olcdbdirectory: /var/lib/ldap/base_dn olcdbindex: ... olcdbmaxsize: 104857600 olclimits: ... olcrootdn: cn=admin,dc=base,dc=dn olcrootpw: {SSHA}....... olcsuffix: dc=base,dc=dn olcsyncrepl: ... olcupdateref: ldap://master.ldap.server/
To this database definition I have added this part to make translucent work.
# ldapadd -Y EXTERNAL -H ldapi:/// << EOF dn: olcOverlay=translucent,olcDatabase={2}mdb,cn=config objectClass: olcConfig objectClass: olcOverlayConfig objectClass: olcTranslucentConfig objectClass: top olcOverlay: translucent EOF
# ldapadd -Y EXTERNAL -H ldapi:/// << EOF dn: olcDatabase=ldap,olcOverlay={3}translucent,olcDatabase={2}mdb,cn=config objectClass: olcConfig objectClass: olcLDAPConfig objectClass: olcTranslucentDatabase objectClass: olcDatabaseConfig olcDbURI: ldap://remote-ldap.server olcDbIDAssertBind: bindmethod="simple" binddn="cn=binddn,dc=base,dc=dn" credentials="onepassword" mode="self" EOF
With this configuration, I can see on the local server, the entries that are available on the remote server only, an ldapsearch does not return entries available on the local server. Is this the normal behavior ?
Another problem is that when I restart slapd, I have an error like this
slapd[3440]: @(#) $OpenLDAP: slapd (Sep 11 2015 15:11:55) $#012#011buildd@babin :/build/openldap-nFTO9j/openldap-2.4.40+dfsg/debian/build/servers/slapd slapd[3441]: syncprov_db_open: invalid config, lastmod must be enabled slapd[3441]: backend_startup_one (type=mdb, suffix="dc=linkeo,dc=com"): bi_db_open failed! (-1) slapd[3441]: DIGEST-MD5 common mech free slapd[3441]: slapd stopped.
I have to reload config without dn: olcOverlay=translucent,olcDatabase={2}mdb,cn=config and dn: olcDatabase=ldap,olcOverlay={3}translucent,olcDatabase={2}mdb,cn=config entries to make slapd start properly.
Can somebody tell me what I have done wrong ?
Thanks,
M. P.
meta backend is one thing I found too when I were looking for some solution and I thought I should also try. I had not yet the time to try it but every where I saw his usage it is to map ou=suffix1 and ou=suffix2 to "a third suffix".
In my case I have my local db with a suffix of dc=base,dc=dn. The remote database has also a suffix of dc=base,dc=dn. And what I want is to have on the unified view a suffix of dc=base,dc=dn too. Something like ou=suffix1,dc=base,dc=dn and ou=suffix2,dc=base,dc=dn is not what I'm looking for.
Maybe you can confirm that it is doable.
Another point in my case is that I have entries that are on both, local and remote databases, with the same dn's. With a search operation, remote entries should be fetched and what is on the local database should append/override remote's entries before being displayed. Have you by chance, be confronted to a case like this with your setup ?
Le 2016-01-08 21:47, jason cafarelli a écrit :
I used suffix massage to combine customer LDAP with my local LDAP server; this allows us to have internal users. Documentation on doing this is very sparse.
Client side; sssd points at dc=local.
# BDB database definitions #######################################################################
#local database b database bdb idlcachesize 50000 suffix "dc=b,dc=com" rootdn "cn=adm,dc=b,dc=com" rootpw {SSHA}xx cachesize 50000 dirtyread dbnosync checkpoint 128 15 idlcachesize 50000 index objectClass eq
#database meta - COMBINES the LDAP DATABASES database meta suffix "dc=local" rootdn "cn=adm,dc=local" rootpw {SSHA}xx
#internal LDAP uri "ldap://127.0.0.1/ou=internal,dc=local [2]" lastmod off suffixmassage "ou=internal,dc=local" "dc=b,dc=com"
#external - customer LDAP #uncomment lines and only change vars inside [] to match env # #uri "ldap://[myldap]/ou=external,dc=local" #lastmod off #suffixmassage "ou=external,dc=local" "[dc=a,dc=a,dc=com]" #
JASON K CAFARELLI Desk: (508) 637-5705 [3] (primary) Mobile: (508) 215-9712 [4] jason.cae@gmail.com
Le 2016-01-08 11:48, M. P. a écrit :
Hi,
We are on a process of merging datas from a remote database to a local database. The two databases have the same base dn. To ease this process, I thought for a way to make a union of the remote database and the local database until remote datas are merged to local database. From my reading I found this thread http://thread.gmane.org/gmane.network.openldap.technical/11893 that is something that correspond I think to what I want.
The practical part is done on a debian jessie 8.2 with openldap 2.4.40+dfsg-1+deb8u1 version. The local database definition is like this.
# Entry 1: olcDatabase={2}mdb,cn=config dn: olcDatabase={2}mdb,cn=config objectclass: olcDatabaseConfig objectclass: olcMdbConfig olcaccess: ... olcdatabase: {2}mdb olcdbdirectory: /var/lib/ldap/base_dn olcdbindex: ... olcdbmaxsize: 104857600 olclimits: ... olcrootdn: cn=admin,dc=base,dc=dn olcrootpw: {SSHA}....... olcsuffix: dc=base,dc=dn olcsyncrepl: ... olcupdateref: ldap://master.ldap.server/
To this database definition I have added this part to make translucent work.
# ldapadd -Y EXTERNAL -H ldapi:/// << EOF dn: olcOverlay=translucent,olcDatabase={2}mdb,cn=config objectClass: olcConfig objectClass: olcOverlayConfig objectClass: olcTranslucentConfig objectClass: top olcOverlay: translucent EOF
# ldapadd -Y EXTERNAL -H ldapi:/// << EOF dn: olcDatabase=ldap,olcOverlay={3}translucent,olcDatabase={2}mdb,cn=config objectClass: olcConfig objectClass: olcLDAPConfig objectClass: olcTranslucentDatabase objectClass: olcDatabaseConfig olcDbURI: ldap://remote-ldap.server olcDbIDAssertBind: bindmethod="simple" binddn="cn=binddn,dc=base,dc=dn" credentials="onepassword" mode="self" EOF
With this configuration, I can see on the local server, the entries that are available on the remote server only, an ldapsearch does not return entries available on the local server. Is this the normal behavior ?
It seems that it is not totally true. Further testing let me say that entries that are present on the local server and not on the remote server, will not be displayed when searching the ldap server. When entries exists in both servers, the remote entries will be appended/overwrited by local entries before being displayed.
I see a workaround to that by creating on the remote server the entries that are missing on remote server but present on the local server but it's certainly not the cleaner possible solution.
It's certainly possible to achieve what I want through a cleaner way.
Another problem is that when I restart slapd, I have an error like this
slapd[3440]: @(#) $OpenLDAP: slapd (Sep 11 2015 15:11:55) $#012#011buildd@babin:/build/openldap-nFTO9j/openldap-2.4.40+dfsg/debian/build/servers/slapd slapd[3441]: syncprov_db_open: invalid config, lastmod must be enabled slapd[3441]: backend_startup_one (type=mdb, suffix="dc=linkeo,dc=com"): bi_db_open failed! (-1) slapd[3441]: DIGEST-MD5 common mech free slapd[3441]: slapd stopped.
I have to reload config without dn: olcOverlay=translucent,olcDatabase={2}mdb,cn=config and dn: olcDatabase=ldap,olcOverlay={3}translucent,olcDatabase={2}mdb,cn=config entries to make slapd start properly.
Can somebody tell me what I have done wrong ?
On a second test machine I don't have this problem. I can stop slapd and start it later without problem. I don't know why, host system, slpad version ,config, all is similar. have to investigate.
Thanks,
M. P. wrote:
Further testing let me say that entries that are present on the local server and not on the remote server, will not be displayed when searching the ldap server. When entries exists in both servers, the remote entries will be appended/overwrited by local entries before being displayed.
So it seems to work as designed and documented (see section DESCRIPTION in slapo-translucent(5)).
Ciao, Michael.
Le 2016-01-11 13:37, Michael Ströder a écrit :
M. P. wrote:
Further testing let me say that entries that are present on the local server and not on the remote server, will not be displayed when searching the ldap server. When entries exists in both servers, the remote entries will be appended/overwrited by local entries before being displayed.
So it seems to work as designed and documented (see section DESCRIPTION in slapo-translucent(5)).
Yep it seems. It was just to add my last results for information.
Do you have any advise for the workaround ? Another way to achieve what I want ?
Ciao, Michael.
Le 2016-01-11 12:52, M. P. a écrit :
Le 2016-01-08 11:48, M. P. a écrit :
Hi,
We are on a process of merging datas from a remote database to a local database. The two databases have the same base dn. To ease this process, I thought for a way to make a union of the remote database and the local database until remote datas are merged to local database. From my reading I found this thread http://thread.gmane.org/gmane.network.openldap.technical/11893 that is something that correspond I think to what I want.
The practical part is done on a debian jessie 8.2 with openldap 2.4.40+dfsg-1+deb8u1 version. The local database definition is like this.
# Entry 1: olcDatabase={2}mdb,cn=config dn: olcDatabase={2}mdb,cn=config objectclass: olcDatabaseConfig objectclass: olcMdbConfig olcaccess: ... olcdatabase: {2}mdb olcdbdirectory: /var/lib/ldap/base_dn olcdbindex: ... olcdbmaxsize: 104857600 olclimits: ... olcrootdn: cn=admin,dc=base,dc=dn olcrootpw: {SSHA}....... olcsuffix: dc=base,dc=dn olcsyncrepl: ... olcupdateref: ldap://master.ldap.server/
To this database definition I have added this part to make translucent work.
# ldapadd -Y EXTERNAL -H ldapi:/// << EOF dn: olcOverlay=translucent,olcDatabase={2}mdb,cn=config objectClass: olcConfig objectClass: olcOverlayConfig objectClass: olcTranslucentConfig objectClass: top olcOverlay: translucent EOF
# ldapadd -Y EXTERNAL -H ldapi:/// << EOF dn: olcDatabase=ldap,olcOverlay={3}translucent,olcDatabase={2}mdb,cn=config objectClass: olcConfig objectClass: olcLDAPConfig objectClass: olcTranslucentDatabase objectClass: olcDatabaseConfig olcDbURI: ldap://remote-ldap.server olcDbIDAssertBind: bindmethod="simple" binddn="cn=binddn,dc=base,dc=dn" credentials="onepassword" mode="self" EOF
With this configuration, I can see on the local server, the entries that are available on the remote server only, an ldapsearch does not return entries available on the local server. Is this the normal behavior ?
It seems that it is not totally true. Further testing let me say that entries that are present on the local server and not on the remote server, will not be displayed when searching the ldap server. When entries exists in both servers, the remote entries will be appended/overwrited by local entries before being displayed.
I see a workaround to that by creating on the remote server the entries that are missing on remote server but present on the local server but it's certainly not the cleaner possible solution.
It's certainly possible to achieve what I want through a cleaner way.
Another problem is that when I restart slapd, I have an error like this
slapd[3440]: @(#) $OpenLDAP: slapd (Sep 11 2015 15:11:55) $#012#011buildd@babin:/build/openldap-nFTO9j/openldap-2.4.40+dfsg/debian/build/servers/slapd slapd[3441]: syncprov_db_open: invalid config, lastmod must be enabled slapd[3441]: backend_startup_one (type=mdb, suffix="dc=linkeo,dc=com"): bi_db_open failed! (-1) slapd[3441]: DIGEST-MD5 common mech free slapd[3441]: slapd stopped.
I have to reload config without dn: olcOverlay=translucent,olcDatabase={2}mdb,cn=config and dn: olcDatabase=ldap,olcOverlay={3}translucent,olcDatabase={2}mdb,cn=config entries to make slapd start properly.
Can somebody tell me what I have done wrong ?
On a second test machine I don't have this problem. I can stop slapd and start it later without problem. I don't know why, host system, slpad version ,config, all is similar. have to investigate.
I have I thing an explaination. It seems syncprov and translucent can't be loaded at the same time when starting slapd. Would it mean that they are not compatible ? What I can tell is that it "works" when syncprov is loaded at startup time and adding translucent at run time, it's just I can't restart slapd, when both modules are in the configuration, for the moment.
Thanks,
openldap-technical@openldap.org
-
jason cafarelli -
M. P. -
Michael Ströder