Greetings all,

I'm trying to figure out why Syncrepl is only syncing part of my provider's database when I use GSSAPI to connect. Both my provider and consumer are on 2.4.40. Here are all the steps I'm taking:

My provider is working fine, I've been using it for months now without any issues. I added this to the provider:

dn: olcOverlay={0}syncprov,olcDatabase={1}mdb,cn=config
objectClass: olcSyncProvConfig
olcOverlay: {0}syncprov
olcSpCheckpoint: 100 10
structuralObjectClass: olcSyncProvConfig
entryUUID: b32ac160-29e6-1036-8d0a-07ef98fd592e
creatorsName: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth
createTimestamp: 20161019012544Z
olcSpSessionlog: 100
entryCSN: 20161024233803.817199Z#000000#000#000000
modifiersName: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth
modifyTimestamp: 20161024233803Z

I also indexed entryCSN and entryUUID on the provider. I have olcAuthzRegexp setup on the provider as well.

olcAuthzRegexp: {0}"uid=admin,cn=harmonywave.com,cn=GSSAPI,cn=auth" "cn=admin,dc=harmonywave,dc=com"
olcAuthzRegexp: {1}"uid=ldap/admin,cn=harmonywave.com,cn=GSSAPI,cn=auth" "dn.exact=gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth"
olcAuthzRegexp: {2}"uid=syncprov,cn=harmonywave.com,cn=GSSAPI,cn=auth" "cn=syncprov,dc=harmonywave,dc=com" #not using this.
olcAuthzRegexp: {3}"uid=.*\/admin,cn=harmonywave.com,cn=GSSAPI,cn=auth" "cn=admin,dc=harmonywave,dc=com"
olcAuthzRegexp: {4}"uid=host\/([^.]*).harmonywave.com,cn=harmonywave.com,cn=GSSAPI,cn=auth" "cn=$1+ipHostNumber=.*,ou=Hosts,dc=harmonywave,dc=com"
olcAuthzRegexp: {5}"uid=([^/]*),cn=harmonywave.com,cn=GSSAPI,cn=auth" "uid=$1,ou=End Users,ou=People,dc=harmonywave,dc=com"

On the consumer I have slapd installed. The first thing I did was change the olcSuffix on my database. I'm not sure if this is required or not.

dn: olcDatabase={1}mdb,cn=config
changetype: modify
replace: olcSuffix
olcSuffix: dc=harmonywave,dc=com
-
replace: olcRootDN
olcRootDN: cn=admin,dc=harmonywave,dc=com

Then I'm adding my ldap keytab for the consumer.

kadmin: ktadd -k /etc/ldap/ldap.keytab ldap/consumer.harmonywave.com
consumer: ~# chown openldap:openldap /etc/ldap/ldap.keytab
consumer: ~# chmod 0640 /etc/ldap/ldap.keytab

I edited my /etc/default/slapd file and pointed the KRB5_KTNAME environment variable to the new keytab then restarted slapd. Next I installed kstart and created a ticket cache.

consumer: ~# k5start -U -f /etc/ldap/ldap.keytab -K 10 -l 24h -k /tmp/krb5cc_108 -o openldap -b

I can see the ldap service's keytab with klist.

consumer: ~# klist /tmp/krb5cc_108

Ticket cache: FILE:/tmp/krb5cc_108
Default principal: ldap/koprulu.harmonywave.com@HARMONYWAVE.COM

Valid starting       Expires              Service principal
10/28/2016 21:18:14  10/29/2016 07:18:14  krbtgt/HARMONYWAVE.COM@HARMONYWAVE.COM
    renew until 10/29/2016 21:18:14

Then I add my olcSaslRealm

dn: cn=config
changetype: modify
add: olcSaslRealm
olcSaslRealm: HARMONYWAVE.COM

Here is what my database looks like right before I add olcSyncrepl:

dn: olcDatabase={1}mdb,cn=config
objectClass: olcDatabaseConfig
objectClass: olcMdbConfig
olcDatabase: {1}mdb
olcDbDirectory: /var/lib/ldap
olcAccess: {0}to attrs=userPassword,shadowLastChange by self write by anonym
 ous auth by * none
olcAccess: {1}to dn.base="" by * read
olcAccess: {2}to * by * read
olcLastMod: TRUE
olcRootPW:: ...
olcDbCheckpoint: 512 30
olcDbMaxSize: 1073741824
structuralObjectClass: olcMdbConfig
entryUUID: 9a091324-2e84-1036-8b7a-73db8891632a
creatorsName: cn=admin,cn=config
createTimestamp: 20161024222607Z
olcSuffix: dc=harmonywave,dc=com
olcRootDN: cn=admin,dc=harmonywave,dc=com
olcDbIndex: cn,uid eq
olcDbIndex: entryCSN eq
olcDbIndex: entryUUID eq
olcDbIndex: member,memberUid eq
olcDbIndex: objectClass eq
olcDbIndex: uidNumber,gidNumber eq
entryCSN: 20161029033105.691204Z#000000#000#000000
modifiersName: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth
modifyTimestamp: 20161029033105Z

then I add olcSyncrepl to the consumer.

dn: olcDatabase={1}mdb,cn=config
changetype: modify
add: olcSyncrepl
olcSyncrepl: {0}rid=000
  provider=ldap://provider.harmonywave.com
  type=RefreshAndPersist
  retry="30 10 1800 +"
  searchbase="dc=harmonywave,dc=com"
  bindmethod=sasl
  saslmech=GSSAPI
  starttls=critical
  tls_cacert=/etc/ssl/certs/ca.harmonywave.com.pem
  tls_reqcert=demand


After that I slapcat on the consumer and I only see about 1/3 of my data from the provider. When I watch the log on the provider this is what I get:

Oct 28 21:39:02 baneling slapd[12540]: conn=4421 fd=36 ACCEPT from IP=10.1.30.19:55992 (IP=0.0.0.0:389)
Oct 28 21:39:02 baneling slapd[12540]: conn=4421 op=0 EXT oid=1.3.6.1.4.1.1466.20037
Oct 28 21:39:02 baneling slapd[12540]: conn=4421 op=0 STARTTLS
Oct 28 21:39:02 baneling slapd[12540]: conn=4421 op=0 RESULT oid= err=0 text=
Oct 28 21:39:02 baneling slapd[12540]: conn=4421 fd=36 TLS established tls_ssf=128 ssf=128
Oct 28 21:39:02 baneling slapd[12540]: conn=1005 op=43768 SRCH base="dc=harmonywave,dc=com" scope=2 deref=0 filter="(&(|(objectClass=krbPrincipalAux)(objectClass=krbPrincipal))(krbPrincipalName=krbtgt/HARMONYWAVE.COM@HARMONYWAVE.COM))"
Oct 28 21:39:02 baneling slapd[12540]: conn=1005 op=43768 SRCH attr=krbprincipalname krbcanonicalname objectclass krbprincipalkey krbmaxrenewableage krbmaxticketlife krbticketflags krbprincipalexpiration krbticketpolicyreference krbUpEnabled krbpwdpolicyreference krbpasswordexpiration krbLastFailedAuth krbLoginFailedCount krbLastSuccessfulAuth krbLastPwdChange krbLastAdminUnlock krbExtraData krbObjectReferences krbAllowedToDelegateTo
Oct 28 21:39:02 baneling slapd[12540]: conn=1005 op=43768 SEARCH RESULT tag=101 err=0 nentries=1 text=
Oct 28 21:39:02 baneling slapd[12540]: conn=1005 op=43769 SRCH base="dc=harmonywave,dc=com" scope=2 deref=0 filter="(&(|(objectClass=krbPrincipalAux)(objectClass=krbPrincipal))(krbPrincipalName=ldap/baneling.harmonywave.com@HARMONYWAVE.COM))"
Oct 28 21:39:02 baneling slapd[12540]: conn=1005 op=43769 SRCH attr=krbprincipalname krbcanonicalname objectclass krbprincipalkey krbmaxrenewableage krbmaxticketlife krbticketflags krbprincipalexpiration krbticketpolicyreference krbUpEnabled krbpwdpolicyreference krbpasswordexpiration krbLastFailedAuth krbLoginFailedCount krbLastSuccessfulAuth krbLastPwdChange krbLastAdminUnlock krbExtraData krbObjectReferences krbAllowedToDelegateTo
Oct 28 21:39:02 baneling slapd[12540]: conn=1005 op=43769 SEARCH RESULT tag=101 err=0 nentries=1 text=
Oct 28 21:39:02 baneling slapd[12540]: conn=1005 op=43770 SRCH base="dc=harmonywave,dc=com" scope=2 deref=0 filter="(&(|(objectClass=krbPrincipalAux)(objectClass=krbPrincipal))(krbPrincipalName=ldap/koprulu.harmonywave.com@HARMONYWAVE.COM))"
Oct 28 21:39:02 baneling slapd[12540]: conn=1005 op=43770 SRCH attr=krbprincipalname krbcanonicalname objectclass krbprincipalkey krbmaxrenewableage krbmaxticketlife krbticketflags krbprincipalexpiration krbticketpolicyreference krbUpEnabled krbpwdpolicyreference krbpasswordexpiration krbLastFailedAuth krbLoginFailedCount krbLastSuccessfulAuth krbLastPwdChange krbLastAdminUnlock krbExtraData krbObjectReferences krbAllowedToDelegateTo
Oct 28 21:39:02 baneling slapd[12540]: conn=1005 op=43770 SEARCH RESULT tag=101 err=0 nentries=1 text=
Oct 28 21:39:02 baneling slapd[12540]: conn=4421 op=1 BIND dn="" method=163
Oct 28 21:39:02 baneling slapd[12540]: conn=4421 op=1 RESULT tag=97 err=14 text=SASL(0): successful result:
Oct 28 21:39:02 baneling slapd[12540]: conn=4421 op=2 BIND dn="" method=163
Oct 28 21:39:02 baneling slapd[12540]: conn=4421 op=2 RESULT tag=97 err=14 text=SASL(0): successful result:
Oct 28 21:39:02 baneling slapd[12540]: conn=4421 op=3 BIND dn="" method=163
Oct 28 21:39:02 baneling slapd[12540]: conn=4421 op=3 BIND authcid="ldap/koprulu.harmonywave.com@HARMONYWAVE.COM" authzid="ldap/koprulu.harmonywave.com@HARMONYWAVE.COM"
Oct 28 21:39:02 baneling slapd[12540]: conn=4421 op=3 BIND dn="uid=ldap/koprulu.harmonywave.com,cn=harmonywave.com,cn=gssapi,cn=auth" mech=GSSAPI sasl_ssf=56 ssf=128
Oct 28 21:39:02 baneling slapd[12540]: conn=4421 op=3 RESULT tag=97 err=0 text=
Oct 28 21:39:02 baneling slapd[12540]: conn=4421 op=4 SRCH base="dc=harmonywave,dc=com" scope=2 deref=0 filter="(objectClass=*)"
Oct 28 21:39:02 baneling slapd[12540]: conn=4421 op=4 SRCH attr=* +
Oct 28 21:39:02 baneling slapd[12540]: conn=4421 op=5 UNBIND
Oct 28 21:39:02 baneling slapd[12540]: conn=4421 fd=36 closed

The only thing I really notice from this is near the end of the file. It when it searches the base with attributes "*+", but then immediately unbinds. I've seen people stating that authzid is required, but when I don't provide it I still get a partial sync, so I'm not sure about this. I've restored my consumer to a clean install of slapd and repeated the above steps with minor variations several times but the consumer always syncs the exact same amount of data and then seems to stop.

Any help to point me in the right direction would be appreciated.

Thanks,
Joshua Schaeffer