openldap-technical

openldap-technical@openldap.org

13 participants
9906 discussions

slapd core dumps on ldap add
by Celalettin KILINÇ 05 Aug '14

05 Aug '14

Hi, I have openldap 2.4.36 installed using mdb backend on Solaris 10. I am using this database for a while and suddenly I started getting core dumps while trying to add any data to database. Disk, memory usages seem fine. Database size is not exceeded, it is almost 1/10 full. What may be the reason for this? Please find the other information about the issue as below: slapd version: # ./slapd -VVV @(#) $OpenLDAP: slapd 2.4.36 (Oct 6 2013 13:56:57) $ myuser@myhost:/some/dir/openldap-2.4.36/servers/slapd Included static overlays: syncprov Included static backends: config ldif mdb slapd.conf for mdb backend: database mdb directory /some/dir/openldap/var/openldap-data suffix "dc=mycomp,dc=com" rootdn "cn=Manager,dc=mycomp,dc=com" rootpw password index objectClass eq maxsize 45899345920 dbnosync checkpoint 0 10 threads 32 listener-threads 16 concurrency 1024 envflags writemap,nometasync slapd debug log: 53e15b8d conn=1001 op=17 do_add ber_scanf fmt ({m) ber: 53e15b8d conn=1001 op=17 do_add: dn (BatchGroupName=test,uid=myuser,OperationType=create,o=cust,dc=mycomp,dc=com) ber_scanf fmt ({m{W}}) ber: ber_scanf fmt ({m{W}}) ber: ber_scanf fmt (}) ber: 53e15b8d >>> dnPrettyNormal: <BatchGroupName=test,uid=myuser,OperationType=create,o=cust,dc=mycomp,dc=com> => ldap_bv2dn(BatchGroupName=test,uid=myuser,OperationType=create,o=cust,dc=mycomp,dc=com,0) <= ldap_bv2dn(BatchGroupName=test,uid=myuser,OperationType=create,o=cust,dc=mycomp,dc=com)=0 => ldap_dn2bv(272) <= ldap_dn2bv(BatchGroupName=test,uid=myuser,OperationType=create,o=cust,dc=mycomp,dc=com)=0 => ldap_dn2bv(272) <= ldap_dn2bv(BatchGroupName=test,uid=myuser,OperationType=create,o=cust,dc=mycomp,dc=com)=0 53e15b8d <<< dnPrettyNormal: <BatchGroupName=test,uid=myuser,OperationType=create,o=cust,dc=mycomp,dc=com>, <BatchGroupName=test,uid=myuser,OperationType=create,o=cust,dc=mycomp,dc=com> 53e15b8d ==> mdb_add: BatchGroupName=test,uid=myuser,OperationType=create,o=cust,dc=mycomp,dc=com 53e15b8d oc_check_required entry (BatchGroupName=test,uid=myuser,OperationType=create,o=cust,dc=mycomp,dc=com), objectClass "RTEBatchGroup" 53e15b8d oc_check_allowed type "BatchGroupName" 53e15b8d oc_check_allowed type "objectClass" 53e15b8d oc_check_allowed type "structuralObjectClass" 53e15b8d mdb_dn2entry("BatchGroupName=test,uid=myuser,OperationType=create,o=cust,dc=mycomp,dc=com") 53e15b8d => mdb_dn2id("BatchGroupName=test,uid=myuser,OperationType=create,o=cust,dc=mycomp,dc=com") 53e15b8d <= mdb_dn2id: get failed: MDB_NOTFOUND: No matching key/data pair found (-30798) 53e15b8d => mdb_entry_decode: 53e15b8d <= mdb_entry_decode 53e15b8d => mdb_dn2id_add 0x1a28eb: "BatchGroupName=test,uid=myuser,OperationType=create,o=cust,dc=mycomp,dc=com" zsh: bus error (core dumped) ./slapd -d5 gdb backtrace of the generated core file: # gdb slapd core GNU gdb 6.8 Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "sparc-sun-solaris2.10"... (no debugging symbols found) Reading symbols from /some/dir/lib/libldap_r-2.4.so.2...(no debugging symbols found)...done. Loaded symbols for /some/dir/lib/libldap_r-2.4.so.2 Reading symbols from /some/dir/lib/liblber-2.4.so.2...(no debugging symbols found)...done. Loaded symbols for /some/dir/lib/liblber-2.4.so.2 Reading symbols from /usr/lib/sparcv9/libicuuc.so.3...(no debugging symbols found)...done. Loaded symbols for /usr/lib/64/libicuuc.so.3 Reading symbols from /usr/lib/sparcv9/libicudata.so.3...(no debugging symbols found)...done. Loaded symbols for /usr/lib/64/libicudata.so.3 Reading symbols from /some/dir/lib/libsasl2.so.2... (no debugging symbols found)...done. Loaded symbols for /some/dir/lib/libsasl2.so.2 Reading symbols from /lib/sparcv9/libdl.so.1... warning: Lowest section in /lib/sparcv9/libdl.so.1 is .hash at 0000000000000120 (no debugging symbols found)...done. Loaded symbols for /lib/64/libdl.so.1 Reading symbols from /opt/csw/lib/sparcv9/libssl.so.0.9.8...(no debugging symbols found)...done. Loaded symbols for /opt/csw/lib/sparcv9/libssl.so.0.9.8 Reading symbols from /opt/csw/lib/sparcv9/libcrypto.so.0.9.8...(no debugging symbols found)...done. Loaded symbols for /opt/csw/lib/sparcv9/libcrypto.so.0.9.8 Reading symbols from /lib/sparcv9/libresolv.so.2... (no debugging symbols found)...done. Loaded symbols for /lib/64/libresolv.so.2 Reading symbols from /lib/sparcv9/libgen.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/64/libgen.so.1 Reading symbols from /lib/sparcv9/libnsl.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/64/libnsl.so.1 Reading symbols from /lib/sparcv9/libsocket.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/64/libsocket.so.1 Reading symbols from /lib/sparcv9/librt.so.1... (no debugging symbols found)...done. Loaded symbols for /lib/64/librt.so.1 Reading symbols from /lib/sparcv9/libc.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/64/libc.so.1 Reading symbols from /opt/csw/gcc4/lib/sparcv9/libgcc_s.so.1...(no debugging symbols found)...done. Loaded symbols for /opt/csw/gcc4/lib/sparcv9/libgcc_s.so.1 Reading symbols from /lib/sparcv9/libpthread.so.1... warning: Lowest section in /lib/sparcv9/libpthread.so.1 is .dynamic at 00000000000000b0 (no debugging symbols found)...done. Loaded symbols for /lib/64/libpthread.so.1 Reading symbols from /lib/sparcv9/libm.so.2... (no debugging symbols found)...done. Loaded symbols for /lib/64/libm.so.2 Reading symbols from /usr/lib/sparcv9/libCrun.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/lib/64/libCrun.so.1 Reading symbols from /lib/sparcv9/libaio.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/64/libaio.so.1 Reading symbols from /lib/sparcv9/libmd.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/64/libmd.so.1 Reading symbols from /platform/sun4v/lib/sparcv9/libc_psr.so.1... (no debugging symbols found)...done. Loaded symbols for /platform/SUNW,Netra-T2000/lib/sparcv9/libc_psr.so.1 Reading symbols from /lib/sparcv9/ld.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/sparcv9/ld.so.1 Core was generated by `./slapd -d5'. Program terminated with signal 10, Bus error. [New process 1335262 ] [New process 90078 ] [New process 155614 ] [New process 221150 ] [New process 286686 ] [New process 352222 ] [New process 417758 ] [New process 483294 ] [New process 548830 ] [New process 614366 ] [New process 679902 ] [New process 745438 ] [New process 810974 ] [New process 876510 ] [New process 942046 ] [New process 1007582 ] [New process 1073118 ] [New process 1138654 ] [New process 1204190 ] [New process 1269726 ] #0 0x0000000100159c3c in mdb_page_search_root () (gdb) bt #0 0x0000000100159c3c in mdb_page_search_root () #1 0x000000010015a2d0 in mdb_page_search () #2 0x000000010015c984 in mdb_cursor_first () #3 0x000000010015d93c in mdb_cursor_get () #4 0x000000010015204c in mdb_page_alloc () #5 0x0000000100152b6c in mdb_page_touch () #6 0x000000010015a264 in mdb_page_search () #7 0x000000010015dba0 in mdb_cursor_touch () #8 0x000000010015e314 in mdb_cursor_put () #9 0x000000010018f25c in mdb_dn2id_add () #10 0x0000000100184e94 in mdb_add () #11 0x0000000100060970 in fe_op_add () #12 0x00000001000601fc in do_add () #13 0x000000010005128c in connection_operation () #14 0x0000000100051a4c in connection_read_thread () #15 0xffffffff7f4164bc in ldap_int_thread_pool_wrapper () from /some/dir/lib/libldap_r-2.4.so.2 #16 0xffffffff7ced7374 in _lwp_start () from /lib/64/libc.so.1 #17 0xffffffff7ced7374 in _lwp_start () from /lib/64/libc.so.1 Backtrace stopped: previous frame identical to this frame (corrupt stack?) Thanks

2 1

RE: ACL doesn't work on my Centos 6
by Dong Xi 05 Aug '14

05 Aug '14

Hi Quanah, I am using RHEL build, so in this case, slaps.conf. Can you please tell me how to apply the ACL in my case (disable anonymous access to any except rootdn, and allow self and bind user write to userPassword)? I am new to openldap, any help is really appreciated. dxi -----Original Message----- From: "Quanah Gibson-Mount" <quanah(a)zimbra.com> Sent: ‎2014-‎08-‎04 12:31 To: "Dong Xi" <dxi188(a)gmail.com>; "openldap-technical(a)openldap.org" <openldap-technical(a)openldap.org> Subject: Re: ACL doesn't work on my Centos 6 --On Saturday, August 02, 2014 10:39 PM -0400 Dong Xi <dxi188(a)gmail.com> wrote: > > > > Hi, > > Just set up openldap on my CentOS 6.5. Are you using the RHEL build of OpenLDAP or your own? If you are using the RHEL build, be aware that it uses cn=config, not slapd.conf, so making changes to slapd.conf will not have any effect. --Quanah -- Quanah Gibson-Mount Server Architect Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration

1 1

Need help in undertsanding server certification verification by the client
by SOMA SEKHAR 05 Aug '14

05 Aug '14

Hi , I have my ca cert in a dir and I am setting the CACERTDIR option in openldap global options. *ldap_set_option(NULL, LDAP_OPT_X_TLS_CACERTDIR, <my dir path>)*. After that , I started the tls connection using '*ldap_start_tls_s*' , followed by '*ldap_bind_s*'. This worked fine. What I did not understand is that , even after removing the ca cert from that directory , ldap bind succeeds. Does it mean that certificate verification is not done for the second time by SSL_connect? I have just started on openldap and gone through the code in version 2.4 and openssl-fips-1.2 , searched in google,stackoverflow etc . Can anyone please help me with some information or pointers on this. -- Thanks&Regards, SomaSekhar.

1 0

Re: Will lmdb suit my needs ?
by Florian Weimer 04 Aug '14

04 Aug '14

* Sankar P.: > SELECT DISTINCT column2 FROM autocomplete WHERE column1 LIKE > '<prefix>%' ORDER BY column3 DESC LIMIT 5 > > where column1 and column2 are strings and column3 is int. > > So, what my query essentially does is get a prefix string from user > and get the top (sorted by column3) 5 records where column1 start with > the input prefix and return the column2 of those records. > > In my sqlite, the first time query for each key (a or b or c ...) is > taking about 350 seconds (with an index on all columns) and about 500 > seconds (with an index on only column1). Do you use PRAGMA case_sensitive_like? How many candidate does the "column1 LIKE '<prefix>%'" expression select?

1 0

ACL doesn't work on my Centos 6
by Dong Xi 04 Aug '14

04 Aug '14

Hi, Just set up openldap on my CentOS 6.5. Looks like very easy to set up. And it is working. However, I found ACL doesn't work as expected. I am trying to disable read access to anonymous, and give userPassword write access to self and binduser. But it doesn't work: anonymous can still read whole ldap entries, and no one except cn=Manager,dc=domain,dc=com can change userPassword. I even change following section to access to * by * none to lock down ldap, but still anonymous can read all. === access to dn.base="" by * read access to dn.base="cn=Subschema" by * read access to attrs=userPassword by self write by dn.exact="cn=binduser,dc=domain,dc=com" write by users read by anonymous auth access to * by users read by anonymous auth === to === access to * by * none === Here is the complete slapd.conf. Look like only default ACL works, whatever ACL policy I am trying to apply. Please help. slapd.conf ==== # # See slapd.conf(5) for details on configuration options. # This file should NOT be world readable. # include /etc/openldap/schema/corba.schema include /etc/openldap/schema/core.schema include /etc/openldap/schema/cosine.schema include /etc/openldap/schema/duaconf.schema include /etc/openldap/schema/dyngroup.schema include /etc/openldap/schema/inetorgperson.schema include /etc/openldap/schema/java.schema include /etc/openldap/schema/misc.schema include /etc/openldap/schema/nis.schema include /etc/openldap/schema/openldap.schema include /etc/openldap/schema/ppolicy.schema include /etc/openldap/schema/collective.schema # Allow LDAPv2 client connections. This is NOT the default. allow bind_v2 # Do not enable referrals until AFTER you have a working directory # service AND an understanding of referrals. #referral ldap://root.openldap.org pidfile /var/run/openldap/slapd.pid argsfile /var/run/openldap/slapd.args # Load dynamic backend modules # - modulepath is architecture dependent value (32/64-bit system) # - back_sql.la overlay requires openldap-server-sql package # - dyngroup.la and dynlist.la cannot be used at the same time # modulepath /usr/lib/openldap # modulepath /usr/lib64/openldap # moduleload accesslog.la # moduleload auditlog.la # moduleload back_sql.la # moduleload chain.la # moduleload collect.la # moduleload constraint.la # moduleload dds.la # moduleload deref.la # moduleload dyngroup.la # moduleload dynlist.la # moduleload memberof.la # moduleload pbind.la # moduleload pcache.la # moduleload ppolicy.la # moduleload refint.la # moduleload retcode.la # moduleload rwm.la # moduleload seqmod.la # moduleload smbk5pwd.la # moduleload sssvlv.la # moduleload syncprov.la # moduleload translucent.la # moduleload unique.la # moduleload valsort.la # The next three lines allow use of TLS for encrypting connections using a # dummy test certificate which you can generate by running # /usr/libexec/openldap/generate-server-cert.sh. Your client software may balk # at self-signed certificates, however. #TLSCACertificatePath /etc/openldap/certs #TLSCertificateFile "\"OpenLDAP Server\"" #TLSCertificateKeyFile /etc/openldap/certs/password # Sample security restrictions # Require integrity protection (prevent hijacking) # Require 112-bit (3DES or better) encryption for updates # Require 63-bit encryption for simple bind # security ssf=1 update_ssf=112 simple_bind=64 # Sample access control policy: # Root DSE: allow anyone to read it # Subschema (sub)entry DSE: allow anyone to read it # Other DSEs: # Allow self write access # Allow authenticated users read access # Allow anonymous users to authenticate # Directives needed to implement policy: #access to dn.base="" by * read #access to dn.base="cn=Subschema" by * read #access to * # by self write # by users read # by anonymous auth access to dn.base="" by * read access to dn.base="cn=Subschema" by * read access to attrs=userPassword by self write by dn.exact="cn=binduser,dc=domain,dc=com" write by users read by anonymous auth access to * by users read by anonymous auth # # if no access controls are present, the default policy # allows anyone and everyone to read anything but restricts # updates to rootdn. (e.g., "access to * by * read") # # rootdn can always read and write EVERYTHING! # enable on-the-fly configuration (cn=config) database config access to * by * none # enable server status monitoring (cn=monitor) database monitor access to * by * none ####################################################################### # database definitions ####################################################################### database bdb suffix "dc=domain,dc=com" checkpoint 1024 15 rootdn "cn=Manager,dc=domain,dc=com" rootpw {SSHA}f3F8TAj9BihNSUtVdMyqrvlcIoawHDgb loglevel 256 sizelimit unlimited # Cleartext passwords, especially for the rootdn, should # be avoided. See slappasswd(8) and slapd.conf(5) for details. # Use of strong authentication encouraged. # rootpw secret #rootpw {SSHA}f3F8TAj9BihNSUtVdMyqrvlcIoawHDgb # The database directory MUST exist prior to running slapd AND # should only be accessible by the slapd and slap tools. # Mode 700 recommended. directory /var/lib/ldap # Indices to maintain for this database index objectClass eq,pres index ou,cn,mail,surname,givenname eq,pres,sub index uidNumber,gidNumber,loginShell eq,pres index uid,memberUid eq,pres,sub index nisMapName,nisMapEntry eq,pres,sub # Replicas of this database #replogfile /var/lib/ldap/openldap-master-replog #replica host=ldap-1.example.com:389 starttls=critical # bindmethod=sasl saslmech=GSSAPI # authcId=host/ldap-master.example.com(a)EXAMPLE.COM

2 1

Groups in ACLs on MDB
by Johannes Löthberg 04 Aug '14

04 Aug '14

Hello, I've been trying to get using groups working in ACLs, but no matter what I do the group ACL isn't applied. It seems it might be a LMDB bug, and I'm planning on switching to hdb to see if it works there when I get the time. I've attached the olcAccess.ldif that doesn't work and the output of slapacl -D uid=kyrias,ou=users,dc=kyriasis,dc=com \ -b ou=users,dc=kyriasis,dc=com -dacl which shows that the group ACL isn't applied to the user uid=kyrias,ou=users,dc=kyriasis,dc=com even tho it is a member of the cn=admins,ou=security,dc=kyriasis,dc=com group and that the 'to *' ACL is above the other ones. -- Sincerely, Johannes Löthberg PGP Key ID: 3A9D0BB5

1 0

Password Failure for one user
by brown wrap 01 Aug '14

01 Aug '14

I am trying to track down why one user all of a sudden can't log into a machine. He can use his LDAP passwd through our network, but gets an immediate login failure on a particular machine. I can 'su' to his account, from my account using his password, but when I try to log into this machine remotely, I get the failure and it terminates the sessions. There have been no changes to the system at all. I am looking for step to troubleshoot this issue. Thanks.

2 2

Q: cn=Connection 0,cn=Connections,cn=Monitor
by Ulrich Windl 01 Aug '14

01 Aug '14

Hi! I'm browsing the monitor database for routine, and I founf this "Connectio 0": # Connection 0, Connections, Monitor dn: cn=Connection 0,cn=Connections,cn=Monitor monitorConnectionNumber: 0 monitorConnectionProtocol: 0 monitorConnectionOpsReceived: 4 monitorConnectionOpsExecuting: 0 monitorConnectionOpsPending: 0 monitorConnectionOpsCompleted: 4 monitorConnectionGet: 6 monitorConnectionRead: 6 monitorConnectionWrite: 0 monitorConnectionMask: L monitorConnectionListener: monitorConnectionPeerDomain: unknown monitorConnectionPeerAddress: unknown monitorConnectionLocalAddress: monitorConnectionStartTime: 19700101000000Z monitorConnectionActivityTime: 19700101000000Z Is it a bug that Start and Activity time are both unset? If the reads and writes of the connection can be counted, the timestamps could be updated as well, I guess. It's 2.4.26 here, so maybe it has been fixed meanwhile... I just saw that I have a second "connection 0": # Connection 0, Connections, Monitor dn: cn=Connection 0,cn=Connections,cn=Monitor monitorConnectionNumber: 0 monitorConnectionProtocol: 0 monitorConnectionOpsReceived: 4 monitorConnectionOpsExecuting: 0 monitorConnectionOpsPending: 0 monitorConnectionOpsCompleted: 4 monitorConnectionGet: 6 monitorConnectionRead: 6 monitorConnectionWrite: 0 monitorConnectionMask: L monitorConnectionListener: monitorConnectionPeerDomain: unknown monitorConnectionPeerAddress: unknown monitorConnectionLocalAddress: monitorConnectionStartTime: 19700101000000Z monitorConnectionActivityTime: 19700101000000Z Doing a simple quick-and-dirty test, I found out that I have multiple "Connection 0"s: sh monitor2.sh |grep '^# Connection' |sort -n SASL/EXTERNAL authentication started SASL username: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth SASL SSF: 0 # Connection 0, Connections, Monitor # Connection 0, Connections, Monitor # Connection 0, Connections, Monitor # Connection 0, Connections, Monitor # Connection 0, Connections, Monitor # Connection 0, Connections, Monitor # Connection 0, Connections, Monitor # Connection 0, Connections, Monitor # Connection 0, Connections, Monitor # Connection 1000, Connections, Monitor # Connection 1001, Connections, Monitor # Connection 1008, Connections, Monitor # Connection 1013, Connections, Monitor # Connection 1016, Connections, Monitor # Connection 1017, Connections, Monitor # Connection 1018, Connections, Monitor # Connection 1020, Connections, Monitor # Connection 1021, Connections, Monitor # Connection 1024, Connections, Monitor # Connection 1036, Connections, Monitor # Connection 1040, Connections, Monitor # Connection 1042, Connections, Monitor # Connection 1043, Connections, Monitor # Connection 1118, Connections, Monitor # Connection 1201, Connections, Monitor # Connection 1247, Connections, Monitor # Connection 1380, Connections, Monitor # Connection 14755, Connections, Monitor # Connection 14758, Connections, Monitor # Connection 15369, Connections, Monitor # Connection 15800, Connections, Monitor # Connection 15886, Connections, Monitor # Connection 16674, Connections, Monitor # Connection 20526, Connections, Monitor # Connection 20626, Connections, Monitor # Connection 21044, Connections, Monitor # Connection 21460, Connections, Monitor # Connection 21474, Connections, Monitor # Connection 21688, Connections, Monitor # Connection 25846, Connections, Monitor # Connection 30396, Connections, Monitor # Connection 30665, Connections, Monitor # Connection 31948, Connections, Monitor # Connection 35407, Connections, Monitor # Connection 35953, Connections, Monitor # Connection 36575, Connections, Monitor # Connection 36576, Connections, Monitor # Connection 36577, Connections, Monitor # Connection 36648, Connections, Monitor # Connection 50382, Connections, Monitor # Connection 50599, Connections, Monitor # Connection 50706, Connections, Monitor # Connection 51477, Connections, Monitor # Connection 55650, Connections, Monitor # Connection 55651, Connections, Monitor # Connection 56206, Connections, Monitor # Connection 61141, Connections, Monitor # Connection 61144, Connections, Monitor # Connection 61152, Connections, Monitor # Connection 61153, Connections, Monitor # Connection 61162, Connections, Monitor # Connection 61163, Connections, Monitor # Connection 61164, Connections, Monitor # Connection 61165, Connections, Monitor # Connection 61166, Connections, Monitor # Connection 61167, Connections, Monitor # Connection 61168, Connections, Monitor # Connection 62161, Connections, Monitor # Connection 65286, Connections, Monitor # Connection 67224, Connections, Monitor # Connection 70109, Connections, Monitor # Connection 70162, Connections, Monitor # Connection 70516, Connections, Monitor # Connection 70528, Connections, Monitor # Connection 70529, Connections, Monitor # Connections, Monitor Regards, Ulrich

2 1

Antw: Setting up Monitor Backend
by Ulrich Windl 31 Jul '14

31 Jul '14

>>> Leonard Smith <lrsmith(a)umich.edu> schrieb am 31.07.2014 um 21:52 in Nachricht <CAP9L-w5pR5z80_aTgt_ux2xUpH4gu-kA_Y5zd2VZuNfE9kgfQg(a)mail.gmail.com>: > To all, > > I'm trying to setup the monitoring backend so I can get metrics in OpenLDAP > 2.4.23 on Centos 6. I have it setup and I can query but I don't seem to be > getting any metrics/values. I check the olcAccess rules and it should be > allowing access so I"m not sure why I cannot see any values for any of the > monitoring points. > > > sh-4.1# ldapsearch -Y EXTERNAL -H ldapi:/// -b > 'cn=Total,cn=Connections,cn=Monitor' Hi! Try " ldapsearch -Y EXTERNAL -H ldapi:/// -b 'cn=Total,cn=Connections,cn=monitor' -s sub '(objectClass=*)' monitorCounter". I got: # Total, Connections, Monitor dn: cn=Total,cn=Connections,cn=Monitor monitorCounter: 70315 Regards, Ulrich > SASL/EXTERNAL authentication started > SASL username: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth > SASL SSF: 0 > # extended LDIF > # > # LDAPv3 > # base <cn=Total,cn=Connections,cn=Monitor> with scope subtree > # filter: (objectclass=*) > # requesting: ALL > # > > # Total, Connections, Monitor > dn: cn=Total,cn=Connections,cn=Monitor > objectClass: monitorCounterObject > cn: Total > > # search result > search: 2 > result: 0 Success > > # numResponses: 2 > # numEntries: 1 > > dn: olcDatabase={1}monitor > objectClass: olcDatabaseConfig > olcDatabase: {1}monitor > olcAddContentAcl: FALSE > olcLastMod: TRUE > olcMaxDerefDepth: 15 > olcReadOnly: FALSE > olcSyncUseSubentry: FALSE > olcMonitoring: FALSE > structuralObjectClass: olcDatabaseConfig > entryUUID: 4313fd2c-b0ec-1032-8c14-c907d74f027d > creatorsName: cn=config > createTimestamp: 20130913181549Z > olcAccess: {0}to * by > dn.base="gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth" read by > dn.base="cn=manager > ,dc=foo,dc=com" read by dn.base="cn=monitor,dc=foo,dc=com" read by * none

1 0

syncrepl_del_nonpresent leading to directory implosion
by Aaron Bennett 10 Jul '14

10 Jul '14

Hi, I've got a two-node MMR setup using syncrepl running 2.4.36 with back-bdb. Last night, between 1/3 and 1/2 of the entries in our directory disappeared. Fortunately I have olcLogLeve=sync set, so I think I have a line on what happened... I see this at 4AM long of the morning before the fun started: Jul 7 04:19:30 zoot slapd[15011]: bdb(dc=clarku,dc=edu): Lock table is out of available lock entries Jul 7 04:19:30 zoot slapd[15011]: => bdb_idl_insert_key: c_get failed: Cannot allocate memory (12) Then many hours later: Jul 7 15:04:30 zoot slapd[15011]: bdb(dc=clarku,dc=edu): Lock table is out of available lock entries Jul 7 15:04:30 zoot slapd[15011]: => bdb_idl_delete_key: c_get failed: Cannot allocate memory (12) Jul 7 15:04:30 zoot slapd[15011]: null_callback : error code 0x50 Jul 7 15:04:30 zoot slapd[15011]: slap_graduate_commit_csn: removing 0x7fa33c248a80 20140707190424.188580Z#000000#001#000000 Jul 7 15:04:30 zoot slapd[15011]: syncrepl_del_nonpresent: rid=001 be_delete cn=parents_ug_students,ou=Other,ou=Groups,dc=clarku,dc=edu (80) This repeats itself several times: Jul 7 16:41:09 zoot slapd[15011]: syncprov_matchops: skipping original sid 001 Jul 7 16:41:09 zoot slapd[15011]: bdb(dc=clarku,dc=edu): Lock table is out of available lock entries Jul 7 16:41:09 zoot slapd[15011]: => bdb_idl_delete_key: c_get failed: Cannot allocate memory (12) Jul 7 16:41:09 zoot slapd[15011]: null_callback : error code 0x50 Jul 7 16:41:09 zoot slapd[15011]: slap_graduate_commit_csn: removing 0x7fa37e4b4260 20140707204059.442906Z#000000#001#000000 Jul 7 16:41:09 zoot slapd[15011]: syncrepl_del_nonpresent: rid=001 be_delete cn=parents_ug_students,ou=Other,ou=Groups,dc=clarku,dc=edu (80) Jul 7 16:52:44 zoot slapd[2179]: bdb(dc=clarku,dc=edu): Lock table is out of available lock entries Jul 7 16:52:44 zoot slapd[2179]: => bdb_idl_delete_key: c_get failed: Cannot allocate memory (12) Jul 7 16:52:44 zoot slapd[2179]: null_callback : error code 0x50 Jul 7 16:52:44 zoot slapd[2179]: slap_graduate_commit_csn: removing 0x7f74581dae70 20140707205225.029175Z#000000#001#000000 Jul 7 16:52:44 zoot slapd[2179]: syncrepl_del_nonpresent: rid=001 be_delete cn=parents_ug_students,ou=Other,ou=Groups,dc=clarku,dc=edu (80) Jul 7 16:52:44 zoot slapd[2179]: slap_queue_csn: queing 0x7f7458be8470 20140707205225.029175Z#000000#001#000000 Jul 7 16:52:44 zoot slapd[2179]: slap_graduate_commit_csn: removing 0x7f74581ad2e0 20140707205225.029175Z#000000#001#000000 Then we get (possibly unrelated) a lengthy list of nonpresent_callbacks (tons more then I list here): Jul 7 17:06:24 zoot slapd[2179]: nonpresent_callback: rid=001 present UUID 5111f346-871c-1033-8d4b-f5dbd10b8ca0, dn uid=rvenkatachalam,ou=Users,dc=clarku,dc=edu Jul 7 17:06:24 zoot slapd[2179]: nonpresent_callback: rid=001 present UUID 1e3d94e4-8ce8-1033-8d6a-f5dbd10b8ca0, dn cn=BIOL399-09-SU14,ou=Courses,ou=Groups,dc=clarku,dc=edu Jul 7 17:06:24 zoot slapd[2179]: nonpresent_callback: rid=001 present UUID 45843aa6-91b8-1033-81a9-ff6ec337b146, dn cn=MSPC3090-01-F14,ou=Courses,ou=Groups,dc=clarku,dc=edu Jul 7 17:06:24 zoot slapd[2179]: nonpresent_callback: rid=001 present UUID 4dd09f16-921b-1033-8d8e-f5dbd10b8ca0, dn cn=msit3710-01-f14,ou=DL,ou=Groups,dc=clarku,dc=edu Jul 7 17:06:24 zoot slapd[2179]: nonpresent_callback: rid=001 present UUID 0b3e8d76-9607-1033-81cf-ff6ec337b146, dn cn=acct5411-01-f14,ou=DL,ou=Groups,dc=clarku,dc=edu Jul 7 17:06:24 zoot slapd[2179]: nonpresent_callback: rid=001 present UUID a698a278-9677-1033-81ee-ff6ec337b146, dn cn=ESL0051-E1-F14,ou=Courses,ou=Groups,dc=clarku,dc=edu Jul 7 17:06:24 zoot slapd[2179]: nonpresent_callback: rid=001 present UUID ccf94c1c-979c-1033-8dc9-f5dbd10b8ca0, dn uid=dgunarajasingam,ou=Users,dc=clarku,dc=edu Followed by another lock table / del_nonpresent: Jul 7 17:06:24 zoot slapd[2179]: bdb(dc=clarku,dc=edu): Lock table is out of available lock entries Jul 7 17:06:24 zoot slapd[2179]: => bdb_idl_delete_key: c_del id failed: Cannot allocate memory (12) Jul 7 17:06:24 zoot slapd[2179]: null_callback : error code 0x50 Jul 7 17:06:24 zoot slapd[2179]: slap_graduate_commit_csn: removing 0x7f73e016cb40 20140707210614.062782Z#000000#001#000000 Jul 7 17:06:24 zoot slapd[2179]: syncrepl_del_nonpresent: rid=001 be_delete cn=parents_ug_students,ou=Other,ou=Groups,dc=clarku,dc=edu (80) Jul 7 17:06:24 zoot slapd[2179]: slap_queue_csn: queing 0x7f73e0f0eaa0 20140707210614.062782Z#000000#001#000000 Jul 7 17:06:24 zoot slapd[2179]: slap_graduate_commit_csn: removing 0x7f73e0f180c0 20140707210614.062782Z#000000#001#000000 That repeats itself @ 18:07, 18:47, 18:55. Then the train departs for crazy town... notice this is the other node: ul 7 18:55:59 animal slapd[10389]: nonpresent_callback: rid=002 nonpresent UUID 0e6fea9e-9279-1033-8d97-f5dbd10b8ca0, dn cn=MSPC3070-01-F14,ou=Courses,ou=Groups,dc=clarku,dc=edu Jul 7 18:55:59 animal slapd[10389]: nonpresent_callback: rid=002 nonpresent UUID f710e364-958c-1033-8da2-f5dbd10b8ca0, dn cn=MKT5900-01-F14,ou=Courses,ou=Groups,dc=clarku,dc=edu Jul 7 18:55:59 animal slapd[10389]: nonpresent_callback: rid=002 nonpresent UUID 815ece60-965e-1033-8dab-f5dbd10b8ca0, dn cn=ESL0067-M1-F14,ou=Courses,ou=Groups,dc=clarku,dc=edu Jul 7 18:55:59 animal slapd[10389]: nonpresent_callback: rid=002 nonpresent UUID f9d8a408-96cf-1033-8db0-f5dbd10b8ca0, dn cn=eng2950-01-su14,ou=DL,ou=Groups,dc=clarku,dc=edu Jul 7 18:55:59 animal slapd[10389]: nonpresent_callback: rid=002 nonpresent UUID 480d8cbc-96d3-1033-8dbf-f5dbd10b8ca0, dn cn=esl0011-e1-f14,ou=DL,ou=Groups,dc=clarku,dc=edu Jul 7 18:55:59 animal slapd[10389]: nonpresent_callback: rid=002 nonpresent UUID ccf94c1c-979c-1033-8dc9-f5dbd10b8ca0, dn uid=dgunarajasingam,ou=Users,dc=clarku,dc=edu Jul 7 18:55:59 animal slapd[10389]: slap_graduate_commit_csn: removing 0x7f59c8eddc00 20140707225554.051198Z#000000#002#000000 Jul 7 18:55:59 animal slapd[10389]: syncrepl_del_nonpresent: rid=002 be_delete cn=MPA3950-03-SU14,ou=Courses,ou=Groups,dc=clarku,dc=edu (0) Jul 7 18:55:59 animal slapd[10389]: syncprov_matchops: skipping original sid 002 Jul 7 18:55:59 animal slapd[10389]: syncrepl_del_nonpresent: rid=002 be_delete cn=MSIT3950-0R-SU14,ou=Courses,ou=Groups,dc=clarku,dc=edu (0) Jul 7 18:55:59 animal slapd[10389]: syncprov_matchops: skipping original sid 002 Jul 7 18:55:59 animal slapd[10389]: syncrepl_del_nonpresent: rid=002 be_delete uid=zbamberger,ou=Users,dc=clarku,dc=edu (0) Jul 7 18:55:59 animal slapd[10389]: syncprov_matchops: skipping original sid 002 Jul 7 18:55:59 animal slapd[10389]: syncrepl_del_nonpresent: rid=002 be_delete uid=dgunarajasingam,ou=Users,dc=clarku,dc=edu (0) Jul 7 18:55:59 animal slapd[10389]: syncprov_matchops: skipping original sid 002 Jul 7 18:55:59 animal slapd[10389]: syncrepl_del_nonpresent: rid=002 be_delete uid=hmunawar,ou=Users,dc=clarku,dc=edu (0) Jul 7 18:55:59 animal slapd[10389]: syncprov_matchops: skipping original sid 002 Jul 7 18:55:59 animal slapd[10389]: syncrepl_del_nonpresent: rid=002 be_delete cn=scrn2940-01-su14,ou=DL,ou=Groups,dc=clarku,dc=edu (0) Jul 7 18:55:59 animal slapd[10389]: syncprov_matchops: skipping original sid 002 Jul 7 18:55:59 animal slapd[10389]: syncrepl_del_nonpresent: rid=002 be_delete cn=eng2940-02-su14,ou=DL,ou=Groups,dc=clarku,dc=edu (0) Jul 7 18:55:59 animal slapd[10389]: syncprov_matchops: skipping original sid 002 Jul 7 18:55:59 animal slapd[10389]: syncrepl_del_nonpresent: rid=002 be_delete cn=id2940-02-su14,ou=DL,ou=Groups,dc=clarku,dc=edu (0) Jul 7 18:55:59 animal slapd[10389]: syncprov_matchops: skipping original sid 002 Jul 7 18:55:59 animal slapd[10389]: syncrepl_del_nonpresent: rid=002 be_delete cn=msit3950-0p-su14,ou=DL,ou=Groups,dc=clarku,dc=edu (0) Jul 7 18:55:59 animal slapd[10389]: syncprov_matchops: skipping original sid 002 Jul 7 18:55:59 animal slapd[10389]: syncrepl_del_nonpresent: rid=002 be_delete cn=mspc3060-0p-su14,ou=DL,ou=Groups,dc=clarku,dc=edu (0) That goes on, deleting things, until my pager exploded when some critical role accounts got nuked and our radius servers died. I pulled an LDIF when I realized what was happening and between 18:55 and 19:35, our LDIF went from 75MB to 45MB - so about 30MB of stuff disappeared. What happened? Did I get hit by an oddball BDB/Syncrepl behavior? Do I have a bad misconfiguration in my replication setup? Thanks for your time, Aaron ---- Manager of Systems Administration Clark University ITS

3 3

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical