Full_Name: Pierangelo Masarati
Version: HEAD/re24
OS: irrelevant
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (129.72.141.24)
Submitted by: ando
When requesting paged results control on glued databases, if a database (either
superior or subordinate) returns one page that ends with the last corresponding
entry in that database, the resulting cookie is empty. Thus, the result
returned to the client corresponds to the final result of a paged response.
However, other databases may have returned data. A fix is being designed. p.
masarati(a)aero.polimi.it wrote:
> OK, I've found the problem, and it seems to be a configuration problem.
> Please read below.
Great. Thanks for taking the time to look at this.
> ^^^ An entry with the same DN exists in the superior database. As a
> consequence, this entry is returned by back-bdb's tool functions, but
> backglue tries to release it using back-perl. Since back-perl does not
> have any tool functions, what is considered a safe replacement is used.
> Unfortunately, the safe replacement doesn't know how to deal with
> read-only entries returned by back-bdb. The solution consists in:
>
> - temporarily commenting the instance of back-perl
>
> - removing this entry from your back-bdb
>
> - re-enabling the instance of back-perl
>
> A follow-on to this ITS could be to devise a means to detect whether a
> superior database contains entries that belong to a subordinate one, and
> warn the administrator of the inconsistency.
Ah I see. I remember thinking at the time whether or not I needed a DN
within my master database too (similar to a FS mount point) but
obviously it has been this that has been causing the problem. At least
now I know that this isn't the case and can work around it.
I'd expect a subordinate database to always override the master (similar
to the mount point idea again) but in the case of slapcat then I believe
it should probably just ignore a subordinate that doesn't have a
physical backing store presence such as back-perl. I'm not sure quite
how you'd mark this though.
An appropriately worded warning would definitely be a good start,
especially as when I first started working with back-perl as a
subordinate, I was confused that putting the subordinate definition
after the master would always fail whereas putting it before would allow
it to work.
ATB,
Mark.
--
Mark Cave-Ayland - Senior Technical Architect
PostgreSQL - PostGIS
Sirius Corporation plc - control through freedom
http://www.siriusit.co.uk
t: +44 870 608 0063
Sirius Labs: http://www.siriusit.co.uk/labs
OK, I've found the problem, and it seems to be a configuration problem.
Please read below.
> Here is the slapd.conf used in the test:
>
>
> #
> # See slapd.conf(5) for details on configuration options.
> # This file should NOT be world readable.
> #
> include /home/build/rel-openldap/etc/openldap/schema/core.schema
>
> # Define global ACLs to disable default read access.
>
> # Do not enable referrals until AFTER you have a working directory
> # service AND an understanding of referrals.
> #referral ldap://root.openldap.org
>
> pidfile /home/build/rel-openldap/var/run/slapd.pid
> argsfile /home/build/rel-openldap/var/run/slapd.args
>
> # Load dynamic backend modules:
> # modulepath /home/build/rel-openldap/libexec/openldap
> # moduleload back_bdb.la
> # moduleload back_hdb.la
> # moduleload back_ldap.la
>
> # Sample security restrictions
> # Require integrity protection (prevent hijacking)
> # Require 112-bit (3DES or better) encryption for updates
> # Require 63-bit encryption for simple bind
> # security ssf=1 update_ssf=112 simple_bind=64
>
> # Sample access control policy:
> # Root DSE: allow anyone to read it
> # Subschema (sub)entry DSE: allow anyone to read it
> # Other DSEs:
> # Allow self write access
> # Allow authenticated users read access
> # Allow anonymous users to authenticate
> # Directives needed to implement policy:
> # access to dn.base="" by * read
> # access to dn.base="cn=Subschema" by * read
> # access to *
> # by self write
> # by users read
> # by anonymous auth
> #
> # if no access controls are present, the default policy
> # allows anyone and everyone to read anything but restricts
> # updates to rootdn. (e.g., "access to * by * read")
> #
> # rootdn can always read and write EVERYTHING!
>
> database perl
> suffix "ou=perl,dc=my-domain,dc=com"
^^ This is the suffix where the instance of back-perl is rooted at
> subordinate
> perlModulePath /home/build/rel-openldap/etc/openldap
> perlModule SampleLDAP
>
> #######################################################################
> # BDB database definitions
> #######################################################################
>
> database bdb
> suffix "dc=my-domain,dc=com"
> rootdn "cn=Manager,dc=my-domain,dc=com"
> # Cleartext passwords, especially for the rootdn, should
> # be avoid. See slappasswd(8) and slapd.conf(5) for details.
> # Use of strong authentication encouraged.
> rootpw secret
> # The database directory MUST exist prior to running slapd AND
> # should only be accessible by the slapd and slap tools.
> # Mode 700 recommended.
> directory /home/build/rel-openldap/var/openldap-data
> # Indices to maintain
> index objectClass eq
>
>
> And here is the slapcat output when the perl database is disabled in
> slapd.conf to prevent the crash:
>
>
> dn: dc=my-domain,dc=com
> objectClass: top
> objectClass: dcObject
> objectClass: organization
> o:: bXlkb21haW4uY29tIA==
> dc: my-domain
> structuralObjectClass: organization
> entryUUID: 0bdc2650-d5c0-102e-9fe7-3350c645132d
> creatorsName: cn=Manager,dc=my-domain,dc=com
> createTimestamp: 20100406120241Z
> entryCSN: 20100406120241.310422Z#000000#000#000000
> modifiersName: cn=Manager,dc=my-domain,dc=com
> modifyTimestamp: 20100406120241Z
>
> dn: ou=real,dc=my-domain,dc=com
> objectClass: organizationalUnit
> objectClass: top
> structuralObjectClass: organizationalUnit
> entryUUID: 5644fb04-d5c0-102e-9c04-81e0ba1bb1f2
> creatorsName: cn=Manager,dc=my-domain,dc=com
> createTimestamp: 20100406120446Z
> ou: real
> entryCSN: 20100406120458.545769Z#000000#000#000000
> modifiersName: cn=Manager,dc=my-domain,dc=com
> modifyTimestamp: 20100406120458Z
>
> dn: ou=perl,dc=my-domain,dc=com
^^^ An entry with the same DN exists in the superior database. As a
consequence, this entry is returned by back-bdb's tool functions, but
backglue tries to release it using back-perl. Since back-perl does not
have any tool functions, what is considered a safe replacement is used.
Unfortunately, the safe replacement doesn't know how to deal with
read-only entries returned by back-bdb. The solution consists in:
- temporarily commenting the instance of back-perl
- removing this entry from your back-bdb
- re-enabling the instance of back-perl
A follow-on to this ITS could be to devise a means to detect whether a
superior database contains entries that belong to a subordinate one, and
warn the administrator of the inconsistency.
p.
> masarati(a)aero.polimi.it wrote:
>
>> ^^^ I think the culprit is here. No point in tracking down further this
>> issue unless you re-link as recommended by this message.
>>
>> p.
>
> Sorry - I experienced the error on a production box running Debian
> packages, and re-built from source on my local machine just to confirm
> the bug still exists. However, my local machine already has other LDAP
> libraries installed which seems to have resulted in some of the wrong
> libraries being picked up at compile time.
>
> I've now removed all pre-built openldap packages, re-built again from
> source and can confirm that the bug still stands since the
> "ldap_int_global_options" warning has now disappeared:
>
>
> zeno:/home/build/rel-openldap# ./sbin/slapcat
> Here in new
> Posix Var 8192 and 4096
> bdb_db_open: database "dc=my-domain,dc=com": unclean shutdown detected;
> attempting recovery.
> bdb_db_open: warning - no DB_CONFIG file found in directory
> /home/build/rel-openldap/var/openldap-data: (2).
> Expect poor performance for suffix "dc=my-domain,dc=com".
> bdb_db_open: database "dc=my-domain,dc=com": recovery skipped in
> read-only mode. Run manual recovery if errors are encountered.
> bdb_monitor_db_open: monitoring disabled; configure monitor database to
> enable
> dn: dc=my-domain,dc=com
> objectClass: top
> objectClass: dcObject
> objectClass: organization
> o:: bXlkb21haW4uY29tIA==
> dc: my-domain
> structuralObjectClass: organization
> entryUUID: 0bdc2650-d5c0-102e-9fe7-3350c645132d
> creatorsName: cn=Manager,dc=my-domain,dc=com
> createTimestamp: 20100406120241Z
> entryCSN: 20100406120241.310422Z#000000#000#000000
> modifiersName: cn=Manager,dc=my-domain,dc=com
> modifyTimestamp: 20100406120241Z
>
> dn: ou=real,dc=my-domain,dc=com
> objectClass: organizationalUnit
> objectClass: top
> structuralObjectClass: organizationalUnit
> entryUUID: 5644fb04-d5c0-102e-9c04-81e0ba1bb1f2
> creatorsName: cn=Manager,dc=my-domain,dc=com
> createTimestamp: 20100406120446Z
> ou: real
> entryCSN: 20100406120458.545769Z#000000#000#000000
> modifiersName: cn=Manager,dc=my-domain,dc=com
> modifyTimestamp: 20100406120458Z
>
> *** glibc detected *** ./sbin/slapcat: free(): invalid pointer:
^^^ this message is not quite informative, nor is what follows below.
Please read <http://www.openldap.org/faq/data/cache/56.html> carefully,
and provide the information required.
p.
> 0x00000000009c4fd3 ***
> ======= Backtrace: =========
> /lib/libc.so.6[0x7f4ea6fd0948]
> /lib/libc.so.6(cfree+0x76)[0x7f4ea6fd2a56]
> ./sbin/slapcat[0x42c11d]
> ./sbin/slapcat[0x42c1c9]
> ./sbin/slapcat[0x481448]
> ./sbin/slapcat[0x484c93]
> ./sbin/slapcat[0x4874b9]
> ./sbin/slapcat[0x408d40]
> /lib/libc.so.6(__libc_start_main+0xe6)[0x7f4ea6f7b1a6]
> ./sbin/slapcat[0x4087d9]
>
>
> Many thanks,
>
> Mark.
>
> --
> Mark Cave-Ayland - Senior Technical Architect
> PostgreSQL - PostGIS
> Sirius Corporation plc - control through freedom
> http://www.siriusit.co.uk
> t: +44 870 608 0063
>
> Sirius Labs: http://www.siriusit.co.uk/labs
>
masarati(a)aero.polimi.it wrote:
> ^^^ I think the culprit is here. No point in tracking down further this
> issue unless you re-link as recommended by this message.
>
> p.
Sorry - I experienced the error on a production box running Debian
packages, and re-built from source on my local machine just to confirm
the bug still exists. However, my local machine already has other LDAP
libraries installed which seems to have resulted in some of the wrong
libraries being picked up at compile time.
I've now removed all pre-built openldap packages, re-built again from
source and can confirm that the bug still stands since the
"ldap_int_global_options" warning has now disappeared:
zeno:/home/build/rel-openldap# ./sbin/slapcat
Here in new
Posix Var 8192 and 4096
bdb_db_open: database "dc=my-domain,dc=com": unclean shutdown detected;
attempting recovery.
bdb_db_open: warning - no DB_CONFIG file found in directory
/home/build/rel-openldap/var/openldap-data: (2).
Expect poor performance for suffix "dc=my-domain,dc=com".
bdb_db_open: database "dc=my-domain,dc=com": recovery skipped in
read-only mode. Run manual recovery if errors are encountered.
bdb_monitor_db_open: monitoring disabled; configure monitor database to
enable
dn: dc=my-domain,dc=com
objectClass: top
objectClass: dcObject
objectClass: organization
o:: bXlkb21haW4uY29tIA==
dc: my-domain
structuralObjectClass: organization
entryUUID: 0bdc2650-d5c0-102e-9fe7-3350c645132d
creatorsName: cn=Manager,dc=my-domain,dc=com
createTimestamp: 20100406120241Z
entryCSN: 20100406120241.310422Z#000000#000#000000
modifiersName: cn=Manager,dc=my-domain,dc=com
modifyTimestamp: 20100406120241Z
dn: ou=real,dc=my-domain,dc=com
objectClass: organizationalUnit
objectClass: top
structuralObjectClass: organizationalUnit
entryUUID: 5644fb04-d5c0-102e-9c04-81e0ba1bb1f2
creatorsName: cn=Manager,dc=my-domain,dc=com
createTimestamp: 20100406120446Z
ou: real
entryCSN: 20100406120458.545769Z#000000#000#000000
modifiersName: cn=Manager,dc=my-domain,dc=com
modifyTimestamp: 20100406120458Z
*** glibc detected *** ./sbin/slapcat: free(): invalid pointer:
0x00000000009c4fd3 ***
======= Backtrace: =========
/lib/libc.so.6[0x7f4ea6fd0948]
/lib/libc.so.6(cfree+0x76)[0x7f4ea6fd2a56]
./sbin/slapcat[0x42c11d]
./sbin/slapcat[0x42c1c9]
./sbin/slapcat[0x481448]
./sbin/slapcat[0x484c93]
./sbin/slapcat[0x4874b9]
./sbin/slapcat[0x408d40]
/lib/libc.so.6(__libc_start_main+0xe6)[0x7f4ea6f7b1a6]
./sbin/slapcat[0x4087d9]
Many thanks,
Mark.
--
Mark Cave-Ayland - Senior Technical Architect
PostgreSQL - PostGIS
Sirius Corporation plc - control through freedom
http://www.siriusit.co.uk
t: +44 870 608 0063
Sirius Labs: http://www.siriusit.co.uk/labs
Here is the slapd.conf used in the test:
#
# See slapd.conf(5) for details on configuration options.
# This file should NOT be world readable.
#
include /home/build/rel-openldap/etc/openldap/schema/core.schema
# Define global ACLs to disable default read access.
# Do not enable referrals until AFTER you have a working directory
# service AND an understanding of referrals.
#referral ldap://root.openldap.org
pidfile /home/build/rel-openldap/var/run/slapd.pid
argsfile /home/build/rel-openldap/var/run/slapd.args
# Load dynamic backend modules:
# modulepath /home/build/rel-openldap/libexec/openldap
# moduleload back_bdb.la
# moduleload back_hdb.la
# moduleload back_ldap.la
# Sample security restrictions
# Require integrity protection (prevent hijacking)
# Require 112-bit (3DES or better) encryption for updates
# Require 63-bit encryption for simple bind
# security ssf=1 update_ssf=112 simple_bind=64
# Sample access control policy:
# Root DSE: allow anyone to read it
# Subschema (sub)entry DSE: allow anyone to read it
# Other DSEs:
# Allow self write access
# Allow authenticated users read access
# Allow anonymous users to authenticate
# Directives needed to implement policy:
# access to dn.base="" by * read
# access to dn.base="cn=Subschema" by * read
# access to *
# by self write
# by users read
# by anonymous auth
#
# if no access controls are present, the default policy
# allows anyone and everyone to read anything but restricts
# updates to rootdn. (e.g., "access to * by * read")
#
# rootdn can always read and write EVERYTHING!
database perl
suffix "ou=perl,dc=my-domain,dc=com"
subordinate
perlModulePath /home/build/rel-openldap/etc/openldap
perlModule SampleLDAP
#######################################################################
# BDB database definitions
#######################################################################
database bdb
suffix "dc=my-domain,dc=com"
rootdn "cn=Manager,dc=my-domain,dc=com"
# Cleartext passwords, especially for the rootdn, should
# be avoid. See slappasswd(8) and slapd.conf(5) for details.
# Use of strong authentication encouraged.
rootpw secret
# The database directory MUST exist prior to running slapd AND
# should only be accessible by the slapd and slap tools.
# Mode 700 recommended.
directory /home/build/rel-openldap/var/openldap-data
# Indices to maintain
index objectClass eq
And here is the slapcat output when the perl database is disabled in
slapd.conf to prevent the crash:
dn: dc=my-domain,dc=com
objectClass: top
objectClass: dcObject
objectClass: organization
o:: bXlkb21haW4uY29tIA==
dc: my-domain
structuralObjectClass: organization
entryUUID: 0bdc2650-d5c0-102e-9fe7-3350c645132d
creatorsName: cn=Manager,dc=my-domain,dc=com
createTimestamp: 20100406120241Z
entryCSN: 20100406120241.310422Z#000000#000#000000
modifiersName: cn=Manager,dc=my-domain,dc=com
modifyTimestamp: 20100406120241Z
dn: ou=real,dc=my-domain,dc=com
objectClass: organizationalUnit
objectClass: top
structuralObjectClass: organizationalUnit
entryUUID: 5644fb04-d5c0-102e-9c04-81e0ba1bb1f2
creatorsName: cn=Manager,dc=my-domain,dc=com
createTimestamp: 20100406120446Z
ou: real
entryCSN: 20100406120458.545769Z#000000#000#000000
modifiersName: cn=Manager,dc=my-domain,dc=com
modifyTimestamp: 20100406120458Z
dn: ou=perl,dc=my-domain,dc=com
objectClass: organizationalUnit
objectClass: top
ou: perl
structuralObjectClass: organizationalUnit
entryUUID: 6438973e-d5c0-102e-9c05-81e0ba1bb1f2
creatorsName: cn=Manager,dc=my-domain,dc=com
createTimestamp: 20100406120509Z
entryCSN: 20100406120509.555750Z#000000#000#000000
modifiersName: cn=Manager,dc=my-domain,dc=com
modifyTimestamp: 20100406120509Z
ATB,
Mark.
--
Mark Cave-Ayland - Senior Technical Architect
PostgreSQL - PostGIS
Sirius Corporation plc - control through freedom
http://www.siriusit.co.uk
t: +44 870 608 0063
Sirius Labs: http://www.siriusit.co.uk/labs
Full_Name: Mark Cave-Ayland
Version: 2.4.21
OS: Linux
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (81.187.123.11)
Hi there,
Having implemented a custom perl module as a subordinate with back-perl, we've
found a segfault when using slapcat to dump the database. The backtrace looks
like this:
zeno:/home/build/rel-openldap# ./sbin/slapcat
./sbin/slapcat: Symbol `ldap_int_global_options' has different size in shared
object, consider re-linking
Here in new
Posix Var 8192 and 4096
bdb_db_open: warning - no DB_CONFIG file found in directory
/home/build/rel-openldap/var/openldap-data: (2).
Expect poor performance for suffix "dc=my-domain,dc=com".
bdb_monitor_db_open: monitoring disabled; configure monitor database to enable
dn: dc=my-domain,dc=com
objectClass: top
objectClass: dcObject
objectClass: organization
o:: bXlkb21haW4uY29tIA==
dc: my-domain
structuralObjectClass: organization
entryUUID: 0bdc2650-d5c0-102e-9fe7-3350c645132d
creatorsName: cn=Manager,dc=my-domain,dc=com
createTimestamp: 20100406120241Z
entryCSN: 20100406120241.310422Z#000000#000#000000
modifiersName: cn=Manager,dc=my-domain,dc=com
modifyTimestamp: 20100406120241Z
dn: ou=real,dc=my-domain,dc=com
objectClass: organizationalUnit
objectClass: top
structuralObjectClass: organizationalUnit
entryUUID: 5644fb04-d5c0-102e-9c04-81e0ba1bb1f2
creatorsName: cn=Manager,dc=my-domain,dc=com
createTimestamp: 20100406120446Z
ou: real
entryCSN: 20100406120458.545769Z#000000#000#000000
modifiersName: cn=Manager,dc=my-domain,dc=com
modifyTimestamp: 20100406120458Z
*** glibc detected *** ./sbin/slapcat: free(): invalid pointer:
0x00000000009c5063 ***
======= Backtrace: =========
/lib/libc.so.6[0x7f1adc150948]
/lib/libc.so.6(cfree+0x76)[0x7f1adc152a56]
./sbin/slapcat[0x42c11d]
./sbin/slapcat[0x42c1c9]
./sbin/slapcat[0x481448]
./sbin/slapcat[0x484c93]
./sbin/slapcat[0x4874b9]
./sbin/slapcat[0x408d40]
/lib/libc.so.6(__libc_start_main+0xe6)[0x7f1adc0fb1a6]
./sbin/slapcat[0x4087d9]
Looks like an invalid pointer is being deferenced when running slapcat.
ATB,
Mark.