Versioning entries in the DIT
by Lorenzo Pastrana
Hi, List
I'm planning to store some 'versioned' data into an ldap directory, and
trying not to reinvent the wheel ... if anybody has any good resource to
share on the subject I'll be glad to hear.
Thanks in advance,
LP
Lorenzo Pastrana - Happy End Vision
--------------------------
Design web
Conception multimédia
Communication visuelle et édition
--------------------------
Tél. : 01 42 47 83 09
Fax : 01 47 70 70 19
E-mail : lorenzo.pastrana(a)happyend.fr
14 years, 9 months
handling forward-only links, DN+Binary and DN+String
by Andrew Bartlett
Having thought I got to the bottom of extended DN behaviour, I've come
across more challenges, that I would like thoughts on.
Handling renames of one-way links: OpenLDAP already does this, but
Samba needs some help here (as we try to infer the rename from the
presence of backlinks, but for one-way links, how should we know we are
being linked to?)
Handling of DN+Binary and DN+String one-way links. For example,
wellKnownObjects:
B:32:22b70c67d56e4efb91e9300fca3dc1aa:CN=ForeignSecurityPrincipals,DC=samba,DC=org
This is a 'DN+Binary' syntax attribute (for resolving well known GUIDs
into a DN), and must therefore follow when the well known target
renames. MS-ADTS 3.1.1.1.6 specifies the behaviour.
The challenge I see here is that I really do need an additional syntax
in OpenLDAP. If I map this to just a binary string (as I do now), then
the rename will not follow though. If I map it to a DN (as I had tried
in the past), then the syntax is invalid. Is it entirely unreasonable
to add an additional syntax?
This is a bit of a 'hit and run' question, as I won't be able to carry
on the discussion during Christmas/New Year, but any thoughts would be
most welcome.
Thanks,
Andrew Bartlett
--
Andrew Bartlett
http://samba.org/~abartlet/
Authentication Developer, Samba Team http://samba.org
Samba Developer, Red Hat Inc.
14 years, 9 months
In memory cache or bdb performance degradation when upgrading from openldap v2.3.27 to v2.4.11 ?
by Chiles, Michael
Openldap Technical folk,
We have inherited an openldap farm that was deployed using openldap v2.3.27.
We have been testing a newly compiled v2.4.11 with same compile flags as a possible replacement due to some replication errors we have seen, but have discovered other bigger problems with the new instance.
We believe the issue may be related to in memory cache not working as expected, or that 2.4.11 does not use the hdb backend as efficiently as before. Can anyone confirm a negative performance difference between these versions, or an issue with cache? We are seeing major significant differences in the db_stat output with orders of magnitude difference in the number of attempted reads against the backend cache. I assume these are unsual and that in memory entry cache would normally prevent this traffic from reach the bdb cache. I assume we simply have something wrong in configuration, but I don't see an obvious explanation. If anyone has a moment to review, we would appreciate your feedback.
Here is the process we followed, with supporting config info:
We have a SLAMD benchmark test based on a real world use case where 400 clients make a "near" simultaneous connection to the directory and execute a search like the following:
ldapsearch -h server1 -x - b "ou=myou,dc=mydc,dc=com" "objectclass=*"
There are nearly 70,000 objects in this ou with 5 attributes each ( 3 of which are objectclass ), and nearly 210,000 objects in the entire directory. We have an objectclass index.
2.4.11 tests were performed on instances compiled on SLES9.3 64bit, 4 way dual core procs, 16GB RAM, using hoard memory manager, bdb 4.6, and cyrus-sasl-2.1.22 .
2.3.27 tests were performaned on SLES9.3 64bit, 2 way single core proc, 8GB RAM, using standard memory manager, and standard bdb ( 4.2 ).
On the v2.3.27 instances, we see all 400 clients get a connection, and get their results.
On the new v2.4.11 instance, we see around 150-175 clients get a connection, and the rest get a failure that they cannot reach the server. After more benchmarking, tcpdump, and loglevel -1 we know that the client traffic is getting to the box, but the openldap listener thread does not pick up the connection. We also see high numbers of processes waiting in the CPU run queue.
Reducing the number of objects in the directory to 100 results in successful connections to all 400 clients, which lead us to believe the issue might be due to differences in read performance between the instances. The same DB_CONFIG was used in both cases, and the slapd.conf was the same, with some minor tweaks due to slightly different cache configuration options between the versions. Please see the DB_CONFIG and the hdb backend stanza from the slapd.conf file included below.
We then did some basic single query tests of both instances and looked at the logs with loglevel -1 and the db_stat output. What we saw was a major difference between both instances on the db_stat results. As mentioned in the summary above, we don't have a good explanation for the difference, although it is significant, and reliable across multiple iterations of test. Please see the db_stat differences shown below. Also seems very unusual that the initial db cache stats would be so high on the new version.
slapd.conf:
2.4.11 hdb stanza ( also tested these with the same cache numbers as below 2.3.27 instance with no difference. We reduced these to reasonable levels as old version config seemed overkill ):
database hdb
directory /local/mnt/ldap.2.4.11/cache-data
threads 32
suffix "dc=mydc,dc=com"
rootdn <<snip>>
rootpw <<snip>>
cachesize 500000
dncachesize 1000000
idlcachesize 30000000
sizelimit 10000000
loglevel stats sync
dirtyread
include /opt/ldap/indexes/my.indexes
2.3.27 hdb stanza
database hdb
directory /local/mnt/ldap/cache-data
threads 32
suffix " dc=mydc,dc=com "
rootdn <<snip>>
rootpw <<snip>>
cachesize 20971520
dbcachesize 20971520 ( not a typo - this one is "dBcachesize. The other is dNcachesize )
idlcachesize 20971520
sizelimit 10000000
loglevel stats sync
dirtyread
include /opt/ldap/indexes/my.indexes
DB_config ( Same for both instances ):
set_cachesize 1 1048576000 12
set_flags DB_LOG_AUTOREMOVE
set_lg_bsize 2097512
set_lg_dir /local/mnt/ldap/cache-data ( this value points to correct directory in both instances )
set_flags DB_TXN_NOSYNC
set_lg_regionmax 500000
set_lk_max_locks 30000
set_lk_max_lockers 30000
set_lk_max_objects 30000
set_tmp_dir /dev/shm
After startup with no client test ( previous database instance was completely deleted and recreated using slapadd ), here are the db_stat -m output. I excluded some of the index db info for brevity:
Version 2.4.11
1GB 1000MB Total cache size
12 Number of caches
12 Maximum number of caches
168MB 688KB Pool individual cache size
0 Maximum memory-mapped file size
0 Maximum open file descriptors
0 Maximum sequential buffer writes
0 Sleep after writing maximum sequential buffers
0 Requested pages mapped into the process' address space
15M Requested pages found in the cache (99%)
24 Requested pages not found in the cache
9225 Pages created in the cache
24 Pages read into the cache
9244 Pages written from the cache to the backing file
0 Clean pages forced from the cache
0 Dirty pages forced from the cache
0 Dirty pages written by trickle-sync thread
9247 Current total page count
9247 Current clean page count
0 Current dirty page count
393252 Number of hash buckets used for page location
14M Total number of times hash chains searched for a page (14773760)
9 The longest hash chain searched for a page
14M Total number of hash chain entries checked for page (14764487)
0 The number of hash bucket locks that required waiting (0%)
0 The maximum number of times any hash bucket lock was waited for (0%)
0 The number of region locks that required waiting (0%)
0 The number of buffers frozen
0 The number of buffers thawed
0 The number of frozen buffers freed
9309 The number of page allocations
0 The number of hash buckets examined during allocations
0 The maximum number of hash buckets examined for an allocation
0 The number of pages examined during allocations
0 The max number of pages examined for an allocation
0 Threads waited on page I/O
Pool File: dn2id.bdb
4096 Page size
0 Requested pages mapped into the process' address space
1005002 Requested pages found in the cache (99%)
2 Requested pages not found in the cache
3062 Pages created in the cache
2 Pages read into the cache
3064 Pages written from the cache to the backing file
Pool File: id2entry.bdb
16384 Page size
0 Requested pages mapped into the process' address space
419925 Requested pages found in the cache (99%)
2 Requested pages not found in the cache
2967 Pages created in the cache
2 Pages read into the cache
2969 Pages written from the cache to the backing file
Version 2.3.27
1GB 1000MB Total cache size.
12 Number of caches.
168MB 688KB Pool individual cache size.
0 Requested pages mapped into the process' address space.
22738 Requested pages found in the cache (99%).
285 Requested pages not found in the cache.
0 Pages created in the cache.
285 Pages read into the cache.
0 Pages written from the cache to the backing file.
0 Clean pages forced from the cache.
0 Dirty pages forced from the cache.
0 Dirty pages written by trickle-sync thread.
285 Current total page count.
285 Current clean page count.
0 Current dirty page count.
393252 Number of hash buckets used for page location.
23308 Total number of times hash chains searched for a page.
12 The longest hash chain searched for a page.
22738 Total number of hash buckets examined for page location.
46616 The number of hash bucket locks granted without waiting.
0 The number of hash bucket locks granted after waiting.
0 The maximum number of times any hash bucket lock was waited for.
641 The number of region locks granted without waiting.
0 The number of region locks granted after waiting.
297 The number of page allocations.
0 The number of hash buckets examined during allocations
0 The max number of hash buckets examined for an allocation
0 The number of pages examined during allocations
0 The max number of pages examined for an allocation
Pool File: dn2id.bdb
4096 Page size.
0 Requested pages mapped into the process' address space.
13076 Requested pages found in the cache (99%).
132 Requested pages not found in the cache.
0 Pages created in the cache.
132 Pages read into the cache.
0 Pages written from the cache to the backing file.
Pool File: id2entry.bdb
16384 Page size.
0 Requested pages mapped into the process' address space.
9659 Requested pages found in the cache (99%).
138 Requested pages not found in the cache.
0 Pages created in the cache.
138 Pages read into the cache.
0 Pages written from the cache to the backing file.
After 1 client query:
Version 2.4.11
1GB 1000MB Total cache size
12 Number of caches
12 Maximum number of caches
168MB 688KB Pool individual cache size
0 Maximum memory-mapped file size
0 Maximum open file descriptors
0 Maximum sequential buffer writes
0 Sleep after writing maximum sequential buffers
0 Requested pages mapped into the process' address space
15M Requested pages found in the cache (99%)
24 Requested pages not found in the cache
9244 Pages created in the cache
24 Pages read into the cache
9263 Pages written from the cache to the backing file
0 Clean pages forced from the cache
0 Dirty pages forced from the cache
0 Dirty pages written by trickle-sync thread
9266 Current total page count
9266 Current clean page count
0 Current dirty page count
393252 Number of hash buckets used for page location
14M Total number of times hash chains searched for a page (14753673)
9 The longest hash chain searched for a page
14M Total number of hash chain entries checked for page (14744381)
0 The number of hash bucket locks that required waiting (0%)
0 The maximum number of times any hash bucket lock was waited for (0%)
0 The number of region locks that required waiting (0%)
0 The number of buffers frozen
0 The number of buffers thawed
0 The number of frozen buffers freed
9328 The number of page allocations
0 The number of hash buckets examined during allocations
0 The maximum number of hash buckets examined for an allocation
0 The number of pages examined during allocations
0 The max number of pages examined for an allocation
0 Threads waited on page I/O
Pool File: dn2id.bdb
4096 Page size
0 Requested pages mapped into the process' address space
997746 Requested pages found in the cache (99%)
2 Requested pages not found in the cache
3062 Pages created in the cache
2 Pages read into the cache
3064 Pages written from the cache to the backing file
Pool File: id2entry.bdb
16384 Page size
0 Requested pages mapped into the process' address space
410855 Requested pages found in the cache (99%)
2 Requested pages not found in the cache
2967 Pages created in the cache
2 Pages read into the cache
2969 Pages written from the cache to the backing file
Version 2.3.27
1GB 1000MB Total cache size.
12 Number of caches.
168MB 688KB Pool individual cache size.
0 Requested pages mapped into the process' address space.
299222 Requested pages found in the cache (98%).
7144 Requested pages not found in the cache.
0 Pages created in the cache.
7144 Pages read into the cache.
0 Pages written from the cache to the backing file.
0 Clean pages forced from the cache.
0 Dirty pages forced from the cache.
0 Dirty pages written by trickle-sync thread.
7144 Current total page count.
7144 Current clean page count.
0 Current dirty page count.
393252 Number of hash buckets used for page location.
313510 Total number of times hash chains searched for a page.
23 The longest hash chain searched for a page.
300752 Total number of hash buckets examined for page location.
627020 The number of hash bucket locks granted without waiting.
0 The number of hash bucket locks granted after waiting.
0 The maximum number of times any hash bucket lock was waited for.
14400 The number of region locks granted without waiting.
0 The number of region locks granted after waiting.
7164 The number of page allocations.
0 The number of hash buckets examined during allocations
0 The max number of hash buckets examined for an allocation
0 The number of pages examined during allocations
0 The max number of pages examined for an allocation
Pool File: dn2id.bdb
4096 Page size.
0 Requested pages mapped into the process' address space.
173225 Requested pages found in the cache (98%).
3233 Requested pages not found in the cache.
0 Pages created in the cache.
3233 Pages read into the cache.
0 Pages written from the cache to the backing file.
Pool File: id2entry.bdb
16384 Page size.
0 Requested pages mapped into the process' address space.
125990 Requested pages found in the cache (97%).
3888 Requested pages not found in the cache.
0 Pages created in the cache.
3888 Pages read into the cache.
0 Pages written from the cache to the backing file.
##########################
Thanks!
-Michael
14 years, 9 months
how to solve invalid structural object class chain
by owen nirvana
I try to a entry like this:
cn = .. ,dc=..,dc=..,dc=..,dc=..
cn = ..
objectClass: person
objectClass: inetOrgPerson
objectClass: organizatinoalRole
objectClass: device
objectClass: pkiUser
roleOccupant: ...
employeeNumber : ..
userCertificate;binary :: BASE64 Encode
icSerilNumber: ..
tel: ...
fax: ...
telex: ..
log reports " invalid structural object class chain in
organizatinoalRole/Person or organizatinoalRole/inetOrgPerson etc"
I try many combination about the order of person ,inetOrgPerson and
organizatinoalRole, and the same error was reported.
thanks for help
gtalk:freeespeech@gmail.com
14 years, 9 months
Re: Re: N-Way Multi-Master replication - delete problem
by Adrien Futschik
Hy everyone,
I have just tested the same procedure with OpenLDAP 2.4.13. The problem
remains the same.
Did I miss something ? Is this supposed to be like this ?
I'm joining the modified script I'm using to setup both masters and the LDIF files I'm using to add and remove an entry (+ attributes).
I did not use access-log, is this supposed to work with N-Way Multi-Master replication ? I thought it was only used in case of Delta Synchronization/Replication.
Adrien
========================================
Message date : Dec 18 2008, 02:24 PM
From : "Miguel Jinez" <miguel.jinez(a)gmail.com>
To : adrien.futschik(a)atosorigin.com
Copy to : openldap-technical(a)openldap.org
Subject : Re: N-Way Multi-Master replication - delete problem
Maybe in openLdap 2.4.13 it would be solved.
I had that problem, and I fixed it using accesslog, but it makes you lose
performance....
2008/12/18 Adrien Futschik <adrien.futschik(a)atosorigin.com>
> I'm testing N-Way Multi-Master replication with OpenLDAP 2.411.
>
> I have setup 2 Masters (m1 & m2) starting form test050-syncrepl-multimaster
> and modifying it.
>
> Every thing seems to work fine except deleting entries.
>
> Let me explain.
> case 1 :
> . When I add an entry on m1 it is successfully replicated on m2.
> . When I try to delete this entry on m1, it is successfully removed from
> m1, but not replicated on m2.
> . When, I try to delete this entry on m2, it is successfully removed from
> m2 & m1.
>
> case 2 :
> . When I add an entry on m2 it is successfully replicated on m1.
> . When I try to delete this entry on m2, it is successfully removed from
> m2, but not replicated on m1.
> . When, I try to delete this entry on m1, it is successfully removed from
> m1 & m2.
>
> I don't have the same problem when I delete an attribute or update an
> entry. Is this normal ?
>
> Adrien Futschik
>
Adrien Futschik
14 years, 9 months
'Has A' vs 'Is A'
by Lorenzo Pastrana
Hi list,
Me again ...
I've got to compose some objects by inheritance of parent classes (Is
A), I'm now on the point of building the actual object tree and I was
wondering what is the preferred mechanism for aggregation (Has A) and
controlling children types ... can I do this in the schema ?
Thanks,
LP.
Lorenzo Pastrana - Happy End Vision
--------------------------
Design web
Conception multimédia
Communication visuelle et édition
--------------------------
Tél. : 01 42 47 83 09
Fax : 01 47 70 70 19
E-mail : lorenzo.pastrana(a)happyend.fr
14 years, 9 months
Workflow / tools / practice working on a new shema : reload to cn=config or editing cn=config in place ?
by Lorenzo Pastrana
Hi, list
Still a genuine noob.. ;P
I'm working on a brand new shema to store my app's data ..
What do you guys use to do when working on a 'new' directory ?
a) work on a shema file / restart server
b) work on a shema file / reload to cn=config
c) edit cn=config /
a) could be ok .. a little brutal but ...
b) sounds tedious
c) what tool do you use ?
Thx.
LP
Lorenzo Pastrana - Happy End Vision
--------------------------
Design web
Conception multimédia
Communication visuelle et édition
--------------------------
Tél. : 01 42 47 83 09
Fax : 01 47 70 70 19
E-mail : lorenzo.pastrana(a)happyend.fr
14 years, 9 months
how to solve invalid structural object class chain
by owen nirvana
I try to a entry like this:
cn = .. ,dc=..,dc=..,dc=..,dc=..
cn = ..
objectClass: person
objectClass: inetOrgPerson
objectClass: organizatinoalRole
objectClass: device
objectClass: pkiUser
roleOccupant: ...
employeeNumber : ..
userCertificate;binary :: BASE64 Encode
icSerilNumber: ..
tel: ...
fax: ...
telex: ..
log reports " invalid structural object class chain in
organizatinoalRole/Person or organizatinoalRole/inetOrgPerson etc"
I try many combination about the order of person ,inetOrgPerson and
organizatinoalRole, and the same error was reported.
thanks for help
gtalk:freeespeech@gmail.com
14 years, 9 months
Unix id command and Openldap
by okossuth＠antel.com.uy
Hi
Does the id command works with a system using OPENLDAP authentication ?
I have implemented a server with openldap 2.4 and several clients use this system to authenticate
users, and works fine except that when I do a "id user" on a client it only gives me the information of the primary
group which the user belongs to and not of the suplementary groups that he is also a member of in the LDAP server...
any ideas??
Saludos,
Oskar Kossuth
Administrador UNIX
ANTEL Telecomunicaciones
El presente correo y cualquier posible archivo adjunto está
dirigido únicamente al destinatario del mensaje y contiene información
que puede ser confidencial. Si Ud. no es el destinatario correcto por
favor notifique al remitente respondiendo anexando este mensaje y elimine
inmediatamente el e-mail y los posibles archivos adjuntos al mismo de su
sistema. Está prohibida cualquier utilización, difusión o copia de este
e-mail por cualquier persona o entidad que no sean las específicas
destinatarias del mensaje. ANTEL no acepta ninguna responsabilidad con
respecto a cualquier comunicación que haya sido emitida incumpliendo
nuestra Política de Seguridad de la Información.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
This e-mail and any attachment is confidential and is intended solely for
the addressee(s). If you are not intended recipient please inform the
sender immediately, answering this e-mail and delete it as well as the
attached files. Any use, circulation or copy of this e-mail by any person
or entity that is not the specific addressee(s) is prohibited. ANTEL is
not responsible for any communication emitted without respecting our
Information Security Policy.
14 years, 9 months
Solaris 10 LDAP Password issue
by John Gee
Hello Community,
i have a problem with Solaris 10 LDAP password encryption to a OpenLDAP Server.
When setting a inital Password with ldapadd login works fine, after the user changed inital Password on a Solaris-Station with 'passwd -r ldap' Solaris commited it with 'password successfully changed for john'. But the user cant login with the new Password.
1) Setting initial Password with ldapadd (Password: 8ASdhXY!Xy)
version: 1
dn: uid=john,ou=people,ou=unix,o=kleinfeld,c=ch
userPassword: {MD5}khVDRrTSYMHjTw7V6VEZwg==
2) User Login and change password with 'passwd -r ldap' (Password: 9DnxSF!dKS)
version: 1
dn: uid=john,ou=people,ou=unix,o=kleinfeld,c=ch
userPassword: {crypt}0vUAwIdPR4X2E
Has someone a idea whats going wrong?
I cant track down this problem.
--( nsswitch.conf )---
passwd: compat
passwd_compat: files ldap
group: files ldap
shadow_compat files ldap
--( pam.conf )---
login auth sufficient pam_unix_auth.so.1
login auth required pam_ldap.so.1
rlogin auth sufficient pam_unix_auth.so.1
rlogin auth required pam_ldap.so.1
other auth sufficient pam_unix_auth.so.1
other auth required pam_ldap.so.1
--( /etc/security/policy.conf )---
CRYPT_ALGORITHMS_ALLOW=1,2a,md5
CRYPT_DEFAULT=__unix__
Regards
John
14 years, 9 months
