Hi Quanah,
We see the same issue with 2.4.58 (compiled from source).
The most common operations are ('attr=authTimestamp', 3890) ('attr=pwdFailureTime', 713) ('attr=userPassword', 29) ('attr=memberuid', 22) ('attr=accountExpiryDate', 21) ('attr=pwdAccountLockedTime pwdFailureTime pwdFailureTime pwdAccountLockedTime', 20) ('attr=pwdAccountLockedTime pwdFailureTime pwdFailureTime', 16) ('dn="cn=DISABLED_USERS,ou=Groups,dc=org,dc=com"', 15) ('attr=member', 15)
I am able to debug that mod operations are fast on a fresh mdb, but after a certain number of operations the mdb size is going from 300 MB to 10 GB. And, the time spent per operation is like 1-2 seconds for every mod, and 5-6 seconds when there are a large number of mods. We also use a group for disabling users, when running mod to add users to that group we are seeing all ldap binds and other mods slowing as well. We have sortvals enabled for all groups.
We have the following indexes:
olcDbIndex: objectClass eq olcDbIndex: entryCSN eq olcDbIndex: entryUUID eq olcDbIndex: uid eq olcDbIndex: uidNumber eq olcDbIndex: uniqueMember eq olcDbIndex: gidNumber eq olcDbIndex: displayName eq olcDbIndex: cn eq olcDbIndex: sn eq olcDbIndex: member eq
Regards Bhanush
On Mon, Jan 9, 2023 at 9:11 PM Quanah Gibson-Mount quanah@fast-mail.org wrote:
--On Friday, December 23, 2022 9:06 PM +0530 Bhanush Mehta bhanush.mehta@flipkart.com wrote:
Hi All,
We are seeing very slow MOD operations on our ldap (250 MB data dump), while using mdb (data.mdb is 6.4 Gb). The average MOD operation is going to 8-9 seconds. We are seeing 1k disk ops and 6-7MB/s writes. The disk is 4096 IOPS Sata SSD, we have seen write speed to be 126 MB/s generally.
The slapd version is slapd (May 14 2022 18:35:44).
The above is not a valid version. At a guess you're using an OpenLDAP 2.4 build from Debian, but you'd have to check your package manager to confirm.
It would be helpful to know what the mod operations are doing (I.e., adding new attribute values to a multi-valued attribute that has a lot of values already, etc)
--Quanah
--On Monday, January 16, 2023 10:47 AM +0530 Bhanush Mehta bhanush.mehta@flipkart.com wrote:
Hi Quanah,
We see the same issue with 2.4.58 (compiled from source).
I am able to debug that mod operations are fast on a fresh mdb, but after a certain number of operations the mdb size is going from 300 MB to 10 GB.
Hello,
Are you sure that the database is actually 10GB in size? Keep in mind that back-mdb uses a sparse file, so the filesystem will often show the database as being the "size" of the MDB maxsize value, regardless of what the actual DB size is.
You can use du -h "file.mdb" to see the actual filesize.
For example, in a test system, I have:
maxsize 1073741824
which is 1GB.
If we look at the output of ls:
total 2704 -rw------- 1 root root 1073741824 May 17 2022 data.mdb
but the actual size on disk is:
du -h data.mdb 2.7M data.mdb
Regards, Quanah
openldap-technical@openldap.org