Hello,
Yes, the member attribute is rewritten every 10 minutes. We are going to modify the batch so that it updates the attribute in differential mode and not in full mode. This will slow down the increase of the base before switching to OpenLDAP 2.5.
Jean-Charles
-----Message d'origine----- De : Ulrich Windl Ulrich.Windl@rz.uni-regensburg.de Envoyé : jeudi 25 mars 2021 07:30 À : Jean-Charles ROGEZ jean-charles.rogez@csnovidys.com; openldap-technical@openldap.org Cc : Cédric MORELLE cedric.morelle@csnovidys.com; Loïc NERTOMB loic.nertomb@csnovidys.com; Pierre@hypatia.openldap.org Objet : Antw: [EXT] MDB_MAP_FULL with plenty of free pages
Jean-Charles ROGEZ jean-charles.rogez@csnovidys.com schrieb am 24.03.2021
um 15:06 in Nachricht PR2P264MB0911634B6ED936442371A372D1639@PR2P264MB0911.FRAP264.PROD.OUTLOOK.COM:
Hello,
We use OpenLDAP 2.4.57 under RHEL8 with a configuration with 2 directories in MM and 2 replicas. We only write on one of the masters. The size of the LMDB database grows following writes of members of large groups (2000 members) by a batch which runs every 10 minutes.
Does that mean you re-write your 2000 members every 10 minutes, or do just just update (add or drop a few) the members?
The base grows regularly and sometimes undergoes significant jumps until it
reaches its maximum size. The lmdb_stat ‑ef command indicates that very few pages are used and everything else is free pages. The used pages are stable.
[cid:image001.jpg@01D720BD.58844450]
And yet, it is no longer possible to write in the directory. Sometimes it's
on the master directory, sometimes on replicas where syncrepl fails.
2021‑03‑24T11: 52: 59.978157 + 01: 00 int‑ohz‑infra1 slapd debug local4
25076 ‑
mdb_id2entry_put: mdb_put failed: MDB_MAP_FULL: Environment mapsize limit reached (‑30792) "uid = us‑00000301, ou = users, dc = bst, dc = ocn, dc = infra, dc = ftgroup " 2021‑03‑24T11: 52: 59.978184 + 01: 00 int‑ohz‑infra1 slapd debug local4
25076 ‑
syncrepl_null_callback: error code 0x50 2021‑03‑24T11: 52: 59.978202 + 01: 00 int‑ohz‑infra1 slapd debug local4
25076 ‑
syncrepl_entry: rid = 001 be_modify failed (80) 2021‑03‑24T11: 52: 59.978780 + 01: 00 int‑ohz‑infra1 slapd debug local4
25076 ‑
do_syncrepl: rid = 001 rc 80 retrying
There is no transaction in progress: mdb_stat ‑r /var/lib/ldap/data/ Reader Table Status pid thread txnid 25076 7fa0e42c5480 ‑ 25076 7fa0977fe700 ‑ 25076 7fa097fff700 ‑ 25076 7fa094cfa700 ‑ 25076 7fa086ffe700 ‑ 25076 7fa0857fb700 ‑ 25076 7fa0867fd700 ‑ 25076 7fa085ffc700 ‑ 25076 7fa074ffc700 ‑
The batch executes a lot of transactions < 2s. Restarting slapd does not resolve the problem. If we compact the database with mdb_copy ‑c, it only makes a few MB and it works again. The problem no longer appears without large groups.
Why are free pages not used? Wouldn't there be a problem with writing many multi‑valued attributes?
Thank you for your help !
Jean‑Charles Rogez [https://marketing.csnovidys.com/Logos/banniere_mail.png]
[twitter icon]https://twitter.com/csnovidys
[linkedin icon]https://www.linkedin.com/company/novidy's
Jean‑Charles ROGEZ
Architecte Système | INTEGRATION SYSTEMES | PLESSIS Standard : +33180848010tel:+33180848010 Ligne directe : tel:r Mobile : tel: Email : jean‑charles.rogez@csnovidys.commailto:jean‑charles.rogez@csnovidys.c om
[https://marketing.csnovidys.com/Logos/banniere_mail.png]
[twitter icon]https://twitter.com/csnovidys
[linkedin icon]https://www.linkedin.com/company/novidy's
Jean-Charles ROGEZ
Architecte Système | INTEGRATION SYSTEMES | PLESSIS Standard : +33180848010tel:+33180848010 Ligne directe : tel:r Mobile : tel: Email : jean-charles.rogez@csnovidys.commailto:jean-charles.rogez@csnovidys.com