February 2018 - openldap-bugs

(ITS#8804) repeated crashes using chaning

by klaus.jungbauer＠is4it.de

Full_Name: Klaus Jungbauer Version: 2.4.45 OS: RHEL 7.4 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (82.135.30.210) I tried to use slapd within a productive environment where the slapd uses its mdb to resolve technical accounts and netgroups while real user accounts have to be resolved at AD. To do this a proxy was configured with chaining to resolve referrals. This configuration leads to very often crashes within an interval from 3 to 60 seconds (!): slapd: chain.c:226: ldap_chain_uri_cmp: Assertion `!((&li2->li_bvuri[ 0 ])->bv_val == ((void *)0))' failed. Using gdb i can see the crashes were caused by chain.c:226 or 227 coming from chain.c:831. Going deeper i saw that the assert checking li_bvuri uses a pointer at li_bvuri.bv_val which was an allocated mem at insertion time of node to tree which gets freed at chain:893. This points to a strange memory (mis)management. As a quick and dirty hack i changed chain:893 and chain.c:620 from 'ldap_memfree( li.li_uri );' to 'if ( temporary ) ldap_memfree( li.li_uri );' and inserted 'if (li2->li_bvuri[0].bv_len == 0 || li2->li_bvuri[0].bv_val == NULL) return -1;' below chain.c:225. With this changes slapd runs about half a day. Adding a Debug statement like 'Debug( 256, "#### uri_cmp: li2->li_bvuri[0].{bv_len=%d, bv_val='%s'}\n", li2->li_bvuri[0].bv_len, li2->li_bvuri[0].bv_val, 0 );' at end of chain.c:225 shows strange outputs like: 5a82a1de #### uri_cmp: li2->li_bvuri[0].{bv_len=-967084369, bv_val='(null)'} I expect another place in code adds nodes to tree too - with unresolved pointers. cn=config was created by slaptest from slapd.conf. I uploaded slapd.conf as slapd.conf_chain_provider__klaus.jungbauer(a)is4it.de to your ftp server. See RedHat Ticket #02027931 too. What do you need on additional files like config, logs, ...? Kind regards Klaus Jungbauer

5 years, 7 months

1
0
0 / 0

Re: (ITS#8486) Syncprov sessionlog is inefficient, kills perf

by quanah＠symas.com

Using the patch in the openldap-scratch repository, this is no longer a problem. Test case: 2 masters with identical DBs Approximately 4 million user DB Modify every user on master 1 (replicated to master 2) Stop master 2, reload it with the original DB Let time pass so the accesslog DB empties out a record of all changes Start master 2 master 2 will then start querying the sesison log to find any deleted entries while this is ongoing, perform 4 concurrent queries against master 1 (in this case, a search against the entire DB with filter "(uid=*)" requesting 1.1 All four searches execute w/o issue once this patch is in place. Master 2 logs: Feb 10 19:09:54 anvil2 systemd[1]: Started LSB: Symas OpenLDAP. Feb 10 19:09:54 anvil2 slapd[5580]: do_syncrep2: rid=100 LDAP_RES_SEARCH_RESULT Feb 10 19:09:54 anvil2 slapd[5580]: do_syncrep2: rid=100 delta-sync lost sync, switching to REFRESH Feb 10 19:09:54 anvil2 slapd[5580]: do_syncrep2: rid=100 (4096) Content Sync Refresh Required --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

5 years, 7 months

1
0
0 / 0

(ITS#8803) [PATCH] Add -L option for using external locking

by nirsof＠gmail.com

Full_Name: Nir Soffer Version: master OS: Fedora 27 URL: https://github.com/nirs/lmdb/commit/e9ffc44797f9884590c480d0f160d8f745ee8ff9 Submission from: (NULL) (217.132.180.29) A user may prefer to provide external locking using the MDB_NOLOCK when using the lmdb library. To use the mdb_dump, mdb_load, and mdb_stat with the same locking, we need an option to disable the library internal locking when running these tools. The attached patch file is derived from lmdb Software. All of the modifications to lmdb represented in the following patch(es) were developed by Nir Soffer nirsof(a)gmail.com. I have not assigned rights and/or interest in this work to any party. The attached modifications to OpenLDAP Software are subject to the following notice: Copyright 2018 Nir Soffer Redistribution and use in source and binary forms, with or without modification, are permitted only as authorized by the OpenLDAP Public License.

5 years, 7 months

1
0
0 / 0

Re: (ITS#6268) multi-master sync replication ldap_add error code 68 bug

by quanah＠symas.com

Hi Barry, I had a few questions to ask in relation to this ITS: a) Can you reproduce this problem with current RE24? and b) Could you provide copies of your shell scripts so I can create a generalized regression test case for this? (And, if this issue is not resolved as of yet, it will help with getting it fixed). Thanks! Regards, Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

5 years, 7 months

1
0
0 / 0

AW: (ITS#8777) [LMDB] Closing read cursor uses already freed transaction (MDB_VL32)

by markus＠greenrobot.de

It's not that trivial. We recycle cursors because they *might* be reused later. At the point when the txn is done it is unknown whether the cursor will be used in the future.

5 years, 7 months

1
0
0 / 0

Re: (ITS#8777) [LMDB] Closing read cursor uses already freed transaction (MDB_VL32)

by hyc＠symas.com

Howard Chu wrote: > Markus Junginger wrote: >>> There was never a bug here. The doc is quite explicit >>> http://www.lmdb.tech/doc/group__mdb.html#ga73a5938ae4c3239ee11efa07eb22b882 >>> >>> After txn_abort the only valid call on a still-open cursor is >>> cursor_renew, not cursor_close. >> >> However you want to call it, are you open to improve this workflow? >> >> API-wise, it would be so much simpler just to free a cursor after it became >> obsolete. It's counter-intuitive to open a new txn, renew the cursor, close >> the cursor, and abort the txn - just to close a cursor. > > API-wise, it is much simpler to just close the cursor before closing its > transaction, if you have no need to reuse the cursor. To be explicit - a typical workflow, as outlined in the Getting Started guide: Open an env Open a txn inside the env Open a cursor inside the txn When finished: Close cursor inside txn Close txn inside env Close env The only reason the API allows a read cursor to exist outside of its creating read txn is to allow it to be reused in a later read txn. If you're not reusing it then you should just close it in the proper order. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

5 years, 7 months

1
0
0 / 0

Re: (ITS#8777) [LMDB] Closing read cursor uses already freed transaction (MDB_VL32)

by hyc＠symas.com

Markus Junginger wrote: >> There was never a bug here. The doc is quite explicit >> http://www.lmdb.tech/doc/group__mdb.html#ga73a5938ae4c3239ee11efa07eb22b882 >> >> After txn_abort the only valid call on a still-open cursor is >> cursor_renew, not cursor_close. > > However you want to call it, are you open to improve this workflow? > > API-wise, it would be so much simpler just to free a cursor after it became > obsolete. It's counter-intuitive to open a new txn, renew the cursor, close > the cursor, and abort the txn - just to close a cursor. API-wise, it is much simpler to just close the cursor before closing its transaction, if you have no need to reuse the cursor. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

5 years, 7 months

1
0
0 / 0

Re: (ITS#8777) [LMDB] Closing read cursor uses already freed transaction (MDB_VL32)

by markus＠greenrobot.de

> There was never a bug here. The doc is quite explicit > http://www.lmdb.tech/doc/group__mdb.html#ga73a5938ae4c3239ee11efa07eb22b882 > > After txn_abort the only valid call on a still-open cursor is > cursor_renew, not cursor_close. However you want to call it, are you open to improve this workflow? API-wise, it would be so much simpler just to free a cursor after it became obsolete. It's counter-intuitive to open a new txn, renew the cursor, close the cursor, and abort the txn - just to close a cursor. And without MDB_VL32, plain cursor closing after txn abort seems to work just fine (I did not consider "closing" as "using" when I read the docs). One could say that the inconsistent behavior is a result of the complexity MDB_VL32 introduces. If you don't want to change the current behavior, it would still be nice for future developers to null the obsolete txn ptr in the cursor on txn abort to make the unwanted close call crash always, not just occasionally.

5 years, 7 months

1
0
0 / 0

Re: AW: AW: (ITS#8777) [LMDB] Closing read cursor uses already freed transaction (MDB_VL32)

by hyc＠symas.com

Markus Junginger wrote: > I perfectly see that MDB_VL32 is the superior as a feature. On the other > hand I'm a bit worried that it may hold up LMDB development for use cases > you refer to as obsolete yourself. > > While there are workarounds for this particular bug (which we applied), There was never a bug here. The doc is quite explicit http://www.lmdb.tech/doc/group__mdb.html#ga73a5938ae4c3239ee11efa07eb22b882 After txn_abort the only valid call on a still-open cursor is cursor_renew, not cursor_close. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

5 years, 7 months

1
0
0 / 0

AW: AW: (ITS#8777) [LMDB] Closing read cursor uses already freed transaction (MDB_VL32)

by markus＠greenrobot.de

I perfectly see that MDB_VL32 is the superior as a feature. On the other hand I'm a bit worried that it may hold up LMDB development for use cases you refer to as obsolete yourself. While there are workarounds for this particular bug (which we applied), I wouldn't be surprised if MDB_VL32 still needs some stabilization after it will be officially released to get real exposure in the real world. Also, for future changes, the MDB_VL32 "fork" will have to be adjusted as well.

5 years, 7 months

1
0
0 / 0

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs February 2018