https://bugs.openldap.org/show_bug.cgi?id=10607

Issue ID: 10607 Summary: slapd: use-after-free in config_back_delete/_modify/_modrdn with concurrent cn=config writes Product: OpenLDAP Version: 2.6.13 Hardware: x86_64 OS: Linux Status: UNCONFIRMED Keywords: needs_review Severity: normal Priority: --- Component: slapd Assignee: bugs@openldap.org Reporter: coding@markus-mazurczak.de Target Milestone: ---

Created attachment 1217 --> https://bugs.openldap.org/attachment.cgi?id=1217&action=edit Patch: re-resolve config entry after pause and write lock (against 2.6.13, -p1)

SUMMARY =======

back-config resolves the target CfEntryInfo *before* it pauses the server. Another cn=config write runs to completion inside that window and frees the entry, so the first operation continues on freed memory. With two clients writing to cn=config concurrently, slapd reliably crashes with SIGSEGV.

This is not a theoretical race. It is the blocking defect for using cn=config as the control plane of a multi-tenant directory service: every tenant we create or remove is a handful of cn=config writes, and two overlapping provisioning flows take the server down.

ANALYSIS ========

config_back_modify, config_back_modrdn and config_back_delete all do:

ce = config_find_base( cfb->cb_root, &op->o_req_ndn, &last, op ); ... cheap validation on ce ... slap_pause_server(); ldap_pvt_thread_rdwr_wlock( &cfb->cb_rwlock ); ... use ce ...

In openldap-2.6.13 (servers/slapd/bconfig.c):

function config_find_base slap_pause_server wlock config_back_modify 6364 6439 6446 config_back_modrdn 6525 6668 6675 config_back_delete 6791 6825 6828

Between the lookup and the pause, this thread holds no lock on the config tree. A second cn=config write that is already past its own pause completes, reaches

ce->ce_entry->e_private = NULL; entry_free( ce->ce_entry ); ch_free( ce ); /* bconfig.c:6946 */

and the first thread then dereferences the freed CfEntryInfo at bconfig.c:6835.

config_back_add is not affected: it resolves the entry after the lock, inside config_add_internal.

The read paths are correct -- config_back_search and config_back_compare take cb_rwlock for reading *before* calling config_find_base.

Note that the thread pool itself is fine. PAUSE_ARG(DO_PAUSE) subtracts ltp_pause, so a second pauser first goes GO_IDLE, waits out the running pause and only then pauses; assert(!pool->ltp_pause) in handle_pause() holds. The problem is purely the stale pointer taken before the pause.

REPRODUCTION ============

Build slapd with AddressSanitizer (glibc; ASan does not work under musl):

CFLAGS="-g -O1 -fno-omit-frame-pointer -fsanitize=address" \ LDFLAGS="-fsanitize=address" \ ./configure --enable-mdb=yes --enable-dynlist=yes --enable-spasswd \ --with-cyrus-sasl --with-tls=openssl --enable-crypt make depend && make

Start slapd with a cn=config database, then run four concurrent loops that each add an olcDatabase=mdb entry, add an olcOverlay=dynlist child, delete olcRootPW, and delete both again. The crash appears within seconds.

AddressSanitizer output:

==15==ERROR: AddressSanitizer: heap-use-after-free READ of size 4 at 0x5060000681a0 thread T11 #0 config_back_delete servers/slapd/bconfig.c:6835 #1 fe_op_delete servers/slapd/delete.c:181 #2 do_delete servers/slapd/delete.c:95 #3 connection_operation servers/slapd/connection.c:1137

freed by thread T9 here: #0 free #1 ber_memfree_x libraries/liblber/memory.c:152 #2 ch_free servers/slapd/ch_malloc.c:139 #3 config_back_delete servers/slapd/bconfig.c:6946

previously allocated by thread T2 here: #0 calloc #1 ber_memcalloc_x libraries/liblber/memory.c:283 #2 ch_calloc servers/slapd/ch_malloc.c:104 #3 config_add_internal servers/slapd/bconfig.c:5615 #4 config_back_add servers/slapd/bconfig.c:5849

Without the sanitizer, the stock 2.6.13 build dies with SIGSEGV after roughly 175 add/delete cycles under the same load. With the patch below applied, the same load ran 14402 cycles in 150 seconds with no crash and a clean ASan run.

PATCH =====

Resolve the entry again once the pause and the write lock are held, and answer noSuchObject if it is gone. The pre-pause lookup is kept, so a request that is going to fail anyway still does not pause the whole server.

The patch is attached as 0001-cn-config-resolve-entry-after-pause.patch and applies with -p1 to openldap-2.6.13.

For config_back_modrdn the re-resolved entry's ce_type is compared with the validated one; ixold is derived from op->o_req_ndn, so it stays consistent.

I am happy to adjust the approach -- taking cb_rwlock for reading around the pre-pause validation as well would close the (much narrower) window in which the early checks themselves touch a freed entry.