opensource@gmx-topmail.de wrote:

We are using LMDB and are running a big number of tests on a CI server (Debian via Docker, Xenon CPU).

Are you running multiple Docker instances on the same database? AFAIK that won't work because of Docker's munging with POSIX namespaces.

The crash happens during opening the database. This is done a lot in the tests without problems (and on Windows this problem does not exist). However, during the CI process we get the crash quite reproducible (~80% crash rate when running the full CI test suite).

This is our basic code triggering the SIGSEGV (minus error checking for simplicity):

mdb_env_create(&dbEnv_);
mdb_env_set_maxdbs(dbEnv_, 8);
mdb_env_set_mapsize(dbEnv_, maxDbSizeInKByte * 1024);
mdb_env_open(dbEnv_, directory, 0, fileMode);
MDB_txn* txn;
mdb_txn_begin(dbEnv_, NULL, 0, &txn);

It crashes inside mdb_txn_begin with this: SIGSEGV (0xb) at pc=0x00007fd2607e4b58, pid=23603, tid=0x00007fd228633700 C [libpthread.so.0+0x5b58] __pthread_mutex_lock_full+0x1a8

I digged into it a bit, and could tracd it down to this LOCK_MUTEX call in mdb_txn_renew0:

    /* Not yet touching txn == env->me_txn0, it may be active */
    if (ti) {
        if (LOCK_MUTEX(rc, env, env->me_wmutex))
            return rc;

I also logged some data about the env and the mutex (which looks normal however): PID: 23603 ENV: 0x7fd1e03e9ab0 TXN: 0x7fd22811b000 COUNT: 0 KIND: 144 LOCK: 0 OWNER: 0

Any idea what could go wrong here?

Thanks! Markus

Some additional crash info:

siginfo: si_signo: 11 (SIGSEGV), si_code: 1 (SEGV_MAPERR), si_addr: 0x00007fd22801a058

Registers: RAX=0x00007fd22801a060, RBX=0x00007fd22811b040, RCX=0x0000000000000000, RDX=0x0000000000000000 RSP=0x00007fd22862ff50, RBP=0x0000000000005c4f, RSI=0x0000000000000080, RDI=0x0000000000005c4f R8 =0x0000000000000080, R9 =0x0000000000000000, R10=0x00007fd22811b060, R11=0x0000000000000000 R12=0x0000000000000000, R13=0x00007fd22813b0f0, R14=0x00007fd2286309d0, R15=0x00007fd25848b000 RIP=0x00007fd2607e4b58, EFLAGS=0x0000000000010206, CSGSFS=0x0000000000000033, ERR=0x0000000000000006 TRAPNO=0x000000000000000e

Top of Stack: (sp=0x00007fd22862ff50) 0x00007fd22862ff50: 00007fd22813b0f0 00007fd228630040 0x00007fd22862ff60: 0000000000000000 00007fd22813b0f0 0x00007fd22862ff70: 00007fd2286309d0 00007fd20c2d05e4 0x00007fd22862ff80: 00007fd228630050 00007fd22862ff90 0x00007fd22862ff90: 0000000000001020 000000000000003c 0x00007fd22862ffa0: 0000000a00000023 0000000a00000000 0x00007fd22862ffb0: 00007fd228633700 00007fd25fee1c29 0x00007fd22862ffc0: 0000000000000000 00007fd25fe7a423 0x00007fd22862ffd0: 00007fd2286309d0 0000000900000000 0x00007fd22862ffe0: 0000000000000008 00007fd244000000 0x00007fd22862fff0: 00007fd22813b0f0 00007fd25fe79ae3 0x00007fd228630000: 0000000000000000 00000000286300c0 0x00007fd228630010: 0000000000000000 00007fd25fe7b255 0x00007fd228630020: 00007fd22811b000 00007fd1e03e9ab0 0x00007fd228630030: 00007fd1e08f3260 00007fd25fe6fb22 0x00007fd228630040: 00007fd2286300c0 00007fd20c2d0e8a 0x00007fd228630050: 00007fd2286300c0 00007fd2601ac2a0 0x00007fd228630060: 0000000000000020 00007fd244000000 0x00007fd228630070: 0000000800000000 00007fd25fe79ae3 0x00007fd228630080: 0000000000000020 00007fd228630660 0x00007fd228630090: 00007fd1e08f3260 00007fd2286301c8 0x00007fd2286300a0: 00000000601ac2a0 0000000000000000 0x00007fd2286300b0: 00007fd1e03e9ab0 00007fd25fe6fb22 0x00007fd2286300c0: 00007fd228630660 00007fd20c23a768 0x00007fd2286300d0: 0000000200f01b00 00007fd20000003b 0x00007fd2286300e0: 00007fd20c2eb3fd 00007fd24921da4c 0x00007fd2286300f0: 00000005cac01420 00007fd22814c210 0x00007fd228630100: 00007fd228630460 00007fd24921f99f 0x00007fd228630110: 00007fd2286301c8 0000002000000000 0x00007fd228630120: 0000000000000000 0000000000000000 0x00007fd228630130: 0000000000000000 0000000000000020 0x00007fd228630140: 0000000000000000 0000000000000000

Instructions: (pc=0x00007fd2607e4b58) 0x00007fd2607e4b38: 00 a9 00 00 00 40 0f 85 02 ff ff ff c7 43 04 01 0x00007fd2607e4b48: 00 00 00 64 48 8b 04 25 e0 02 00 00 48 83 e0 fe 0x00007fd2607e4b58: 4c 89 50 f8 64 48 8b 04 25 e0 02 00 00 48 89 43 0x00007fd2607e4b68: 20 64 48 8b 04 25 10 00 00 00 48 05 e0 02 00 00

Register to memory mapping:

RAX=0x00007fd22801a060 is an unknown value RBX=0x00007fd22811b040 is an unknown value RCX=0x0000000000000000 is an unknown value RDX=0x0000000000000000 is an unknown value RSP=0x00007fd22862ff50 is pointing into the stack for thread: 0x00007fd25848b000 RBP=0x0000000000005c4f is an unknown value RSI=0x0000000000000080 is an unknown value RDI=0x0000000000005c4f is an unknown value R8 =0x0000000000000080 is an unknown value R9 =0x0000000000000000 is an unknown value R10=0x00007fd22811b060 is an unknown value R11=0x0000000000000000 is an unknown value R12=0x0000000000000000 is an unknown value R13={method} {0x00007fd22813b0f0} 'nativeCreate' '(Ljava/lang/String;J[B)J' in 'MyDb' R14=0x00007fd2286309d0 is pointing into the stack for thread: 0x00007fd25848b000 R15=0x00007fd25848b000 is a thread