mark jayson wrote:
> Hello,
>
> I've created a program that writes data to an lmdb database using python bindings. I've written about 300gb worth of data but when I was re-reading the database
> by running my script again, it crashes immediately. Did a backtrace in gdb and this is what I got:
>
> Really have no clue what's going on here. I suspect that the server has rebooted (or was rebooted by someone while I was running the script but I thought lmdb
> is crash-proof.

It is. Extensively tested so, but storage devices can still lie about whether they successfully wrote data.

> Do you have any idea how to best troubleshoot this? The database is worth 300gb+ so it's not easy to share it, not to mention it contains sensitive information.

Try using mdb_dump to see if it will backup the contents without crashing.

Also, when asking for help you should specify exactly what version of liblmdb you used.

The backtrace is a bit odd because normally just opening an environment shouldn't require committing any write txns.
>
> Starting program: /usr/bin/python3 read_events.py -d jul012022-23/
> [Thread debugging using libthread_db enabled]
> Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
> build/lib/mdb.c:3274: Assertion 'len >= 0 && id <= env->me_pglast' failed in mdb_freelist_save()
>
> Program received signal SIGABRT, Aborted.
> __pthread_kill_implementation (no_tid=0, signo=6, threadid=140737350283264) at ./nptl/pthread_kill.c:44
> 44./nptl/pthread_kill.c: No such file or directory.
> (gdb) backtrace
> #0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=140737350283264) at ./nptl/pthread_kill.c:44
> #1  __pthread_kill_internal (signo=6, threadid=140737350283264) at ./nptl/pthread_kill.c:78
> #2  __GI___pthread_kill (threadid=140737350283264, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
> #3  0x00007ffff7c96476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
> #4  0x00007ffff7c7c7f3 in __GI_abort () at ./stdlib/abort.c:79
> #5  0x00007ffff62fb3e2 in mdb_assert_fail (env=0x5555565b49f0, expr_txt=expr_txt@entry=0x7ffff62fe308 "len >= 0 && id <= env->me_pglast", 
>     func=func@entry=0x7ffff62fe930 <__func__.13> "mdb_freelist_save", line=line@entry=3274, file=0x7ffff62fe010 "build/lib/mdb.c") at build/lib/mdb.c:1545
> #6  0x00007ffff62f10df in mdb_freelist_save (txn=0x555556752950) at build/lib/mdb.c:3274
> #7  mdb_txn_commit (txn=0x555556752950) at build/lib/mdb.c:3646
> #8  0x00007ffff62f369b in txn_db_from_name (env=env@entry=0x7ffff61c8090, name=<optimized out>, flags=262144) at lmdb/cpython.c:1017
> #9  0x00007ffff62f6e1d in env_open_db (self=0x7ffff61c8090, args=<optimized out>, kwds=<optimized out>) at lmdb/cpython.c:1665
> #10 0x00005555556b23a9 in ?? ()
> #11 0x0000555555699c14 in _PyEval_EvalFrameDefault ()
> #12 0x0000555555696176 in ?? ()
> #13 0x000055555578bc56 in PyEval_EvalCode ()
> #14 0x00005555557b8b18 in ?? ()
> #15 0x00005555557b196b in ?? ()
> #16 0x00005555557b8865 in ?? ()
> #17 0x00005555557b7d48 in _PyRun_SimpleFileObject ()
> #18 0x00005555557b7a43 in _PyRun_AnyFileObject ()
> #19 0x00005555557a8c3e in Py_RunMain ()
> #20 0x000055555577ebcd in Py_BytesMain ()
> #21 0x00007ffff7c7dd90 in __libc_start_call_main (main=main@entry=0x55555577eb90, argc=argc@entry=14, argv=argv@entry=0x7fffffffe378) at
> ../sysdeps/nptl/libc_start_call_main.h:58
> #22 0x00007ffff7c7de40 in __libc_start_main_impl (main=0x55555577eb90, argc=14, argv=0x7fffffffe378, init=<optimized out>, fini=<optimized out>,
> rtld_fini=<optimized out>, stack_end=0x7fffffffe368)
>     at ../csu/libc-start.c:392
> #23 0x000055555577eac5 in _start ()
> (gdb) client_loop: send disconnect: Broken pipe
>
>
> For what it's worth, this is how I've opened the database environment:
>
> dbenv = lmdb.open(db_dir, map_size=1099511627776, max_dbs=11, readahead=False) 
>
>


--
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/