matthew@wellquite.org wrote:

Full_Name: Matthew Sackman Version: LMDB 0.9.18 OS: Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (109.149.185.170)

Linux koba 4.4.0-1-amd64 #1 SMP Debian 4.4.6-1 (2016-03-17) x86_64 GNU/Linux LMDB 0.9.18 (compiled locally)

I'm currently writing a program that iterates through multiple LMDB databases in order to check differences between them. On each database I have one long-running read-only txn with a cursor, and I'm walking through all the databases at the same time.

I'm seeing some very odd behaviour. For example, cursor.Get with FIRST is not always returning the first item in the table (which I happen to know exactly what it should be). The behaviour is non-deterministic - different runs produce different results.

I've searched through the docs and I can't find any statements regarding opening multiple dbs from the same process. Is this supported?

I'm writing this in Go, using my own bindings. I'm pretty certain that I'm doing all the threading correctly and I certainly get this same issue regardless of whether I'm using NOTLS or not (with my bindings, it shouldn't be necessary because I'm locking go-routines to os-threads), plus these are the same bindings that I've used throughout GoshawkDB and I've not come across this problem until now.

I can guarantee that each DB that I open will be opened in a different OS thread and that no single OS thread will ever access different DBs.

Is this entire task read-only, nobody is modifying any of the DBs while these readers are iterating thru the DBs?

I've only used Go very briefly. I would be quite suspicious of that goroutine behavior but don't have any insight there.

You might consider compiling LMDB with MDB_DEBUG defined, enabling debug output, and then watching the cursor activity. Of course, the act of writing debug messages will impact thread scheduling, so it may not reveal what's going on.