matthew(a)wellquite.org wrote:
Full_Name: Matthew Sackman
Version: LMDB 0.9.18
OS: Linux
URL:
ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (109.149.185.170)
Linux koba 4.4.0-1-amd64 #1 SMP Debian 4.4.6-1 (2016-03-17) x86_64 GNU/Linux
LMDB 0.9.18 (compiled locally)
I'm currently writing a program that iterates through multiple LMDB databases in
order to check differences between them. On each database I have one
long-running read-only txn with a cursor, and I'm walking through all the
databases at the same time.
I'm seeing some very odd behaviour. For example, cursor.Get with FIRST is not
always returning the first item in the table (which I happen to know exactly
what it should be). The behaviour is non-deterministic - different runs produce
different results.
I've searched through the docs and I can't find any statements regarding opening
multiple dbs from the same process. Is this supported?
I'm writing this in Go, using my own bindings. I'm pretty certain that I'm
doing
all the threading correctly and I certainly get this same issue regardless of
whether I'm using NOTLS or not (with my bindings, it shouldn't be necessary
because I'm locking go-routines to os-threads), plus these are the same bindings
that I've used throughout GoshawkDB and I've not come across this problem until
now.
I can guarantee that each DB that I open will be opened in a different OS thread
and that no single OS thread will ever access different DBs.
Is this entire task read-only, nobody is modifying any of the DBs while these
readers are iterating thru the DBs?
I've only used Go very briefly. I would be quite suspicious of that goroutine
behavior but don't have any insight there.
You might consider compiling LMDB with MDB_DEBUG defined, enabling debug
output, and then watching the cursor activity. Of course, the act of writing
debug messages will impact thread scheduling, so it may not reveal what's
going on.
--
-- Howard Chu
CTO, Symas Corp.
http://www.symas.com
Director, Highland Sun
http://highlandsun.com/hyc/
Chief Architect, OpenLDAP
http://www.openldap.org/project/