(ITS#7775) LMDB terminates Postfix daemon process without logfile record
by wietse@porcupine.org
Full_Name: Wietse Venema
Version: All LMDB versions
OS: Linux, BSD
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (70.104.130.26)
Subject: LMDB terminates Postfix daemon process without logfile record
The LMDB library uses assert() which will abort() the program after
writing a message to stderr. In the case of the Postfix MTA, the
symptom is a that a daemon process falls out of the sky without
logfile record.
- This frustrates sysadmins (no logfile message).
- This frustrates my efforts to make Postfix automatically recover
from non-catastrophic error. In many cases the database is only
a cache, a mere optimization. I want the option to just blow it
away when LMDB has a problem with it.
- Library modules should not abort programs after error. They should
return an error result such as EINVAL, or use some other strategy
that leaves control with the application.
- Writing diagnostics to stderr is not useful for daemon programs
in the general case.
To fix this problem I suggest that LMDB provide way to notify the
application that it was unable to complete a request, so that the
application can make proper arrangements.
A few alternatives:
- When a request fails, return a distinct error code such as
MDB_PANIC, and allow the application to look up the text for the
error.
- Invoke an application call-back function with the error code and
problem description text. If the error code is MDB_PANIC then
the application knows that it needs to make final arrangements.
The second option preserves 100% backwards compatibility. The first
option may cause functions to return a result that they didn't
return previously.
Wietse
9 years, 8 months
Re: (ITS#7773) slapo-constraint falsely restricts modrdn with count 1
by pierangelo.masarati@polimi.it
On 12/28/2013 01:52 PM, michael(a)stroeder.com wrote:
> Full_Name:
> Version: 2.4.38
> OS:
> URL:
> Submission from: (NULL) (212.227.35.93)
>
>
> Sometimes it's useful to restrict an existing attribute type declared
> multi-valued to a single value with slapo-constraint to avoid modifying the
> standard schema.
>
> Example:
>
> constraint_attribute
> uid
> count 1
> restrict="ldap:///ou=example??sub?(objectClass=account)"
>
> Unfortunately when using uid in the RDN one cannot rename the entry anymore even
> when deleteoldrdn is set to true.
>
> slapo-constraint should handle this case exactly like SINGLE-VALUE RDN
> attributes are handled.
Should be fixed in git master. Nothing specific to rename; rather,
slapo-constraint was not handling internal modifications like
softadd/softdel.
p.
--
Pierangelo Masarati
Associate Professor
Dipartimento di Scienze e Tecnologie Aerospaziali
Politecnico di Milano
9 years, 8 months
(ITS#7773) slapo-constraint falsely restricts modrdn with count 1
by michael@stroeder.com
Full_Name:
Version: 2.4.38
OS:
URL:
Submission from: (NULL) (212.227.35.93)
Sometimes it's useful to restrict an existing attribute type declared
multi-valued to a single value with slapo-constraint to avoid modifying the
standard schema.
Example:
constraint_attribute
uid
count 1
restrict="ldap:///ou=example??sub?(objectClass=account)"
Unfortunately when using uid in the RDN one cannot rename the entry anymore even
when deleteoldrdn is set to true.
slapo-constraint should handle this case exactly like SINGLE-VALUE RDN
attributes are handled.
9 years, 8 months
Re: (ITS#7665) LMDB: Documentation issue or mistake: MDB_NOMETASYNC vs MDB_CREATE
by h.b.furuseth@usit.uio.no
Salvador Ortiz writes:
> The documentation states that the Environment's flags should not overlap with
> the Database's ones, but MDB_NOMETASYNC and MDB_CREATE share the same value.
That doc is a current implementation detail for liblmdb developers
adding new flags. I should not have put it in ldmb.h. I'll drop
it and add an #if/#error check in mdb.c instead.
--
Hallvard
9 years, 8 months
Re: (ITS#7682) LMDB: [PATCH] mdb_env_copy should retry open() if O_DIRECT fails
by hyc@symas.com
h.b.furuseth(a)usit.uio.no wrote:
> This was fixed in LMDB 0.9.9, except: If both O_DIRECT and
> F_NOCACHE are #defined, both are used. Looks a bit strange.
> Maybe just one should be used in that case. (Which?) Also I'm
> still wondering if env_open needs to fail if F_NOCACHE fails.
>
> Anyway, maybe it should be something like this. The "else {}"
> is in case O_DIRECT is an unsupported dummy definiton:
Isn't that in the wrong order then? What you suggest here will use O_DIRECT
even if it's a dummy.
I don't see any reason to fail the operation. They just want to make a
copy/backup; even if the optimization of avoiding pollution of the buffer
cache can't be done, they still want the copy to happen.
> #ifdef O_DIRECT
> if ((rc = fcntl(newfd, F_GETFL)) != -1)
> (void) fcntl(newfd, F_SETFL, rc | O_DIRECT);
> else
> #endif
> {
> #ifdef F_NOCACHE /* __APPLE__ */
> (void) fcntl(newfd, F_NOCACHE, 1);
> #endif
> }
>
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
9 years, 9 months
(ITS#7772) liblmdb sub-page growth when adding existing data
by h.b.furuseth@usit.uio.no
Full_Name: Hallvard B Furuseth
Version: mdb.master, 4c8f57615c5ca7b014c038e59c1045182e74f5ad
OS: Linux x86_64
URL: ftp://ftp.openldap.org/incoming/Hallvard-Furuseth-growth-131223.c
Submission from: (NULL) (81.191.45.35)
Submitted by: hallvard
A non-LEAF2 sub-page grows if you put() a data item which already
exists. The space remains unused while the sub-page exists.
(A LEAF2 page grows reusable space, and only if it is full.)
Demo program enclosed. Edit mdb.c to set mdb_debug=1 and build
with -DMDB_DEBUG. Watch the 1st DPRINTF() in mdb_node_add():
bash$ ./a.out 2>&1 | perl -lne '/add to (.*) key size/ && print $1'
leaf page 2 index 0, data size 404 # node with 1st item
leaf page 2 index 0, data size 840 # convert to sub-page for 2nd item
leaf sub-page 2 index 0, data size 0
leaf sub-page 2 index 1, data size 0
leaf page 2 index 0, data size 1250 # space for re-adding 2nd item
leaf page 2 index 0, data size 1660 # space for re-adding 2nd item
leaf page 2 index 0, data size 48 # ...too big, moved to sub-DB.
Fix: lmdb could position cursor+xcursor fully before spill/touch,
with an MDB_GET_BOTH variant which returns different codes for "no
such key" and "no such data". And position it before MDB_MULTIPLE
loops up. The current comparisons to see if the data item exists,
go away. Maybe mdb_cursor_put() gets split in several functions,
since much of the work in the recursive put()s will be unneeded.
9 years, 9 months
(ITS#7771) liblmdb cursor issues
by h.b.furuseth@usit.uio.no
Full_Name: Hallvard B Furuseth
Version: mdb.master, 4c8f57615c5ca7b014c038e59c1045182e74f5ad
OS: Linux x86_64
URL: ftp://ftp.openldap.org/incoming/Hallvard-Furuseth-131222.c
Submission from: (NULL) (2001:700:100:556::233)
Submitted by: hallvard
mdb_cursor_put() doc bug in lmdb.h:
"If the function fails for any reason, the state of the
cursor will be unchanged."
That's quite untrue. To do that, the function must backup
the cursor before moving it. (What is a "failure" anyway
- are MDB_KEYEXIST and MDB_NOTFOUND failures?)
Cursor tracking issues with MDB_DUPSORT:
Cursor tracking sometimes ignores xcursors referring to
sub-pages, yet code which uses an xcursor does not always
fix that first. The enclosed program has several crashes.
I.e. lots more code needs something like
if ((leaf->mn_flags & (F_DUPDATA|F_SUBDATA)) == F_DUPDATA)
m2->mc_xcursor->mx_cursor.mc_pg[0] = NODEDATA(leaf);
before using an xcursor. I tried that everywhere before
using xcursors: That helped, but it was not enough. E.g.
after mdb_cursor_del() calls mdb_node_shrink(), it tracks
cursors in the same sub-page but not nearby sub-pages.
Also the fix suggested above means readers do more work
that writers did not. Maybe instead all code which
modified a database, should fix or invalidate that DB's
cursors before returning (even at failure).
9 years, 9 months
Re: (ITS#7682) LMDB: [PATCH] mdb_env_copy should retry open() if O_DIRECT fails
by h.b.furuseth@usit.uio.no
This was fixed in LMDB 0.9.9, except: If both O_DIRECT and
F_NOCACHE are #defined, both are used. Looks a bit strange.
Maybe just one should be used in that case. (Which?) Also I'm
still wondering if env_open needs to fail if F_NOCACHE fails.
Anyway, maybe it should be something like this. The "else {}"
is in case O_DIRECT is an unsupported dummy definiton:
#ifdef O_DIRECT
if ((rc = fcntl(newfd, F_GETFL)) != -1)
(void) fcntl(newfd, F_SETFL, rc | O_DIRECT);
else
#endif
{
#ifdef F_NOCACHE /* __APPLE__ */
(void) fcntl(newfd, F_NOCACHE, 1);
#endif
}
--
Hallvard
9 years, 9 months