(ITS#8174) mdb_drop(,MAIN_DBI,) leaks pages of named DBs
by h.b.furuseth＠usit.uio.no
Full_Name: Hallvard B Furuseth
Version: LMDB_0.9.14
OS:
URL:
Submission from: (NULL) (81.191.45.5)
Submitted by: hallvard
mdb_drop(,MAIN_DBI,) leaks pages of named DBs
It does not drop the pages of named DBs, but it does
drop the mainDB pages which refer to these DBs.
3 possible fixes + 1 hybrid (from old IRC chats):
(1) Fail if there are named DBs. The minimal fix.
(2) Reset both metapages. Reject new write txns while old
readers remain. Needs MDB_DATA_{VERSION/FORMAT} > 1.
(3) Recursive: Drop named DBs too.
(4) Try (3), but fail when that gets complicated.
What to do with open DBIs of named databases:
(A) Fail if there are any.
(B) Close them, like a recursive mdb_drop() would do.
The user must likely "stop the world" first.
(C) Keep them open but useless, as when another process
dropped the DB. All the user can do is close them.
API:
I think all variants but "fail" can be dangerous and/or
possibly surprising, and should all be options - even if
we only implement one choice. I.e. fail if there are
named DBs/DBIs and no options say what to do about it.
Would that be a new mdb_drop2() function? It's unfortunate
to change the meaning of mdb_drop() param 'del' > 1. OTOH
'del' > 1 has been an error since 0.9.7, so I'm OK with it.
Details:
(2) Reset both metapages:
mdb_drop() merely sets a txn flag so mdb_txn_commit() will
reset the metapages. The flag also prevents most operations,
like MDB_TXN_ERROR does. We can include it in MDB_TXN_BLOCKED
from my "mdb/safer" branch.
An MDB_meta flag then tells mdb_txn_renew0() to re-commit
the metapage if Commit failed to write the 2nd metapage,
and to reject writers when too old readers exist.
mdb_txn_renew0() needs a flags parameter, so it can react
to the new txn flags MDB_NO[META]SYNC. The caller cannot
set them before mdb_txn_renew0(), since txn == me_txn0.
(3) Recursive drop:
mdb_drop0() must locate and use the up-to-date mt_dbs[]
record when it sees a named DB record in the mainDB. Either:
- Search mt_dbxs[]. Then drop is O(n*n) unless we make
mt_dbxs[] a tree/hash first. OK for most users, but a
few people have said they use lots of DBs.
- Or mdb_drop() can first save dirty named DBs, like Commit
does. Wasteful, but less new code.
To make mdb_drop0() recursive: Factor out the DB-walking code
in mdb_env_cwalk(), at least for non-stale DBIs?
If searching mdb_dbxs[] for DBIs, it can also call mdb_drop().
Except it should not close any DBIs until everything succeeds.
(4) Recursive or fail:
Fail if there are dirty named DB (so mdb_drop0 need not searchA0Amt_dbxs), or
maybe non-DB_STALE ones (so it need not reset DB
records), or if called in a nested txn with DB_NEW DBIs or
stuff like that (because it makes my head hurt to try to
figure it out). Well, it looks much safer now than before we
had mt_dbiseqs[]. Maybe dirty DBs are the only problem.
(B) Close DBIs:
Simple, but makes drop(MAIN DBI) a "stop the world" operation.
(C) Keep DBIs open:
Reset mt_dbs[] contents and mark the DBIs as DB_STALE, then
attempts to use them will fail. Probably just a TODO item
until there is a general "drop DB without closing it" option.
Related issues:
There is no way to fully reset the main DB (including flags)
and close it. I think that should be an option too - someday.
Would need the main DBI to not be so special. E.g. currently
DB_NEW doesn't apply to it.
Until then, that's just something to think about with the API
for the rest of this stuff. The cleanest API would be if the
main DBI is just like other DBIs, instead of the default "do
not close it" being different.
IIRC it's possible for the user to overwrite/delete a named
DB as plain data and maybe use plain data as a named DB. I
have an old branch to catch that, which we were a bit nervious
about but should be fine.
8 years, 5 months
(ITS#8173) back-ldap slapd crash segfault
by adrian.raemy＠vtg.admin.ch
Full_Name: Adrian Raemy
Version: 2.4.40
OS: SLES 11 SP3
URL:
Submission from: (NULL) (193.5.216.100)
Hi,
The Openldap backend "ldap" slapd crashing under heavy load.
Tested with 2.4.26 and also 2.4.40.
We get an segfault:
Jun 18 00:00:36 serverxx1 kernel: [9079587.309374] slapd[6193]: segfault at 40
ip 00007f184cf6b2c6 sp 00007f1838ff85c0 error 4 in slapd[7f184cdda000+26c000]
Jun 18 09:40:12 serverxx1 kernel: [9114294.985888] slapd[10309]: segfault at 40
ip 00007f3b31f492c6 sp 00007f3b1f7f55c0 error 4 in slapd[7f3b31db8000+26c000]
Jun 18 12:38:29 serverxx1 kernel: [9124971.271300] slapd[15868]: segfault 4 40
ip 00007fc1555512c6 sp 00007fc141ffa5c0 error 4 in slapd[7fc1553c0000+26c000]
OpenLdap Proxy Server has:
Type = VM Guest
mem = 4GB
CPU = 2
Cores = 2
We don't get any other errors or warning before, CPU load and MEM etc all is
green and OK. Also files open etc is set unlimited.
Scenario: Linux/ Unix Clients and Applications via ssl start_tls and ssl over
Openldap Proxy using on the Proxy the backend "ldap" to proxing to Openldap
Master.
When the Proxy gets heavy load due to activity from clients and Applications,
the slapd crashes unexpected with the above slapd segfault.
This is only happen with the backend-ldap. The Openldap Master doesn't crash and
is stable at all. So it must be the backend ldap which has the problem.
We perform Search-, Modify and add/remove Operations. So the entire funtionality
will be used.
Only with lot of searches sent over proxy to master we couldn't reproduce the
segfault. It needs realy a heavy load generated (read/itite- modify operations
at all) i guess to reproduce this. We don't have this possibility in our very
restricted environment.
We can't provide a core dump of the slapd due to confidentially restrictions.
We would test with the backend "meta" but this backend doesn't support the
"extended operations" like "LDAP Password Modify Extended Operation". So we have
only two possibilities, the one that ldap backend will be fixed or the extended
operations at meta backend will be added.
Please, we need help/ a fix asap, e e authentication at all is compromised and
the slapd crashes often on the Openldap Proxy.
Thanks for your help
8 years, 5 months
(ITS#8172)
by michael＠stroeder.com
This is a cryptographically signed message in MIME format.
--------------ms000203090009040808090702
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
To clarify: The listener would only return the parameters with CONTINUE w=
hich
it wants to modify.
--------------ms000203090009040808090702
Content-Type: application/pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"
Content-Description: S/MIME Cryptographic Signature
MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgEFADCABgkqhkiG9w0BBwEAAKCC
DIEwggY0MIIEHKADAgECAgEeMA0GCSqGSIb3DQEBBQUAMH0xCzAJBgNVBAYTAklMMRYwFAYD
VQQKEw1TdGFydENvbSBMdGQuMSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBDZXJ0aWZpY2F0
ZSBTaWduaW5nMSkwJwYDVQQDEyBTdGFydENvbSBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0eTAe
Fw0wNzEwMjQyMTAxNTVaFw0xNzEwMjQyMTAxNTVaMIGMMQswCQYDVQQGEwJJTDEWMBQGA1UE
ChMNU3RhcnRDb20gTHRkLjErMCkGA1UECxMiU2VjdXJlIERpZ2l0YWwgQ2VydGlmaWNhdGUg
U2lnbmluZzE4MDYGA1UEAxMvU3RhcnRDb20gQ2xhc3MgMSBQcmltYXJ5IEludGVybWVkaWF0
ZSBDbGllbnQgQ0EwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDHCYPMzi3YGrEp
pC4Tq5a+ijKDjKaIQZZVR63UbxIP6uq/I0fhCu+cQhoUfE6ERKKnu8zPf1Jwuk0tsvVCk6U9
b+0UjM0dLep3ZdE1gblK/1FwYT5Pipsu2yOMluLqwvsuz9/9f1+1PKHG/FaR/wpbfuIqu54q
zHDYeqiUfsYzoVflR80DAC7hmJ+SmZnNTWyUGHJbBpA8Q89lGxahNvuryGaC/o2/ceD2uYDX
9U8Eg5DpIpGQdcbQeGarV04WgAUjjXX5r/2dabmtxWMZwhZna//jdiSyrrSMTGKkDiXm6/3/
4ebfeZuCYKzN2P8O2F/Xe2AC/Y7zeEsnR7FOp+uXAgMBAAGjggGtMIIBqTAPBgNVHRMBAf8E
BTADAQH/MA4GA1UdDwEB/wQEAwIBBjAdBgNVHQ4EFgQUU3Ltkpzg2ssBXHx+ljVO8tS4UYIw
HwYDVR0jBBgwFoAUTgvvGqRAW6UXaYcwyjRoQ9BBrvIwZgYIKwYBBQUHAQEEWjBYMCcGCCsG
AQUFBzABhhtodHRwOi8vb2NzcC5zdGFydHNzbC5jb20vY2EwLQYIKwYBBQUHMAKGIWh0dHA6
Ly93d3cuc3RhcnRzc2wuY29tL3Nmc2NhLmNydDBbBgNVHR8EVDBSMCegJaAjhiFodHRwOi8v
d3d3LnN0YXJ0c3NsLmNvbS9zZnNjYS5jcmwwJ6AloCOGIWh0dHA6Ly9jcmwuc3RhcnRzc2wu
Y29tL3Nmc2NhLmNybDCBgAYDVR0gBHkwdzB1BgsrBgEEAYG1NwECATBmMC4GCCsGAQUFBwIB
FiJodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9wb2xpY3kucGRmMDQGCCsGAQUFBwIBFihodHRw
Oi8vd3d3LnN0YXJ0c3NsLmNvbS9pbnRlcm1lZGlhdGUucGRmMA0GCSqGSIb3DQEBBQUAA4IC
AQAKgwh9eKssBly4Y4xerhy5I3dNoXHYfYa8PlVLL/qtXnkFgdtY1o95CfegFJTwqBBmf8py
TUnFsukDFUI22zF5bVHzuJ+GxhnSqN2sD1qetbYwBYK2iyYA5Pg7Er1A+hKMIzEzcduRkIMm
CeUTyMyikfbUFvIBivtvkR8ZFAk22BZy+pJfAoedO61HTz4qSfQoCRcLN5A0t4DkuVhTMXIz
uQ8CnykhExD6x4e6ebIbrjZLb7L+ocR0y4YjCl/Pd4MXU91y0vTipgr/O75CDUHDRHCCKBVm
z/Rzkc/b970MEeHt5LC3NiWTgBSvrLEuVzBKM586YoRD9Dy3OHQgWI270g+5MYA8GfgI/EPT
5G7xPbCDz+zjdH89PeR3U4So4lSXur6H6vp+m9TQXPF3a0LwZrp8MQ+Z77U1uL7TelWO5lAp
sbAonrqASfTpaprFVkL4nyGH+NHST2ZJPWIBk81i6Vw0ny0qZW2Niy/QvVNKbb43A43ny076
khXO7cNbBIRdJ/6qQNq9Bqb5C0Q5nEsFcj75oxQRqlKf6TcvGbjxkJh8BYtv9ePsXklAxtm8
J7GCUBthHSQgepbkOexhJ0wP8imUkyiPHQ0GvEnd83129fZjoEhdGwXV27ioRKbj/cIq7JRX
un0NbeY+UdMYu9jGfIpDLtUUGSgsg2zMGs5R4jCCBkUwggUtoAMCAQICAwtNUDANBgkqhkiG
9w0BAQUFADCBjDELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNV
BAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcxODA2BgNVBAMTL1N0YXJ0
Q29tIENsYXNzIDEgUHJpbWFyeSBJbnRlcm1lZGlhdGUgQ2xpZW50IENBMB4XDTE0MDkyMzIw
NDc1NVoXDTE1MDkyNDIyMDUxOFowXzEZMBcGA1UEDRMQNk0yWTdpOXpEdGU2alV3MDEdMBsG
A1UEAwwUbWljaGFlbEBzdHJvZWRlci5jb20xIzAhBgkqhkiG9w0BCQEWFG1pY2hhZWxAc3Ry
b2VkZXIuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAynIH09fU597v4fg4
uwqFJDPUxQHV9qxrfM/c3veStPyYl0JorqKHrD+hfCNZ+Toy65NN9f9vO26WnnLoDF+FE2Dk
Qi61iTgK5jZTr5dJG1WQkk698UNWO87lUWBRYYiUM7wC3ek2E0rzR99qIxE4dG9wws19F3KK
JvNN8tMTyoPw5Vkw4qx2SW54WEBMx7oCXBIZPPDD8ovled6vDVweVSaYXFUbkxbKJR87Msih
34Ba+cM5SAHQNQ11jaSJjFeqbQnXjf0nESnvo0XIckc9w240w3jcgu6b5SIQBI2vv5TaIv7v
KqNk0o+cGc9NnCw5/xD/OAB9Aj/qDca6NheHCQIDAQABo4IC2jCCAtYwCQYDVR0TBAIwADAL
BgNVHQ8EBAMCBLAwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMEMB0GA1UdDgQWBBTj
z1BhELZO3zEJfmZQ4I+c6NCXIjAfBgNVHSMEGDAWgBRTcu2SnODaywFcfH6WNU7y1LhRgjAf
BgNVHREEGDAWgRRtaWNoYWVsQHN0cm9lZGVyLmNvbTCCAUwGA1UdIASCAUMwggE/MIIBOwYL
KwYBBAGBtTcBAgMwggEqMC4GCCsGAQUFBwIBFiJodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9w
b2xpY3kucGRmMIH3BggrBgEFBQcCAjCB6jAnFiBTdGFydENvbSBDZXJ0aWZpY2F0aW9uIEF1
dGhvcml0eTADAgEBGoG+VGhpcyBjZXJ0aWZpY2F0ZSB3YXMgaXNzdWVkIGFjY29yZGluZyB0
byB0aGUgQ2xhc3MgMSBWYWxpZGF0aW9uIHJlcXVpcmVtZW50cyBvZiB0aGUgU3RhcnRDb20g
Q0EgcG9saWN5LCByZWxpYW5jZSBvbmx5IGZvciB0aGUgaW50ZW5kZWQgcHVycG9zZSBpbiBj
b21wbGlhbmNlIG9mIHRoZSByZWx5aW5nIHBhcnR5IG9ibGlnYXRpb25zLjA2BgNVHR8ELzAt
MCugKaAnhiVodHRwOi8vY3JsLnN0YXJ0c3NsLmNvbS9jcnR1MS1jcmwuY3JsMIGOBggrBgEF
BQcBAQSBgTB/MDkGCCsGAQUFBzABhi1odHRwOi8vb2NzcC5zdGFydHNzbC5jb20vc3ViL2Ns
YXNzMS9jbGllbnQvY2EwQgYIKwYBBQUHMAKGNmh0dHA6Ly9haWEuc3RhcnRzc2wuY29tL2Nl
cnRzL3N1Yi5jbGFzczEuY2xpZW50LmNhLmNydDAjBgNVHRIEHDAahhhodHRwOi8vd3d3LnN0
YXJ0c3NsLmNvbS8wDQYJKoZIhvcNAQEFBQADggEBAAfByd9CEZ5pYuRa3XuE5xeJoIpDol22
A1mIGuRqxaBFIummmDoYr/tVgFod/3evQJO+ad8T5wpooY422HkHN4a1UI7ujCKcU/PrQSxm
v28AAsl4Fo/InY1nSFjMy8Ywj3EG+Edj1ZpCkzTRNGZjBa6Uj7UY7UW71kYcSdCBe8vc9Zi3
6OnHGkXROWIii3wBKLDEZqxknw0Cj9TTy5lyllYzyHku4aXLDPhiYrTzhWiwgYmweaLvW/yq
YVsHRpW8udzqOr6xvUVcuRmfMTENM7RSlRZVw2+5+I+zn7jqOrrULO1FP7Lo5cPilsMpSMOw
oJnPLQgNZvXOPFO6VaDCOboxggPtMIID6QIBATCBlDCBjDELMAkGA1UEBhMCSUwxFjAUBgNV
BAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRl
IFNpZ25pbmcxODA2BgNVBAMTL1N0YXJ0Q29tIENsYXNzIDEgUHJpbWFyeSBJbnRlcm1lZGlh
dGUgQ2xpZW50IENBAgMLTVAwDQYJYIZIAWUDBAIBBQCgggIpMBgGCSqGSIb3DQEJAzELBgkq
hkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTE1MDYxMzE0MjcwNFowLwYJKoZIhvcNAQkEMSIE
IF/GMLW63vIhNpJPnfnKsX0XJHMHRPEXp4I1Yl+UAcdYMGwGCSqGSIb3DQEJDzFfMF0wCwYJ
YIZIAWUDBAEqMAsGCWCGSAFlAwQBAjAKBggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYI
KoZIhvcNAwICAUAwBwYFKw4DAgcwDQYIKoZIhvcNAwICASgwgaUGCSsGAQQBgjcQBDGBlzCB
lDCBjDELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNl
Y3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcxODA2BgNVBAMTL1N0YXJ0Q29tIENs
YXNzIDEgUHJpbWFyeSBJbnRlcm1lZGlhdGUgQ2xpZW50IENBAgMLTVAwgacGCyqGSIb3DQEJ
EAILMYGXoIGUMIGMMQswCQYDVQQGEwJJTDEWMBQGA1UEChMNU3RhcnRDb20gTHRkLjErMCkG
A1UECxMiU2VjdXJlIERpZ2l0YWwgQ2VydGlmaWNhdGUgU2lnbmluZzE4MDYGA1UEAxMvU3Rh
cnRDb20gQ2xhc3MgMSBQcmltYXJ5IEludGVybWVkaWF0ZSBDbGllbnQgQ0ECAwtNUDANBgkq
hkiG9w0BAQEFAASCAQB96W6VFDsEY2SNlQe9wdgKdqxerHGIhS/IuXvLhVDajOASd1BbNL0S
cRAmV/4VI6cZL1diG85aIWmkXz6aftocbZ6ZE5wMLpGswGGSFShzDTUKten15MvF518rjJXy
DtsB/Yq8QkLgoGzQmaKA6Pm5GmI43lR30i3ZeykyeTYd/e52JG06sVFolgTJb/+qrrBF5AxI
nNVXKX9/iAlBNiEzs35TBGOAtdvjJB4meHLxsBqGjDlOmx1HfDedb4+//6y4QT6Y3BqR8yWl
tdNI53auTjKTBVa0ijdUfZw88BkO9X1kfIpC0MdyNSU44n4veF+/IuoGX+3Ogmo1V96iuS/C
AAAAAAAA
--------------ms000203090009040808090702--
8 years, 5 months
(ITS#8172) RFE: back-sock CONTINUE to get modified parameters
by michael＠stroeder.com
Full_Name:
Version:
OS:
URL:
Submission from: (NULL) (84.178.19.235)
It would be very nice if back-sock configured as overlay could make use of
modified parameters returned with CONTINUE.
Example for an incoming search request:
SEARCH
msgid: 42
suffix: dc=example,dc=com
base: dc=example,dc=com
scope: 2
[..]
filter: (objectClass=*)
[..]
The external listener could return:
CONTINUE
msgid: 42
suffix: dc=example,dc=com
base: ou=test,dc=example,dc=com
scope: 1
[..]
filter: (objectClass=account)
[..]
This would make request rewriting possible without having to use slapo-rwm.
8 years, 5 months
(ITS#8171) slapd-sock(5): Details for CONTINUE response missing
by michael＠stroeder.com
Full_Name:
Version: HEAD
OS:
URL:
Submission from: (NULL) (79.223.49.129)
It's not clear from the description in slapd-sock(5) whether the CONTINUE
response in overlay mode should contain the msgid or not.
While there's the detailed syntax for RESULT
RESULT
msgid: <message id>
code: <integer>
matched: <matched DN>
info: <text>
<blank line>
there is no such syntax for CONTINUE.
Is this correct?
CONTINUE
msgid: <message id>
<blank line>
8 years, 5 months
Re: (ITS#8170) ldapsearch -Z appears not to work
by quanah＠zimbra.com
--On Thursday, June 11, 2015 11:03 PM +0000 meta(a)pobox.com wrote:
> Full_Name: mathew murphy
> Version: 2.4.40
> OS: CentOS 7
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (2605:6000:ee08:3200:56ee:75ff:fe4a:3f67)
>
>
> % ldapsearch --help
> [...]
> -Z Start TLS request (-ZZ to require successful response)
> % ldapsearch -x -Z -h localhost -p 636 -b 'dc=mydc' '(objectclass=*)'
> ldap_start_tls: Can't contact LDAP server (-1)
> ldap_sasl_bind(SIMPLE): Can't contact LDAP server (-1)
-Z is for startTLS. LDAPS on port 636 is not startTLS. I.e., they are not
compatible things.
This ITS is invalid and will be closed.
--Quanah
--
Quanah Gibson-Mount
Platform Architect
Zimbra, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
8 years, 5 months
(ITS#8170) ldapsearch -Z appears not to work
by meta＠pobox.com
Full_Name: mathew murphy
Version: 2.4.40
OS: CentOS 7
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (2605:6000:ee08:3200:56ee:75ff:fe4a:3f67)
% ldapsearch --help
[...]
-Z Start TLS request (-ZZ to require successful response)
% ldapsearch -x -Z -h localhost -p 636 -b 'dc=mydc' '(objectclass=*)'
ldap_start_tls: Can't contact LDAP server (-1)
ldap_sasl_bind(SIMPLE): Can't contact LDAP server (-1)
However:
% ldapsearch -x -H ldaps://localhost:636 -b 'dc=mydc' '(objectclass=*)'
[lots of data]
So -Z isn't needed if you use -H ldaps://..., and doesn't work if you don't.
I realize that the FAQ mentions in passing that you need to use -H, but it's
really well hidden compared to the bad advice given by ldapsearch --help.
8 years, 5 months
