Re: (ITS#7024) OpenLDAP does not save the attribute "jpegPhoto" in the sql database
by masarati＠aero.polimi.it
>
>> back-sql calls backsql_BindParamBerVal(), which indeed considers the
>> data as
>> SQL_C_CHAR, SQL_VARCHAR. A binary variant can be easily defined, but
>> back-sql
>> will need to know what attribute values need to be treated as binaries.
>> A
>> general approach to this requires to modify the attribute mapping, to
>> add a
>> "type" flag.
>>
>> p.
>>
> I have carefully read the documentation (man slapd-sql) but could not
> find how to modify the attribute mapping and add a "type" flag.
I meant: modify the code. Since back-sql is basically unmaintained (any
volunteers out there?) this feature won't likely be added, not to mention
the fact that it would break all existing installations.
p.
11 years, 9 months
(ITS#7025) "the backglue code doesn't install a handler for the Abandon operation"
by hans.moser＠ofd-z.niedersachsen.de
Full_Name: Marc Patermann
Version: 2.4.26
OS: SLES 11 SP1 x86_64
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (195.37.205.30)
Hi,
thankfully Howard tracked down the issue I posted on openldap-technical as
"provider crash on high replication load" down to:
"I believe the problem is much simpler - the backglue code doesn't install a
handler for the Abandon operation, and this is preventing syncprov's abandon
handler from running, so it never gets to clean up when a consumer connection
closes.
By now you should submit a bug report to the ITS so we can track this further.
It should only require a small patch to the backglue code."
The provider has three databases, which are glued together under one root
database.
Each of the three subordinates is replicated by about 60 consumers.
On heavy replication load (all 180 consumers do initial sync), I can crash the
server with on little mod on an object in one of the subordinate databases.
If I can provide any further information on this, please let me know.
11 years, 9 months
Re: (ITS#7024) OpenLDAP does not save the attribute "jpegPhoto" in the sql database
by daniil＠chics.ru
> back-sql calls backsql_BindParamBerVal(), which indeed considers the data as
> SQL_C_CHAR, SQL_VARCHAR. A binary variant can be easily defined, but back-sql
> will need to know what attribute values need to be treated as binaries. A
> general approach to this requires to modify the attribute mapping, to add a
> "type" flag.
>
> p.
>
I have carefully read the documentation (man slapd-sql) but could not
find how to modify the attribute mapping and add a "type" flag.
11 years, 9 months
Re: (ITS#7024) OpenLDAP does not save the attribute "jpegPhoto" in the sql database
by Pierangelo Masarati
back-sql calls backsql_BindParamBerVal(), which indeed considers the data as
SQL_C_CHAR, SQL_VARCHAR. A binary variant can be easily defined, but back-sql
will need to know what attribute values need to be treated as binaries. A
general approach to this requires to modify the attribute mapping, to add a
"type" flag.
p.
11 years, 9 months
Re: (ITS#6993)
by masarati＠aero.polimi.it
> Hello
>
> This problem occurs at the end of the connection init.
> At the end of the connection_init, the connection is provided to the
> listener list (through slapd_add_internal call) and then, the connection
> mutex is freed.
>
> ==> At this point, the connection is available to the listener but the
> back end initialization has not been done.
>
> The backend_connection_init call is done out of the connection mutex
> protection.
> If the connection mutex is freed after the connection_init call, all the
> back end should performed their initialization before it could be used
> in the operation
Thank you. I understand your point, and I agree that the connection
initialization is not over until all backends had an opportunity to deal
with it. My concern is about any harm this change may cause to existing
code.
A quick search in official backends and overlays shows that only
slapo-rwm(5) is currently setting this hook. Let alone this possible bug,
this fact triggers something in my mind: bi_connection_init() might not be
the best place for your initialization to occur.
In fact, this is intended to provide something to *all* connections, but a
connection being created does not imply that your backend will be used, so
it may be a waste of resources to initialize your backend for *all*
connections.
You might be better off initializing your backend's connections only when
they are actually needed, i.e. each operation within your backend should
(mutex-protected) check whether initialization occurred and, if it didn't,
do it; if it's ongoing, just pause waiting for it to finish.
p.
11 years, 9 months
(ITS#6993)
by yann.carre＠alcatel-lucent.com
Hello
This problem occurs at the end of the connection init.
At the end of the connection_init, the connection is provided to the
listener list (through slapd_add_internal call) and then, the connection
mutex is freed.
==> At this point, the connection is available to the listener but the
back end initialization has not been done.
The backend_connection_init call is done out of the connection mutex
protection.
If the connection mutex is freed after the connection_init call, all the
back end should performed their initialization before it could be used
in the operation
Regards
Yann Carre
11 years, 9 months
Re: (ITS#6993) Backend Connection problem
by masarati＠aero.polimi.it
[you must reply to the ITS to have your messages injected in the system]
> Hello
>
> This problem occurs at the end of the connection init.
> At the end of the connection_init, the connection is provided to the
> listener list (through slapd_add_internal call) and then, the connection
> mutex is freed.
>
> ==> At this point, the connection is available to the listener but the
> back end initialization has not been done.
>
> The backend_connection_init call is done out of the connection mutex
> protection.
> If the connection mutex is freed after the connection_init call, all the
> back end should performed their initialization before it could be used
> in the operation
11 years, 9 months
(ITS#7024) OpenLDAP does not save the attribute "jpegPhoto" in the sql database
by daniil＠chics.ru
Full_Name: Daniil Harun
Version: 2.4.26
OS: FreeBSD, Linux
URL:
Submission from: (NULL) (80.85.151.246)
OpenLDAP does not save the attribute "jpegphoto" in the sql database.
Use back-sql, unixodbc and postgresql database. The reason for the problem of
the use of the data type SQL_VARCHAR instead of SQL_LONGVARBINARY.
slapd debug output:
backsql_modify_internal(): adding new values for attribute "jpegPhoto"
backsql_modify_internal(): arg(2)=1
backsql_modify_internal(): arg(1)="����";
executing "UPDATE personal SET jpegphoto=? WHERE id=?"
backsql_modify_internal(): add_proc execution failed (rc=-1, prc=0)
Return code: -1
nativeErrCode=7 SQLengineState=22021 msg="ERROR: invalid byte sequence for
encoding "UTF8": 0xff;
Error while executing the query"
SQL tracing is enabled:
[ODBC][23880][1313994179.641693][SQLPrepare.c][196]
Entry:
Statement = 0x293eae00
SQL = [UPDATE personal SET jpegphoto=? WHERE
id=?][length = 42 (SQL_NTS)]
[ODBC][23880][1313994179.641791][SQLPrepare.c][371]
Exit:[SQL_SUCCESS]
[ODBC][23880][1313994179.641886][SQLBindParameter.c][217]
Entry:
Statement = 0x293eae00
Param Number = 2
Param Type = 1
C Type = -27 SQL_C_UBIGINT
SQL Type = 4 SQL_INTEGER
Col Def = 0
Scale = 0
Rgb Value = 0xbf1fc7a4
Value Max = 0
StrLen Or Ind = 0x0
[ODBC][23880][1313994179.641988][SQLBindParameter.c][397]
Exit:[SQL_SUCCESS]
[ODBC][23880][1313994179.642182][SQLBindParamet[ODBC][23880][1313994179.641693][SQLPrepare.c][196]
Entry:
Statement = 0x293eae00
SQL = [UPDATE personal SET jpegphoto=? WHERE
id=?][length = 42 (SQL_NTS)]
[ODBC][23880][1313994179.641791][SQLPrepare.c][371]
Exit:[SQL_SUCCESS]
[ODBC][23880][1313994179.641886][SQLBindParameter.c][217]
Entry:
Statement = 0x293eae00
Param Number = 2
Param Type = 1
C Type = -27 SQL_C_UBIGINT
SQL Type = 4 SQL_INTEGER
Col Def = 0
Scale = 0
Rgb Value = 0xbf1fc7a4
Value Max = 0
StrLen Or Ind = 0x0
[ODBC][23880][1313994179.641988][SQLBindParameter.c][397]
Exit:[SQL_SUCCESS]
[ODBC][23880][1313994179.642182][SQLBindParameter.c][217]
Entry:
Statement = 0x293eae00
Param Number = 1
Param Type = 1
C Type = 1 SQL_C_CHAR
SQL Type = 12 SQL_VARCHAR
Col Def = 2790
Scale = 0
Rgb Value = 0x29529700
Value Max = 2790
StrLen Or Ind = 0x0
[ODBC][23880][1313994179.642284][SQLBindParameter.c][397]
Exit:[SQL_SUCCESS]
[ODBC][23880][1313994179.642741][SQLExecute.c][187]
Entry:
Statement = 0x293eae00
[ODBC][23880][1313994179.645733][SQLExecute.c][348]
Exit:[SQL_ERROR]
DIAG [22021] ERROR: invalid byte sequence for encoding "UTF8":
0xff;
Error while executing the query
er.c][217]
Entry:
Statement = 0x293eae00
Param Number = 1
Param Type = 1
C Type = 1 SQL_C_CHAR
SQL Type = 12 SQL_VARCHAR
Col Def = 2790
Scale = 0
Rgb Value = 0x29529700
Value Max = 2790
StrLen Or Ind = 0x0
[ODBC][23880][1313994179.642284][SQLBindParameter.c][397]
Exit:[SQL_SUCCESS]
[ODBC][23880][1313994179.642741][SQLExecute.c][187]
Entry:
Statement = 0x293eae00
[ODBC][23880][1313994179.645733][SQLExecute.c][348]
Exit:[SQL_ERROR]
DIAG [22021] ERROR: invalid byte sequence for encoding "UTF8":
0xff;
Error while executing the query
OpenLDAP uses the wrong data - types SQL_C_CHAR,SQL_VARCHAR instead of
SQL_C_BINARY,SQL_LONGVARBINARY.
How to make use of the correct data types?
11 years, 9 months
ITS#7021
by masarati＠aero.polimi.it
All in all, I think it is better to stick with the spec and with the
documentation, although I also added some notes to slapo-ppolicy(5).
p.
11 years, 9 months
Re: (ITS#6997) Segmentation fault when using slapo-rwm and slapo-chain
by masarati＠aero.polimi.it
> Full_Name: Manuel Gaupp
> Version: 2.4.26
> OS: Linux 2.6.18 (CentOS 5.6 x86_64)
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (93.208.112.253)
>
>
> Under the following circumstances OpenLDAP crashes when a chained search
> does not return with result code 0:
> - "rwm" overlay used in a global context
> - "chain" overlay used in a database specifix context
>
> (gdb) bt
> #0 0x0000000400040004 in ?? ()
> #1 0x000000000042e5c7 in slap_cleanup_play (op=0x197834b0, rs=0x42a07ca0)
> at result.c:539
> #2 0x000000000042ef9b in send_ldap_response (op=0x197834b0,
> rs=0x42a07ca0)
> at result.c:724
> #3 0x000000000042fcf9 in slap_send_ldap_result (op=0x197834b0,
> rs=0x42a07ca0) at result.c:851
> #4 0x00000000004d982a in ldap_chain_response (op=0x197834b0,
> rs=0x42a07ca0) at chain.c:1165
> #5 0x00000000004820ca in over_back_response (op=0x197834b0,
> rs=0x42a07ca0)
> at backover.c:237
> #6 0x000000000042e535 in slap_response_play (op=0x197834b0,
> rs=0x42a07ca0)
> at result.c:505
> #7 0x000000000042ef78 in send_ldap_response (op=0x197834b0,
> rs=0x42a07ca0)
> at result.c:579
> #8 0x000000000042fcf9 in slap_send_ldap_result (op=0x197834b0,
> rs=0x42a07ca0) at result.c:851
> #9 0x00000000004928f0 in bdb_search (op=0x197834b0, rs=0x42a07ca0)
> at search.c:499
> #10 0x0000000000482352 in overlay_op_walk (op=0x197834b0, rs=0x42a07ca0,
> which=op_search, oi=0x196f3bb0, on=0x0) at backover.c:671
> #11 0x00000000004828b7 in over_op_func (op=0x197834b0, rs=0x42a07ca0,
> which=op_search) at backover.c:723
> #12 0x0000000000421cf9 in fe_op_search (op=0x197834b0, rs=0x42a07ca0)
> at search.c:402
> #13 0x0000000000482352 in overlay_op_walk (op=0x197834b0, rs=0x42a07ca0,
> which=op_search, oi=0x1970fd30, on=0x0) at backover.c:671
> #14 0x00000000004828b7 in over_op_func (op=0x197834b0, rs=0x42a07ca0,
> which=op_search) at backover.c:723
> #15 0x0000000000422505 in do_search (op=0x197834b0, rs=0x42a07ca0)
> at search.c:247
> #16 0x000000000041fa95 in connection_operation (ctx=0x42a07d70,
> arg_v=<value optimized out>) at connection.c:1138
> #17 0x0000000000503d5c in ldap_int_thread_pool_wrapper (xpool=0x196beee0)
> at tpool.c:685
> #18 0x000000316820673d in start_thread () from /lib64/libpthread.so.0
> #19 0x0000003167ad44bd in clone () from /lib64/libc.so.6
>
> OpenLDAP 2.4.26 was built using:
> ./configure --enable-rewrite --enable-ldap --enable-rwm --enable-bdb
>
> This issue can be reproduced using the following configuration
> (referred as 'slapd.conf.proxy')
> ------------------------------------------
> overlay rwm
> # There might be some rewrite directives
> database bdb
> suffix "o=example"
> rootdn "cn=admin,o=example"
> rootpw secret
> directory /root/db.proxy
> overlay chain
> chain-return-error true
> chain-uri "ldap://localhost:38977"
> ------------------------------------------
> The process is started on port 389 with plain ldap
> /usr/local/libexec/slapd -f slapd.conf.proxy -h "ldap://localhost:389"
>
> Create the following entry in this server:
> dn: cn=ldap,o=example
> objectclass: extensibleObject
> objectclass: referral
> ref: ldap://localhost:38977/cn=real,o=example
>
> To reproduce the error there has to be another server where the referral
> points to. It contains the following minimal configuration:
> (referred as 'slapd.conf.server')
> ------------------------------------------
> database bdb
> suffix "cn=real,o=example"
> rootdn "cn=admin,cn=real,o=example"
> rootpw secret
> directory /root/db.server
> ------------------------------------------
> The process is started on port 38977 with plain ldap
> /usr/local/libexec/slapd -f slapd.conf.server -h "ldap://localhost:38977"
>
> Using the following ldapsearch, slapd crashes:
> ldapsearch -h localhost -p 389 -D "cn=admin,o=example" -w secret \
> -x -b "ou=abc,cn=ldap,o=example" 1.1
>
> Whereas the following ldapsearch returns with "success"
> ldapsearch -h localhost -p 389 -D "cn=admin,o=example" -w secret \
> -x -b "cn=ldap,o=example" 1.1
>
> Deactivating the "rwm" overlay prevents from this issue. So I think the
> problem can be found somewhere in the rwm sources.
I'm afraid that's unavoidable given the stacking order of the overlays.
This will likely be fixed only when ITS#6166 is addressed, i.e. in
OpenLDAP 2.5.
p.
11 years, 9 months
