Re: (ITS#9088) Concurrency not seen in OpenLDAP 2.4.44
by ondra＠mistotebe.net
On Tue, Oct 08, 2019 at 12:56:04PM +0000, jrajalakshmi(a)juniper.net wrote:
> Hi,
>
> Thanks for the reply. There is an implementation already in our product.
Hi,
as mentioned previously, please provide a sample C program that shows
what's happening and preferably one that also behaves better with 2.4.23
from your perspective so we can help you investigate. I don't believe
packet captures would help us here at this point.
> To summarize on the behavior observed
>
> Scenario#1:
>
> Single LDAP session with LDAP Bind does authentication in Sequential
> Order. Due to sequential execution of OpenLDAP Search API after
> version 2.4.24 we observe packet drops in LDAP Authentication Request.
> This is resulting in Low performance of our product.
Previously you mentioned only searches, which should work over the same
connection depending how you do things. Not sure how binds fit in the
picture here? Just a note that you can only have certain types of
operations pending at the same time and binds are not one of them.
Also don't know what you mean by packet drops here. Is that packet drops
in your product or packet drops on the LDAP session?
> Scenario #2: (this is a sample test case tried to compare the behavior
> with Scenario#1)
>
> There are say 10 LDAP instances configured, all pointing to the same
> LDAP server. The request-response is handled properly, performance is
> improved when compared to scenario#1. But the drawback is the multiple
> threads for every LDAP client session, utilizing more memory. This
> design may not be recommended for our product.
You can have a single thread taking care of the session (or however many
sessions you want) if you use a non-blocking socket for the underlying
connection(s) (and use ldap_result and friends) which is what the
asynchronous API has been designed for.
> Pl. reply back for any further queries / clarifications. The packet
> captures file size is big (>6MB), can try to share the relevant
> request-response details......
Regards,
--
Ondřej Kuzník
Senior Software Engineer
Symas Corporation http://www.symas.com
Packaged, certified, and supported LDAP solutions powered by OpenLDAP
4 years, 2 months
- 1
- 0
RE: (ITS#9088) Concurrency not seen in OpenLDAP 2.4.44
by jrajalakshmi＠juniper.net
--_000_CH2PR05MB69514B8448808C2D1086ABB7DF9A0CH2PR05MB6951namp_
Content-Type: text/plain; charset="iso-8859-2"
Content-Transfer-Encoding: quoted-printable
Hi,
Thanks for the reply. There is an implementation already in our product.
To summarize on the behavior observed
Scenario#1:
Single LDAP session with LDAP Bind does authentication in Sequential Order.=
Due to sequential execution of OpenLDAP Search API after version 2.4.24 we=
observe packet drops in LDAP Authentication Request. This is resulting in =
Low performance of our product.
Scenario #2: (this is a sample test case tried to compare the behavior with=
Scenario#1)
There are say 10 LDAP instances configured, all pointing to the same LDAP s=
erver. The request-response is handled properly, performance is improved wh=
en compared to scenario#1. But the drawback is the multiple threads for eve=
ry LDAP client session, utilizing more memory. This design may not be recom=
mended for our product.
Pl. reply back for any further queries / clarifications. The packet capture=
s file size is big (>6MB), can try to share the relevant request-response d=
etails......
Thanks,
Raji
-----Original Message-----
From: Ond=F8ej Kuzn=EDk <ondra(a)mistotebe.net>
Sent: Monday, September 30, 2019 8:18 PM
To: Rajalakshmi Jayaraman <jrajalakshmi(a)juniper.net>
Cc: openldap-its(a)OpenLDAP.org
Subject: Re: (ITS#9088) Concurrency not seen in OpenLDAP 2.4.44
On Mon, Sep 23, 2019 at 09:22:45AM +0000, jrajalakshmi(a)juniper.net<mailto:j=
rajalakshmi(a)juniper.net> wrote:
> (1) What problem/issue/behavior are you having trouble with? What do
> you expect to see?
> Using OpenLDAP version 2.4.44 we are not seeing Concurrency whereas in
> OpenLDAP Version 2.4.23 we are seeing Concurrent LDAP Requests.
> In our product OpenLDAP 2.4.23 was previously used and the concurrent
> request for search using the API ' ldap_search_ext_s() ' was working
> as expected and our product scalability was better.
> We have recently upgraded LDAP to OpenLDAP 2.4.44. But with this
> version, the api 'ldap_search_ext_s()' does not seem to work as
> expected, i.e concurrent requests handling is not happening.
Hi,
if you want the project to investigate this, it would be useful if you prov=
ide a sample program that exhibits this behaviour with a vanilla build of 2=
.4.48 and doesn't exhibit that with an analogous build of 2.4.23.
I would like to point out that since 2.4.23 is a very old release, it is al=
so possible you have been relying on undocumented behaviour or a libldap_r =
bug that has since been fixed. That is what a sample program would also hel=
p to rule out.
Regards,
--
Ond=F8ej Kuzn=EDk
Senior Software Engineer
Symas Corporation https://urldefense.com/v3/__http://=
www.symas.com__;!8WoA6RjC81c!Xs72RxgyQ_p-CmwcbkhCu7cCMZXgmihqyjqC563sunVF3O=
tjroYFsOTBsDbndrE4VnYu$<https://urldefense.com/v3/__http:/www.symas.com__;!=
8WoA6RjC81c!Xs72RxgyQ_p-CmwcbkhCu7cCMZXgmihqyjqC563sunVF3OtjroYFsOTBsDbndrE=
4VnYu$>
Packaged, certified, and supported LDAP solutions powered by OpenLDAP
Juniper Business Use Only
--_000_CH2PR05MB69514B8448808C2D1086ABB7DF9A0CH2PR05MB6951namp_
Content-Type: text/html; charset="iso-8859-2"
Content-Transfer-Encoding: quoted-printable
<html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr=
osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:=
//www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Diso-8859-=
2">
<meta name=3D"Generator" content=3D"Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
p.MsoPlainText, li.MsoPlainText, div.MsoPlainText
{mso-style-priority:99;
mso-style-link:"Plain Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.PlainTextChar
{mso-style-name:"Plain Text Char";
mso-style-priority:99;
mso-style-link:"Plain Text";
font-family:"Calibri",sans-serif;}
p.msipfooter30b3d538, li.msipfooter30b3d538, div.msipfooter30b3d538
{mso-style-name:msipfooter30b3d538;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle21
{mso-style-type:personal-compose;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext=3D"edit">
<o:idmap v:ext=3D"edit" data=3D"1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=3D"EN-US" link=3D"#0563C1" vlink=3D"#954F72">
<div class=3D"WordSection1">
<p class=3D"MsoPlainText">Hi,<o:p></o:p></p>
<p class=3D"MsoPlainText">Thanks for the reply. There is an implementation =
already in our product.<o:p></o:p></p>
<p class=3D"MsoPlainText"><o:p> </o:p></p>
<p class=3D"MsoPlainText">To summarize on the behavior observed<o:p></o:p><=
/p>
<p class=3D"MsoPlainText">Scenario#1:<o:p></o:p></p>
<p class=3D"MsoPlainText">Single LDAP session with LDAP Bind does authentic=
ation in Sequential Order. Due to sequential execution of OpenLDAP Search A=
PI after version 2.4.24 we observe packet drops in LDAP Authentication Requ=
est. This is resulting in Low performance
of our product. <o:p></o:p></p>
<p class=3D"MsoPlainText"><o:p> </o:p></p>
<p class=3D"MsoPlainText">Scenario #2: (this is a sample test case tried to=
compare the behavior with Scenario#1)<o:p></o:p></p>
<p class=3D"MsoPlainText">There are say 10 LDAP instances configured, all p=
ointing to the same LDAP server. The request-response is handled properly, =
performance is improved when compared to scenario#1. But the drawback is th=
e multiple threads for every LDAP
client session, utilizing more memory. This design may not be recommended =
for our product.<o:p></o:p></p>
<p class=3D"MsoPlainText"><o:p> </o:p></p>
<p class=3D"MsoPlainText">Pl. reply back for any further queries / clarific=
ations. The packet captures file size is big (>6MB), can try to share th=
e relevant request-response details……<o:p></o:p></p>
<p class=3D"MsoPlainText"><o:p> </o:p></p>
<p class=3D"MsoPlainText">Thanks,<br>
Raji<o:p></o:p></p>
<p class=3D"MsoPlainText"><o:p> </o:p></p>
<p class=3D"MsoPlainText"><o:p> </o:p></p>
<p class=3D"MsoPlainText"><o:p> </o:p></p>
<p class=3D"MsoPlainText"><o:p> </o:p></p>
<p class=3D"MsoPlainText"><o:p> </o:p></p>
<p class=3D"MsoPlainText"><o:p> </o:p></p>
<p class=3D"MsoPlainText">-----Original Message-----<br>
From: Ond=F8ej Kuzn=EDk <ondra(a)mistotebe.net> <br>
Sent: Monday, September 30, 2019 8:18 PM<br>
To: Rajalakshmi Jayaraman <jrajalakshmi(a)juniper.net><br>
Cc: openldap-its(a)OpenLDAP.org<br>
Subject: Re: (ITS#9088) Concurrency not seen in OpenLDAP 2.4.44<o:p></o:p><=
/p>
<p class=3D"MsoPlainText"><o:p> </o:p></p>
<p class=3D"MsoPlainText">On Mon, Sep 23, 2019 at 09:22:45AM +0000, <a =
href=3D"mailto:jrajalakshmi@juniper.net">
<span style=3D"color:windowtext;text-decoration:none">jrajalakshmi(a)juniper.=
net</span></a> wrote:<o:p></o:p></p>
<p class=3D"MsoPlainText">> (1) What problem/issue/behavior are you havi=
ng trouble with? What do
<o:p></o:p></p>
<p class=3D"MsoPlainText">> you expect to see?<o:p></o:p></p>
<p class=3D"MsoPlainText">> Using OpenLDAP version 2.4.44 we are not see=
ing Concurrency whereas in
<o:p></o:p></p>
<p class=3D"MsoPlainText">> OpenLDAP Version 2.4.23 we are seeing Concur=
rent LDAP Requests.<o:p></o:p></p>
<p class=3D"MsoPlainText">> In our product OpenLDAP 2.4.23 was previousl=
y used and the concurrent
<o:p></o:p></p>
<p class=3D"MsoPlainText">> request for search using the API ' ldap_sear=
ch_ext_s() ' was working
<o:p></o:p></p>
<p class=3D"MsoPlainText">> as expected and our product scalability was =
better.<o:p></o:p></p>
<p class=3D"MsoPlainText">> We have recently upgraded LDAP to OpenLDAP 2=
.4.44. But with this
<o:p></o:p></p>
<p class=3D"MsoPlainText">> version, the api 'ldap_search_ext_s()' does =
not seem to work as
<o:p></o:p></p>
<p class=3D"MsoPlainText">> expected, i.e concurrent requests handling i=
s not happening.<o:p></o:p></p>
<p class=3D"MsoPlainText"><o:p> </o:p></p>
<p class=3D"MsoPlainText">Hi,<o:p></o:p></p>
<p class=3D"MsoPlainText">if you want the project to investigate this, it w=
ould be useful if you provide a sample program that exhibits this behaviour=
with a vanilla build of 2.4.48 and doesn't exhibit that with an analogous =
build of 2.4.23.<o:p></o:p></p>
<p class=3D"MsoPlainText"><o:p> </o:p></p>
<p class=3D"MsoPlainText">I would like to point out that since 2.4.23 is a =
very old release, it is also possible you have been relying on undocumented=
behaviour or a libldap_r bug that has since been fixed. That is what a sam=
ple program would also help to rule
out.<o:p></o:p></p>
<p class=3D"MsoPlainText"><o:p> </o:p></p>
<p class=3D"MsoPlainText">Regards,<o:p></o:p></p>
<p class=3D"MsoPlainText"><o:p> </o:p></p>
<p class=3D"MsoPlainText">--<o:p></o:p></p>
<p class=3D"MsoPlainText">Ond=F8ej Kuzn=EDk<o:p></o:p></p>
<p class=3D"MsoPlainText">Senior Software Engineer<o:p></o:p></p>
<p class=3D"MsoPlainText">Symas Corporation &n=
bsp;  =
; <a href=3D"https://urldefense.com/v3/__http:/www.=
symas.com__;!8WoA6RjC81c!Xs72RxgyQ_p-CmwcbkhCu7cCMZXgmihqyjqC563sunVF3Otjro=
YFsOTBsDbndrE4VnYu$">
<span style=3D"color:windowtext;text-decoration:none">https://urldefense.co=
m/v3/__http://www.symas.com__;!8WoA6RjC81c!Xs72RxgyQ_p-CmwcbkhCu7cCMZXgmihq=
yjqC563sunVF3OtjroYFsOTBsDbndrE4VnYu$</span></a>
<o:p></o:p></p>
<p class=3D"MsoPlainText">Packaged, certified, and supported LDAP solutions=
powered by OpenLDAP<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"msipfooter30b3d538" align=3D"center" style=3D"margin:0in;margin=
-bottom:.0001pt;text-align:center">
<span style=3D"font-size:7.0pt;color:black">Juniper Business Use Only</span=
><o:p></o:p></p>
</div>
</body>
</html>
--_000_CH2PR05MB69514B8448808C2D1086ABB7DF9A0CH2PR05MB6951namp_--
4 years, 2 months
- 1
- 0
Re: (ITS#8996) Please supply a pkg-config file for libldap
by hugh.mcmaster＠outlook.com
Hi Ryan,
Thank you for your review.
On Thu, 3 Oct 2019 at 13:05, Ryan Tandy wrote:
> Thanks for providing the patches. I have reviewed and tested them,
> everything I could think of works properly.
Excellent! :)
> As a personal opinion I might prefer shorter descriptions, for example
> "OpenLDAP ASN.1 Basic Encoding Rules library for LDAP" and "OpenLDAP
> Lightweight Directory Access Protocol library (reentrant version)". Only
> an opinion, there's nothing wrong with what you wrote.
I'm happy for the descriptions to be whatever describes the libraries best.
I particularly like your shortened ASN.1 description.
> I'd like to see this included, however a more senior team member should
> approve it first. Good luck!
Thanks. Hopefully that will happen soon.
Kind regards,
Hugh
4 years, 2 months
- 1
- 0
Re: (ITS#8996) Please supply a pkg-config file for libldap
by ryan＠openldap.org
Hello Hugh,
Thanks for providing the patches. I have reviewed and tested them,
everything I could think of works properly.
As a personal opinion I might prefer shorter descriptions, for example
"OpenLDAP ASN.1 Basic Encoding Rules library for LDAP" and "OpenLDAP
Lightweight Directory Access Protocol library (reentrant version)". Only
an opinion, there's nothing wrong with what you wrote.
I'd like to see this included, however a more senior team member should
approve it first. Good luck!
cheers,
Ryan
4 years, 2 months
- 1
- 0