openldap-bugs

openldap-bugs@openldap.org

1 participants
21065 discussions

(ITS#9085) Test suite is failing
by kloczko.tomasz＠gmail.com 19 Sep '19

19 Sep '19

Full_Name: Tomasz Kloczko Version: 2.4.48 OS: Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (92.25.145.71) + /usr/bin/make -O -j48 V=1 VERBOSE=1 check -j1 cd tests; /usr/bin/make test make[1]: Entering directory '/home/tkloczko/rpmbuild/BUILD/openldap-2.4.48/tests' make[2]: Entering directory '/home/tkloczko/rpmbuild/BUILD/openldap-2.4.48/tests' Initiating LDAP tests for BDB... Running ./scripts/all for bdb... >>>>> Executing all LDAP tests for bdb >>>>> Starting test000-rootdse for bdb... running defines.sh Starting slapd on TCP/IP port 9011... Using ldapsearch to retrieve the root DSE... Using ldapsearch to retrieve the cn=Subschema... Using ldapsearch to retrieve the cn=Monitor... dn: objectClass: top objectClass: OpenLDAProotDSE structuralObjectClass: OpenLDAProotDSE configContext: cn=config namingContexts: o=OpenLDAP Project,l=Internet monitorContext: cn=Monitor supportedControl: 2.16.840.1.113730.3.4.18 supportedControl: 2.16.840.1.113730.3.4.2 supportedControl: 1.3.6.1.4.1.4203.1.10.1 supportedControl: 1.3.6.1.1.22 supportedControl: 1.2.840.113556.1.4.319 supportedControl: 1.2.826.0.1.3344810.2.3 supportedControl: 1.3.6.1.1.13.2 supportedControl: 1.3.6.1.1.13.1 supportedControl: 1.3.6.1.1.12 supportedExtension: 1.3.6.1.4.1.4203.1.11.1 supportedExtension: 1.3.6.1.4.1.4203.1.11.3 supportedExtension: 1.3.6.1.1.8 supportedFeatures: 1.3.6.1.1.14 supportedFeatures: 1.3.6.1.4.1.4203.1.5.1 supportedFeatures: 1.3.6.1.4.1.4203.1.5.2 supportedFeatures: 1.3.6.1.4.1.4203.1.5.3 supportedFeatures: 1.3.6.1.4.1.4203.1.5.4 supportedFeatures: 1.3.6.1.4.1.4203.1.5.5 supportedLDAPVersion: 3 supportedSASLMechanisms: GSS-SPNEGO supportedSASLMechanisms: GSSAPI vendorName: The OpenLDAP Project <http://www.openldap.org/> entryDN: subschemaSubentry: cn=Subschema dn: cn=Subschema objectClass: top objectClass: subentry objectClass: subschema objectClass: extensibleObject cn: Subschema dn: cn=Monitor objectClass: monitorServer cn: Monitor description: This subtree contains monitoring/managing objects. description: This object contains information about this server. description: Most of the information is held in operational attributes, which must be explicitly requested. monitoredInfo: OpenLDAP: slapd 2.4.48 (Sep 20 2019 05:49:42) ./scripts/test000-rootdse: line 68: 1709599 Hangup $SLAPD -f $CONF1 -h $URI1 -d $LVL $TIMING > $LOG1 2>&1 >>>>> Test succeeded >>>>> test000-rootdse completed OK for bdb. >>>>> Starting test001-slapadd for bdb... running defines.sh Running slapadd to build slapd database... Starting slapd on TCP/IP port 9011... Using ldapsearch to retrieve all the entries... Waiting 5 seconds for slapd to start... Waiting 5 seconds for slapd to start... Waiting 5 seconds for slapd to start... Waiting 5 seconds for slapd to start... Waiting 5 seconds for slapd to start... Waiting 5 seconds for slapd to start... ldapsearch failed (32)! ./scripts/test001-slapadd: line 55: kill: (1709740) - No such process >>>>> test001-slapadd failed for bdb (exit 32) make[2]: *** [Makefile:294: bdb-yes] Error 32 make[2]: Leaving directory '/home/tkloczko/rpmbuild/BUILD/openldap-2.4.48/tests' make[1]: *** [Makefile:279: test] Error 2 make[1]: Leaving directory '/home/tkloczko/rpmbuild/BUILD/openldap-2.4.48/tests' make: *** [Makefile:291: test] Error 2 error: Bad exit status from /var/tmp/rpm-tmp.aVXfhD (%check)

1 0

Re: (ITS#9017) Improving performance of commit sync in Windows
by kriszyp＠gmail.com 18 Sep '19

18 Sep '19

--000000000000d8a73d0592d86418 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Checking on this again, is this still a possibility for merging into LMDB? This fix is still working great (improved performance) on our systems. Thanks, Kris On Mon, Jun 17, 2019 at 1:04 PM Kris Zyp <kriszyp(a)gmail.com> wrote: > Is this still being considered/reviewed? Let me know if there are any > other changes you would like me to make. This patch has continued to yiel= d > significant and reliable performance improvements for us, and seems like = it > would be nice for this to be available for other Windows users. > > On Fri, May 3, 2019 at 3:52 PM Kris Zyp <kriszyp(a)gmail.com> wrote: > >> For the sake of putting this in the email thread (other code discussion >> in GitHub), here is the latest squashed commit of the proposed patch (wi= th >> the on-demand, retained overlapped array to reduce re-malloc and opening >> event handles): >> https://github.com/kriszyp/node-lmdb/commit/726a9156662c703bf3d453aab75e= e222072b990f >> >> >> >> Thanks, >> Kris >> >> >> >> *From: *Kris Zyp <kriszyp(a)gmail.com> >> *Sent: *April 30, 2019 12:43 PM >> *To: *Howard Chu <hyc(a)symas.com>; openldap-its(a)OpenLDAP.org >> *Subject: *RE: (ITS#9017) Improving performance of commit sync in Window= s >> >> >> >> > What is the point of using writemap mode if you still need to use >> WriteFile >> >> > on every individual page? >> >> >> >> As I understood from the documentation, and have observed, using writema= p >> mode is faster (and uses less temporary memory) because it doesn=E2=80= =99t require >> mallocs to allocate pages (docs: =E2=80=9CThis is faster and uses fewer = mallocs=E2=80=9D). >> To be clear though, LMDB is so incredibly fast and efficient, that in >> sync-mode, it takes enormous transactions before the time spent allocati= ng >> and creating the dirty pages with the updated b-tree is anywhere even >> remotely close to the time it takes to wait for disk flushing, even with= an >> SSD. But the more pertinent question is efficiency, and measuring CPU >> cycles rather than time spent (efficiency is more important than just ti= me >> spent). When I ran my tests this morning of 100 (sync) transactions with >> 100 puts per transaction, times varied quite a bit, but it seemed like >> running with writemap enabled typically averages about 500ms of CPU and >> with writemap disabled it typically averages around 600ms. Not a huge >> difference, but still definitely worthwhile, I think. >> >> >> >> Caveat emptor: Measuring LMDB performance with sync interactions on >> Windows is one of the most frustratingly erratic things to measure. It i= s >> sunny outside right now, times could be different when it starts raining >> later, but, this is what I saw this morning... >> >> >> >> > What is the performance difference between your patch using writemap, >> and just >> >> > not using writemap in the first place? >> >> >> >> Running 1000 sync transactions on 3GB db with a single put per >> transaction, without writemap map, without the patch took about 60 secon= ds. >> And it took about 1 second with the patch with writemap mode enabled! >> (there is no significant difference in sync times with writemap enabled = or >> disabled with the patch.) So the difference was huge in my test. And not >> only that, without the patch, the CPU usage was actually _*higher*_ >> during that 60 seconds (close to 100% of a core) than during the executi= on >> with the patch for one second (close to 50%). Anyway, there are certain= ly >> tests I have run where the differences are not as large (doing small >> commits on large dbs accentuates the differences), but the patch always >> seems to win. It could also be that my particular configuration causes >> bigger differences (on an SSD drive, and maybe a more fragmented file?). >> >> >> >> Anyway, I added error handling for the malloc, and fixed/changed the >> other things you suggested. Be happy to make any other changes you want. >> The updated patch is here: >> >> >> https://github.com/kriszyp/node-lmdb/commit/25366dea9453749cf6637f43ec17= b9b62094acde >> >> >> >> > OVERLAPPED* ov =3D malloc((pagecount - keep) * sizeof(OVERLAPPED)); >> >> > Probably this ought to just be pre-allocated based on the maximum >> number of dirty pages a txn allows. >> >> >> >> I wasn=E2=80=99t sure I understood this comment. Are you suggesting we m= alloc(MDB_IDL_UM_MAX >> * sizeof(OVERLAPPED)) for each environment, and retain it for the life o= f >> the environment? I think that is 4MB, if my math is right, which seems l= ike >> a lot of memory to keep allocated (we usually have a lot of open >> environments). If the goal is to reduce the number of mallocs, how about= we >> retain the OVERLAPPED array, and only free and re-malloc if the previous >> allocation wasn=E2=80=99t large enough? Then there isn=E2=80=99t unneces= sary allocation, >> and we only malloc when there is a bigger transaction than any previous.= I >> put this together in a separate commit, as I wasn=E2=80=99t sure if this= what you >> wanted (can squash if you prefer): >> https://github.com/kriszyp/node-lmdb/commit/2fe68fb5269c843e2e789746a17a= 4b2adefaac40 >> >> >> >> Thank you for the review! >> >> >> >> Thanks, >> Kris >> >> >> >> *From: *Howard Chu <hyc(a)symas.com> >> *Sent: *April 30, 2019 7:12 AM >> *To: *kriszyp(a)gmail.com; openldap-its(a)OpenLDAP.org >> *Subject: *Re: (ITS#9017) Improving performance of commit sync in Window= s >> >> >> >> kriszyp(a)gmail.com wrote: >> >> > Full_Name: Kristopher William Zyp >> >> > Version: LMDB 0.9.23 >> >> > OS: Windows >> >> > URL: >> https://github.com/kriszyp/node-lmdb/commit/7ff525ae57684a163d32af74a0ab= 9332b7fc4ce9 >> >> > Submission from: (NULL) (71.199.6.148) >> >> > >> >> > >> >> > We have seen very poor performance on the sync of commits on large >> databases in >> >> > Windows. On databases with 2GB of data, in writemap mode, the sync of >> even small >> >> > commits is consistently well over 100ms (without writemap it is faster= , >> but >> >> > still slow). It is expected that a sync should take some time while >> waiting for >> >> > disk confirmation of the writes, but more concerning is that these syn= c >> >> > operations (in writemap mode) are instead dominated by nearly 100% >> system CPU >> >> > utilization, so operations that requires sub-millisecond b-tree update >> >> > operations are then dominated by very large amounts of system CPU >> cycles during >> >> > the sync phase. >> >> > >> >> > I think that the fundamental problem is that FlushViewOfFile seems to >> be an O(n) >> >> > operation where n is the size of the file (or map). I presume that >> Windows is >> >> > scanning the entire map/file for dirty pages to flush, I'm guessing >> because it >> >> > doesn't have an internal index of all the dirty pages for every >> file/map-view in >> >> > the OS disk cache. Therefore, the turns into an extremely expensive, >> CPU-bound >> >> > operation to find the dirty pages for (large file) and initiate their >> writes, >> >> > which, of course, is contrary to the whole goal of a scalable database >> system. >> >> > And FlushFileBuffers is also relatively slow as well. We have attempte= d >> to batch >> >> > as many operations into single transaction as possible, but this is >> still a very >> >> > large overhead. >> >> > >> >> > The Windows docs for FlushFileBuffers itself warns about the >> inefficiencies of >> >> > this function ( >> https://docs.microsoft.com/en-us/windows/desktop/api/fileapi/nf-fileapi-= flushfilebuffers >> ). >> >> > Which also points to the solution: it is much faster to write out the >> dirty >> >> > pages with WriteFile through a sync file handle >> (FILE_FLAG_WRITE_THROUGH). >> >> > >> >> > The associated patch >> >> > ( >> https://github.com/kriszyp/node-lmdb/commit/7ff525ae57684a163d32af74a0ab= 9332b7fc4ce9 >> ) >> >> > is my attempt at implementing this solution, for Windows. Fortunately, >> with the >> >> > design of LMDB, this is relatively straightforward. LMDB already >> supports >> >> > writing out dirty pages with WriteFile calls. I added a write-through >> handle for >> >> > sending these writes directly to disk. I then made that file-handle >> >> > overlapped/asynchronously, so all the writes for a commit could be >> started in >> >> > overlap mode, and (at least theoretically) transfer in parallel to the >> drive and >> >> > then used GetOverlappedResult to wait for the completion. So basically >> >> > mdb_page_flush becomes the sync. I extended the writing of dirty pages >> through >> >> > WriteFile to writemap mode as well (for writing meta too), so that >> WriteFile >> >> > with write-through can be used to flush the data without ever needing >> to call >> >> > FlushViewOfFile or FlushFileBuffers. I also implemented support for >> write >> >> > gathering in writemap mode where contiguous file positions infers >> contiguous >> >> > memory (by tracking the starting position with wdp and writing >> contiguous pages >> >> > in single operations). Sorting of the dirty list is maintained even in >> writemap >> >> > mode for this purpose. >> >> >> >> What is the point of using writemap mode if you still need to use >> WriteFile >> >> on every individual page? >> >> >> >> > The performance benefits of this patch, in my testing, are >> considerable. Writing >> >> > out/syncing transactions is typically over 5x faster in writemap mode, >> and 2x >> >> > faster in standard mode. And perhaps more importantly (especially in >> environment >> >> > with many threads/processes), the efficiency benefits are even larger, >> >> > particularly in writemap mode, where there can be a 50-100x reduction >> in the >> >> > system CPU usage by using this patch. This brings windows performance >> with >> >> > sync'ed transactions in LMDB back into the range of "lightning" >> performance :). >> >> >> >> What is the performance difference between your patch using writemap, an= d >> just >> >> not using writemap in the first place? >> >> >> >> -- >> >> -- Howard Chu >> >> CTO, Symas Corp. http://www.symas.com >> >> Director, Highland Sun http://highlandsun.com/hyc/ >> >> Chief Architect, OpenLDAP http://www.openldap.org/project/ >> >> >> >> >> > --000000000000d8a73d0592d86418 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">Checking on this again, is this still a possibility for me= rging into LMDB? This fix is still working great (improved performance) on = our systems.<div>Thanks,</div><div>Kris</div></div> <div class=3D"gmail_= quote"><div dir=3D"ltr" class=3D"gmail_attr">On Mon, Jun 17, 2019 at 1:04 P= M Kris Zyp <<a href=3D"mailto:kriszyp@gmail.com">kriszyp(a)gmail.com</a>&g= t; wrote: </div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0p= x 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div d= ir=3D"ltr">Is this still being considered/reviewed? Let me know if there ar= e any other changes you would like me to make. This patch has continued to = yield significant and reliable performance improvements for us, and seems l= ike it would be nice for this to be available for other Windows users.</div= > <div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Fr= i, May 3, 2019 at 3:52 PM Kris Zyp <<a href=3D"mailto:kriszyp@gmail.com"= target=3D"_blank">kriszyp(a)gmail.com</a>> wrote: </div><blockquote cl= ass=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid= rgb(204,204,204);padding-left:1ex"><div lang=3D"EN-CA"><div class=3D"gmail= -m_-3761698270430747347gmail-m_5242437559147988140WordSection1">For the sake of putting this in the email thread (other code di= scussion in GitHub), here is the latest squashed commit of the proposed pat= ch (with the on-demand, retained overlapped array to reduce re-malloc and o= pening event handles): <a href=3D"https://github.com/kriszyp/node-lmdb/comm= it/726a9156662c703bf3d453aab75ee222072b990f" target=3D"_blank">https://gith= ub.com/kriszyp/node-lmdb/commit/726a9156662c703bf3d453aab75ee222072b990f</a= >=C2=A0= Thanks, Kris=C2=A0<div styl= e=3D"border-right:none;border-bottom:none;border-left:none;border-top:1pt s= olid rgb(225,225,225);padding:3pt 0cm 0cm">From: <a href=3D"mailto:kriszyp@gmail.com" = target=3D"_blank">Kris Zyp</a> Sent: April 30, 2019 12:43 PM <= b>To: <a href=3D"mailto:hyc@symas.com" target=3D"_blank">Howard Chu</a>= ; <a href=3D"mailto:openldap-its@OpenLDAP.org" target=3D"_blank">openldap-i= ts(a)OpenLDAP.org</a> Subject: RE: (ITS#9017) Improving performance= of commit sync in Windows</div>=C2=A0= > What is the point of using writemap mod= e if you still need to use WriteFile> on every individual page?<u= >=C2=A0As I understood from the docum= entation, and have observed, using writemap mode is faster (and uses less t= emporary memory) because it doesn=E2=80=99t require mallocs to allocate pag= es (docs: =E2=80=9CThis is faster and uses fewer mallocs=E2=80=9D). To be c= lear though, LMDB is so incredibly fast and efficient, that in sync-mode, i= t takes enormous transactions before the time spent allocating and creating= the dirty pages with the updated b-tree is anywhere even remotely close to= the time it takes to wait for disk flushing, even with an SSD. But the mor= e pertinent question is efficiency, and measuring CPU cycles rather than ti= me spent (efficiency is more important than just time spent). When I ran my= tests this morning of 100 (sync) transactions with 100 puts per transactio= n, times varied quite a bit, but it seemed like running with writemap enabl= ed typically averages about 500ms of CPU and with writemap disabled it typi= cally averages around 600ms. Not a huge difference, but still definitely wo= rthwhile, I think.=C2=A0= Caveat emptor: Measuring LMDB performance wi= th sync interactions on Windows is one of the most frustratingly erratic th= ings to measure. It is sunny outside right now, times could be different wh= en it starts raining later, but, this is what I saw this morning...<= u>=C2=A0> What is the performance difference between your patch using write= map, and just> not using writem= ap in the first place?=C2= =A0Running 1000 sync transactions on 3GB = db with a single put per transaction, without writemap map, without the pat= ch took about 60 seconds. And it took about 1 second with the patch with wr= itemap mode enabled! (there is no significant difference in sync times with= writemap enabled or disabled with the patch.) So the difference was huge i= n my test. And not only that, without the patch, the CPU usage was actually= _higher_ during that 60 seconds (close to 100% of a core) than duri= ng the execution with the patch for one second (close to 50%).=C2=A0 Anyway= , there are certainly tests I have run where the differences are not as lar= ge (doing small commits on large dbs accentuates the differences), but the = patch always seems to win. It could also be that my particular configuratio= n causes bigger differences (on an SSD drive, and maybe a more fragmented f= ile?).=C2=A0Anyway, I added error handling for the malloc, and fixed= /changed the other things you suggested. Be happy to make any other changes= you want. The updated patch is here:<a href=3D"https://github.com/kriszyp/node-lmdb/commit/25366dea9453749cf= 6637f43ec17b9b62094acde" target=3D"_blank">https://github.com/kriszyp/node-= lmdb/commit/25366dea9453749cf6637f43ec17b9b62094acde</a><= p class=3D"MsoNormal">=C2=A0><s= pan class=3D"gmail-m_-3761698270430747347gmail-m_5242437559147988140blob-co= de-inner"> OVERLAPPED* ov =3D malloc<span= style=3D"font-size:9pt;font-family:Consolas;color:rgb(36,41,46)">((pagecou= nt - keep) * sizeof(OVERLAPPED));<= /span>> Probably this o= ught to just be pre-allocated based on the maximum number of dirty pages a = txn allows.=C2=A0</span= >I wasn= =E2=80=99t sure I understood this comment. Are you suggesting we mal= loc(MDB_IDL_UM_MAX * sizeof(OVERLAPPED)) for each environment, and retain i= t for the life of the environment? I think that is 4MB, if my math is right= , which seems like a lot of memory to keep allocated (we usually have a lot= of open environments). If the goal is to reduce the number of mallocs, how= about we retain the OVERLAPPED array, and only free and re-malloc if the p= revious allocation wasn=E2=80=99t large enough? Then there isn=E2=80=99t un= necessary allocation, and we only malloc when there is a bigger transaction= than any previous. I put this together in a separate commit, as I wasn=E2= =80=99t sure if this what you wanted (can squash if you prefer): <a href=3D= "https://github.com/kriszyp/node-lmdb/commit/2fe68fb5269c843e2e789746a17a4b= 2adefaac40" target=3D"_blank">https://github.com/kriszyp/node-lmdb/commit/2= fe68fb5269c843e2e789746a17a4b2adefaac40</a>=C2=A0Thank you for the r= eview! =C2=A0Thanks,<b= r>Kris=C2=A0<div= style=3D"border-right:none;border-bottom:none;border-left:none;border-top:= 1pt solid rgb(225,225,225);padding:3pt 0cm 0cm">F= rom: <a href=3D"mailto:hyc@symas.com" target=3D"_blank">Howard Chu</a><= br>Sent: April 30, 2019 7:12 AM To: <a href=3D"mailto:kris= zyp(a)gmail.com" target=3D"_blank">kriszyp(a)gmail.com</a>; <a href=3D"mailto:o= penldap-its(a)OpenLDAP.org" target=3D"_blank">openldap-its(a)OpenLDAP.org</a><b= r>Subject: Re: (ITS#9017) Improving performance of commit sync in Wi= ndows</div>=C2=A0</p= ><a href=3D"mailto:kriszyp@gmail.com" target=3D"_bla= nk">kriszyp(a)gmail.com</a> wrote:&g= t; Full_Name: Kristopher William Zyp> Version: LMDB 0.9.23> OS= : Windows> URL: <a href=3D"http= s://github.com/kriszyp/node-lmdb/commit/7ff525ae57684a163d32af74a0ab9332b7f= c4ce9" target=3D"_blank">https://github.com/kriszyp/node-lmdb/commit/7ff525= ae57684a163d32af74a0ab9332b7fc4ce9</a>> Submission from: (NULL) (71.199.6.148)> > <u= >> We have seen very poor performance on = the sync of commits on large databases in> Windows. On databases with 2GB of data, in writemap mode, the s= ync of even small> commits is c= onsistently well over 100ms (without writemap it is faster, but</= u>> still slow). It is expected that a sync s= hould take some time while waiting for> disk confirmation of the writes, but more concerning is that these= sync> operations (in writemap = mode) are instead dominated by nearly 100% system CPU> utilization, so operations that requires sub-millis= econd b-tree update> operations= are then dominated by very large amounts of system CPU cycles during</u= >> the sync phase.<p= class=3D"MsoNormal">> > I t= hink that the fundamental problem is that FlushViewOfFile seems to be an O(= n)> operation where n is the si= ze of the file (or map). I presume that Windows is> scanning the entire map/file for dirty pages to flush,= I'm guessing because it> d= oesn't have an internal index of all the dirty pages for every file/map= -view in> the OS disk cache. Th= erefore, the turns into an extremely expensive, CPU-bound= > operation to find the dirty pages for (large fi= le) and initiate their writes,>= which, of course, is contrary to the whole goal of a scalable database sys= tem.> And FlushFileBuffers is a= lso relatively slow as well. We have attempted to batch<p= class=3D"MsoNormal">> as many operations into single transaction as pos= sible, but this is still a very&gt= ; large overhead.> </= u>> The Windows docs for FlushFileBuffers its= elf warns about the inefficiencies of> this function (<a href=3D"https://docs.microsoft.com/en-us/windows/= desktop/api/fileapi/nf-fileapi-flushfilebuffers" target=3D"_blank">https://= docs.microsoft.com/en-us/windows/desktop/api/fileapi/nf-fileapi-flushfilebu= ffers</a>).> Which also points = to the solution: it is much faster to write out the dirty= > pages with WriteFile through a sync file handle= (FILE_FLAG_WRITE_THROUGH).> <u= >> The associated patch<= /u>> (<a href=3D"https://github.com/kriszyp/n= ode-lmdb/commit/7ff525ae57684a163d32af74a0ab9332b7fc4ce9" target=3D"_blank"= >https://github.com/kriszyp/node-lmdb/commit/7ff525ae57684a163d32af74a0ab93= 32b7fc4ce9</a>)> is my attempt = at implementing this solution, for Windows. Fortunately, with the= > design of LMDB, this is relatively stra= ightforward. LMDB already supports= > writing out dirty pages with WriteFile calls. I added a write-through = handle for> sending these write= s directly to disk. I then made that file-handle> overlapped/asynchronously, so all the writes for a comm= it could be started in> overlap= mode, and (at least theoretically) transfer in parallel to the drive and<u= >> then used GetOverlappedResult t= o wait for the completion. So basically> mdb_page_flush becomes the sync. I extended the writing of dirty = pages through> WriteFile to wri= temap mode as well (for writing meta too), so that WriteFile<= /p>> with write-through can be used to flush the = data without ever needing to call&= gt; FlushViewOfFile or FlushFileBuffers. I also implemented support for wri= te> gathering in writemap mode = where contiguous file positions infers contiguous> memory (by tracking the starting position with wdp and = writing contiguous pages> in si= ngle operations). Sorting of the dirty list is maintained even in writemap<= u>> mode for this purpose.<= u>=C2=A0What is the point of using writemap mode if you still need to use Writ= eFileon every individual page?<= /u>=C2=A0> The performance benefits of this patch, in my testing, are co= nsiderable. Writing> out/syncin= g transactions is typically over 5x faster in writemap mode, and 2x<= u>> faster in standard mode. And perhaps = more importantly (especially in environment> with many threads/processes), the efficiency benefits are eve= n larger,> particularly in writ= emap mode, where there can be a 50-100x reduction in the<= p class=3D"MsoNormal">> system CPU usage by using this patch. This bring= s windows performance with> syn= c'ed transactions in LMDB back into the range of "lightning" = performance :).=C2=A0</u= >What is the performance difference between your= patch using writemap, and justnot= using writemap in the first place?=C2=A0-- =C2=A0=C2=A0-- Howard Chu=C2=A0 CTO, Symas Corp.=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 <a href=3D"http://www.symas.com" target=3D"_blank">http:= //www.symas.com</a>=C2=A0 Director= , Highland Sun=C2=A0=C2=A0=C2=A0=C2=A0 <a href=3D"http://highlandsun.com/hy= c/" target=3D"_blank">http://highlandsun.com/hyc/</a>=C2=A0 Chief Architect, OpenLDAP=C2=A0 <a href=3D"http:/= /www.openldap.org/project/" target=3D"_blank">http://www.openldap.org/proje= ct/</a>=C2=A0=C2=A0</div></div></blockquote></div> </blockquote></div> --000000000000d8a73d0592d86418--

1 0

(ITS#9084) Build issue when slaps is not included
by aixtools＠felt.demon.nl 18 Sep '19

18 Sep '19

Full_Name: Michael Felt Version: 2.4.48 OS: AIX URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (109.104.126.147) Build of 2.4.46 proceeds with no issue. When using the same command to build 2.4.48 the build stops with an undefined variable error: root@x066:[/data/prj/aixtools/src]grep LDAP_OPT_X_TLS *46/* *48/* openldap-2.4.46/CHANGES: Added libldap LDAP_OPT_X_TLS_PACKAGE (ITS#6969) openldap-2.4.48/CHANGES: Added libldap LDAP_OPT_X_TLS_PACKAGE (ITS#6969) root@x066:[/data/prj/aixtools/src]find *46 *48 | xargs grep LDAP_OPT_X_TLS_ECNAME openldap-2.4.48/include/ldap.h:#define LDAP_OPT_X_TLS_ECNAME 0x6012 openldap-2.4.48/libraries/libldap/tls2.c: case LDAP_OPT_X_TLS_ECNAME: openldap-2.4.48/libraries/libldap/tls2.c: case LDAP_OPT_X_TLS_ECNAME: openldap-2.4.48/servers/slapd/bconfig.c: case CFG_TLS_ECNAME: flag = LDAP_OPT_X_TLS_ECNAME; break; >From above, it appears that LDAP_OPT_X_TLS_ECNAME should be defined, but I question that as I get the following error when compiling... Entering subdirectory libldap make[2]: Entering directory '/data/prj/aixtools/static/openldap-2.4.48/libraries/libldap' /bin/sh ../../libtool --mode=compile xlc_r -I/opt/include -O2 -qmaxmem=-1 -I../../include -I../../../../src/openldap-2.4.48/include -I/opt/include -DLDAP_LIBRARY -c ../../../../src/openldap-2.4.48/libraries/libldap/tls2.c xlc_r -I/opt/include -O2 -qmaxmem=-1 -I../../include -I../../../../src/openldap-2.4.48/include -I/opt/include -DLDAP_LIBRARY -c ../../../../src/openldap-2.4.48/libraries/libldap/tls2.c -o tls2.o "../../../../src/openldap-2.4.48/libraries/libldap/tls2.c", line 658.14: 1506-045 (S) Undeclared identifier LDAP_OPT_X_TLS_ECNAME. "../../../../src/openldap-2.4.48/libraries/libldap/tls2.c", line 658.14: 1506-051 (S) Case expression must be a valid integral constant. "../../../../src/openldap-2.4.48/libraries/libldap/tls2.c", line 781.14: 1506-045 (S) Undeclared identifier LDAP_OPT_X_TLS_ECNAME. "../../../../src/openldap-2.4.48/libraries/libldap/tls2.c", line 781.14: 1506-051 (S) Case expression must be a valid integral constant. make[2]: *** [Makefile:413: tls2.lo] Error 1 make[2]: Leaving directory '/data/prj/aixtools/static/openldap-2.4.48/libraries/libldap' make[1]: *** [Makefile:297: all-common] Error 1 make[1]: Leaving directory '/data/prj/aixtools/static/openldap-2.4.48/libraries' make: *** [Makefile:313: all-common] Error 1 Researching further... Michael

1 0

(ITS#9083) Broken link in the list of commercial support partners
by marketing＠daasi.de 18 Sep '19

18 Sep '19

Full_Name: Romy Hoffart Version: - OS: - URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (37.24.13.226) Hey everyone, I am with the marketing department of DAASI International, one of the commercial support partners for OpenLDAP listed on your website. Unfortunately I noticed that the link to our website doesn't work anymore. Could you replace the link here (https://www.openldap.org/support/) with a new link to our website (https://daasi.de/en/)? I was in touch with Quanah from Symas before, she recommended me to submit this issue here. Thank you so much for your time & help, Romy

1 0

Re: (ITS#9082) Issues while upgrading to 2.4.48
by quanah＠symas.com 17 Sep '19

17 Sep '19

--On Tuesday, September 17, 2019 6:47 PM +0000 krishradhe11(a)gmail.com wrote: > Full_Name: Radhe Krish > Version: 2.4.44 > OS: Centos 7 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (139.85.223.11) Hello, The ITS system is for reporting bugs, not usage questions. Please redirect your query to the openldap-technical(a)openldap.org email list for questions such as yours. I will note that this question has been asked and answered there already for other individuals, so if you look at the list archives, you'll find your answer. <https://www.openldap.org/lists/mm/listinfo/openldap-technical> <https://www.openldap.org/lists/openldap-technical/> <https://www.openldap.org/lists/openldap-technical/201908/msg00157.html> This ITS will be closed. Regards, Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

(ITS#9082) Issues while upgrading to 2.4.48
by krishradhe11＠gmail.com 17 Sep '19

17 Sep '19

Full_Name: Radhe Krish Version: 2.4.44 OS: Centos 7 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (139.85.223.11) Hello ! I 'm using Openldap version 2.4.44 running on my centos 7 machine. In process to upgrade to the recently released Openldap version 2.4.48, I'm facing some difficulties while choosing the concerned folder to save executable libraries. I followed the instructions as in official website but I'm not positive about how to upgrade from already installed Openldap 2.4.44 to 2.4.48. I tried to find RPMs supporting this but still no luck.Can you help me with clear instructions to proceed upgrade from 2.4.44 to 2.4.48 or help me with the corresponding RPM, that will be really helpful.

1 0

RE: (ITS#9023) crash using ppolicy chaining from slave to master
by jpayanides＠prosodie.com 17 Sep '19

17 Sep '19

--_000_AM0PR0202MB3553BA671B52D50E489F7ACBBA8F0AM0PR0202MB3553_ Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 SGVsbG8gT25kxZllaiAsDQp0aGFua3MgYSBsb3QgZm9yIHdhcm5pbmcgbWUuDQpJIHdpbGwgdGVz dCB0aGUgbmV3IGNvZGUgd2l0aGluIGEgbW9udGggKEkgY2Fubm90IGRvIGl0IGF0IHRoaXMgdGlt ZSkuDQoNCklsIGtlZXAgeW91IGluZm9ybWVkLg0KDQpraW5kIHJlZ2FyZHMsDQoNCg0KSmVhbi1Q aGlsaXBwZSBBeWFuaWRlcw0KQ29uY2VwdGlvbiBldCBhcmNoaXRlY3R1cmUgc8OpY3VyaXTDqQ0K U2VydmljZSAmIE9wZXJhdGlvbiAvIEluZnJhc3RydWN0dXJlIE5ldHdvcmsgU2VjdXJpdHkgQmFj a2JvbmUNCnTDqWwgKzMzMTM0NDkwNjI4DQpQcm9zb2RpZS1DYXBnZW1pbmkNClBlb3BsZSBtYXR0 ZXIsIHJlc3VsdHMgY291bnQuDQpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXw0KRGUg OiBPbmTFmWVqIEt1em7DrWsgPG9uZHJhQG1pc3RvdGViZS5uZXQ+DQpFbnZvecOpIDogbWVyY3Jl ZGkgMTEgc2VwdGVtYnJlIDIwMTkgMTg6MDQNCsOAIDogQVlBTklERVMsIEpFQU4tUEhJTElQUEUg PGpwYXlhbmlkZXNAcHJvc29kaWUuY29tPg0KQ2MgOiBvcGVubGRhcC1pdHNAT3BlbkxEQVAub3Jn IDxvcGVubGRhcC1pdHNAT3BlbkxEQVAub3JnPg0KT2JqZXQgOiBSZTogKElUUyM5MDIzKSBjcmFz aCB1c2luZyBwcG9saWN5IGNoYWluaW5nIGZyb20gc2xhdmUgdG8gbWFzdGVyDQoNCkhpIEplYW4t UGhpbGlwcGUsDQpJIGhhdmUganVzdCBzZWVuIGFuZCBmaXhlZCBhIGNyYXNoIHRoYXQgbWlnaHQg YmUgc2ltaWxhciB0byB5b3Vycw0KKElUUyM5MDc2KSwgY291bGQgeW91IHRyeSBhbmQgc2VlIGlm IHRoZSBjcmFzaCBnb2VzIGF3YXkgaWYgeW91IGFwcGx5DQpjb21taXQgYTE0ZmI3MzFhYzZlMzlj ZDAzNzUxMmNhNjNiY2IwMjFmMDE5NmU1Yj8NCg0KVGhhbmtzLA0KDQotLQ0KT25kxZllaiBLdXpu w61rDQpTZW5pb3IgU29mdHdhcmUgRW5naW5lZXINClN5bWFzIENvcnBvcmF0aW9uICAgICAgICAg ICAgICAgICAgICAgICBodHRwOi8vd3d3LnN5bWFzLmNvbQ0KUGFja2FnZWQsIGNlcnRpZmllZCwg YW5kIHN1cHBvcnRlZCBMREFQIHNvbHV0aW9ucyBwb3dlcmVkIGJ5IE9wZW5MREFQDQpUaGlzIG1l c3NhZ2UgY29udGFpbnMgaW5mb3JtYXRpb24gdGhhdCBtYXkgYmUgcHJpdmlsZWdlZCBvciBjb25m aWRlbnRpYWwgYW5kIGlzIHRoZSBwcm9wZXJ0eSBvZiB0aGUgQ2FwZ2VtaW5pIEdyb3VwLiBJdCBp cyBpbnRlbmRlZCBvbmx5IGZvciB0aGUgcGVyc29uIHRvIHdob20gaXQgaXMgYWRkcmVzc2VkLiBJ ZiB5b3UgYXJlIG5vdCB0aGUgaW50ZW5kZWQgcmVjaXBpZW50LCB5b3UgYXJlIG5vdCBhdXRob3Jp emVkIHRvIHJlYWQsIHByaW50LCByZXRhaW4sIGNvcHksIGRpc3NlbWluYXRlLCBkaXN0cmlidXRl LCBvciB1c2UgdGhpcyBtZXNzYWdlIG9yIGFueSBwYXJ0IHRoZXJlb2YuIElmIHlvdSByZWNlaXZl IHRoaXMgbWVzc2FnZSBpbiBlcnJvciwgcGxlYXNlIG5vdGlmeSB0aGUgc2VuZGVyIGltbWVkaWF0 ZWx5IGFuZCBkZWxldGUgYWxsIGNvcGllcyBvZiB0aGlzIG1lc3NhZ2UuCg== --_000_AM0PR0202MB3553BA671B52D50E489F7ACBBA8F0AM0PR0202MB3553_ Content-Type: text/html; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 PGh0bWw+DQo8aGVhZD4NCjxtZXRhIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSIgY29udGVudD0i dGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04Ij4NCjxzdHlsZSB0eXBlPSJ0ZXh0L2NzcyIgc3R5bGU9 ImRpc3BsYXk6bm9uZTsiPiBQIHttYXJnaW4tdG9wOjA7bWFyZ2luLWJvdHRvbTowO30gPC9zdHls ZT4NCjwvaGVhZD4NCjxib2R5IGRpcj0ibHRyIj4NCjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OiBD YWxpYnJpLCBBcmlhbCwgSGVsdmV0aWNhLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDEycHQ7IGNv bG9yOiByZ2IoMCwgMCwgMCk7Ij4NCkhlbGxvIDxmb250IHNpemU9IjIiPjxzcGFuIHN0eWxlPSJm b250LXNpemU6MTFwdCI+T25kxZllaiA8L3NwYW4+PC9mb250Piw8L2Rpdj4NCjxkaXYgc3R5bGU9 ImZvbnQtZmFtaWx5OiBDYWxpYnJpLCBBcmlhbCwgSGVsdmV0aWNhLCBzYW5zLXNlcmlmOyBmb250 LXNpemU6IDEycHQ7IGNvbG9yOiByZ2IoMCwgMCwgMCk7Ij4NCnRoYW5rcyBhIGxvdCBmb3Igd2Fy bmluZyBtZS48L2Rpdj4NCjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OiBDYWxpYnJpLCBBcmlhbCwg SGVsdmV0aWNhLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDEycHQ7IGNvbG9yOiByZ2IoMCwgMCwg MCk7Ij4NCkkgd2lsbCB0ZXN0IHRoZSBuZXcgY29kZSB3aXRoaW4gYSBtb250aCAoSSBjYW5ub3Qg ZG8gaXQgYXQgdGhpcyB0aW1lKS48L2Rpdj4NCjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OiBDYWxp YnJpLCBBcmlhbCwgSGVsdmV0aWNhLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDEycHQ7IGNvbG9y OiByZ2IoMCwgMCwgMCk7Ij4NCjxicj4NCjwvZGl2Pg0KPGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6 IENhbGlicmksIEFyaWFsLCBIZWx2ZXRpY2EsIHNhbnMtc2VyaWY7IGZvbnQtc2l6ZTogMTJwdDsg Y29sb3I6IHJnYigwLCAwLCAwKTsiPg0KSWwga2VlcCB5b3UgaW5mb3JtZWQuPC9kaXY+DQo8ZGl2 IHN0eWxlPSJmb250LWZhbWlseTogQ2FsaWJyaSwgQXJpYWwsIEhlbHZldGljYSwgc2Fucy1zZXJp ZjsgZm9udC1zaXplOiAxMnB0OyBjb2xvcjogcmdiKDAsIDAsIDApOyI+DQo8YnI+DQo8L2Rpdj4N CjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OiBDYWxpYnJpLCBBcmlhbCwgSGVsdmV0aWNhLCBzYW5z LXNlcmlmOyBmb250LXNpemU6IDEycHQ7IGNvbG9yOiByZ2IoMCwgMCwgMCk7Ij4NCmtpbmQgcmVn YXJkcyw8YnI+DQo8L2Rpdj4NCjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OiBDYWxpYnJpLCBBcmlh bCwgSGVsdmV0aWNhLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDEycHQ7IGNvbG9yOiByZ2IoMCwg MCwgMCk7Ij4NCjxicj4NCjwvZGl2Pg0KPGRpdiBpZD0iU2lnbmF0dXJlIj4NCjxkaXYgaWQ9ImRp dnRhZ2RlZmF1bHR3cmFwcGVyIiBkaXI9Imx0ciIgc3R5bGU9ImZvbnQtc2l6ZToxMnB0OyBjb2xv cjojMDAwMDAwOyBmb250LWZhbWlseTpDYWxpYnJpLEhlbHZldGljYSxzYW5zLXNlcmlmIj4NCjxk aXYgaWQ9ImRpdnRhZ2RlZmF1bHR3cmFwcGVyIiBkaXI9Imx0ciIgc3R5bGU9ImZvbnQtc2l6ZTox MnB0OyBjb2xvcjojMDAwMDAwOyBiYWNrZ3JvdW5kLWNvbG9yOiNGRkZGRkY7IGZvbnQtZmFtaWx5 OkNhbGlicmksQXJpYWwsSGVsdmV0aWNhLHNhbnMtc2VyaWYiPg0KPHAgc3R5bGU9Im1hcmdpbi10 b3A6IDBweDsgbWFyZ2luLWJvdHRvbTogMHB4OyI+PHNwYW4gbGFuZz0iZnIiPjwvc3Bhbj48L3A+ DQo8ZGl2IHN0eWxlPSJtYXJnaW46MCI+PGZvbnQgc2l6ZT0iMyIgZmFjZT0iVGltZXMgTmV3IFJv bWFuLHNlcmlmIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEycHQiPjxmb250IHNpemU9IjIiIGZh Y2U9IlZlcmRhbmEsc2Fucy1zZXJpZiIgY29sb3I9IiMwMDQ2OEMiPjxzcGFuIHN0eWxlPSJmb250 LXNpemU6OXB0Ij48Yj5KZWFuLVBoaWxpcHBlIEF5YW5pZGVzPGJyPg0KPC9iPjwvc3Bhbj48L2Zv bnQ+PC9zcGFuPjwvZm9udD48L2Rpdj4NCjxzcGFuIHN0eWxlPSJmb250LXNpemU6Ny41cHQ7IGZv bnQtZmFtaWx5OiZxdW90O0FyaWFsJnF1b3Q7LCZxdW90O3NhbnMtc2VyaWYmcXVvdDs7IGNvbG9y OmJsYWNrIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjcuNXB0OyBmb250LWZhbWlseTomcXVvdDtB cmlhbCZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7OyBjb2xvcjpibGFjayI+Q29uY2VwdGlv biBldCBhcmNoaXRlY3R1cmUgc8OpY3VyaXTDqTwvc3Bhbj48YnI+DQo8c3BhbiBzdHlsZT0iZm9u dC1zaXplOjhwdDsgZm9udC1mYW1pbHk6QXJpYWwsSGVsdmV0aWNhLHNhbnMtc2VyaWYiPlNlcnZp Y2UgJmFtcDsgT3BlcmF0aW9uIC8gSW5mcmFzdHJ1Y3R1cmU8L3NwYW4+PC9zcGFuPjxzcGFuIHN0 eWxlPSJmb250LXNpemU6OS4wcHQ7IGZvbnQtZmFtaWx5OiZxdW90O0FyaWFsJnF1b3Q7LCZxdW90 O3NhbnMtc2VyaWYmcXVvdDs7IGNvbG9yOmJsYWNrIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjhw dDsgZm9udC1mYW1pbHk6QXJpYWwsSGVsdmV0aWNhLHNhbnMtc2VyaWYiPg0KIE5ldHdvcmsgU2Vj dXJpdHkgQmFja2JvbmU8L3NwYW4+PGJyPg0KPGEgY2xhc3M9Im8zNjVidXR0b24iPjxzcGFuIGNs YXNzPSJtcy1mb250LXMgbXMtZm9udC1jb2xvci10aGVtZVByaW1hcnkiIHRpdGxlPSImIzQzOzMz MTM0NDkwNjI4Ij50w6lsICYjNDM7MzMxMzQ0OTA2Mjg8L3NwYW4+PC9hPjxicj4NClByb3NvZGll LUNhcGdlbWluaTxicj4NCjwvc3Bhbj48c3Ryb25nPjxzcGFuIHN0eWxlPSJmb250LWZhbWlseTom cXVvdDtBcmlhbCZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7OyBjb2xvcjojMDA5OENDIj5Q ZW9wbGUgbWF0dGVyLCByZXN1bHRzIGNvdW50Ljwvc3Bhbj48L3N0cm9uZz48L2Rpdj4NCjxzdHJv bmc+PHNwYW4gc3R5bGU9ImZvbnQtZmFtaWx5OiZxdW90O0FyaWFsJnF1b3Q7LCZxdW90O3NhbnMt c2VyaWYmcXVvdDs7IGNvbG9yOiMwMDk4Q0MiPjwvc3Bhbj48L3N0cm9uZz48L2Rpdj4NCjwvZGl2 Pg0KPGRpdiBpZD0iYXBwZW5kb25zZW5kIj48L2Rpdj4NCjxociBzdHlsZT0iZGlzcGxheTppbmxp bmUtYmxvY2s7d2lkdGg6OTglIiB0YWJpbmRleD0iLTEiPg0KPGRpdiBpZD0iZGl2UnBseUZ3ZE1z ZyIgZGlyPSJsdHIiPjxmb250IGZhY2U9IkNhbGlicmksIHNhbnMtc2VyaWYiIHN0eWxlPSJmb250 LXNpemU6MTFwdCIgY29sb3I9IiMwMDAwMDAiPjxiPkRlIDo8L2I+IE9uZMWZZWogS3V6bsOtayAm bHQ7b25kcmFAbWlzdG90ZWJlLm5ldCZndDs8YnI+DQo8Yj5FbnZvecOpIDo8L2I+IG1lcmNyZWRp IDExIHNlcHRlbWJyZSAyMDE5IDE4OjA0PGJyPg0KPGI+w4AgOjwvYj4gQVlBTklERVMsIEpFQU4t UEhJTElQUEUgJmx0O2pwYXlhbmlkZXNAcHJvc29kaWUuY29tJmd0Ozxicj4NCjxiPkNjJm5ic3A7 OjwvYj4gb3BlbmxkYXAtaXRzQE9wZW5MREFQLm9yZyAmbHQ7b3BlbmxkYXAtaXRzQE9wZW5MREFQ Lm9yZyZndDs8YnI+DQo8Yj5PYmpldCA6PC9iPiBSZTogKElUUyM5MDIzKSBjcmFzaCB1c2luZyBw cG9saWN5IGNoYWluaW5nIGZyb20gc2xhdmUgdG8gbWFzdGVyPC9mb250Pg0KPGRpdj4mbmJzcDs8 L2Rpdj4NCjwvZGl2Pg0KPGRpdiBjbGFzcz0iQm9keUZyYWdtZW50Ij48Zm9udCBzaXplPSIyIj48 c3BhbiBzdHlsZT0iZm9udC1zaXplOjExcHQ7Ij4NCjxkaXYgY2xhc3M9IlBsYWluVGV4dCI+SGkg SmVhbi1QaGlsaXBwZSw8YnI+DQpJIGhhdmUganVzdCBzZWVuIGFuZCBmaXhlZCBhIGNyYXNoIHRo YXQgbWlnaHQgYmUgc2ltaWxhciB0byB5b3Vyczxicj4NCihJVFMjOTA3NiksIGNvdWxkIHlvdSB0 cnkgYW5kIHNlZSBpZiB0aGUgY3Jhc2ggZ29lcyBhd2F5IGlmIHlvdSBhcHBseTxicj4NCmNvbW1p dCBhMTRmYjczMWFjNmUzOWNkMDM3NTEyY2E2M2JjYjAyMWYwMTk2ZTViPzxicj4NCjxicj4NClRo YW5rcyw8YnI+DQo8YnI+DQotLSA8YnI+DQpPbmTFmWVqIEt1em7DrWs8YnI+DQpTZW5pb3IgU29m dHdhcmUgRW5naW5lZXI8YnI+DQpTeW1hcyBDb3Jwb3JhdGlvbiZuYnNwOyZuYnNwOyZuYnNwOyZu YnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNw OyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyA8 YSBocmVmPSJodHRwOi8vd3d3LnN5bWFzLmNvbSI+aHR0cDovL3d3dy5zeW1hcy5jb208L2E+PGJy Pg0KUGFja2FnZWQsIGNlcnRpZmllZCwgYW5kIHN1cHBvcnRlZCBMREFQIHNvbHV0aW9ucyBwb3dl cmVkIGJ5IE9wZW5MREFQPGJyPg0KPC9kaXY+DQo8L3NwYW4+PC9mb250PjwvZGl2Pg0KPHNwYW4g c3R5bGU9ImZvbnQtc2l6ZTogOXB4OyBsaW5lLWhlaWdodDogMTBweDsiPlRoaXMgbWVzc2FnZSBj b250YWlucyBpbmZvcm1hdGlvbiB0aGF0IG1heSBiZSBwcml2aWxlZ2VkIG9yIGNvbmZpZGVudGlh bCBhbmQgaXMgdGhlIHByb3BlcnR5IG9mIHRoZSBDYXBnZW1pbmkgR3JvdXAuIEl0IGlzIGludGVu ZGVkIG9ubHkgZm9yIHRoZSBwZXJzb24gdG8gd2hvbSBpdCBpcyBhZGRyZXNzZWQuIElmIHlvdSBh cmUgbm90IHRoZSBpbnRlbmRlZCByZWNpcGllbnQsIHlvdSBhcmUgbm90IGF1dGhvcml6ZWQgdG8g cmVhZCwgcHJpbnQsIHJldGFpbiwgY29weSwgZGlzc2VtaW5hdGUsIGRpc3RyaWJ1dGUsIG9yIHVz ZSB0aGlzIG1lc3NhZ2Ugb3IgYW55IHBhcnQgdGhlcmVvZi4gSWYgeW91IHJlY2VpdmUgdGhpcyBt ZXNzYWdlIGluIGVycm9yLCBwbGVhc2Ugbm90aWZ5IHRoZSBzZW5kZXIgaW1tZWRpYXRlbHkgYW5k IGRlbGV0ZSBhbGwgY29waWVzIG9mIHRoaXMgbWVzc2FnZS48L3NwYW4+PC9ib2R5Pg0KPC9odG1s Pg0K --_000_AM0PR0202MB3553BA671B52D50E489F7ACBBA8F0AM0PR0202MB3553_--

1 0

(ITS#9081) Memory leak in ldap_initialize/ldap_unbind
by jengelh＠inai.de 16 Sep '19

16 Sep '19

Full_Name: Jan Engelhardt Version: 2.4.48 OS: openSUSE Tumbleweed; Linux 5.3.0-rc7 x86_64 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (2a02:8108:96c0:15fc:2137:8684:9608:51f8) » cat x.c #define LDAP_DEPRECATED 1 #include <ldap.h> int main() { LDAP *ld; ldap_initialize(&ld, "ldapi:///"); ldap_unbind(ld); } » gcc x.c -Wall -lldap -ggdb3 » valgrind --leak-check=full ./a.out ==25779== HEAP SUMMARY: ==25779== in use at exit: 26,395 bytes in 85 blocks ==25779== total heap usage: 233 allocs, 148 frees, 177,134 bytes allocated ==25779== ==25779== 40 bytes in 1 blocks are definitely lost in loss record 11 of 24 ==25779== at 0x4838B65: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==25779== by 0x4A8310C: ber_memcalloc_x (memory.c:283) ==25779== by 0x4A84EC0: ber_sockbuf_alloc (sockbuf.c:60) ==25779== by 0x487396A: ldap_create (open.c:172) ==25779== by 0x4873C8D: ldap_initialize (open.c:241) ==25779== by 0x10915F: main (x.c:6) ==25779== ==25779== LEAK SUMMARY: ==25779== definitely lost: 40 bytes in 1 blocks ==25779== indirectly lost: 0 bytes in 0 blocks ==25779== possibly lost: 0 bytes in 0 blocks ==25779== still reachable: 26,355 bytes in 84 blocks ==25779== suppressed: 0 bytes in 0 blocks ==25779== Reachable blocks (those to which a pointer was found) are not shown. ==25779== To see them, rerun with: --leak-check=full --show-leak-kinds=all ==25779== ==25779== For lists of detected and suppressed errors, rerun with: -s ==25779== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0) unbind.c: 123 for ( lm = ld->ld_responses; lm != NULL; lm = next ) { 128 if ( ld->ld_abandoned != NULL ) { 132 LDAP_MUTEX_UNLOCK( &ld->ld_res_mutex ); 136 ber_int_sb_destroy( ld->ld_sb ); Should this probably be LBER_FREE rather than ber_int_sb_destroy?

1 0

ITS#9080 a further relevant ITS
by Norman.Gray＠glasgow.ac.uk 14 Sep '19

14 Sep '19

I should also note that ITS#4996 (2007, closed) appears to be the issue=20 which resulted in the support for `-H "<proto>:///DN"` being added to=20 the command-line clients. --=20 Norman Gray : https://nxg.me.uk SUPA School of Physics and Astronomy, University of Glasgow, UK

1 0

(ITS#9080) Feature request: support DNS SRV lookups in ldap.conf
by gray＠nxg.name 14 Sep '19

14 Sep '19

Full_Name: Norman Gray Version: 2.4.47 OS: FreeBSD 12.0 (but not OS-soecific) URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (2001:8b0:df5:af53:c1a5:cbb2:6a6a:3390) Feature request: support specification of DNS SRV records in ldap.conf, matching ldapsearch -H The `ldapsearch` tools supports specifying lookups of SRV records via a special case syntax in the argument to the `-H` option. A URI such as ldap:///dc=ldap,dc=example,dc=com (with the commas and equals signs suitably escaped) prompts ldapsearch to do a lookup of an SRV record for ldap.example.com. However the URI synatx in ldap.conf doesn't have a corresponding special case, so it's not possible to put such a spec in a ldap.conf file. The ldap.conf documentation doesn't claim any support for SRV records, so there isn't a bug here, but on a Principle of Least Astonishment it would be very useful if the same syntax that -H respects were respected by ldap.conf as well. Other LDAP clients have different ways of specifying this (eg, nslcd.conf supports a `DNS:<domain>` syntax; Linux automount's `autofs.conf` has a `ldap_uri` attribute which supports a very similar dc=xxx syntax), so an alternative ldap.conf syntax would be a good second best. As an auxiliary point, when `ldapsearch` sees `URI ldap:///dc%3D...` in the ldap.conf file, it silently ignores it, rather than producing an error. It doesn't even produce a warning when `ldapsearch` is invoked with `-d-1`. I had to use strace to reassure myself that the config file was actually being read. I feel that a library should make a _lot_ more noise about an attribute in a configuration file being seen but ignored (I can see that the `ldap://dc...` URI, which is of course syntactically OK, _might_ be inadvertently meaningful, and thus not necessarily a detectable error, but even in that case `-d-1` should produce _something_). A scan through ITS reports found the following: * ITS#5919 (2009, still open) discusses a similar request, and discusses a variety of issues with it. This is a useful cross-reference. * ITS#6462 (2010, open) and ITS#8610 (2017, open) both touch on SRV records, but aren't particularly relevant, since they're both about the handling of ldaps:// URIs specifying a SRV record. * ITS#7027 (2011, closed) and ITS#8196 (open) are concerned with the internal handling of SRV records, but not their configuration $ ldapsearch -VV ldapsearch: @(#) $OpenLDAP: ldapsearch 2.4.47 (Jul 25 2019 01:30:14) $ root@120amd64-quarterly-job-16:/wrkdirs/usr/ports/net/openldap24-sasl-client/work/openldap-2.4.47/clients/tools (LDAP library: OpenLDAP 20447)

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs