--On August 6, 2009 9:35:18 AM -0700 Quanah Gibson-Mount quanah@zimbra.com wrote:
In line with checking the perf issues between 2.3 and 2.4, I've started looking at how well each release scales. The previous benchmarks were taken from saturating the server with 48 slamd clients starting with 1 thread, and then increasing the number of threads per client across each iteration. I.e., iteration 1 has 48 connections doing queries. Iteration 2 has 96 connections doing queries, etc.
For the scalability test, I switched to 9 slamd clients, incrementing 1 thread per iteration. What I see as a result is that 2.3 scales better, and keeps substantially more consistent results as the number of clients increases, whereas 2.4 gets consistently worse as the number of clients increases. I.e., 2.4 does not scale particularly well. The test ends after 3 consecutive non-improving increments.
Howard suggested I try this test series using back-null rather than back-bdb, so that we could determine whether or not the connection manager was at fault. It is clear from the results that the connection manager is *not* the issue, as OL 2.4 vastly surpasses OL 2.3 when using back-null. In fact, I ran the test a second time on 2.3 to be sure it wasn't a bad run the first time, and ended up with virtually identical results. In summary:
OL 2.3 maxed at 18 clients (2 iterations) OL 2.4.7 maxed at 153 clients (17 iterations) OL 2.4.17 maxed at 234 clients (26 iterations)
This would imply that the tremendous slowdown is directly a result of something being done in back-bdb/back-hdb.
OL2.3: Job Iterations 20090806173944-87277079-1 39373.546 Stopped because the maximum duration had been reached 20090806173944-87277079-2 47256.904 Stopped because the maximum duration had been reached 20090806173944-87277079-3 46438.308 Stopped because the maximum duration had been reached 20090806173944-87277079-4 45903.442 Stopped because the maximum duration had been reached 20090806173944-87277079-5 45995.692 Stopped because the maximum duration had been reached
OL 2.4.7: Job Iterations 20090806135128-77860977-1 36435.079 Stopped because the maximum duration had been reached 20090806135128-77860977-2 48183.404 Stopped because the maximum duration had been reached 20090806135128-77860977-3 48647.312 Stopped because the maximum duration had been reached 20090806135128-77860977-4 49940.312 Stopped because the maximum duration had been reached 20090806135128-77860977-5 50594.942 Stopped because the maximum duration had been reached 20090806135128-77860977-6 51580.254 Stopped because the maximum duration had been reached 20090806135128-77860977-7 51938.067 Stopped because the maximum duration had been reached 20090806135128-77860977-8 52452.596 Stopped because the maximum duration had been reached 20090806135128-77860977-9 52942.708 Stopped because the maximum duration had been reached 20090806135128-77860977-10 53301.662 Stopped because the maximum duration had been reached 20090806135128-77860977-11 53608.279 Stopped because the maximum duration had been reached 20090806135128-77860977-12 53585.854 Stopped because the maximum duration had been reached 20090806135128-77860977-13 53898.312 Stopped because the maximum duration had been reached 20090806135128-77860977-14 54304.983 Stopped because the maximum duration had been reached 20090806135128-77860977-15 54326.067 Stopped because the maximum duration had been reached 20090806135128-77860977-16 54532.996 Stopped because the maximum duration had been reached 20090806135128-77860977-17 54716.604 Stopped because the maximum duration had been reached 20090806135128-77860977-18 54605.483 Stopped because the maximum duration had been reached 20090806135128-77860977-19 54297.504 Stopped because the maximum duration had been reached 20090806135128-77860977-20 54403.079 Stopped because the maximum duration had been reached
OL 2.4.17: Job Iterations 20090806153211-90890978-1 37449.696 Stopped because the maximum duration had been reached 20090806153211-90890978-2 48954.121 Stopped because the maximum duration had been reached 20090806153211-90890978-3 50264.058 Stopped because the maximum duration had been reached 20090806153211-90890978-4 51336.096 Stopped because the maximum duration had been reached 20090806153211-90890978-5 52332.388 Stopped because the maximum duration had been reached 20090806153211-90890978-6 53114.296 Stopped because the maximum duration had been reached 20090806153211-90890978-7 53675.608 Stopped because the maximum duration had been reached 20090806153211-90890978-8 54105.358 Stopped because the maximum duration had been reached 20090806153211-90890978-9 54442.021 Stopped because the maximum duration had been reached 20090806153211-90890978-10 54734.250 Stopped because the maximum duration had been reached 20090806153211-90890978-11 54827.133 Stopped because the maximum duration had been reached 20090806153211-90890978-12 54873.817 Stopped because the maximum duration had been reached 20090806153211-90890978-13 55318.479 Stopped because the maximum duration had been reached 20090806153211-90890978-14 55244.354 Stopped because the maximum duration had been reached 20090806153211-90890978-15 55370.579 Stopped because the maximum duration had been reached 20090806153211-90890978-16 55413.208 Stopped because the maximum duration had been reached 20090806153211-90890978-17 55463.596 Stopped because the maximum duration had been reached 20090806153211-90890978-18 55423.079 Stopped because the maximum duration had been reached 20090806153211-90890978-19 55655.408 Stopped because the maximum duration had been reached 20090806153211-90890978-20 55412.900 Stopped because the maximum duration had been reached 20090806153211-90890978-21 55656.488 Stopped because the maximum duration had been reached 20090806153211-90890978-22 56010.821 Stopped because the maximum duration had been reached 20090806153211-90890978-23 55971.654 Stopped because the maximum duration had been reached 20090806153211-90890978-24 56168.250 Stopped because the maximum duration had been reached 20090806153211-90890978-25 56076.054 Stopped because the maximum duration had been reached 20090806153211-90890978-26 56179.308 Stopped because the maximum duration had been reached 20090806153211-90890978-27 56018.808 Stopped because the maximum duration had been reached 20090806153211-90890978-28 55944.858 Stopped because the maximum duration had been reached 20090806153211-90890978-29 55578.896 Stopped because the maximum duration had been reached
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration