https://bugs.openldap.org/show_bug.cgi?id=9891

--- Comment #2 from Ondřej Kuzník ondra@mistotebe.net --- On Tue, Aug 02, 2022 at 09:07:00AM +0000, openldap-its@openldap.org wrote:

In addition, relying on sleep seems to make test case execution unstable, often failing when my environment is busy.

Yes, you are rehashing points I made a while ago[0]. Nowadays we have a nascent Python test suite in the master branch that, among other things, does not have to rely on sleeping in the same way. We welcome any help making it mature and porting the current test suite to this.

Until that's done, you can keep adjusting the environment variables SLEEP{0,1,2} to deal with the false positives.

[0]. https://lists.openldap.org/hyperkitty/list/openldap-devel@openldap.org/threa...

Thanks,

https://bugs.openldap.org/show_bug.cgi?id=9891

--- Comment #3 from Ondřej Kuzník ondra@mistotebe.net --- On Tue, Aug 02, 2022 at 03:12:31PM +0100, Howard Chu wrote:

...

Yes, you are rehashing points I made a while ago[0]. Nowadays we have a nascent Python test suite in the master branch that, among other things, does not have to rely on sleeping in the same way. We welcome any help making it mature and porting the current test suite to this.

That can speed up a few points when testing on an idle system, but it can't speed anything up when testing on a busy system. You still need to set some upper bound on waiting for events to complete, to detect when the event isn't happening at all.

Aside from that, the simple sleeps in the current test suite could be replaced by loops of short delay checks instead. Again, you still need a reasonable upper bound on timeouts for these checks, and those upper bounds will still need to be extended when running on a busy (or slow) system.

Most[0] of our delays can be avoided altogether by asking something to notify us as soon as it happens. That way we use a long (even hours if we wanted) timeout in case the notification never arrives to catch actual bugs. That's certainly true for startup and replication, which together make up 50-90% of our test suite waits.

All of a sudden that safety timeout is no longer something we'd ever need to tweak even on the slowest of systems and test suite runs would reflect the minimum time the host could actually run the test suite in. Wait-repoll loops would virtually disappear and machines that can do it will run each test even in milliseconds where at all possible, even with multiprovider replication[1].

[0]. With the obvious exception of delays where the functionality dictates certain timing, like retries, purges and expiries [1]. The following test routinely runs in ~30ms on my machine and even that can still be improved on - once syncmonitor is easier to use as a library, the extra fork/exec/interpreter startup disappears https://git.openldap.org/openldap/openldap/-/blob/master/tests/python/syncre...

Regards,

https://bugs.openldap.org/show_bug.cgi?id=9891

--- Comment #5 from Ondřej Kuzník ondra@mistotebe.net --- Sure, it's important to make that point just as it will be important to make sure there's at least one test that doesn't rely on syncprov being configured everywhere or running with mod-harness.

https://bugs.openldap.org/show_bug.cgi?id=9891

Quanah Gibson-Mount quanah@openldap.org changed:

What |Removed |Added ---------------------------------------------------------------------------- Status|RESOLVED |VERIFIED