https://bugs.openldap.org/show_bug.cgi?id=9463
--- Comment #4 from Ondřej Kuzník ondra@mistotebe.net --- On Tue, Aug 17, 2021 at 04:27:09AM +0000, openldap-its@openldap.org wrote:
Which tests are more likely to fail?
I've seen test034 failing at least. You can run specific tests this way: ``` cd tests ./run -b <backend> -l <iterations> <test-script> ```
e.g. ./run -b wt -l 50 test034
So far, many failures have been timing issues. I will work on improving the test.
The point of CI is to test the code repeatedly, catching obvious bugs immediately and those that are not reproducible all the time at some point. Re-enable the backend in your fork (making sure you set appropriate CI timeout) and see if you can make it more reliable. If you find see what you thing is a timing issue, first explain how it pans out, why it's not a bug in the code being tested or an intrinsic issue of the test scenario - e.g. the second commit in MR!282 still lacks a decent explanation what it is that happens that you're trying to fix.
By the way, testing all backends is very expensive, so How about doing manual testing about minor backend instead of per-commit? You will be able to do a full test at the release point.
Symas do "full tests" at release point, which include a more thorough replication test, those that are not easily achievable in the current CI infrastructure. Also MR!347 attempts to introduce a new test platform that could run more comprehensive and robust tests much quicker.
Regards,