The database already exists as an mdb, I could dump and restore it with slapadd, but I expect that’d take as much time as the slapindex.
If I load the mdb from an x86 instance of openldap on an arm64 index, queries return zero results until I do a slapindex, I suspect the way indexes is encoded is inconsistent between architectures.
It certainly would’ve saved me some time experimenting with tool threads if I was aware of that limitation.
From:
Quanah Gibson-Mount <quanah@fast-mail.org>
Date: Thursday, 20 July 2023 at 15:52
To: Maud Parratt <Maud.Parratt@bjss.com>, openldap-technical@openldap.org <openldap-technical@openldap.org>
Subject: Re: slapindex a 60GB mdb in reasonable time
--On Thursday, July 20, 2023 3:28 PM +0000 Maud Parratt
<Maud.Parratt@bjss.com> wrote:
>
>
> 2.6.4 (alpine linux, run in a container)
>
> I'm unable to share the indexing schema, I'm under strict rules on
> exposing anything project specific.
>
> I've been using a tmpfs memory mounted file system for the slapindex
>
>
>
> slcToolThreads I've experimented with a bunch of settings, from 4-48
> with 16 physical cores, but no matter what I do it only ever pins one
> core to 100%.
It looks like the documentation needs updating. For back-mdb, any value
above 2 is set to 2, as it has no improvement in indexing performance.
A better question would be, is why are you running slapindex at all?
Generally one loads their database with slapadd, which includes indexing
the database. If you need to change the indexing for a specific attribute,
then you slapindex *just that attribute*.
If you are doing slapadd followed by slapindex you're wasting resources,
assuming your indexing hasn't changed.
It's unfortunate you cannot share your indexing since it's possible that
there unnecessary indexing involved.
--Quanah
The information included in this email and any files transmitted with it may contain information that is confidential and it must not be used by, or its contents or attachments copied or disclosed to, persons other than the intended addressee. If you have received this email in error, please notify BJSS. In the absence of written agreement to the contrary BJSS' relevant standard terms of contract for any work to be undertaken will apply. Please carry out virus or such other checks as you consider appropriate in respect of this email. BJSS does not accept responsibility for any adverse effect upon your system or data in relation to this email or any files transmitted with it. BJSS Limited, a company registered in England and Wales (Company Number 2777575), VAT Registration Number 613295452, Registered Office Address, 1 Whitehall Quay, Leeds, LS1 4HR