Derek Zhou wrote:
Karsten
Unfortunately, down time is even worse than slow.
I wish OpenLDAP has an integrated search and delete operation, like delete entries based
on a search filter, all executed in the same transaction in the server. That should be
more efficient than doing search and delete from the client side because the B+ trees are
just traversed once. Also the server can impose size limit on how many entries can be
deleted in one go, just like the search, to avoid someone hogging the server.
For the LDAP gurus here, the above idea should be technically possible, right?
In normal operation, as opposed to simple experimentation, why do you actually need to
delete
a large number of entries quickly? If you just need to make them unavailable to a bunch
of
clients, you can first do a subtree rename to move them out of the way, and then delete
incrementally at whatever speed.
Derek
On Monday, November 26, 2018 03:36:16 PM Karsten Heymann wrote:
> Hi Derek,
>
> no idea about the deletion speed, but if a short downtime is an option
> and the entries to be deleted can be matched by a search filter, you
> could do a
>
> slapcat -a '(!(<filter of what should be deleted>))'
>
> and the slapadd the resulting file into an empty database. Depending
> on the amount of to entries be deleted in comparison to the total
> number of entries in the database, this could be quicker than live
> deletes.
>
> best regards
> Karsten
> Am Sa., 24. Nov. 2018 um 18:56 Uhr schrieb Derek Zhou
<derek(a)shannon-data.com>:
>>
>> dear list,
>>
>> I was experimenting with slapd by adding a lot of entries then deleting them.
What I found is adding speed is not bad but deletion speed is lacking. With or without
dbsync, delete speed is around half of that of adding. Naively, I thought deleting should
be easier than adding, because in adding you actually need to pass and write whole entry
of data, while in delete you can just mark the database page as free? The DB file did not
shrink after massive deletion; which kind of suggest that deleting is only marking page as
free, not really return them to the OS. I am using the latest git tree and the mdb
backend.
>>
>> Another related idea is delete a whole branch from the DIT. LDAP is already
hierarchical, to delete all entries under a branch, one would assume that there must be an
better way than deleting entries with a client side script, like what I am doing? With SQL
you can drop a table. With LDAP, can I delete a whole branch?
>>
>> --
>> Derek Zhou
>> Shannon Systems
>>
http://www.shannon-sys.com
>>
--
-- Howard Chu
CTO, Symas Corp.
http://www.symas.com
Director, Highland Sun
http://highlandsun.com/hyc/
Chief Architect, OpenLDAP
http://www.openldap.org/project/