Howard Chu writes:
I had considered MD5 before (especially since we already had code
it) but it was slower, and we're not looking for cryptographic
assurances or hash distribution anyway.
Yes, I was thinking mostly of killing unnecessary code and exposed
features. After a brief test, I could see no good reason to keep CRC:
Neither speed nor quality. On my machine, anyway.
Basically all of these crypto hash functions are overkill, in terms
hash size and computation. We're only looking to detect casual misuse
or corruption, not malicious deception.
Yes. The main reason I see to use either CRC or MD5 is that they are
likely to be installed somewhere so non-programmers can use them. We
could use a faster hash like MurmurHash3, but would then need to provide
a 'slaputil hash' command or something for use by shell programs.
I didn't really spend a lot of time comparing the two
speed. But even with the memory access bottleneck, I would guess that
on a loaded system with many threads running, the algorithm with fewer
instructions is the better choice. Have you measured the throughput
when multiple threads are executing?
On my 32-bit host, MD5 on a threaded test program had 90-95% of CRC's
throughput instead of 105% or whatever it was unthreaded. OTOH crc32()
from linking -lz gave ~275%. OTOH I'd expect MD5 to be more costly on
an older or cheaper machine where the which hasn't outpaced memory speed
as much as modern workhorses.