Hallvard B Furuseth wrote:
I wrote:
It makes more sense to check for ridiculous-sized numbers before parsing them and just output a min/max value depending on sign. (Or right-truncate e.g. n*12 digits and add n*5 to the length.)
Committed. Chopping n*7 digits and adding n*3 to the binary-integer length instead. More wasteful of the exponent bits, but leaves fewer digits to parse.
This code is wrong:
/* Chop least significant digits, increase length instead */ if ( val.bv_len > k ) { chop = (val.bv_len - k + 2) / 7; /* 2 fewer digits */ val.bv_len -= chop * INDEX_INTLEN_CHOP; /* #digits chopped */ chop *= 3; /* >#key bytes chopped: 256**3 > 10**7 */ if ( chop > 0x7fffffff ) { memset( key->bv_val, neg ^ 0xff, index_intlen ); return 0; } }
On most 32 bit machines the "if( chop ...)" comparison must always fail, since ber_slen_t is only 32bits. You should do the test before multiplying chop by 3.