Date: Fri, 9 Sep 2016 19:55:14 -0400 From: Rich Felker <dalias@...c.org> To: musl@...ts.openwall.com Subject: Re: [PATCH][RFC] fix strtod int optimization in non-nearest rounding mode On Sun, Sep 04, 2016 at 04:51:03AM +0200, Szabolcs Nagy wrote: > the mid-sized integer optimization relies on lnz set up properly > to mark the last non-zero decimal digit, but this was not done > if the non-zero digit lied outside the KMAX digits of the base > 10^9 number representation. > > so if the fractional part was a very long list of zeros (>2048*9 on > x86) followed by non-zero digits then the integer optimization could > kick in discarding the tiny non-zero fraction which can mean wrong > result on non-nearest rounding mode. > > strtof, strtod and strtold were all affected. And *scanf. :-) > --- > the dc counter is long long, but lnz is int, so this is > not correct on 64bit targets where dc can be >INT_MAX, > probably lnz should be just set to KMAX*9 or similar in > this case. The issue is not exclusive to 64-bit targets. You can scanf a >2GB decimal string on 32-bit too. We could just set lnz to match the location where the |=1 takes place (on the order of KMAX*9), or we could just make lnz long long (it's only used one other place). Your approach is probably nicer though. Rich
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.