Date: Sun, 10 Jan 2016 11:33:22 +0100 From: Szabolcs Nagy <nsz@...t70.net> To: musl@...ts.openwall.com Subject: Re: Possible infinite loop in qsort() * Rich Felker <dalias@...c.org> [2016-01-09 23:05:16 -0500]: > On Sat, Jan 09, 2016 at 10:07:19AM +0100, Felix Janda wrote: > > Markus Wichmann wrote: > > > This is the Leonardo number precompute loop in qsort(): > > > > > > for(lp=lp=width, i=2; (lp[i]=lp[i-2]+lp[i-1]+width) < size; i++); > > > ... > > > > musl enforces that object sizes should not be greater than PTRDIFF_MAX. > > See for example the discussion at > > > > http://www.openwall.com/lists/musl/2013/06/27/7 > > > > So there will not be objects of size 3GB with musl on x32. Since the > > Leonardo numbers grow slower than 2^n in general no overflow should > > happen if "size" is valid. Otherwise, UB was invoked. > > Note also that if you do want to use this code on an implementation > without such a guarantee, only the case where the member size is 1 can > possibly have >SIZE_MAX/2 members. In that case, you can massively > optimize out the whole sort by just counting the number of times each > byte appears (in size_t[UCHAR_MAX+1] space which is tiny), sorting the > pairs (value,count) using the comparison function, then writing out > each value the appropriate number of times. one element array would have to be special cased too: qsort(p, 1, SIZE_MAX, cmp) would oob access lp in musl qsort(p, 1, SIZE_MAX/2, cmp) would loop for more than necessary before lp[i] reaches >=size but it's harmless because lp is large enough.
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.