Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Sun, 2 Oct 2016 12:59:50 +0200
Subject: "non-float" malloc (was: incompatibility between libtheora/mmx and

Hello Szabolcs,

On Wed, Sep 14, 2016 at 04:41:45PM +0200, Szabolcs Nagy wrote:
> this might be an issue:
> musl uses float instructions in malloc,
> if mmx needs different fpu state and
> they don't change it back for a malloc
> call that can corrupt the heap.
> to test this, try to use the 'non-float bin index'
> code in musl from here:

Thanks again for the solution to the libtheora mystery.

Now I have reported the same issue affecting ffmpeg, it contains quite
a bit of simd assembler code and assumed until now among others that
malloc() never uses floating point.

Hopefully a fix is on the way there.

For the time being, to have a working ffmpeg build,
I tried to use your "non-float" malloc.

Unfortunately this leads to a segfault when used with the explicit runtime
loader, because brk() happily grows over the dynamic loader's data.
The modified variant of the malloc code seems to not apply
the traverses_stack_p() check.

I guess as long as you want to keep the special malloc.c
you would like it to be robust.

Would you mind adding the brk()/stack overlap checking to this variant
of the code?


Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.