Date: Wed, 14 Sep 2016 12:32:53 +0200 From: u-uy74@...ey.se To: musl@...ts.openwall.com Subject: Re: incompatibility between libtheora/mmx and musl ? On Tue, Sep 13, 2016 at 04:43:04PM -0400, Rich Felker wrote: > > Libtheora contains an assembler part for ia32 and x86_64 which has been > > in use for many years, with at least both glibc and uclibc. > > > > Now, musl-based builds of libtheora for ia32 with this code enabled > > lead to the encoder segfaulting. No problem when using the C version. > > https://trac.xiph.org/ticket/2287 > > > > There is also a corresponding > > > > https://bugs.alpinelinux.org/issues/6132 > > The most likely explanation is that they're overflowing a heap buffer. > Perhaps it would be possible to pad all their malloc/realloc calls > with +64 or so to see if that makes the problem go away. If so that > gives a good starting point for tracking down the bug. Thanks for the suggestion, indeed libtheora seems to consistently use ogg_malloc() which is redefinable. Building with #define _ogg_malloc(x) malloc((x)+256) #define _ogg_calloc(x,y) calloc((x)+256,(y)) #define _ogg_realloc(y,x) realloc((y),(x)+256) #define _ogg_free free instead of the default #define _ogg_malloc malloc #define _ogg_calloc calloc #define _ogg_realloc realloc #define _ogg_free free did not make any difference. The crash on a test file occurs in the same way and the resulting partial output file is as long as otherwise. This may mean that this is not a simple overflowing but rather overwriting or reading distant "random" places (?) (register corruption?) Rune
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.