Date: Tue, 20 Sep 2022 01:40:48 +0800 From: baiyang <baiyang@...il.com> To: "James Y Knight" <jyknight@...gle.com>, musl <musl@...ts.openwall.com> Cc: "Florian Weimer" <fweimer@...hat.com> Subject: Re: Re: The heap memory performance (malloc/free/realloc) is significantly degraded in musl 1.2 (compared to 1.1) Hi James, I looked at the code of tcmalloc, but I didn't find any of the problems you mentioned in the implementation of malloc_usable_size (see: https://github.com/google/tcmalloc/blob/9179bb884848c30616667ba129bcf9afee114c32/tcmalloc/tcmalloc.cc#L1099 ). On the contrary, similar to musl, tcmalloc also directly uses the return value of malloc_usable_size in its realloc implementation to determine whether memory needs to be reallocated: https://github.com/google/tcmalloc/blob/9179bb884848c30616667ba129bcf9afee114c32/tcmalloc/tcmalloc.cc#L1499 I think this is enough to show that the return value of malloc_usable_size in tcmalloc is accurate and reliable, otherwise its own realloc will cause a segment fault. Thanks :-) -- Best Regards BaiYang baiyang@...il.com http://i.baiy.cn **** < END OF EMAIL > **** From: James Y Knight Date: 2022-09-19 21:53 To: musl CC: Florian Weimer; baiyang Subject: Re: [musl] The heap memory performance (malloc/free/realloc) is significantly degraded in musl 1.2 (compared to 1.1) Indeed. RedHat mentioned that problem in their recent post about _FORTIFY_SOURCE=3, here https://developers.redhat.com/articles/2022/09/17/gccs-new-fortification-level """ _FORTIFY_SOURCE=3 revealed another pattern. Applications such as systemd used malloc_usable_size to determine available space in objects and then used the residual space. The glibc manual discourages this type of usage, dictating that malloc_usable_size is for diagnostic purposes only. But applications use the function as a hack to avoid reallocating buffers when there is space in the underlying malloc chunk. The implementation of malloc_usable_size needs to be fixed to return the allocated object size instead of the chunk size in non-diagnostic use. Alternatively, another solution is to deprecate the function. But that is a topic for discussion by the glibc community. """ On Mon, Sep 19, 2022 at 9:47 AM Rich Felker <dalias@...c.org> wrote: On Mon, Sep 19, 2022 at 02:36:41PM +0200, Florian Weimer wrote: > * Szabolcs Nagy: > > > unlike musl those implementations don't return exact size nor have the > > same security and memory fragmentation guarantees, so bad comparision. > > > > tcmalloc: > > // Returns the actual number N of bytes reserved by tcmalloc for the pointer > > // p. This number may be equal to or greater than the number of bytes > > // requested when p was allocated. > > // > > // This function is just useful for statistics collection. The client must > > // *not* read or write from the extra bytes that are indicated by this call. > > > > jemalloc: > > <para>The <function>malloc_usable_size()</function> function > > returns the usable size of the allocation pointed to by > > <parameter>ptr</parameter>. The return value may be larger than the size > > that was requested during allocation. The > > <function>malloc_usable_size()</function> function is not a > > mechanism for in-place <function>realloc()</function>; rather > > it is provided solely as a tool for introspection purposes. Any > > discrepancy between the requested allocation size and the size reported > > by <function>malloc_usable_size()</function> should not be > > depended on, since such behavior is entirely implementation-dependent. > > These implementations are buggy or at least mis-documented. The > interface contract is clearly that for that particular object, the extra > bytes in the allocation are available for reading and writing. It is > not guaranteed that the allocator will always provide the same number of > extra bytes for the same requested size, but they must be there for the > allocation being examined. It's even in the name of the function! I'm not sure I understand what you're saying, but the core problem that really can't be solved is potential discrepancy between the malloc implementation's idea of usable and the compiler's. For example: char *p = malloc(1); if (malloc_usable_size(p)>1) p = 42; will cause a compiler that's actively detecting UB to abort the program when malloc_usable_size returns a value larger than 1. Rich Content of type "text/html" skipped
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.