Date: Mon, 10 Oct 2022 16:13:23 +0200 From: Florian Weimer <fweimer@...hat.com> To: Siddhesh Poyarekar <siddhesh@...hat.com> Cc: Rich Felker <dalias@...c.org>, baiyang <baiyang@...il.com>, musl <musl@...ts.openwall.com> Subject: Re: The heap memory performance (malloc/free/realloc) is significantly degraded in musl 1.2 (compared to 1.1) * Siddhesh Poyarekar: > On Tue, Sep 20, 2022 at 4:34 AM Florian Weimer <fweimer@...hat.com> wrote: >> The compiler needs to treat malloc_usable_size similar to realloc and >> just the size information for the buffer based on the return value from >> malloc_usable_size. This is admittedly harder to do than a comparable >> analysis for realloc if the compiler interprets the standard in such a >> way that after a successful realloc, any access to the original pointer >> value is undefined. >> >> malloc_usable_size is not actually *that* useful with allocators that do >> not have strict size classes because they do not over-allocate that >> much. For these allocators, it may be possible to increase the size of >> allocation significantly without moving it, but that is not reflected in >> the return value of malloc_usable_size at all. > > So the glibc manual does not document malloc_usable_size semantics > (which is weird since it is, well, a GNU extension!) I think we got it via dlmalloc, which says this: /* malloc_usable_size(void* p); Returns the number of bytes you can actually use in an allocated chunk, which may be more than you requested (although often not) due to alignment and minimum size constraints. You can use this many bytes without worrying about overwriting other allocated objects. This is not a particularly great programming practice. malloc_usable_size can be more useful in debugging and assertions, for example: p = malloc(n); assert(malloc_usable_size(p) >= 256); */ I don't think it's a GNU invention. The GNU malloc used to be Mike Haertel's malloc, as far as I can tell, and that didn't have the function. GNU malloc had a malloc_object_allocation_size function at one point, it seems, but I don't know if that was before it was a rebranded dlmalloc. The malloc subsystem wasn't part of the glibc CVS at the time. > Adding support for something that's already declared as bad > programming practice seems like a step backwards. Instead, I hope we > find a way to discourage active use of malloc_usable_size more > strongly. At least based on the systemd experience, the problem they > try to solve is that of glibc realloc being too slow for paths where > the reallocation should return the same block and that should be easy > to special-case. Is there any other valid reason to use > malloc_usable_size instead of simply using realloc? Do you know which case systemd tries to optimize? Increasing the allocation size or lowering it? Maybe we should define a non-copying version of realloc that if that's what programmers want. I've got a write-up somewhere what an a replacement for class-free allocators of jemalloc's xallocx function would look like. I think it has to involve arithmetic progressions (but no L-series), so it's not pretty. Anyway, if there is no real use case for malloc_usable_size with the current glibc realloc implementation, we should really deprecate it. Especially since C++ is moving into an incompatible direction. Thanks, Florian
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.