Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Sat, 24 Sep 2022 01:55:09 -0700
From: Fangrui Song <>
To: James Y Knight <>,,
	Florian Weimer <>, Rich Felker <>,
	baiyang <>
Subject: Re: The heap memory performance (malloc/free/realloc) is
 significantly degraded in musl 1.2 (compared to 1.1)

On 2022-09-20, Szabolcs Nagy wrote:
>* James Y Knight <> [2022-09-20 12:59:00 -0400]:
>> On Tue, Sep 20, 2022 at 9:58 AM Siddhesh Poyarekar <>
>> wrote:
>> > Adding support for something that's already declared as bad
>> > programming practice seems like a step backwards.  Instead, I hope we
>> > find a way to discourage active use of malloc_usable_size more
>> > strongly.
>> BTW, if folks aren't aware, there is already work on the C++ side to expose
>> an API which lets you request a heap allocation of _at least_ the given
>> size, which rounds the actual size up in whatever way the allocator likes,
>> and returns the pointer and actual size allocated. With this API, you
>> declare an explicit intent that all of the memory -- up to the returned
>> size -- is valid to use without needing to go back to the allocator to ask
>> for more.
>> The proposal is still making its way through the standardization process,
>> but hopefully it'll make it into the next version of C++ after C++23.  (Of
>> course, that's not a sure thing until it happens.) Here's the doc, with
>> more rationale/etc:
>this does not seem to discuss how existing applications
>that override new() would cope with this.
>nor how existing implementations on top of c allocators
>would implement it (given that we just agreed that
>malloc_usable_size is not suitable for such use).
>nor how existing allocator tooling (interposers, profilers)
>would handle the new interface.
>> Also, as noted in the doc, jemalloc experimentally implemented this
>> functionality in its non-standard API, via a function it called "smallocx"
>> -- though jemalloc hides the API so it can't be used by default. The API is
>> effectively:
>>   typedef struct { void *ptr; size_t size; } smallocx_return_t;
>>   smallocx_return_t smallocx(size_t size, int flags);
>> (That's consistent with jemalloc's other non-standard APIs, which stick
>> alignment/etc into a "flags" argument, but probably not suitable for a
>> more-standardized cross-implementation API)
>> tcmalloc implements similar functionality, as well, with family of
>> functions named "tcmalloc_size_returning_operator_new":
>so there are already incompatible c apis, which means this
>should not be considered a viable proposal at this point.

Small addition: (allocate_at_least) has made it into C++23. libc++ has implemented it in the
trivial way that just returns the user-requested size. 

>> which of course also isn't a suitable API to support cross-implementation.
>> If someone wants to push forward this area, IMO, it would be really great
>> to have an API exposing this functionality designed to be implemented in a
>> common way across libc malloc implementations -- and eventually added to
>> POSIX or C.
>this is done the wrong way around.

Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.