[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Mon, 19 Sep 2022 15:18:20 -0400
From: Rich Felker <dalias@...c.org>
To: baiyang <baiyang@...il.com>
Cc: musl <musl@...ts.openwall.com>
Subject: Re: Re: The heap memory performance (malloc/free/realloc) is
significantly degraded in musl 1.2 (compared to 1.1)
On Tue, Sep 20, 2022 at 02:44:58AM +0800, baiyang wrote:
> > Is there a reason you're relying on an unreliable and nonstandard
> > function (malloc_usable_size) to do this rather than your program
> > keeping track of its own knowledge of the allocated size? This is what
> > the C language expects you to do. For example if you have a structure
> > that contains a pointer to a dynamically sized buffer, normally you
> > store the size in a size_t member right next to that pointer, allowing
> > you to make these kind of decisions without having to probe anything.
>
> Yes, as I have been said, by comparing the number of bytes that
> realloc needs to copy in the worst case (the return value of
> malloc_usable_size), and the number of bytes we actually need to
> copy, we can optimize the performance of realloc in real scenarios
> and avoid unnecessary memory copies.
You can do exactly the same keeping track of the size yourself. The
only correct value malloc_usable_size can return is the value you
passed to the allocator. If it's returning a different size, your
malloc implementation has a problem that will make you commit UB when
you use the result of malloc_usable_size. Many real-world ones do have
this problem.
> In fact, in scenarios including glibc, tcmalloc, windows crt, mac os
> x, uclibc and musl 1.1, we did achieve good optimization results.
>
> On the other hand, of course we keep the number of bytes actually
> allocated, but it doesn't really reflect objectively the number of
> bytes to be copied by realloc when the memcpy actually occurs. And
> malloc_usable_size() more accurately reflects how many bytes realloc
> needs to copy when it degenerates back to malloc-memcpy-free mode.
I don't understand your claim here. The size you would store is
exactly the size that realloc would have to copy. It if copies more,
that's just realloc being inefficient, but the difference is not going
to be material anyway.
> So our expectation is as mentioned in the man page for linux, mac os
> or windows: "The value returned by malloc_usable_size() may be
> **greater than** the requested size of the allocation" or "The
> memory block size is always at least as large as the allocation it
> backs, **and may be larger**." - We expect to get its internal size
> to evaluate the cost of memory copying.
It's sounding more and more like you did premature optimization
without measuring any of this, since there is *no way* the possible
amount of excess copying a realloc implementation might make
internally could cost more than an extra external function call to
malloc_usable_size (even if it did nothing but return).
Rich
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
