[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20250621024258.GI1827@brightrain.aerifal.cx>
Date: Fri, 20 Jun 2025 22:42:58 -0400
From: Rich Felker <dalias@...c.org>
To: Paul Eggert <eggert@...ucla.edu>
Cc: Alejandro Colomar <alx@...nel.org>, Thorsten Glaser <tg@...bsd.de>,
libc-alpha@...rceware.org, bug-gnulib@....org,
musl@...ts.openwall.com,
наб <nabijaczleweli@...ijaczleweli.xyz>,
Douglas McIlroy <douglas.mcilroy@...tmouth.edu>,
Robert Seacord <rcseacord@...il.com>,
Elliott Hughes <enh@...gle.com>, Bruno Haible <bruno@...sp.org>,
JeanHeyd Meneide <phdofthehouse@...il.com>,
Adhemerval Zanella Netto <adhemerval.zanella@...aro.org>,
Joseph Myers <josmyers@...hat.com>,
Florian Weimer <fweimer@...hat.com>,
Laurent Bercot <ska-dietlibc@...rnet.org>,
Andreas Schwab <schwab@...e.de>, Eric Blake <eblake@...hat.com>,
Vincent Lefevre <vincent@...c17.net>,
Mark Harris <mark.hsj@...il.com>,
Collin Funk <collin.funk1@...il.com>,
Wilco Dijkstra <Wilco.Dijkstra@....com>, DJ Delorie <dj@...hat.com>,
Cristian Rodríguez <cristian@...riguez.im>,
Siddhesh Poyarekar <siddhesh@...plt.org>,
Sam James <sam@...too.org>, Mark Wielaard <mark@...mp.org>,
"Maciej W. Rozycki" <macro@...hat.com>,
Martin Uecker <ma.uecker@...il.com>,
Christopher Bazley <chris.bazley.wg14@...il.com>,
eskil@...ession.se
Subject: Re: Re: alx-0029r1 - Restore the traditional realloc(3)
specification
On Fri, Jun 20, 2025 at 06:34:25PM -0700, Paul Eggert wrote:
> On 2025-06-20 18:06, Alejandro Colomar wrote:
> > + to both return a null pointer and set errno to ENOMEM.
>
> Given the proposed change, I don't see why ENOMEM must be added to the C
> spec. The spec changes malloc and realloc so that returning null means
> failure (i.e., out of memory) and returning non-null means success. In that
> case, there seems to be no pressing need for ENOMEM in the C standard (as
> opposed to POSIX).
>
> If ENOMEM is really needed, its presence should be justified in the
> proposal. Alternatively, the proposal could be divided into two: the main
> proposal is the null-if-and-only-if-failure part, and ENOMEM could be the
> secondary proposal.
>
> One other thing: Doug McIlroy's point was that realloc(p,n) should never
> fail when p already addresses storage of size n or greater. As a corollary,
> realloc(p,0) should never fail when p is non-null. It might be helpful to
> add this as a third proposal. A lot of apps already assume McIlroy's point,
> after all.
I'm very much opposed to such a change. Many (I would say most good)
allocator designs do not admit resizing down in-place due to
segregating allocations by size in slab-like strategies where the size
of the object may even be implicit in its address. While it's possible
to make them simply decline to resize-down, leaving the original
full-sized allocation, this has multiple very negative properties:
1. It exacerbates OOM conditions by preventing the application from
knowing about them and being able to react. If the application
thinks N operations of resizing allocations each of some large size
M down to some small size K succeeded, it has tied up N*(M-k) bytes
more than it expected, and it does not know it can free these
objects to reclaim this memory.
2. It undermines hardening. Our malloc implementation guarantees
catching even single-nonzero-byte buffer overflows past the nominal
allocation size. This requires having the ability to represent the
exact size. Because the difference between the slot size and
nominal allocation size is bounded and small, the representation of
the difference can be stored very compactly. This cannot be done as
efficiently or in the same manner if the difference can be
arbitrarily large.
3. It breaks the common extension function malloc_usable_size. As in
(2) above, we have malloc_usable_size return the exact nominal
size. This is necessary to prevent inconsistencies where the
compiler optimizes with knowledge that the object size is at most
N1 but malloc_usable_size returns a larger value N2, breaking
catastrophically. (Admittedly having malloc_usable_size to begin
with was the mistake here, but that ship has sailed.)
Applications have always needed to be prepared for the possibility
that realloc can always fail, including when reducing the size. Unless
I'm mistaken, this happens even on dlmalloc-like allocators when the
old size is above the "mmap threshold" and the new size is below. A
good allocator is not going to leave an object that was originally
allocated at a large 256k buffer using at least a whole page when it's
resized to 16 bytes.
As this proposal is completely unrelated to fixing the UB and
inconsistency of realloc(p,0) on some implementations, and it's a
breaking change as described above, I would like to see it dropped
from the agenda.
Rich
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
