[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <yi4tvgu27ehkowla2z73qss5xcd5oxwjffqkszuaebgqxwydxw@25ng4ykjwqxb>
Date: Fri, 20 Jun 2025 20:03:00 +0200
From: Alejandro Colomar <alx@...nel.org>
To: Eric Blake <eblake@...hat.com>
Cc: Rich Felker <dalias@...c.org>, enh <enh@...gle.com>,
Florian Weimer <fweimer@...hat.com>, Adhemerval Zanella Netto <adhemerval.zanella@...aro.org>,
musl@...ts.openwall.com, libc-alpha@...rceware.org, Joseph Myers <josmyers@...hat.com>,
наб <nabijaczleweli@...ijaczleweli.xyz>, Paul Eggert <eggert@...ucla.edu>,
Robert Seacord <rcseacord@...il.com>, Bruno Haible <bruno@...sp.org>, bug-gnulib@....org,
JeanHeyd Meneide <phdofthehouse@...il.com>, Thorsten Glaser <tg@...bsd.de>
Subject: Re: Re: BUG: realloc(p,0) should be consistent with malloc(0)
Hi Eric,
On Fri, Jun 20, 2025 at 11:30:59AM -0500, Eric Blake wrote:
> On Fri, Jun 20, 2025 at 01:37:58AM +0200, Alejandro Colomar wrote:
> > Hey Eric!
> >
> > Thanks a lot for the detailed reply! Comments below.
>
> Ditto.
>
[...]
> > See <https://nabijaczleweli.xyz/content/blogn_t/017-malloc0.html>.
>
> Thank you for starting this. It will probably need some revisions
> before it is ready for the C committee, but hopefully this thread helps.
I'll CC you when I write a proposal for realloc(3) for C2y. I have a
clear idea of the wording we want for realloc(3) to be safe again. The
main issue I see is convoncing implementations to agree with it.
And hopefully this thread helps with that.
[...]
> > All standards since C89 have been buggy. If you are pedantic reading
> > C89, the BSDs and all the historic implementations back to the original
> > Unix V7 are non-conforming:
> >
> > <https://port70.net/~nsz/c/c89/c89-draft.html#4.10.3.4>
> >
> > Which says:
> >
> > | If size is zero and ptr is not a null pointer, the object it points to
> > | is freed.
> >
> > It's not clear whether this means that the whole action of realloc(p,0)
> > is to free(3) the pointer, or if it can also allocate a new object.
> > Under the former interpretation, the standard is at odds with reality.
> > Under the latter interpretation, I'd interpret it as saying that
> > realloc(p,0) cannot fail (and thus must free(p)), which would be an
> > interesting guarantee. I guess we'll never know what was the intended
> > reading.
>
> C89 also says (4.10.3):
> "If the size of the space requested is zero, the behavior is
> implementation-defined; the value returned shall be either a null
> pointer or a unique pointer."
>
> Putting those two sentences together, I can make a very strong case
> that an implementation where "realloc(p,0)" frees p, and then returns
> NULL, and documents that it does so, complies (the old object pointed
> to by p is free, and the size being zero means that the new object
> being NULL rather than a distinct pointer to non-dereferenceable
> storage is what the implementation documented); and this is true
> whether or not malloc(0) and realloc(p,0) differ on whether they
> return NULL, as long as both of them document their behavior on zero
> size.
>
> Another observation on C89 - it has different wording in most places
> about "if the space has been deallocated by a call to the free or
> realloc function"; where free() documents that "The free function
> causes the space pointed to by ptr to be deallocated, that is, made
> available for further allocation." The phrase "is deallocated" is
> thus precisely defined. However, the only use of the phrase "is
> freed" in that document is the one sentence you quoted about realloc
> with non-null pointer and zero size. I can argue that as an
> undocumented term, "is freed" is NOT intended to be synonymous with
> "is free()d", and is instead distinct in meaning from "is deallocated"
> (ie. "is deallocated" is how a pointer can be reused by a future
> malloc, and is no longer a distinct memory location; but "is freed"
> could be defined as contents are no longer referenceable but the
> pointer might still be allocated as a distinct location in memory and
> still safe to pass to a later "free()"). But then, you might ask, why
> does the standard talk about space that "has been deallocated by a
> call to the free or realloc function" if "realloc(p,0)" is not
> deallocating? My answer: there is another case where it is obvious
> that realloc does deallocation: namely, when realloc(p,non-zero)
> returns a new pointer that was (presumably larger) than the contents
> of the old p - the old value of p "was deallocated" and can now be
> reused by a future malloc. With that definition in hand, I can now
> argue that whether realloc(p,0) returns p (truncated and freed of its
> contents, but p is still allocated), or returns a new non-NULL pointer
> (p was freed of its contents AND deallocated in order to return the
> new pointer), an implementation where realloc(p,0) returns a non-NULL
> pointer has successfully freed p. Strenuous logic, perhaps, but we're
> already in the weeds.
>
> So, I think we can make arguments that ALL of the following can be
> considered compliant under C89 rules (although the argument is easier
> for some cases than others):
>
> 1. malloc(0) returns non-NULL, realloc(0,0) returns non-NULL,
> realloc(p,0) returns p [free(p) is still safe, but it is no longer
> safe to access contents of p]
>
> 2. malloc(0) returns non-NULL, realloc(0,0) returns non-NULL,
> realloc(p,0) returns non-NULL other than p [free(p) is unsafe]
>
> 3. malloc(0) returns NULL [0-sized objects are unsupported; presumably
> errno=EINVAL but C89 is silent on that], realloc(0,0) returns NULL,
> realloc(p,0) deallocates p and then returns NULL [presumably with
> errno set, at any rate free(p) is unsafe]
>
> 4. malloc(0) returns NULL, realloc(0,0) returns NULL, realloc(p,0)
> returns p unchanged [free(p) is still safe, but dereferencing its
> contents is no longer safe]
>
> 5. malloc(0) returns non-NULL, realloc(0,0) returns non-NULL,
> realloc(p,0) deallocates p and returns NULL [free(p) is unsafe]
>
> Of those, it looks like glibc 2.1 would be case 1 (the same pointer is
> returned, but truncated down to minimum size), glibc 2.2 to present
> would be case 5 (the pointer is freed, the function returns NULL even
> though it inconsistent with malloc(0) being able to return zero-sized
> objects), and other traditional implementations could be either case 2
> (a non-NULL pointer is returned because zero-sized objects are always
> possible, but because it was distinct from p it also met the rule
> about having freed p) or case 1 (the call "freed" the contents of p,
> but did not deallocate it).
Agreed. It seems then that C89 allowed basically everything. I'll
update the manual page patch to reflect that.
[...]
> > C99 changed the specification, probably because of how ambiguous it was.
> >
> > glibc was also buggy, as it differed from every other Unix-like system.
> > All Unix systems behaved as if free(p) and malloc(n). glibc is the only
> > one that didn't follow this obvious consistency rule.
>
> Maybe the intended wording was that "if realloc(p,s) returns a
> non-NULL value distinct from p, then p was deallocated and the new
> value obtained as if by malloc(n)". After all, the reason realloc()
> exists is for the cases where realloc(p,s) can return p (ie. resized
> in place, whether by truncating and optionally handing back an unused
> tail to the system, or by expanding where the new tail was already
> accessible in place even though it has unspecified contents); it's
> only when the resize-in-place can't happen that realloc() must then
> arrange to copy contents from the old pointer to the new. In fact,
> even though portable code must not expect specific contents in the new
> pointer if the sequence free(p);malloc(s) happens to reuse p, I could
> totally see how some (possibly-older) implementations of malloc() have
> sufficient locking in place where it may be easier to try and resize
> pointer p by free(p)malloc(s) and only if the resize changed locations
> then do the copying - as long as the rest of the application can't
> corrupt the contents in the old location before the new location is
> finally returned to the user. But even if it was the intended wording
> (or if that is the wording that we hope to have in place in the
> future), unfortunately it is not the actual wording.
Indeed, the Unix V7 implementation did something like
free(p);
malloc(s);
and _after_ that it copied the contents if necessary. I'm talking from
memory, but I remember having had fun reading that code.
[...]
> In fact, I'm almost inclined to say that it was C89's wording (and not
> C99's) that was the reason that glibc flipped their default to having
> realloc(p,0) return NULL (because it was C89's wording that made it
> into POSIX); and it was C99's debate on newer wording that brought the
> issue to light. And yet, here we are, STILL trying to get better
> wording into both C and POSIX.
Fully agree. That seems the most likely reason, after reading the
mailing list archives that Paul shared. I suspect they saw a draft of
C9x that was still essentially C89 (so before the changes that ended up
in C99), and they thought it was new text, while it was actually old C89
text that they weren't aware of.
> > > > Indeed, glibc is non-conforming to C99 too. Although, I don't like the
> > > > wording from C99, either; it allows weird stuff: it allows an
> > > > implementation where malloc(0) returns NULL and realloc(p,0) non-null
> > > > (so, the opposite of glibc).
> > > >
> > > > C11 is essentially identical to C99 in that regard, so glibc is also
> > > > non-conforming to C11.
>
> Here, I'm inclined to argue the opposite: glibc IS compliant with C99
> and C11, and the commit history in glibc shows that the change to have
> realloc(p,0) return NULL was made at the time of C99 on the grounds of
> a compliance argument, even if it might have been misguided. It was
> C89, not C99, that explicitly required p to be freed; and C99 was
> clarifying that the old object is deallocated before the new object
> (if any) is returned, even if the pointer is the same. And again, it
> stems back to the fact that C says it is implementation-defined
> whether a size of 0 returns NULL or a distinct pointer, and has no
> requirements on errno being set. Presumably, as long as you are
> willing to set errno=0, call realloc(p,0), and then on NULL check if
> errno==EINVAL (p is still valid) or still unset (p was freed), then
> glibc's implementation complies, even though it does not match
> historical behavior of either the implemenations where malloc(0)
> always fails (a zero-sized object is not possible) nor the
> implementations where realloc(p,0) always returns non-NULL. POSIX
> then tried to add the rules to be able to distinguish between NULL
> meaning success and being an EINVAL error (since C99 didn't).
I disagree with this. When you have a chance to read the rest of my
previous email, you'll see why I think glibc doesn't conform to C99.
> I'm out of time today to reply to anything later in your email.
Okay; thanks! Please reply to the rest when you have time.
Have a lovely day!
Alex
--
<https://www.alejandro-colomar.es/>
Download attachment "signature.asc" of type "application/pgp-signature" (834 bytes)
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
