Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Wed, 27 Aug 2014 12:34:13 -0400
From: Rich Felker <dalias@...c.org>
To: musl@...ts.openwall.com
Subject: Re: variadic args (was: compiling musl on x86_64 linux
 with pcc)

On Wed, Aug 27, 2014 at 10:52:54AM +0200, u-igbb@...ey.se wrote:
> On Wed, Aug 27, 2014 at 03:54:07AM -0400, Rich Felker wrote:
> > > I am possibly not realizing some crucial bit but I do not see why the
> > > implementation of stdarg.h must be the business of the library. If the
> > 
> > Because other headers need to know how to get va_list defined for use
> > in function prototypes, and they can't include stdarg.h because it
> > would expose more than they want. glibc handles this by making use of
> 
> I guess this is the matter which everything revolves around.
> What is it that stdarg.h would expose too much?

va_arg, va_start, va_end, va_copy.

> > a special inclusion protocol between the glibc headers and the gcc
> > headers: it defines a special macro before including stdarg.h telling
> > gcc that it just wants the definition of va_list and nothing else.
> 
> Wonder whether it would be reasonable to say that musl expects from
> every compiler to be used with the library either
> a stdarg.h with following properties: ...
>  <which ones?>
> otherwise expecting the compiler to implement the following builtins:
>  <the current contents of musl's stdarg.h>

This would require changes from the compiler anyway (and then the
compiler would be using musl-header-internal interfaces that we do not
define as stable and do not intend for outside use), or require
encoding assumptions about specific compilers' headers in several
places in musl's headers. It would also

> I do not see how this would hurt in any way.
> 
> By the way, this - in any case - *is* an interface against the compiler.
> 
> As a matter of fact, the bultins naming differs between compilers.

Could you provide an example where the naming differs? I have not seen
it. (It should be a working C compiler that could otherwise reasonably
be used with musl, not something like MSVC, or a 16-bit x86 compiler,
or anything else that's definitely not usable. :)

> So the current musl approach is to just boldly assume a *certain*
> relatively arbitrary interface, not "the only possible" nor "the only
> reasonable one".

The same is necessary for things like INF/NAN, complex.h I, etc.;
there is no standard way to express them in C, so we choose the
universally-agreed-upon way to do it.

> > > A compiler used *together* with the library may lack the builtins but it
> > > still can be fully compatible even if in some situations less efficient.
> > 
> > No, it can't. There is simply no way to express, in C, the concept of
> > "get the next variadic argument" without __builtin_va_arg or something
> > 100% equivalent to it. If you have a compiler than chooses to be
> 
> Wait, this is not a library busyness, this is something which happens
> _inside_ a function. Passing variadic arguments between internal functions
> can also be left to the compiler and this should not have to be reflected
> in the library's public headers.

It does matter to the library because the types in the headers have to
match. va_list is not used only in stdarg.h.

> > Again, this has nothing to do with ABI between external variadic
> > functions. It's a matter of the inability to express to the compiler
> > what you want it to do (get the next variadic argument) in plain C.
> 
> It is of course only the compiler who knows how to do va_arg, but its
> internal knowledge can be expressed *either* by __builtin_va_arg *or*
> by an include file defining va_arg in a compiler-specific fashion
> *or even* by implementing va_arg as a function like tcc does.

Setting up your compiler to target musl already requires some specific
tuning like -Wl,-dynamic-linker,... if you're using dynamic linking
(GCC requires more tweaking to remove some glibc assumptions); ideally
such tweaking is added in the compiler's profile for musl, or in
something like spec files or whatever. There's no reason you can't use
'-D__builtin_va_XXX(x)=...' in such a context, unless the compiler
reserves this exact same name but uses it for something completely
different that's impossible to override, and in that case the same
argument applies to lots of musl-internal stuff beginning with __ in
the public headers. This is because __builtin_va_list, etc. are in the
reserved namespace.

> I do not argue that not-builtins are "good" but frankly I do not see
> how these non-builtins-ways to express the compiler specific details
> are contradicting standards or compatibility.

There is simply no way a decent compiler could use the non-builtin
definitions. As I've stated before, they preclude optimization, and
since the way they are expressed overlaps with highly invalid pointer
arithmetic for which a good compiler would want to produce warnings,
they also preclude warning about buggy code.

Note that the nonsense legacy definition would not even be in musl's
i386 bits/stdarg.h if I were writing it today. That header was written
something like a decade ago when i was using it with a much older
toolchain and system, and basically left and forgotten.

> The difference from a builtin is that a definition in a header is "less
> opaque", but the preprocessor macros are a natural abstraction layer in
> C and are not supposed to be looked behind.

I really don't want to argue about this anymore. It's not achieving
anything productive. Your proposed changes on musl's side would not be
likely to help anyone achieve anything they can't already achieve in
simpler ways (-D) and would uglify the headers and make new
cross-component (compiler and libc) dependencies between internal
headers inclusion-control conventions -- one of the original things
musl was intended to avoid.

Also, in the time you've spent arguing for supporting something that's
obviously broken (using illegal pointer arithmetic to represent
variadic args) you could probably have gotten a patch adding
__builtin_va_* into tcc, using predefined macros with the current bad
definitions if nothing else.

Rich

Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.