[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Sat, 9 Nov 2013 00:20:10 -0500
From: Rich Felker <dalias@...ifal.cx>
To: musl@...ts.openwall.com
Subject: Re: Latest pkgsrc results [What builds on musl 0.9.13?]
On Sat, Nov 09, 2013 at 05:02:52AM +0100, Szabolcs Nagy wrote:
> * Rich Felker <dalias@...ifal.cx> [2013-11-08 20:01:30 -0500]:
> > > top missing symbols (compile or link time):
> > >
> > > libintl_*
> > > postgres*-client, libmusicbrainz5, weechat-curses, scmxx, scmgit-base,..
> >
> > Which symbols? glibc doesn't seem to have any symbols matching
> > libintl_*.
>
> these are the gettext symbols (libintl_dgettext etc)
> for some reason they are without prefix in the libc
> but you can get them with prefix as well with -lintl
Yes, Gregor said on IRC that it's something like this...
> > > getloadavg
> > > pbzip2, tokyotyrant, distccmon-gnome
> >
> > This would probably be trivial to add.
>
> return -1?
>
> we can add it i guess but these packages could
> use configure checks (i think they are autoconfed)
Indeed. Not sure if it's useful; just reading from /proc is just as
good.
> > > NO_ADDRESS
> > > apache22, bglibs, cvsps, vanessa_logger, net6
> >
> > This I would rather not add, since we don't support it, and providing
> > the macro but never returning it could be misleading.
>
> actually NO_ADDRESS is an alias for NO_DATA in glibc and the bsds
> (for almost 20years now), but it's not critical
In that case it's probably not harmful to add. I was worried about
switch statements, but if they don't break on glibc/BSD, we should be
fine. This one really could go either way, but if the apps can be
convinced to fix it on their end, that would be nice.
> > > optreset
> > > dar, freetds
> > > (several configure checks found it incorrectly, but then it was not used)
> >
> > Hm? I think we have optreset...
>
> oops, my bad
>
> some packages didnt find optreset in configure while others did
>
> i guess the ones which failed tried unistd.h instead of getopt.h
So it sounds like this is an invalid configure check (looking for just
the symbol rather than symbol+declaration)...
> > > semtimedop
> > > sqlrelay
> >
> > This is a Linux-specific sysvsem operation we're missing, right? If so
> > I think the syscall wrapper should be added.
>
> yes
>
> IPCOP_semtimedop is 4 for SYS_ipc archs
> and the other archs have SYS_semtimedop
Thanks. Added.
> > > herror
> > > opencm
> >
> > No idea what this even is.
>
> "The (obsolete) herror() function prints the error message
> associated with the current value of h_errno on stderr."
>
> ie for printing errors of gethostbyname() and gethostbyaddr()
Ugly, especially since there seems to be no way to get these strings
without printing them to stderr...
> > > mallinfo
> > > libfirm
> >
> > Deprecated API but we could provide a dummy anyway.
>
> it's so ugly with the signed int counters, i wouldnt
> support if it's not widely used
>
> i think libfirm can be fixed: it can either do the accounting
> itself or has no business providing process-wide heap info
> statistics (it is only used for a public diagnostic api)
I thought I remembered other programs using mallinfo too, but maybe
I'm mistaken... The main reason I'd want to provide a dummy is if
there are glibc binaries using it. The range of the counters is fairly
irrelevant; keeping track of the information is prohibitively slow
anyway since it would have to be synchronized between threads on each
call to malloc/free. (glibc does some poor hack keeping this info
per-arena and summarizing it when mallinfo is called, I think...)
> > > TIMESPEC_TO_TIMEVAL
> > > libusbx
> >
> > Perhaps worth adding?
>
> TIMEVAL_TO_TIMESPEC
> TIMESPEC_TO_TIMEVAL
>
> are bsd visible in freebsd, default visible in openbsd
> and gnu visible in glibc in sys/time.h
>
> can be added but only one pkg used it from 10k..
Agreed, it's low priority and we should see if we can just get this
fixed upstream.
> > > c_ispeed, c_ospeed (struct termios members)
> > > icdprog
> >
> > If c_* is reserved we could possibly add these.
>
> c_* is reserved
I'm a bit hesitant to do anything now for fear of breaking more stuff,
but perhaps just #define __c_[io]speed c_[io]speed would be okay...
> > > libiconv_*
> > > odt2txt, ted
> >
> > Specific symbols ??
>
> i've seen some iconv failures but these are probably not libc issues
> (similar to the libintl_* case)
>
> i guess this is just missing -liconv
There should never be a libiconv except on systems that lack iconv in
libc... Maybe this is more mess created by pkgsrc though.
> gcc -o odt2txt -L/usr/lib -Wl,-R/usr/lib odt2txt.o regex.o mem.o strbuf.o kunzip/fileio.o kunzip/zipfile.o -lz
> odt2txt.o: In function `main':
> odt2txt.c:(.text.startup+0x25f): undefined reference to `libiconv_open'
> odt2txt.c:(.text.startup+0x370): undefined reference to `libiconv'
> odt2txt.c:(.text.startup+0x4eb): undefined reference to `libiconv_close'
> odt2txt.c:(.text.startup+0x7e3): undefined reference to `libiconv_open'
> odt2txt.c:(.text.startup+0x8ad): undefined reference to `libiconv'
> odt2txt.c:(.text.startup+0x95a): undefined reference to `libiconvlist'
> collect2: error: ld returned 1 exit status
>
> and there is hex-a-hop:
> ....
> configure: WARNING: unrecognized options: --with-libiconv-prefix, --with-libintl-prefix
Maybe these options are being passed to lots of (or all) packages in
pkgsrc?
> ....
> text.cpp:447:77: error: invalid conversion from const char** to char** [-fpermissive]
> iconv(cd, (const char **)&in_buf, &text_length, &out_buf, &text_utf8_length);
> ^
Odd that this works anywhere if the type is wrong... Maybe it's gcc
4.8 being more strict than previous versions?
> In file included from /usr/include/SDL/SDL_stdinc.h:74:0,
> from /usr/include/SDL/SDL_main.h:26,
> from /usr/include/SDL/SDL.h:30,
> from video.h:4,
> from text.cpp:8:
> /var/pkgsrc/games/hex-a-hop/work/.buildlink/include/iconv.h:81:15: error: initializing argument 2 of size_t libiconv(libiconv_t, char**, size_t*, char**, size_t*) [-fpermissive]
> #define iconv libiconv
> ^
> /var/pkgsrc/games/hex-a-hop/work/.buildlink/include/iconv.h:83:15: note: in expansion of macro iconv
> extern size_t iconv (iconv_t cd, char* * inbuf, size_t *inbytesleft, char* * outbuf, size_t *outbytesleft);
>
> gnuplot misses -liconv as well:
Well, rather it's wrongly trying to use libiconv. My guess is that
pkgsrc is setup for a situation where iconv is not available in libc
and you need libiconv. So the configure mess to prefix libiconv_ to
symbols gets set, but then configure detects iconv in libc and thus
doesn't add -liconv. Then they end up with missing symbols at link
time.
> ....
> /usr/lib/gcc/x86_64-linux-musl/4.8.1/../../../../x86_64-linux-musl/bin/ld: set.o: undefined reference to symbol 'libiconv_open'
> /usr/lib/gcc/x86_64-linux-musl/4.8.1/../../../../x86_64-linux-musl/bin/ld: note: 'libiconv_open' is defined in DSO /usr/lib/libiconv.so.2 so try adding it to the linker command line
> /usr/lib/libiconv.so.2: could not read symbols: Invalid operation
>
> > > KIOCSOUND, TIOCCDTR, TIOCSDTR, TIOCGSIZE (ioctl)
> > > minicom, modemd, beav
> >
> > These should almost surely be added somewhere...
>
> there are some other ioctl failures
> but there might be some bsd only things in pkgsrc
Ah, maybe so...
>
> > > execinfo.h
> > > redis, virtuoso, powerdns, packagekit, qt4-creator
> >
> > Backtrace stuff, right?
>
> yes
>
> now i see that redis also uses jemalloc which uses ffsl
> "warning: implicit declaration of function ffsl"
> that's another missing symbol
Probably worth adding.
> > > sys/bitypes.h
> > > libbind, libdnsres
> >
> > We could add this as an alias for sys/types.h with a #warning, if
> > desired. I'm fairly indifferent.
>
> it is only used by bind (i guess dnsres does the same)
> i wouldnt be surprised if that had other incurable brokenness
> but alias with warning is fine
>
> > > sys/unistd.h
> > > mlview, foobillard, galaxa, slurm
> >
> > Ditto.
>
> it seems this is a subset of unistd.h on the bsds
> with just macro definitions on glibc it is an alias
> to unistd.h
Bleh.
> > > sys/endian.h
> > > swarm
> >
> > This could probably be added even without a #warning since it's
> > nonstandard and sys/endian.h is just as "valid" as endian.h, but I'm
> > not clear whether it's worth it for ONE obscure program that could
> > just be fixed.
>
> yes
>
> > > net/ppp_defs.h
> > > poptop, slurm, xfce4-netload-plugin
> >
> > Don't know enough to comment.
>
> glibc includes linux/ppp_defs.h (plus defines some libc types)
I don't have any fundamental objection to having nonstandard headers
like this whose applicability is limited to specialized programs rely
on linux kernel headers (we already have soundcard.h or something),
but it might be better to move them ALL out of musl and into a
separate header package at some point...
> > > sys/dirent.h
> > > heirloom-libcommon
> >
> > Possibly another #warning redirect?
>
> can be, bsds have it (a subset of dirent.h)
> but glibc does not have it as far as i can tell
> so heirloom should be more clever about it
> (i guess it has some broken ifdef around it)
Perhaps __GLIBC__ :-)
> > > xlocale.h
> > > liblive
> >
> > Since these interfaces were declared in xlocale.h before being
> > standardized in POSIX 2008, it might make sense to provide the header;
> > however, the fact that only one program was found to be needing it
> > makes me doubt the usefulness.
>
> note that vlc and some other media packages
> depend on liblive, so ppl might want it
>
> glibc only defines __local_t (and local_t)
> types there
Hmm, so this header doesn't do anything useful... In that case we
should probably just see if liblive can be fixed.
> > > net/if_ether.h
> > > cia
> >
> > Is this a duplicate of the one in netinet or something different?
>
> as far as i can see it's like netinet/if_ether.h on glibc
> but glibc, freebsd, openbsd dont have it (i can only see
> it in netbsd)
>
> however netinet/if_ether.h includes net/ethernet.h on
> glibc, but it's the other way around in musl
> (and on glibc both headers include linux/if_ether.h)
>
> that's probably a bug
In musl or in glibc? :)
> > > fstab.h
> > > samba35
> >
> > Yet another hideous non-threadsafe, non-library-safe *ent API. As long
> > as there's only one program using it my feeling is to omit it.
>
> samba is probably an important one so it should be
> fixed upstream if not yet done
Yes, I agree.
> > > sys/perm.h
> > > xosview
> >
> > Seems redundant with sys/io.h?
>
> yes, we could add an alias
> (although on glibc sys/io.h has a bit more stuff)
Yes, I just don't see any use for sys/perm.h without sys/io.h.
> > > gcc-4.8.1 bugs:
> > > epic4: exec.c:118:18: internal compiler error: in set_lattice_value, at tree-ssa-ccp.c:451
> > > bullet: btInternalEdgeUtility.cpp:310:2: internal compiler error: Segmentation fault
> > > aegis fails because in libstdc++ cinttypes header is broken without -std=c++11 or -std=gnu++11
> > > boost-libs fail because mm_malloc.h has incompatible posix_memalign decl with throw()
> >
> > Was gcc built aginst musl? Assuming so, it's _possible_ that some of
> > these are musl bugs, though gcc sounds more likely.
>
> the mm_malloc.h thing is ugly, i'm not sure who else
> uses that header, but the incompatible prototype
> might bite others (i still dont quite understand
> why extern "C" api needs throw() in glibc)
It's an optimization to tell GCC that the function won't throw an
exception that reaches back to the caller. GCC should assume this by
default, but because GNU C has support for exceptions in C, it
doesn't. So they have to work around their own misfeature to avoid
suboptimal code generation. In musl I just consider that a GCC bug; I
looked for a way to work around it with global compiler flags rather
than ugly header hacks, and was told none exists, so until GCC gets
their act together and fixes this, we can just live with slightly
suboptimal C++ code generation.
> > > at least mysql5-client config uses getconf
> >
> > Hm? We should probably provide a getconf that works with musl
> > somewhere...
>
> maybe landley could do one for toybox.. :)
I'm not clear on how easy it is to do in a libc-agnostic way, but if
possible that would definitely be a nice solution.
> > > other
> > > qt3-libs, ptlib: using signed int for socklen_t
> >
> > glibc also has unsigned socklen_t, so I'm confused how they work with
> > glibc but not musl...
>
> yes, this is a mystery, actually the warning is shown for
> many packages but most of them dont use Werror for it
> maybe it's a recent gcc warning (passing int* instead of
> socklen_t* aka unsigned* to interfaces like accept)
Yes, this mismatch is a constraint violation and requires a
diagnostic, but GCC used to fail to issue even a warning for it. Maybe
recent GCC makes it an error by default?
> (and qt3-libs has quite a lot of dependencies, i think
> it is the most wanted failing package on pkgsrc now)
Yes, I would suspect so. If we get it working, we might find lots more
failures in things that depend on it...
> > > binutils, mingw-binutils: empty character constant ''
> >
> > Cause? musl-specific?
>
> from the pkgsrc logs my guess is a $ got eaten by some tool
> (maybe busybox?)
>
> binutils builds here just fine (but i build it without bfd)
>
> earm_epoc_pe.c: In function 'gld_arm_epoc_pe_place_orphan':
> earm_epoc_pe.c:1512:37: error: empty character constant
> && (dollar = strchr (secname, '')) != NULL)
> ^
> earm_epoc_pe.c:1646:25: error: empty character constant
> if (strchr (lname, '') == NULL)
> ^
I found the offending code in ld/emultempl/pe.em, but I can't figure
out why it's breaking. The line in question is:
&& (dollar = strchr (secname, '\$')) != NULL)
and running that as a here-document on Busybox ash works fine for me;
the literal $ (without the \ before it) makes it into the output.
> > > dosbox: uses undefined typename in offsetof
> >
> > A type musl is missing?
>
> rechecked and it is missing offsetof
> (so it probably fails to include stddef.h)
So another case of expecting implicit inclusion, most likely.
> > > boost/integer_traits.hpp: misses WCHAR_MIN and WCHAR_MAX because of ifdef hell
> >
> > Oh? Is this their fault or something weird we're doing?
>
> their fault, ugly ifdef hell instead of just
> including wchar.h to work around broken platforms
>
> http://boost.cvs.sourceforge.net/viewvc/boost/boost/boost/integer_traits.hpp?view=markup
I guess we should report this upstream. If they don't want to
unconditionally include wchar.h, they could use #ifndef WCHAR_MIN
instead of their silly hard-coded platform list.
> > > libjit: #error "Don't know how to determine if floating point numbers are finite"
> >
> > Is it doing stupid #ifdefs based on glibc version, or failing for some
> > semi-readonable reason?
>
> dont know, the log only has this message
OK, I guess we'll have to look into this later.
> > > tk: tkPanedWindow.o: file not recognized: File truncated
> >
> > Weird...
>
> yes the log has no context for it
> and tk is a wanted package on pkgsrc
My best guess is something odd in the makefile that caused the file to
get overwritten with junk, touched as a zero-length file, or similar.
> > > cc65, gnats, sandy: undefined siginfo_t because sys/wait.h is used with -std=c89
> >
> > This is something we need to fix. I'm thinking sys/wait.h should just
> > omit the stuff that needs siginfo_t if the right features are not
> > enabled. (Really, omitting the whole file would be valid, but I
> > suspect this would break programs worse.)
>
> makes sense
Fixed.
Rich
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
