Date: Fri, 8 Nov 2013 20:01:30 -0500 From: Rich Felker <dalias@...ifal.cx> To: musl@...ts.openwall.com Cc: Gregor Richards <gr@...due.edu> Subject: Re: Latest pkgsrc results [What builds on musl 0.9.13?] On Fri, Nov 08, 2013 at 12:46:31AM +0100, Szabolcs Nagy wrote: > * Szabolcs Nagy <nsz@...t70.net> [2013-11-07 03:46:52 +0100]: > > * Gregor Richards <gr@...due.edu> [2013-11-06 21:22:14 -0500]: > > > I've uploaded the complete log for this run to Google Drive: https://googledrive.com/host/0BwnS5DMB0YQ6Tjd4Z2RXeTdlaDA/pkgsrc-results-2013Q2-b44694ce2dd6-musl-0.9.13-log.txt.xz > > > . If there's demand, I'll find a less silly solution in the future. > > > > thanks, that is good enough for me > > i processed a large chunk of the logs, here is a summary: Thanks! Do you have any recommended actions on any of them (aside from fixing applications)? > top missing symbols (compile or link time): > > libintl_* > postgres*-client, libmusicbrainz5, weechat-curses, scmxx, scmgit-base,.. Which symbols? glibc doesn't seem to have any symbols matching libintl_*. > getloadavg > pbzip2, tokyotyrant, distccmon-gnome This would probably be trivial to add. > NO_ADDRESS > apache22, bglibs, cvsps, vanessa_logger, net6 This I would rather not add, since we don't support it, and providing the macro but never returning it could be misleading. > res_mkquery, res_send > hesiod, openldap-server, queryperf, traceroute-nanog, avenger, mtr These were on the low-priority agenda for stuff leading up to 1.0, but got pushed back because I want to focus on fixing bugs and such. > optreset > dar, freetds > (several configure checks found it incorrectly, but then it was not used) Hm? I think we have optreset... > strtoq > tdb, ia64sim I'd considered adding the alias, but if only 2 programs were found using it, I think we should just get them fixed. No need to support legacy crap for 2 obscure programs. > GLOB_NOMAGIC, GLOB_BRACE > asterisk18 These can't be defined without actually supporting the functionality, which is heavyweight/ugly. > semtimedop > sqlrelay This is a Linux-specific sysvsem operation we're missing, right? If so I think the syscall wrapper should be added. > sys_errlist > dmake > sys_siglist > xemacs These are both harmful and definitely should not be supported. (They introduce copy relocations which encode the SIZE of these lists as part of the application's ABI.) However, I'm not clear that sys_siglist is covered by strsignal; I think it may be SIG___ names rather than the descriptive strings. > herror > opencm No idea what this even is. > mallinfo > libfirm Deprecated API but we could provide a dummy anyway. > TIMESPEC_TO_TIMEVAL > libusbx Perhaps worth adding? > POSIX_SPAWN_USEVFORK > ruby-posix-spawn Should be fixed in ruby with #ifdef around the usage. This is not useful with musl. In principle we could just provide the flag as a no-op, but it's a really bad API design (exposing the implementation-detail rather than the semantics as the flag name) and since it doesn't affect ABI, I'd rather get apps which are using it fixed at the source level rather than adding a no-op flag. > ruserok > u9fs .....wow. Can't believe anything is still using the rhosts APIs... > struct ip_mreq_source > mad-flute There's an unresolved thread about adding these somewhere, possibly with a patch. > F_EXLCK > qemu Need more info on what it's about. Maybe something we should add. > c_ispeed, c_ospeed (struct termios members) > icdprog If c_* is reserved we could possibly add these. > d_namlen (struct dirent member) > icb This seems to be a BSD-ism, not present on glibc either, and there's no way to add it. > st_ctimensec (struct stat member) > libuv st_ctim.tv_nsec > setpgrp (apparently sysv and bsd prototype differ) > unrealircd Yes, this is a BSD-ism and just needs to be fixed in the application. musl's conforms to the POSIX requirements. > libiconv_* > odt2txt, ted Specific symbols ?? > KIOCSOUND, TIOCCDTR, TIOCSDTR, TIOCGSIZE (ioctl) > minicom, modemd, beav These should almost surely be added somewhere... > __STRING, __P, __BEGIN_DECLS (cdefs.h) > libmp4v2, ctrlproxy, irssi-xmpp, fortran-utils, cook Apps should be fixed. > missing internal symbols: > howl: __sighandler_t > xetex, icu: __timezone > h8300-hms-gcc: __mempcpy > erlang: __uint32_t > libstatgrab: __uint8_t > applyppf, makeppf: __off64_t > qemu: __sigset_t Ditto. > top missing headers: > > sys/timeb.h > unalz, audacity, gplcver, ng-spice, gammu, hdf5, libpgm, mico, dasher,.. This has been added in git (it was mandatory in POSIX-2001). > sys/cdefs.h > asterisk*, xulrunner* (jemalloc), libmilter, sendmail, dhcpcd,.. This has been an ongoing issue, but my leaning is still to omit it. If somebody really wants it, it can be installed as a third-party header; it has no libc dependencies. However, the apps should really just be fixed. > termio.h > blackened, gnatpython, pardiff, emacs*-nox11, ski, bastet, craft, top,.. We could consider this for addition, but I'd rather get apps fixed. > rpc/types.h rpc/rpc.h > hdf, glusterfs, mrstat, lsof, snort, portmap, sharity-light, net-snmp, unfs3, ywho, PAM,.. Non-libc. > sys/queue.h > libstree, openrcs, miniupnpd, mtftpd, pim6dd, transmission,.. This can also be treated as a third-party library to be installed if desired. > sgtty.h > kermit, mg2a, ve, 7plus, xjdic glibc has the header, but the functions are hard to find any Linux documentation for; they seem to be "unimplemented" syscalls. So this is probably a BSD-ism. > execinfo.h > redis, virtuoso, powerdns, packagekit, qt4-creator Backtrace stuff, right? > sys/bitypes.h > libbind, libdnsres We could add this as an alias for sys/types.h with a #warning, if desired. I'm fairly indifferent. > sys/unistd.h > mlview, foobillard, galaxa, slurm Ditto. > sys/endian.h > swarm This could probably be added even without a #warning since it's nonstandard and sys/endian.h is just as "valid" as endian.h, but I'm not clear whether it's worth it for ONE obscure program that could just be fixed. > machine/endian.h > estic, opendis Making a new dir is probably not worth it either. > net/ppp_defs.h > poptop, slurm, xfce4-netload-plugin Don't know enough to comment. > a.out.h > bcc, ntp4 Probably does not make sense; musl does not (out of the box, anyway) support a.out binaries. > asm/page.h > clisp, s10sh, same Isn't this a kernel header? > sys/debugreg.h > gdb6 Hmm, looks like another case of gdb internals in libc (or vice versa)... > sys/dirent.h > heirloom-libcommon Possibly another #warning redirect? > obstack.h > p5-Proc-ProcessTable musl does not provide GNU obstack functions, but _might_ in the future if it's deemed useful for ABI-compat. > gnu/lib-names.h > ruby-ffi I don't think this one makes sense at all to have in musl. > fpu_control.h > jamvm Looks like an ugly non-abstracted legacy version of fenv.h. > xlocale.h > liblive Since these interfaces were declared in xlocale.h before being standardized in POSIX 2008, it might make sense to provide the header; however, the fact that only one program was found to be needing it makes me doubt the usefulness. > net/if_ether.h > cia Is this a duplicate of the one in netinet or something different? > net/if_packet.h > isc-dhcp4 And netpacket/packet.h? > netinet/igmp.h > ngrep Perhaps should be added. > re_comp.h > rdist6 I think this is part of the GNU regex API, not supported by musl. > fstab.h > samba35 Yet another hideous non-threadsafe, non-library-safe *ent API. As long as there's only one program using it my feeling is to omit it. > error.h > vpnc Wow, only one? There's been a fair amount of demand for error.h so I might add this one sometime... > fts.h > slocate fts.h is fundamentally broken on glibc (it #errors with 64bit off_t) and thus any program using it is broken. There is a non-broken version in gnulib or BSD which program should copy into their source trees and use if they need it. So there's no reason for musl to provide it; we couldn't match the glibc ABI anyway since the glibc ABI is wrong (32-bit). > sys/perm.h > xosview Seems redundant with sys/io.h? > gcc-4.8.1 bugs: > epic4: exec.c:118:18: internal compiler error: in set_lattice_value, at tree-ssa-ccp.c:451 > bullet: btInternalEdgeUtility.cpp:310:2: internal compiler error: Segmentation fault > aegis fails because in libstdc++ cinttypes header is broken without -std=c++11 or -std=gnu++11 > boost-libs fail because mm_malloc.h has incompatible posix_memalign decl with throw() Was gcc built aginst musl? Assuming so, it's _possible_ that some of these are musl bugs, though gcc sounds more likely. > probably busybox failures: > ocaml-mysql: sed fails with out of memory > hugs: cat: illegal option: s > rcs: Could not find a sane line-by-line differences program (tried: gdiff rdiff diff) rcs probably wants non-unified diff; I think busybox only provides unified. > sdcc: /bin/sh: syntax error: unexpected redirection Bashism? > at least mysql5-client config uses getconf Hm? We should probably provide a getconf that works with musl somewhere... > accessing FILE internals: > kermit: ckucmd.c:7395:48: debug(F101,"cmdconchk stdin->_cnt","",stdin->_cnt); > icon: fpoll.c:75:9: if (f->_cnt > 0) > emacs24: print.c:760:3: stderr = initial_stderr_stream; > db46: checking for FILE... no configure: error: No FILE type. The last one is a C99 conformance issue, but I'm not sure we want to change it since C11 dropped the requirement. > collision between libc header and linux header: > libnet10: redefinition of struct ether_addr > libnet11: linux/if.h: redefinition of struct ifmap No really good solution to these. > -Werror suicide: > upx, mame, mess, tme, sfslite If the warnings were musl-specific, they might be issues we should "fix" though. > other > qt3-libs, ptlib: using signed int for socklen_t glibc also has unsigned socklen_t, so I'm confused how they work with glibc but not musl... > mysql5-client config tries to check 'getconf GNU_LIBPTHREAD_VERSION' > many audio pkg: alsa/global.h redefines struct timespec if _POSIX_C_SOURCE is unset This is stupid and should just be fixed in alsa. timespec is likely to be defined (e.g. by default) without _POSIX_C_SOURCE being defined, and in C11 it's even required to be defined. > pulseaudio, esound: broken /dev/dsp emulation with LD_PRELOAD hack > clang: DynamicLibrary.cpp has broken ifdef logic and stdin etc access > easytag, libpeas: NULL sentinel BTW, I think we may need to re-think the NULL issue. POSIX seems to require it to have pointer type... > pth, jabberd: #error "Unsupported Linux (g)libc version and/or platform" Hahah. Well in the case of pth this is probably the best it can do. pth is exactly the opposite of portable. Thankfully it's deprecated and somebody's working on a drop-in replacement built on top of POSIX threads for the few programs that actually need it. > binutils, mingw-binutils: empty character constant '' Cause? musl-specific? > p5-GD: CORE/perl.h uses off64_t without gnu or lfs feature test App bug. > dosbox: uses undefined typename in offsetof A type musl is missing? > valgrind: configure: error: Valgrind requires glibc version 2.2 - 2.16 To be fair, valgrind might depend enough on glibc internals to make this kind of specificity necessary. We should work with the valgrind team to get musl support fixed, ideally without anything invasive on the musl side. > emacs20: configure: error: Emacs hasn't been ported to `x86_64-unknown-linux-gnu' systems. Haha. emacs20 still? Really? That's way too old to be useful; it doesn't even support UTF-8. emacs23 is the first version to care about. > rrdtool: configure: error: I need a copy of *nroff to format the documentation > afs, openafs: configure: error: Couldn't guess your Linux version. > nvi: configure: error: No writeable preserve directory found. > gtk-doc: configure: error: could not find DocBook XML DTD V4.3 in XML catalog Most of these look non-musl-specific. > boost/integer_traits.hpp: misses WCHAR_MIN and WCHAR_MAX because of ifdef hell Oh? Is this their fault or something weird we're doing? > cstore: static const int PAGE_SIZE=65000; Namespace violation. > zzuf: redefinition of open: int NEW(open64)(const char *file, int oflag, ...) I guess it's defining _GNU_SOURCE or the whole pkgsrc run is. open64 should not be defined by default in musl. > libjit: #error "Don't know how to determine if floating point numbers are finite" Is it doing stupid #ifdefs based on glibc version, or failing for some semi-readonable reason? > tk: tkPanedWindow.o: file not recognized: File truncated Weird... > ex: conflicing prototype: char *memalign(size_t alignment, size_t size); App bug. > dia: configure: error: Can't find neither a definition of isinf nor ieeefp.h Um, math.h? > cc65, gnats, sandy: undefined siginfo_t because sys/wait.h is used with -std=c89 This is something we need to fix. I'm thinking sys/wait.h should just omit the stuff that needs siginfo_t if the right features are not enabled. (Really, omitting the whole file would be valid, but I suspect this would break programs worse.) > scrollz: undefined curses symbols (tputs, tgetstr) > gnuchess4, camediaplay, sc, sysmon: cannot find curses.h Improper or failed ncurses installation? > ocaml-zip, edb: cannot find zlib.h > poco: cannot find sqlite3.h Unrelated to musl. > sdbm: cannot find ndbm.h Presence or absence of ndbm in libc varies (it's supposedly required as part of XSI, IIRC) but I think we should leave it as a third-party thing. The API is so broken as to be utterly useless. > include fails: > libbinio: EOF was not declared in this scope Assumption that some random header includes stdio.h, maybe? > libmpdclient: unknown type name fd_set Probably the same, but for sys/select.h and sys/time.h. > glib: poll undeclared here I never hit this one before; wonder why it's happening? > libspiff: printf was not declared in this scope > libproxy: fd_set was not declared Probably same as above. > climm: unknown type name int8_t > sirc: incompatible implicit declaration of built-in function strlen > uisp: NULL was not declared in this scope > xqilla: ptrdiff_t does not name a type > gnome-keyring: unknown type name fd_set > libhfs: unknown type name time_t > libpgm: unknown type name ssize_t > log4shib: int64_t does not name a type > nsis: close was not declared in this scope > pyfann: ptrdiff_t does not name a type > rpc2: F_GETFL undeclared > tvision: intptr_t has not been declared > vanessa_logger: undefined CODE > mg: unknown type name mode_t > notecase: basename was not declared in this scope > xemacs: NI_MAXHOST undeclared > aranym: caddr_t was not declared in this scope > gpsim-devel: puts was not declared in this scope > pcemu: O_RDONLY undeclared > vba: uint32_t does not name a type > oto: incompatible implicit declaration of built-in function calloc > cdparanoia: u_int32_t undeclared > anope: unknown type name u_int32_t > conserver: u_short undeclared > gmp3info: uint undeclared > libbegemot: unknown type name u_int > aliados: ushort has not been declared Pretty much all of these look like the same basic issue: invalid assumptions that one header includes another. This means they could probably easily be fixed upstream if somebody's willing to do the analysis to determine where the invalid assumptions are. Since these should all be non-controversial fixes, I think this would be a great "patch flood" project to precede or go along with the upcoming 1.0 release, as discussed in the publicity plan thread. Rich
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.