Date: Mon, 10 Nov 2014 23:50:46 -0500 From: Rich Felker <dalias@...c.org> To: musl@...ts.openwall.com Subject: Nonstandard functions with callbacks The topic of fopencookie has just come up on #musl again in regards to Asterisk's dependence on it (or the similar funopen function), and in response I'd like to offer a few ideas on this kind of function in general. Supporting nonstandard functions is always a potential pitfall. Unlike functions with a rigorous or semi-rigorous specification in one or more standards documents, they inevitably have all sorts of underspecified or unspecified corner cases that some software ends up depending on. And when they come from a single origin (e.g. glibc) rather than various historical systems that all had their own quirks, it's arguably reasonable for applications to expect the exact behavior of the original (e.g. glibc) implementation. In this regard, functions that take a caller-provided callback function are just about the worst possible. The documentation (e.g. man pages or glibc manual) for the original version rarely specifies constraints on what the callback can do, such as: - Does it have to return? - Can it call longjmp? - What happens if it causes pthread cancellation to be acted upon? - Can it change the floating point environment? - Does it run in the same thread as the caller? - Does it need to take special precautions to avoid deadlock? - What reentrancy requirements does it have? - Are there specific standard library functions it can't call? - Can it unwind or backtrace out of the callback context? - Etc. One feature musl intentionally does not yet support is "IFUNC" resolvers in the dynamic linker. These are a feature by which a program can arrange for a symbol to resolve to different versions of a function at runtime depending on cpu capabilities or similar. What's blocking IFUNC support in musl is a specification for the constraints on the resolver function. Since it runs in a context that happens prior to execution of global ctors for the containing module, in a context where relocations have not yet completed, and with locks held in the dynamic linker, there are A LOT of implementation-imposed constraints on what such a function can do. And sweeping those under the rug and just saying "you can do whatever seems to work" is not a reasonable approach for a high-quality implementation; from my perspective, it's better not to have the feature at all than to have a version where internal changes might break something that seems legitimate. The fopencookie situation is fairly similar. The callbacks to provide a custom FILE type would run in a context with the relevant FILE locked. It's likely that the underlying reads or writes performed on the 'cookie' would be somewhat different under musl than under glibc due to musl's readv/writev type model for stdio buffering, and these differences could potentially break applications' assumptions about how the underlying operations would be performed. Such assumptions in turn may be valid if glibc is considered 'authoritative' for the fopencookie function. There are a number of other considerations too with regard to the FILE locking. In a naive fopencookie implementation for musl, the callbacks would run with the FILE lock potentially held in musl's light-locking mode rather than with a true recursive lock, meaning that strange things could happen if the callback tries to perform any operation on its own FILE. Avoiding that seems like it would require some complex dance to 'upgrade' the lock type, which in turn might impose long-term costs on maintaining the FILE locking system. My feeling is that "involves callbacks" should be an indication for exclusion of nonstandard functions. In terms of what I've written above, I think this follows from the existing principles of exclusion based on cost of implementation complexity and high risk of compatibility issues with applications. Rich
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.