[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Date: Mon, 10 Nov 2014 23:50:46 -0500
From: Rich Felker <dalias@...c.org>
To: musl@...ts.openwall.com
Subject: Nonstandard functions with callbacks
The topic of fopencookie has just come up on #musl again in regards to
Asterisk's dependence on it (or the similar funopen function), and in
response I'd like to offer a few ideas on this kind of function in
general.
Supporting nonstandard functions is always a potential pitfall. Unlike
functions with a rigorous or semi-rigorous specification in one or
more standards documents, they inevitably have all sorts of
underspecified or unspecified corner cases that some software ends up
depending on. And when they come from a single origin (e.g. glibc)
rather than various historical systems that all had their own quirks,
it's arguably reasonable for applications to expect the exact behavior
of the original (e.g. glibc) implementation.
In this regard, functions that take a caller-provided callback
function are just about the worst possible. The documentation (e.g.
man pages or glibc manual) for the original version rarely specifies
constraints on what the callback can do, such as:
- Does it have to return?
- Can it call longjmp?
- What happens if it causes pthread cancellation to be acted upon?
- Can it change the floating point environment?
- Does it run in the same thread as the caller?
- Does it need to take special precautions to avoid deadlock?
- What reentrancy requirements does it have?
- Are there specific standard library functions it can't call?
- Can it unwind or backtrace out of the callback context?
- Etc.
One feature musl intentionally does not yet support is "IFUNC"
resolvers in the dynamic linker. These are a feature by which a
program can arrange for a symbol to resolve to different versions of a
function at runtime depending on cpu capabilities or similar. What's
blocking IFUNC support in musl is a specification for the constraints
on the resolver function. Since it runs in a context that happens
prior to execution of global ctors for the containing module, in a
context where relocations have not yet completed, and with locks held
in the dynamic linker, there are A LOT of implementation-imposed
constraints on what such a function can do. And sweeping those under
the rug and just saying "you can do whatever seems to work" is not a
reasonable approach for a high-quality implementation; from my
perspective, it's better not to have the feature at all than to have a
version where internal changes might break something that seems
legitimate.
The fopencookie situation is fairly similar. The callbacks to provide
a custom FILE type would run in a context with the relevant FILE
locked. It's likely that the underlying reads or writes performed on
the 'cookie' would be somewhat different under musl than under glibc
due to musl's readv/writev type model for stdio buffering, and these
differences could potentially break applications' assumptions about
how the underlying operations would be performed. Such assumptions in
turn may be valid if glibc is considered 'authoritative' for the
fopencookie function. There are a number of other considerations too
with regard to the FILE locking. In a naive fopencookie implementation
for musl, the callbacks would run with the FILE lock potentially held
in musl's light-locking mode rather than with a true recursive lock,
meaning that strange things could happen if the callback tries to
perform any operation on its own FILE. Avoiding that seems like it
would require some complex dance to 'upgrade' the lock type, which in
turn might impose long-term costs on maintaining the FILE locking
system.
My feeling is that "involves callbacks" should be an indication for
exclusion of nonstandard functions. In terms of what I've written
above, I think this follows from the existing principles of exclusion
based on cost of implementation complexity and high risk of
compatibility issues with applications.
Rich
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
