[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Tue, 11 Nov 2014 09:22:53 +0200
From: Timo Teras <timo.teras@....fi>
To: Rich Felker <dalias@...c.org>
Cc: musl@...ts.openwall.com
Subject: Re: Nonstandard functions with callbacks
On Mon, 10 Nov 2014 23:50:46 -0500
Rich Felker <dalias@...c.org> wrote:
> The topic of fopencookie has just come up on #musl again in regards to
> Asterisk's dependence on it (or the similar funopen function), and in
> response I'd like to offer a few ideas on this kind of function in
> general.
>
> Supporting nonstandard functions is always a potential pitfall. Unlike
> functions with a rigorous or semi-rigorous specification in one or
> more standards documents, they inevitably have all sorts of
> underspecified or unspecified corner cases that some software ends up
> depending on. And when they come from a single origin (e.g. glibc)
> rather than various historical systems that all had their own quirks,
> it's arguably reasonable for applications to expect the exact behavior
> of the original (e.g. glibc) implementation.
Understand. Especially for musl which aims to be mostly ABI compatible.
Apparently both fopencookie() and funopen() had ABI issues. Most
horribly fopencookie() changed the callback pointer struct
layout/contents.
funopen() seems to have some versions with fpos_t for seek; and others
use off_t.
NetBSD current seems to have funopen2() with alternative signatures for
the callbacks using ssize_t/size_t for read/write instead of int.
So yeah, it's a royal mess.
> In this regard, functions that take a caller-provided callback
> function are just about the worst possible. The documentation (e.g.
> man pages or glibc manual) for the original version rarely specifies
> constraints on what the callback can do, such as:
>
> - Does it have to return?
> - Can it call longjmp?
> - What happens if it causes pthread cancellation to be acted upon?
> - Can it change the floating point environment?
> - Does it run in the same thread as the caller?
> - Does it need to take special precautions to avoid deadlock?
> - What reentrancy requirements does it have?
> - Are there specific standard library functions it can't call?
> - Can it unwind or backtrace out of the callback context?
> - Etc.
>[snip]
> The fopencookie situation is fairly similar. The callbacks to provide
> a custom FILE type would run in a context with the relevant FILE
> locked. It's likely that the underlying reads or writes performed on
> the 'cookie' would be somewhat different under musl than under glibc
> due to musl's readv/writev type model for stdio buffering, and these
> differences could potentially break applications' assumptions about
> how the underlying operations would be performed. Such assumptions in
> turn may be valid if glibc is considered 'authoritative' for the
> fopencookie function. There are a number of other considerations too
> with regard to the FILE locking. In a naive fopencookie implementation
> for musl, the callbacks would run with the FILE lock potentially held
> in musl's light-locking mode rather than with a true recursive lock,
> meaning that strange things could happen if the callback tries to
> perform any operation on its own FILE. Avoiding that seems like it
> would require some complex dance to 'upgrade' the lock type, which in
> turn might impose long-term costs on maintaining the FILE locking
> system.
Generally applications seem to write things to work with funopen() or
fopencookie(); so they do not do too many assumptions. But yes, having
identical emulation seems hard and unfeasible to accomplish. Given, the
vagueness of the situation and the above mentioned hassle I perfectly
understand why it's not suitable for musl upstreaming. At least without
having additional specification.
Though the readv/writev architecture is no excuse the not implement
them. It can always be split to individual read()/write() callbacks.
But yes, the sizes (and alignment of sizes) may vary. And yes, it might
mean the performance will be non-optimal.
> My feeling is that "involves callbacks" should be an indication for
> exclusion of nonstandard functions. In terms of what I've written
> above, I think this follows from the existing principles of exclusion
> based on cost of implementation complexity and high risk of
> compatibility issues with applications.
As distribution, we want things to work. And we can limit support to
certain applications that use the specific API. And as we compile
everything against musl, we can also make additional ABI
considerations. So I think we'll still consider doing fopencookie() or
funopen() as distro-mainted patch. These are additional constraints we
can live with - compared to musl upstream which is committed to
maintain ABI between releases.
Though, we'll probably also file portability bugs against those apps
that rely on this.
/Timo
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
