[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Fri, 22 Dec 2017 11:43:40 -0500
From: Rich Felker <dalias@...c.org>
To: musl@...ts.openwall.com
Subject: Re: Feature request: building musl in a portable way
On Fri, Dec 22, 2017 at 05:09:06PM +0100, ardi wrote:
> On Thu, Dec 21, 2017 at 10:38 PM, Rich Felker <dalias@...c.org> wrote:
> [...]
> >
> > I'm not clear what you want to do.
>
> I'm looking for a C runtime with a MIT-like license that can be
> compiled for several architectures, in 32bit and 64bit, mostly Intel,
> PowerPC, ARM, and MIPS, and is endian-safe, and written in a tidy
> code. I need that such runtime is able to be retargeted to different
> OSs by changing the layer where syscalls are made. At the moment of
> writing this, the OSs I'm interested in are Linux and MacOS. In the
> future I'll likely be interested in other OSs as well.
Changing the layer by which syscalls are made is a lot different from
building with no syscalls. You can change the syscall layer just by
making a new arch that defines (in syscall_arch.h and bits/syscall.h)
your mechanism. See the recent wasm thread or midipix for examples of
more exotic ways this can be done. But be aware that the ability to
get a POSIX-conforming implementation where EINTR and thread
cancellation work requires some sort of atomic boundary that
determines when a syscall has passed the point of having successful
side effects, which makes implementing the syscalls just as functions
hard.
Stepping back a bit, I suspect what you want is to just be able to
implement functions like open(), read(), etc. on your own instead of
implement something like actual syscalls, based on a notion that
open(), read(), etc. "are the syscalls". This is a classical notion
(e.g. based on the organization of man pages into section 2 vs 3) but
it doesn't really correspond to reality. Many/most of the functions
you think of as syscalls _need to actually be library functions_ for
various conformance/subtle-behavior reasons, and the syscalls by the
same names are low-level primitives that are useful in implementing
them. Also, it would be impossible to have a (valid) implementation
where it works to just replace these functions, because for example
stdio _can't_ call open(), read(), etc. for namespace reasons.
> I don't know of any C runtime that meets all these requirements. The
> only two that get close are the different BSDs C runtimes, and musl,
> but both lack the last requirement (i.e.: syscalls are not
> encapsulated in some confined files so that you could rewrite such
> "syscall layer" for each OS --instead, syscalls can be issued from any
> place in code, and the only way to locate and encapsulate the proper
> functions is to manually search for invocations of syscalls in the
> source tree).
>
> So, only options are BSDs and musl (unless I forget any). But both
> BSDs and musl require "heavy editing" if you want to encapsulate
> syscalls, and by doing such editing, you place yourself out of easily
> updating to newer versions without considerable merging work.
As long as you do it the way that acknowledges the above, the "heavy
editing" is isolated to the arch directories you add.
> > Even if it were, configurable builds that exclude
> > functionality are intentionally outside the scope of musl; instead,
> > the project provides fine-grained linking so that you just get what
> > you need; ports to systems where some underlying functionality is not
> > possible simply need to make the relevant syscalls fail with ENOSYS.
>
> There's more to that: musl assumes syscalls are invoked following the
> Linux kernel protocol for syscalls. It's not only a matter of
> translating syscalls numbers and their arguments, but about how the
> syscalls are triggered. So, writing the "compatibility layer" that I
> explained in the previous paragraphs is much harder if it has to
> intercept and translate syscalls, than if you could edit the musl
> files where syscalls are invoked, redirecting them in a more
> comfortable way, and without the tough code for intercepting syscalls.
There are indeed a small number of places where workarounds or other
considerations for Linux-specific parts of the syscall interface
boundary are in general source files rather than in the syscall glue
layer, but I think the number is quite small. If there are
particularly egregious ones that you think could be improved upon,
please let me know.
> > With that said, my guess is that you're really asking for a way to
> > take the "pure" code out of musl and make it a library that you can
> > use on an existing C/POSIX (or non-POSIX C) implementation. This is
> > interesting, but currently outside the scope of musl, and probably
> > covers less interesting code than you might expect [...]
>
> I'm not interested in taking the "pure" code only, but as much code as
> possible, only having to rewrite the syscall retargeting layer.
Yes, I see now that my guess was wrong.
> > [...] Making it easier to use them outside musl
> > is an interesting problem but I'm afraid not one I have resources to
> > devote to at present..
>
> A somewhat crazy idea came to my mind: libraries like ElectricFence,
> that substitute libc functions by their own version if you link them
> before the C runtime. This would let me replace functions that make
> syscalls by my own version. However, it sounds quite hack-ish and
> error-prone in the long run.
It wouldn't do what you want for one of the reasons I described above.
The important things in musl that want to open files, for instance,
don't call open(). They use sys_open which expands to a syscall.
> After thinking about this, I believe the best way would be to be able
> to confine all the musl syscalls invocations in a reduced place that I
> can take control of. In the current directory tree, musl has a
> function for making syscalls, and this would be exactly what I need: a
> place for easily intercepting syscalls and redirecting them. However,
> several functions in musl invoke syscalls directly in assembly,
> without going through that function, so it's not easy to intercept all
> syscalls in this moment.
In a few places, for some archs, that's necessary, but I'm trying to
reduce that so that the amount of required arch-specific code is
minimized. An irreducible example is vfork which fundamentally cannot
be done from C because it has to return twice. clone is treated that
way too, but I think it's wrong; it should be possible to do clone
from C since the function (not the syscall) never has to return twice.
Rich
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
