Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Wed, 19 Dec 2018 19:30:44 -0500
From: Rich Felker <dalias@...c.org>
To: musl@...ts.openwall.com
Subject: Re: Data structures defined by both linux and musl

On Tue, Dec 18, 2018 at 08:41:53PM +0100, Arnd Bergmann wrote:
> I recently discussed with Rich about the work needed to get 64-bit time_t
> support into musl. One of the first steps he identified was to find out which
> interfaces we would want to abstract or wrap for a new ABI given that we
> have to make a binary incompatible interface anyway.
> 
> I have found all the data structures that are provided by both the kernel
> headers and the musl headers now, and annotated what I think we the
> path forward could be. I already provided the same list on IRC, but
> here is a (slightly updated) copy for everyone else.

Thank you. For those just joining now, the context of this is that
support for 64-bit time_t on 32-bit archs probably requires, and at
least is best done with, a new ABI, so far known as the ".2" ABI
(because the ldso would end in ".2" instead of ".1"). This would be an
opportunity to fix lots of ABI mistakes where extensibility of even
support for current functionality is lacking. If done, this would not
be a fork of musl and would not be dropping existing ABIs. Rather,
internal refactoring would eliminate the assumption that the
ABI-with-application types match the ABI-with-kernel/syscall types,
performing translation back and forth where needed. On the existing
".1" ABIs, this translation would mostly be the identity
transformation, but on archs where we're already doing some hacks to
fix up kernel ABI bugs (sysvipc on big endian, mips stat structure,
x32 stuff, etc.) the hacks could be replaced by used of this
translation infrastructure.

It may also be possible that we can add 64-bit time_t on the existing
ABIs (preserving app-to-libc ABI, not app/library-to-library ABI when
types involving time_t are part of their interface) just by remapping
all of the affected interfaces in the headers. I'm not sure if this is
a good idea. It's imperfect and might derail adoption of the better
ABI, but it would probably bring 64-bit time_t to mainstream use more
quickly.

BTW regarding 64-bit time_t on 32-bit archs, Arnd has been working to
make this happen for a long time. I believe it was over 3 years ago we
first spoke about working on it in musl. Basically we've reached the
point where 32-bit archs are a dead-end for developing embedded stuff
that needs to run indefinitely without the ability to upgrade, and
this domain is the main place where 32-bit archs are still very
relevant. Once nice thing about making a new clean ABI is that
embedded users who don't care about binary ecosystems can switch
immediately, and desktop/server distros can take their time and switch
from .1 to .2 when it works best for them.

I indicated to Arnd that, in order for this to move forward, we need
to be able to evaluate the size of the interface surface affected by
time_t change, to evaluate whether the .2 ABI makes sense, and further
get an idea of what other types are ABI dead-ends that should be fixed
at the same time. The list I'm replying to is the start of that work.

Now, a few comments on findings so far. These won't be complete but
they're a start:

> The takeaway is that we probably need to add new definitions for
> flock64, statfs, stat, termios, {msg,sem,shm}{buf,info,id_ds}, ipc_perm,

Not clear on how flock[64?] is affected.

stat and ipc structures contain time_t's and definitely need to
change.

I think termios is listed here because .2 ABI overhaul is a great
opportunity to switch to the "termios2" interfaces, unify the
userspace types, and make support for custom baud work right.

> rlimit, rusage, sched_param, time_t, timeval, timespec, itimerval,
> itimerspec, and timex, and then wrap all kernel interfaces that
> use those.

Not clear on how rlimit is affected, but most of these definitely are.

sched_param is another opportunity. Ours is already a bit larger than
glibc's due to my (unmotivated, inadvertent) inclusion of fields for
the optional POSIX sporadic stuff, but not a lot. glibc already hit
new kernel features they can't provide with the existing ABI.

Rich

Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.