Openwall GNU/*/Linux - a small security-enhanced Linux distro for servers
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Tue, 9 Jun 2015 23:30:50 -0400
From: Rich Felker <>
Subject: Re: Moving forward with sh2/nommu

On Mon, Jun 01, 2015 at 11:11:07AM -0400, Rich Felker wrote:
> [resent to musl list]
> Here's a summary of the issues we need to work through to get a modern
> SH2/nommu-targetted musl/toolchain out of the proof-of-concept stage
> and to the point where it's something people can use roughly 'out of
> the box':
> Kernel issues:
> 1. Kernel should support loading plain ELF directly, unmodified. Right

I have a patch to do this, not polished but it works. It also...

> 2. Kernel insists on having a stack size set in the PT_GNU_STACK
>    program header; if it's 0 (the default ld produces) then execve
>    fails. It should just provide a default, probably 128k (equal to
>    MMU-ful Linux).

...uses a default stack size of 128k if the header is 0, and...

> 3. Kernel uses the stack for brk too, growing brk from the opposite
>    end. This is horribly buggy/dangerous. Just dummying out brk to

...set the brk size to zero so that brk will always fail.

> 4. Syscall trap numbers differ on SH2 vs SH3/4. Presumably the reason
>    is that these two SH2A hardware traps overlap with the syscall
>    range used by SH3/4 ABI:

I haven't patched this yet. I'd like to use 31 (0x1f) as the new
universal SH syscall trap number, instead of 22. More details on the
reasons later.

> musl issues:
> 1. We need runtime detection for the right trap number to use for
>    syscalls. Right now I've got the trap numbers hard-coded for SH2 in
>    my local tree.

I've written the runtime detection, but I'd rather not have to use it.
I managed to avoid inlining a big conditional at each syscall, but
there are still multiple ugly issues:

- The cancellable syscall code has two different instruction addresses
  where the actual syscall could take place (depending on which trap
  instruction is used), whereas the current musl C code assumes that
  there's exactly one instruction pointer range where cancellation may
  be acted upon. I have code to allow a second range but it's ugly.

- The sigreturn (sa_restorer) functions need to use runtime selection,
  which precludes putting them in the canonical form they usually
  have. At some point in the past, not using the exact instruction
  sequences expected for these functions confused gdb, but I suspect
  that's a non-issue on modern versions. Alternatively we could have
  sigreturn.c pass different versions of the sigreturn functions to
  the kernel depending on which syscall ABI is in use at runtime, but
  this requires bolting on new arch-specific logic to something that's
  normally not arch-specific, so I prefer just having the conditionals
  in the sigreturn functions.

- The syscall asm requires an extra register clobber to be able to do
  the runtime trap number switching.

An alternate design I considered was simply patching the trap number
in the .text for SH2, since there's no memory protection, but of
course this would preclude execute-in-place from ROM, etc. so I don't
think it's a proper solution. It is much less invasive, though.

> 2. We need additional runtime detection options for atomics: interrupt
>    masking for plain SH2, and the new CAS instruction for SH2J.

This is the one thing I haven't done, so currently the atomic macros
are using GUSA which is non-atomic and unsafe on SH2 (if an interrupt
happens with invalid stack pointer, memory will be corrupted). This
could be part of the random crashing I've been experiencing (although
I reproduced it without musl) so I'll try to add them next.

> 3. We need sh/vfork.s since the default vfork.c just uses fork, which
>    won't work. I have a version locally but it doesn't make sense to
>    commit without runtime trap number selection.

Done and updated to use runtime selection in the (ugly) patch.

> 4. As long as we're using the FDPIC ELF header flag to get
>    binfmt_elf_fdpic.c to load binaries, the startup code needs to call
>    the personality() syscall to switch back. I have a local hack for
>    doing this in rcrt1.o which is probably not worth upstreaming if we
>    can just make the kernel do it right.

No longer needed because of the kernel patch to load normal ELF.

> 5. The brk workaround I'm doing now can't be upstreamed without a
>    reliable runtime way to distinguish nommu. To put it in malloc.c
>    this would have to be a cross-arch solution. What might make more
>    sense is putting it in syscall_arch.h for sh, where we already
>    have to check for SH2 to determine the right trap number; the
>    inline syscall code can just do if (nr==SYS_brk&&IS_SH2) return 0;

Commit 276904c2f6bde3a31a24ebfa201482601d18b4f9 in musl solves this in
a general manner, even though it's no longer needed with my kernel
patch applied.

One more musl-side issue I neglected to mention is the __unmapself.s
can't work on SH2 because the SH2 trap/interrupt mechanism requires
the userspace stack pointer to be valid at all times. This is now
solved upstream in commit c30cbcb0a646b1f13a22c645616dce624465b883,
but activating it for SH2 requires removing
src/thread/sh/__unmapself.s so the generic C file gets used.

The attached patch covers everything described above that's not
already upstream, and is sufficient to build musl for sh2 with
musl-cross targeting "sheb-linux-musl". I used gcc 4.7.3 because later
versions break the kernel. The attached config.mak for musl shows the
configure options I used. The attached sheb.specs file is how I got
gcc to do always-PIE without breaking the kernel.


View attachment "fdpic_elf_loader.diff" of type "text/plain" (1969 bytes)

View attachment "musl_sh2_v2.diff" of type "text/plain" (13622 bytes)

View attachment "config.mak" of type "text/plain" (1039 bytes)

View attachment "specs.sheb" of type "text/plain" (5187 bytes)

Powered by blists - more mailing lists

Your e-mail address:

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.