Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Wed, 11 Mar 2020 03:08:22 +0100
From: Andreas Dröscher <>
Subject: Re: mips32 little endian -ENOSYS is not -(-ENOSYS)

Am 11.03.20 um 02:40 schrieb Rich Felker:
> On Wed, Mar 11, 2020 at 02:19:31AM +0100, Andreas Dröscher wrote:
>> Am 11.03.20 um 01:55 schrieb Rich Felker:
>>> On Tue, Mar 10, 2020 at 10:10:15PM +0100, Andreas Dröscher wrote:
>>>> Hi
>>>> I'm building a new toolchain for a very old hardware with a very old
>>>> Linux Kernel (2.6.20). The CPU is a Alchemy (now AMD) AU1100
>>>> (production was discontinued).
>>>> Obviously the Kernel lacks a lot of the modern system calls. I
>>>> however expect the general system call interface to be consistent.
>>>> Moreover, musl has fallbacks for many system-calls in place, kudos!
>>>> However, the fallback is never triggered. I will present the issue
>>>> on one example (epoll):
>>>> excerpt from src/linux/epoll.c:
>>>> int epoll_create1(int flags)
>>>> {
>>>> int r = __syscall(SYS_epoll_create1, flags);
>>>> #ifdef SYS_epoll_create
>>>> if (r==-ENOSYS && !flags) r = __syscall(SYS_epoll_create, 1);
>>>> #endif
>>>> return __syscall_ret(r);
>>>> }
>>>> If r is -89 (negative ENOSYS) the fallback is triggered else the
>>>> result is returned as it is. However, in my case __syscall returnes
>>>> 89 (positive ENOSYS).
>>>> I've tracked the return into the kernel and there the negative value
>>>> is returned. The Kernel additionally sets r7 to 1.
>>>> excerpt from arch/mips/syscall_arch.h:
>>>> static inline long __syscall1(long n, long a)
>>>> {
>>>> register long r4 __asm__("$4") = a;
>>>> register long r7 __asm__("$7");
>>>> register long r2 __asm__("$2") = n;
>>>> __asm__ __volatile__ (
>>>> "syscall"
>>>> : "+r"(r2), "=r"(r7)
>>>> : "r"(r4)
>>>> : SYSCALL_CLOBBERLIST, "$8", "$9", "$10");
>>>> return r7 ? -r2 : r2;
>>>> }
>>>> I assume the "bug" is triggered by __syscall1 If r7 is set it will
>>>> change the sign of r2. I can patch that by replacing:
>>>> return r7 ? -r2 : r2;
>>>> with
>>>> return (r7 && r2 > 0) ? -r2 : r2;
>>>> However I've no idea if I'm triggering any side effects or if I
>>>> selected the wrong implementation for my architecture.
>>> It sounds like what you're saying is that the ENOSYS codepath for
>>> mips, at least on your old kernel, is not setting the error flag in r7
>>> and returning ENOSYS in r2, but is instead returning -ENOSYS already
>>> (and not clear whether it's setting r7 at all or just leaving a stale
>>> value there).
>>> Can anyone else confirm this, or point to kernel history that might
>>> suggest it's a real bug? Your workaround looks like it should at least
>>> be *safe* to do, and probably the right thing if this was/is a real
>>> kernel bug in the official kernel rather than something some vendor
>>> broke in their fork.
>>> Rich
>> Sorry for not including that excerpt in the first place:
>> illegal_syscall:
>> 	li	v0, -ENOSYS			# error
>> 	sw	v0, PT_R2(sp)
>> 	li	t0, 1				# set error flag
>> 	sw	t0, PT_R7(sp)
>> 	j	o32_syscall_exit
>> 	END(handle_sys)
>> Source:
> OK, this was fixed by commit bda8229bdd087167f463ad5e74299987924f8137
> in 2008. But it looks like there's still another path, called
> "einval" from before commit fb498e2570eedc6c9c3d165e370624dfc3aed97b,
> that returns -ENOSYS. All of this is awful, and I think your fix is
> probably the right thing to do.
> Rich

Thank you very much for your review.

Just as a side note. I’ve just figured out that there is a second issue with old 

The current implementation of __syscall5, __syscall6 and __syscall7 (those use 
caller saved registers) violate the calling conventions of MIPS32 Linux Kernels 
prior 2.6.35. Those were assuming that the instruction immediately preceding the 
SYSCALL instruction was an instruction for loading the syscall number.

I’ll will try to rearrange the stack pushes to accommodate this requirement and 
report back if I manage to come up with something presentable.


Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.