Openwall GNU/*/Linux - a small security-enhanced Linux distro for servers
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Tue, 26 Jun 2018 20:00:49 +0200 (CEST)
From: Pavel Kankovsky <>
cc: Vasily Averin <>
Subject: Re: 32-bit syscall breakage in -431 kernel with KAISER

On Mon, 25 Jun 2018, Pavel Kankovsky wrote:

> (I know I should learn how to use builtin GDB server in Qemu but...)

I have figured how to do some basic machine-code-level debugging of 
ia32_syscall (the int 0x80 handler) using Qemu:

1. Make sure Qemu is running with -gdb tcp: (or -s).
2. Get vmlinux for the running kernel.
3. Run the debugger:

$ gdb vmlinux
(gdb) target remote tcp:
(gdb) set disassemble-next-line on
(gdb) b ia32_syscall
(gdb) c

4. Run a 32-bit binary in the vm.
5. Use "si" to step through the debugged function.

It turns out the handler is invoked with an unexpected stack frame in the 
"trampoline stack" (the top of the "trampoline stack", as stored in 
TSS.rsp0, is 0xffff810001a36278 in this example):

(gdb) x/6g $rsp
0xffff810001a36248:     0x00000000b7f81ea1      0x0000000000000073
0xffff810001a36258:     0x0000000000000246      0x00000000bfa78274
0xffff810001a36268:     0x000000000000007b      0x000000000000007b

There should be five entries (rip at the bottom, cs, eflags, rsp, and ss 
at the top) only but the actual stack contains one mysterious and bogus 
extra entry (the other 0x000000000000007b) at the top, shifting everything 
else down by 8 bytes.

This stack frame (including the bogus extra entry) is copied to the 
regular kernel stack. As far as I can tell, the same extra entry appears 
in other interrupt handlers, e.g. common_interrupt.

Functions that get an explicit pointer to struct pt_regs are not affected 
but anything that expects to find pt_regs at the top of the stack (e.g. 
compat_alloc_user_space via task_pt_regs(current)) breaks.

It seems the extra entry is added because the top of the trampoline stack 
is not aligned to 16 bytes and the CPU does not like it and enforces the 
alignment, shifting the whole stack down wrt. the expected layout as a 

I have modified struct tss_struct in include/asm-x86_64/processor.h to
include an extra "unsigned long stack_padding" between stack_canary and 
stack to make stack 16-byte aligned...

--- include/asm-x86_64/processor.h.orig	2018-06-26 12:11:17 +0000
+++ include/asm-x86_64/processor.h	2018-06-26 16:50:55 +0000
@@ -269,6 +269,7 @@
  #ifndef __GENKSYMS__
  	unsigned long		stack_canary;
+	unsigned long		stack_padding;
  	unsigned long           stack[64];
  } __attribute__((packed, __aligned__(PAGE_SIZE)));

...and lo! 32-bit ifconfig works as expected.

Pavel Kankovsky aka Peak                      "Que s├žay-je?"

Powered by blists - more mailing lists

Your e-mail address:

Powered by Openwall GNU/*/Linux - Powered by OpenVZ