Date: Wed, 12 Sep 2018 17:37:38 +0200 From: Julian Stecklina <jsteckli@...zon.de> To: Linus Torvalds <torvalds@...ux-foundation.org> Cc: David Woodhouse <dwmw@...zon.co.uk>, Konrad Rzeszutek Wilk <konrad.wilk@...cle.com>, juerg.haefliger@....com, deepa.srinivasan@...cle.com, Jim Mattson <jmattson@...gle.com>, Andrew Cooper <andrew.cooper3@...rix.com>, Linux Kernel Mailing List <linux-kernel@...r.kernel.org>, Boris Ostrovsky <boris.ostrovsky@...cle.com>, linux-mm <linux-mm@...ck.org>, Thomas Gleixner <tglx@...utronix.de>, joao.m.martins@...cle.com, pradeep.vincent@...cle.com, Andi Kleen <ak@...ux.intel.com>, Khalid Aziz <khalid.aziz@...cle.com>, kanth.ghatraju@...cle.com, Liran Alon <liran.alon@...cle.com>, Kees Cook <keescook@...gle.com>, Kernel Hardening <kernel-hardening@...ts.openwall.com>, chris.hyser@...cle.com, Tyler Hicks <tyhicks@...onical.com>, John Haxby <john.haxby@...cle.com>, Jon Masters <jcm@...hat.com> Subject: Re: Redoing eXclusive Page Frame Ownership (XPFO) with isolated CPUs in mind (for KVM to isolate its guests per CPU) Julian Stecklina <jsteckli@...zon.de> writes: > Linus Torvalds <torvalds@...ux-foundation.org> writes: > >> On Fri, Aug 31, 2018 at 12:45 AM Julian Stecklina <jsteckli@...zon.de> wrote: >>> >>> I've been spending some cycles on the XPFO patch set this week. For the >>> patch set as it was posted for v4.13, the performance overhead of >>> compiling a Linux kernel is ~40% on x86_64. The overhead comes almost >>> completely from TLB flushing. If we can live with stale TLB entries >>> allowing temporary access (which I think is reasonable), we can remove >>> all TLB flushing (on x86). This reduces the overhead to 2-3% for >>> kernel compile. >> >> I have to say, even 2-3% for a kernel compile sounds absolutely horrendous. > > Well, it's at least in a range where it doesn't look hopeless. > >> Kernel bullds are 90% user space at least for me, so a 2-3% slowdown >> from a kernel is not some small unnoticeable thing. > > The overhead seems to come from the hooks that XPFO adds to > alloc/free_pages. These hooks add a couple of atomic operations per > allocated (4K) page for book keeping. Some of these atomic ops are only > for debugging and could be removed. There is also some opportunity to > streamline the per-page space overhead of XPFO. I've updated my XPFO branch to make some of the debugging optional and also integrated the XPFO bookkeeping with struct page, instead of requiring CONFIG_PAGE_EXTENSION, which removes some checks in the hot path. These changes push the overhead down to somewhere between 1.5 and 2% for my quad core box in kernel compile. This is close to the measurement noise, so I take suggestions for a better benchmark here. Of course, if you hit contention on the xpfo spinlock then performance will suffer. I guess this is what happened on Khalid's large box. I'll try to remove the spinlocks and add fixup code to the pagefault handler to see whether this improves the situation on large boxes. This might turn out to be ugly, though. Julian  http://git.infradead.org/users/jsteckli/linux-xpfo.git/shortlog/refs/heads/xpfo-master -- Amazon Development Center Germany GmbH Berlin - Dresden - Aachen main office: Krausenstr. 38, 10117 Berlin Geschaeftsfuehrer: Dr. Ralf Herbrich, Christian Schlaeger Ust-ID: DE289237879 Eingetragen am Amtsgericht Charlottenburg HRB 149173 B
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.