[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Wed, 13 Feb 2019 17:01:35 -0700
From: Khalid Aziz <khalid.aziz@...cle.com>
To: juergh@...il.com, tycho@...ho.ws, jsteckli@...zon.de, ak@...ux.intel.com,
torvalds@...ux-foundation.org, liran.alon@...cle.com,
keescook@...gle.com, akpm@...ux-foundation.org, mhocko@...e.com,
catalin.marinas@....com, will.deacon@....com, jmorris@...ei.org,
konrad.wilk@...cle.com
Cc: deepa.srinivasan@...cle.com, chris.hyser@...cle.com, tyhicks@...onical.com,
dwmw@...zon.co.uk, andrew.cooper3@...rix.com, jcm@...hat.com,
boris.ostrovsky@...cle.com, kanth.ghatraju@...cle.com,
oao.m.martins@...cle.com, jmattson@...gle.com,
pradeep.vincent@...cle.com, john.haxby@...cle.com, tglx@...utronix.de,
kirill.shutemov@...ux.intel.com, hch@....de, steven.sistare@...cle.com,
labbott@...hat.com, luto@...nel.org, dave.hansen@...el.com,
peterz@...radead.org, kernel-hardening@...ts.openwall.com,
linux-mm@...ck.org, x86@...nel.org,
linux-arm-kernel@...ts.infradead.org, linux-kernel@...r.kernel.org,
Khalid Aziz <khalid.aziz@...cle.com>,
"Vasileios P . Kemerlis" <vpk@...columbia.edu>,
Juerg Haefliger <juerg.haefliger@...onical.com>,
Tycho Andersen <tycho@...ker.com>,
Marco Benatto <marco.antonio.780@...il.com>,
David Woodhouse <dwmw2@...radead.org>
Subject: [RFC PATCH v8 12/14] xpfo, mm: optimize spinlock usage in xpfo_kunmap
From: Julian Stecklina <jsteckli@...zon.de>
Only the xpfo_kunmap call that needs to actually unmap the page
needs to be serialized. We need to be careful to handle the case,
where after the atomic decrement of the mapcount, a xpfo_kmap
increased the mapcount again. In this case, we can safely skip
modifying the page table.
Model-checked with up to 4 concurrent callers with Spin.
Signed-off-by: Julian Stecklina <jsteckli@...zon.de>
Signed-off-by: Khalid Aziz <khalid.aziz@...cle.com>
Cc: x86@...nel.org
Cc: kernel-hardening@...ts.openwall.com
Cc: Vasileios P. Kemerlis <vpk@...columbia.edu>
Cc: Juerg Haefliger <juerg.haefliger@...onical.com>
Cc: Tycho Andersen <tycho@...ker.com>
Cc: Marco Benatto <marco.antonio.780@...il.com>
Cc: David Woodhouse <dwmw2@...radead.org>
---
mm/xpfo.c | 25 ++++++++++++++++---------
1 file changed, 16 insertions(+), 9 deletions(-)
diff --git a/mm/xpfo.c b/mm/xpfo.c
index dc03c423c52f..5157cbebce4b 100644
--- a/mm/xpfo.c
+++ b/mm/xpfo.c
@@ -124,28 +124,35 @@ EXPORT_SYMBOL(xpfo_kmap);
void xpfo_kunmap(void *kaddr, struct page *page)
{
+ bool flush_tlb = false;
+
if (!static_branch_unlikely(&xpfo_inited))
return;
if (!PageXpfoUser(page))
return;
- spin_lock(&page->xpfo_lock);
-
/*
* The page is to be allocated back to user space, so unmap it from the
* kernel, flush the TLB and tag it as a user page.
*/
if (atomic_dec_return(&page->xpfo_mapcount) == 0) {
-#ifdef CONFIG_XPFO_DEBUG
- BUG_ON(PageXpfoUnmapped(page));
-#endif
- SetPageXpfoUnmapped(page);
- set_kpte(kaddr, page, __pgprot(0));
- xpfo_flush_kernel_tlb(page, 0);
+ spin_lock(&page->xpfo_lock);
+
+ /*
+ * In the case, where we raced with kmap after the
+ * atomic_dec_return, we must not nuke the mapping.
+ */
+ if (atomic_read(&page->xpfo_mapcount) == 0) {
+ SetPageXpfoUnmapped(page);
+ set_kpte(kaddr, page, __pgprot(0));
+ flush_tlb = true;
+ }
+ spin_unlock(&page->xpfo_lock);
}
- spin_unlock(&page->xpfo_lock);
+ if (flush_tlb)
+ xpfo_flush_kernel_tlb(page, 0);
}
EXPORT_SYMBOL(xpfo_kunmap);
--
2.17.1
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
