Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Date: Fri, 14 Dec 2018 18:45:09 +0100
From: Solar Designer <>
Cc: Jann Horn <>
Subject: Re: Linux kernel: userfaultfd bypasses tmpfs file permissions (CVE-2018-18397; since 4.11; fixed in 4.14.87 and 4.19.7)

On Thu, Dec 13, 2018 at 02:59:43PM +0100, Greg KH wrote:
> On Wed, Dec 12, 2018 at 03:24:15PM +0100, Solar Designer wrote:
> > On Wed, Dec 12, 2018 at 01:27:13AM +0100, Jann Horn wrote:
> > > NOTE: I have requested a CVE identifier, and I'm sending this message,
> > > to make tracking of the fix easier; however, to avoid missing security
> > > fixes without CVE identifiers, you should *NOT* be cherry-picking a
> > > specific patch in response to a notification about a kernel security
> > > bug.
> > 
> > (I resisted the urge to comment on this piece in previous postings.)
> > 
> > What should distros/users do, then?  Use latest mainline or upstream
> > stable kernels?  That would expose them to the many recent bugs like
> > this one, but which haven't yet been found (or not yet made public,
> > which is worse).
> Which is better, to be running a system with unkown or known bugs? :)
> I'd pick unknown, as you are a _bit_ safer that way.

I agree, assuming it's likely also unknown to at least a subset of
potential attackers, whereas the known would also be known to them.

However, a major aspect is how often and how quickly you update in
response to new bug findings.  What I'm saying is that if you run a "too
recent" kernel, you have to update far more often than if you run a
"sufficiently old" kernel.  If you're not sure to update often and
quickly enough, then you're safer running a "sufficiently old" kernel
and updating it in response to the less frequent relevant findings.

> > As far as I can tell, by far most Linux kernel vulnerabilities (that are
> > eventually found and made public) are in relatively recent (as of that
> > time) kernel versions.  So a user or a distro would avoid most
> > vulnerabilities (that are eventually found and made public) by staying
> > sufficiently behind current versions, and relying on backports, even if
> > at risk of missing untracked vulnerabilities.
> Who are you relying on for those backports?

Users rely on their distro.  Distro relies on their upstream distro (if
any) and themselves.  Yes, it can take a while.  Usually the wait time
negatively correlates with issue importance and publicity, as it should.

> And what about all of the backports that do not get made?

An acknowledged risk, and a drawback.

> Just look a the spectre patches for loads of examples of that.

Spectre isn't high+ severity at this time, in my opinion.  It's probably
a higher severity for web browsers with JIT than it is for Linux kernel.

We'd need to see exploitation of Spectre against Linux kernel in the
wild for it to become higher overall severity.  It's great that Spectre
is treated seriously upstream, so that we have mature approaches at
dealing with it when and if it starts being actively exploited in the
wild.  Meanwhile:

Having some more Spectre fixes than are e.g. in a RHEL7 kernel isn't
worth the risk of also having more high+ severity vulnerabilities and
needing to update more often as they're discovered.

> > Currently this can be
> > achieved e.g. by using RHEL7'ish kernels forked by Red Hat off 3.10, but
> > probably not anything newer than that yet.  (And when RHEL7 was just
> > released, its kernels were not quite ready for such use.  It takes
> > even RHEL kernels a few years and a few hundred revisions to mature and
> > become a lower security risk.  Fortunately, there's a previous RHEL at a
> > few years and a few hundred revisions old yet still maintained during
> > that time.)
> If you want to rely on RHEL, that's wonderful, but you are tying
> yourself to some unknown developers doing some unknown work (figuring
> out what they have, and have not applied, is non-trivial.)  Those
> developers do great work, and I strongly recommend people use enterprise
> distros if you can afford them,

I agree there are drawbacks.

> but it will cost you both time and money to do so.

These kernels can actually save a downstream distro and a sysadmin time
on not needing to update as often.

As to monetary cost, there are free distros deriving from RHEL.  In this
discussion, I primarily say RHEL just to give credit where it's due.

> If you don't have the time or money, then I strongly recommend using the
> latest stable updates, as they are faster and free :)

... but not mature enough.  (I'll comment on "not wine" below.)

> [...] you can look at the work that the syzbot people are doing for
> concrete evidence that they are finding, and fixing bugs in newer
> kernels that do not get fixed in older kernels, and are still present in
> those older kernels.  I think they are averaging about 10k bugs found a
> year so far.

This is weird.  Brad brought this quote of yours to Twitter, and no one
appears to know how to make proper sense of it.  Dmitry Vyukov tweeted:

"I don't have numbers for old/new bug fix ratio. But there are 10000+
bugs/year fixed. Kernel is 27 years old. This means that either we also
introduce 10000+ bugs/year (25/day), or initial Linus kernel source
contained 25 bugs per each and every line of code"

> And don't think of software as "mature", this isn't wine.  Code gets
> worse with age as the environment changes from when it was written.
> Older code is worse as it is harder to maintain and takes more effort
> over time.  Again, if you have the time and money to do so, wonderful,
> but you might want to reconsider your use of that time and money, given
> that many other large groups consider using the latest kernel a better
> use of their time.
> Also note that older kernels do not work well on newer hardware for the
> obvious reason.  And newer kernels usually run _faster_ on older
> hardware, we have the benchmarks to prove that.  So you can do more
> work, with older hardware, just by updating your kernel, saving you
> money and time :)

That's why I don't run e.g. 2.0.40 kernels anymore. ;-)  They were so
much smaller and in some ways safer (e.g., safe from Meltdown on x86) -
but we need new hardware support, many of the new features, filesystems,
protocols, and better scalability.

So my preference for mature software is counter-balanced by needing
newer functionality and needing to fit in the changing ecosystem.
That's why I mentioned RHEL7 and not older - as the latest that is
already (barely) mature enough.

> > The recommendation to use latest mainline or upstream stable kernels is
> > safe to give (and in a way even the most responsible one to give), but
> > not necessarily the best to follow.
> It all depends on your environment and situation.
> As you say, it is the safest to give, but everyone is different and
> needs to do things differently.  I wrote a whole long essay on this
> thing a while ago if people are interested that tries to provide some
> nuance depending on your situation:

I agree with this part:

"Hierarchy of what kernel to use, from best solution to worst:

 * Supported kernel from your favorite Linux distribution
 * Latest stable release
 * Latest LTS release
 * Older LTS release that is still being maintained

What kernel to never use:

 * Unmaintained kernel release"

However, I think this recommendation isn't great for servers:

"Personally, I prefer the community based Linux distributions that
constantly roll along with the latest updated kernel and it is supported
by that developer community. Distributions in this category are Fedora,
openSUSE, Arch, Gentoo, CoreOS, and others."

Reboot-less live kernel updates from the distro might change this, but
even then each update is effort for the distro and a risk, so reducing
their frequency is preferable.

> I would say we fix more old bugs than
> newer ones by far, and the syzbot numbers back that up.

I'd be interested to see this backed up with the syzbot numbers.


Powered by blists - more mailing lists

Please check out the Open Source Software Security Wiki, which is counterpart to this mailing list.

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.