Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Wed, 18 Nov 2015 10:49:58 -0500
From: Steve Grubb <>
Cc: Florian Weimer <>, Solar Designer <>,
        Jeff Law <>, Bernd Schmidt <>
Subject: Re: Re: Fwd: x86 ROP mitigation

On Wednesday, November 18, 2015 11:51:23 AM Florian Weimer wrote:
> On 11/18/2015 03:10 AM, Solar Designer wrote:
> > This approach makes sense to me, but I think we should have a better
> > idea of whether and how "a point where ROP gadgets are reasonably hard
> > to find & exploit" is potentially reachable.  If it is not even
> > potentially reachable, then this undermines the effort, unfortunately.
> This came up in other discussions as well.  We even got to the point
> where someone ran a ROP gadget finding tool on a core library, which did
> not find any gadgets at all, and someone else found a useful one in a
> few minutes with objdump and no other tool support (and this did not
> even include jumping into the middle of instructions).

This was something that I was involved in. What I did was get the latest 
source code of ROPgadget. [1]  I have no idea how good it is compared to other 
tools. But it does have a command line switch that builds a full ROP chain so 
that you have a working exploit.

Next, I wrote a small script to iterate over the directories on my Fedora 22 
system that should hold programs or libraries the attacker might exploit and 
check each and every one of them using ROPgadget. I was curious what the size 
of the elephant is that we have.

What I found was that the list of libraries or programs that ROPgadget could 
build a chain for is fairly small. I thought about reasons why that might be 
the case and then considered that maybe if the gadgets from several libraries 
were combined, maybe it would find more. But I think ASLR would make too many 
moving parts for that to be practical. If you use a whole library or 
application, then everything moves together up or down as a unit to the new 

Another thought in explaining why the list was so small is that the quality of 
the chaining that ROPgadget has needs a lot of improvement. Could someone more 
clever piece together gadgets that make a chain that ROPgadget didn't see? I 
don't have the expertise to do this by hand. So, I'll do what others would do 
and look for another tool. There is only one other tools that I could find, 
ropper [2]. It dies due to a programming bug. So, I doubt its being used.

The following files are the ones that ROPgadget was able to build a chain for:


This is on a desktop with a lot of server and software development packages 
that total up to approx 3800 rpms. If we are going to try to spoil ROP 
gadgets, I would suggest that we as a community pick one tool and give it some 
love so that it finds all kinds of gadgets. This way we know how effective any 
mitigations are.

During this study, Florian had suggested checking -fstack-protector-all. This 
defeated ROPgadget. It was not able to find any ROP gadgets in anything 
compiled that way. If it were better at finding gadgets I would like to retry 
the study to see if that still holds true.

> In the end, this boils down to lack of concrete goals.  “Blinding ROP
> gadget finder X“ is easy (just change the ELF format in such a way that
> it's no longer recognized by the tool), but probably not very useful if
> you want to improve security, for any useful definition of “security”.
> We face the problem that I and my immediate colleagues (on the Red Hat
> tools team) do not have access to information about successful
> compromises, and what attackers actually do today, on GNU/Linux systems,
> both to achieve initial access

There is information about this scattered around. It largely depends on what 
the role of the system is, what exploit is recently circulating, and external 
vs internal threat actors. Fishing around for the top uses of Linux servers 
[3] reveals probably what we all knew its used for: virtualization, database 
servers, web servers, application servers, etc.

For web servers, there are studies [4] that show what people do. TL;DR: they 
find a hole in the web software to issue a wget command to pull down software, 
this lands in /tmp, they then execute the software downloaded.

There's 3 different points where this could have been defeated. 1) mod_security 
probably would have blocked whatever weird URL or hole they found. 2) /tmp 
should be mounted noexec. But noexec is easy to defeat by invoking the 
interpreter or directly. 3) This is the hard one and yet so simple to 
fix....make all interpreters check the execute bit before executing. They need 
to be a policy enforcement point for the noexec mount option. Otherwise we may 
as well ask the kernel guys to remove the noexec mount option because its 

For other servers, its a similar pattern.

> and to maintain a presence afterwards.

This is something I am also interested in. There are groups of people studying 
this. One such project is ATT&CK [5] run by MITRE. I have been collecting 
information for Linux systems to add to their project. The idea of that 
project is to enumerate the various ways that an attacker can perform actions 
post exploit. With a catalogue, you can then go build tools that check the 
hiding places. If they get a rootkit installed, you might not be able to 
detect it on the host, but rather by its actions on the network.

>From this catalogue, you can the create indicators of compromise to look for. 
Mandiant has one method [6], but I would rather see something based around 
SCAP tooling so that its standardized.

> Under these conditions, anything we implement is, to some degree,
> arbitrary and a shot in the dark.  We can still use our best judgment to
> set priorities, but we are very far from being guided by empirical evidence.

I hope I filled in some of the blanks. I am sure that others can point to more 
information to help fill in more gaps.


1 -
2 -
3 -
4 -
5 -
6 -

Powered by blists - more mailing lists

Please check out the Open Source Software Security Wiki, which is counterpart to this mailing list.

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.