[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Date: Wed, 2 Apr 2014 17:08:05 -0400
From: Rich Felker <dalias@...ifal.cx>
To: musl@...ts.openwall.com
Subject: malloc & uncommitting memory
Adding the ability for malloc to "uncommit" memory that it's already
obtained has been a goal for a long time, but I haven't found a way to
make it efficient. On the kernel side there's been the whole "vrange"
idea floating around for a while, but it's only going to work on
really new kernels, and only on mainstream archs, even if it gets
adopted, and so far the interface seems ugly and unlikely to work very
well. Here's a potential alternative approach:
Whenever a chunk is added (merged, updated, and re-added) to the
bin-63 free list (the largest bin, usually not used directly for
satisfying allocation requests but as a source for trimming when no
smaller-size bins have free chunks that can satisfy the allocation),
move it to the end of the bin rather than the beginning if it's larger
than the chunk currently at the end. This allows us to track the
largest free area.
If the number of free chunks in bin 63 exceeds a certain threshold
(possibly very small), allocate the largest one (unbin it), mmap a new
PROT_NONE zero region over top of the middle of it, and store it in a
new linked list of "frozen" memory regions.
When no free chunks are available in bin 63 and one is needed, before
trying to expand the brk or use mmap, remove a chunk from the frozen
list, mprotect it accessible, and use it. If it's excessively large,
split it and only mprotect a reasonable-size part, re-adding the
remainder to the frozen list. (Note: mprotect can fail here due to
commit limit, and in that case, the frozen chunk would be stuck
frozen and malloc would have to fail the allocation.)
As long as thresholds are chosen right, I don't think this approach
would be costly from a performance standpoint. But it might have
consequences for fragmentation. So I'd like to discuss it further, see
what ideas emerge, and whether it looks reasonable before trying to
implement anything.
It's also possible that this whole goal is undesirable, since it would
create _more_ situations where malloc could fail in the app that's
being nice about "uncommitting" its memory; the benefit is that it
reduces the chance of malloc failure in other apps when one app has
momentarily used huge amounts of memory (allocated in small units, so
that mmap wasn't used) then returned it.
Rich
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
