Date: Wed, 2 Apr 2014 17:08:05 -0400 From: Rich Felker <dalias@...ifal.cx> To: musl@...ts.openwall.com Subject: malloc & uncommitting memory Adding the ability for malloc to "uncommit" memory that it's already obtained has been a goal for a long time, but I haven't found a way to make it efficient. On the kernel side there's been the whole "vrange" idea floating around for a while, but it's only going to work on really new kernels, and only on mainstream archs, even if it gets adopted, and so far the interface seems ugly and unlikely to work very well. Here's a potential alternative approach: Whenever a chunk is added (merged, updated, and re-added) to the bin-63 free list (the largest bin, usually not used directly for satisfying allocation requests but as a source for trimming when no smaller-size bins have free chunks that can satisfy the allocation), move it to the end of the bin rather than the beginning if it's larger than the chunk currently at the end. This allows us to track the largest free area. If the number of free chunks in bin 63 exceeds a certain threshold (possibly very small), allocate the largest one (unbin it), mmap a new PROT_NONE zero region over top of the middle of it, and store it in a new linked list of "frozen" memory regions. When no free chunks are available in bin 63 and one is needed, before trying to expand the brk or use mmap, remove a chunk from the frozen list, mprotect it accessible, and use it. If it's excessively large, split it and only mprotect a reasonable-size part, re-adding the remainder to the frozen list. (Note: mprotect can fail here due to commit limit, and in that case, the frozen chunk would be stuck frozen and malloc would have to fail the allocation.) As long as thresholds are chosen right, I don't think this approach would be costly from a performance standpoint. But it might have consequences for fragmentation. So I'd like to discuss it further, see what ideas emerge, and whether it looks reasonable before trying to implement anything. It's also possible that this whole goal is undesirable, since it would create _more_ situations where malloc could fail in the app that's being nice about "uncommitting" its memory; the benefit is that it reduces the chance of malloc failure in other apps when one app has momentarily used huge amounts of memory (allocated in small units, so that mmap wasn't used) then returned it. Rich
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.