[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Date: Thu, 3 Apr 2014 15:17:53 -0400
From: Rich Felker <dalias@...ifal.cx>
To: musl@...ts.openwall.com
Subject: Re: malloc & uncommitting memory
On Thu, Apr 03, 2014 at 08:17:25PM +0200, M. Ziebell wrote:
> As far as i understand "uncommitted" memory is allocated address space.
> A region in memory which is not used by the program, but the address
> space expanded to make a smaller region available to the programm for
> use.
> Please correct me if I am wrong.
I'm not clear on exactly what you're asking so I'll try to just
clarify the issue. There are basically three types of memory usage
that matter:
- Virtual address space: Process-local resource. Aside from relatively
low overhead in kernel bookkeeping, this only matters in that, if
you run out of addresses, you can't allocate anything else.
- Anonymous pages: Global resource. These are pages that actually
consume (rather than just temporarily using as cache) physical
storage (ram or swap) because they're not backed by a file or device
or the zero page or shared (via COW) with another mapping.
- Commit charge: Global resource. A superset of anonymous pages, also
counting mapped pages that are presently shared (COW) with the zero
page, a file, another process, etc. but which will need their own
storage if/when they're written.
At present, musl's malloc can deallocate only the following:
* All three, only when resizing or freeing an allocation that was
large enough to be serviced individually by mmap.
* Anonymous pages only, by calling madvise with MADV_DONTNEED to reset
them to COW copies of the zero page when a large free range
coalesces in the heap.
In particular, neither commit charge nor virtual address space from
the heap used for small allocations is ever freed. Avoiding freeing
the virtual address space is generally a good thing, since doing so
could result in pathological fragmentation. But not freeing the commit
charge means that an application which momentarily used a large amount
of memory by allocating a small bit at a time (think hideous tree/list
structures used in C++ and glib-based code) then freeing it all will
continue to tie part of the commit limit and reduce the amount of
memory available to other programs (note: this only applies with
overcommit disabled).
> Based on that understanding, the idea or your proposal sounds not
> really desirable.
> You have to take care of tons of exceptions and special cases.
> I'm pretty sure I don't understand what uncommitted memory is for, but
> I'm heary is hundreds of lines of complex code, if-else constructs
> everywhere and bugs.
The whole of malloc is under 600 lines, so anything measured in
"hundreds of lines" is definitely inappropriate as an addition.
However the basic concept (frozen chunk list, described in the first
mail) is at most tens of lines of code, and the strategies 1-3 for
avoiding fragmentation would each be somewhere on the same order of
magnitude, I think.
Rich
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
