Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Tue, 10 Apr 2018 16:44:11 -0400
From: Rich Felker <>
Cc: Florian Weimer <>
Subject: Re: tcmalloc compatibility

On Tue, Apr 10, 2018 at 10:33:55PM +0200, Szabolcs Nagy wrote:
> * Florian Weimer <> [2018-04-10 19:53:46 +0200]:
> > We have some documentation nowadays:
> > 
> > <>
> > 
> > The remaining undocumented aspects concern cyclic dependencies, such
> > as the suitability of certain TLS models for implementing a custom
> > malloc, or using memory-allocating glibc functions such as fopen or
> > backtrace from the allocator itself.
> > 
> it's not documented what the interposed malloc may or may not do.
> cyclic dependency (because malloc calls a memory-allocating libc
> function) is one issue, but in general the libc can be in an
> inconsistent state at the point of an internal malloc call (e.g.
> some lock is held that should never be held when user code is
> running) and thus certain other libc functions may not operate
> as expected in an interposer.
> documenting the set of libc apis that work as expected in an
> interposer is difficult.

I think limiting to AS-safe plus a small list of additional allowed
functions is reasonably easy to guarantee works.

> > In practice, malloc interposition works extremely well and causes few
> > issues due to interposition itself.  Obviously, there are bugs, but
> > most of them would still be bugs if the allocator was non-interposing.
> > (Examples are lots of initial-exec TLS data, and incorrect alignment
> > for allocations.)
> even with the very small set of libc apis that a malloc
> interposer may want to use, there known (but undocumented)
> caveats:
> - glibc dlsym calls calloc (bites anybody who tries to wrap
> calloc in the usual way)

Yes. While I think it's unreasonable that glibc calls calloc from
dlsym, I also think it's unreasonable to be calling dlsym from a
malloc replacement. I don't think I would include dlsym or other
heavy, AS-unsafe functions in an allowed-list for musl.

> - general dynamic tls may call malloc and uses global dl
> lock so using tls can cause recursion or deadlock.

Yes but this is a known glibc bug that's hopefully on the way to
being fixed. A robust/correct implementation (where TLS works in
signal handlers and without possibility of OOM-crashing) can't
allocate at access time.

> - using stdio for logging is problematic because of stdio
> locks and recursive malloc calls.


> - malloc may get called before the first ctor and after
> the last dtor.

An implementation could avoid this, but yes, I agree, there should be
no contract that it won't.

> - malloc may be called on a thread with small stack
> (e.g. gai helper thread) so it should not have excessive
> stack usage.

Certainly increasing the default thread stack size could be a
requirement to use malloc replacements that have large stack usage,
but I tend to doubt that would happen. If malloc has large stack
usage, it's likely to be very slow.

> - glibc fork tries to reset the malloc state before
> calling the child atfork handler in an attempt to support
> non-as-safe fork handlers, code that relies on this can
> get broken with malloc interposition.

Interposed mallocs probably try to support this with pthread_atfork,
but state after MT-fork is permanently an AS context anyway.

> interposition only seems to work extremely well because
> users already fixed their code and things dont change
> much in this space.

Maybe. :)


Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.