[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Wed, 27 May 2015 13:18:42 +0200
From: Alex Dowad <alexinbeijing@...il.com>
To: musl@...ts.openwall.com
Subject: Re: Question re: dynamic linking in musl
On 27/05/15 12:40, Alexander Monakov wrote:
> On Wed, 27 May 2015, Alex Dowad wrote:
>> I'm just dealing with a problem whereby rrdtool is spending >90% of its runtime in
>> musl's dynamic linker (this is on Alpine Linux). This is making it too slow to do
>> what we need it to do.
> That implies that rrdtool spawns an executable with many dynamic dependencies
> over and over again, with each individual invocation doing extremely little
> work. That sounds wasteful. Do you know why it does that?
rrdtool is a monolithic binary which can do various different jobs,
depending on the 1st CL arg. It can be invoked as "rrdtool create",
"rrdtool update", "rrdtool graph", and so on. Yes, some of those
commands do very little work.
Because the same binary does a variety of different things, a lot of
those dynamic dependencies are not needed on most runs. So yes, it is
wasteful. Why did the author design it that way? I don't know. It works
quite well for the intended purpose, aside from this point of using a
lot of dynamic libraries.
>
> Can you also provide exact figures? For example your callgrind logs? Or just
> 'time' statistics for executables that spend much time in the dynamic linker.
Callgrind data attached.
>
>> NOW: Is there any reason why dso->deps could not be used to search for symbols *only*
>> in the dependencies of the binary which is currently being relocated?
> That is not how ELF symbol lookup works. Simply put, the symbol should be
> taken from whatever first module in the whole lookup chain provides it; e.g.
> if the executable exports a function that is also exported in libX11, the
> executable's definition prevails.
OK, I get it. So dynamic linking has O(n^2) runtime, with "n" being the
number of dynamic libraries you link against!
>
>
> Recently I was working on some dynamic linker speedups for musl. On my
> testcase, Clang/LLVM with ~100-200 dynamic libraries and ~20000 symbols that
> need to be resolved in the dynamic linker, I obtained a speedup from 240 ms to
> 110 ms, while glibc needs 50 ms with lazy binding and 140 without. So while
> musl's dynamic linker speed can be improved, still rrdtool is doing something
> odd in the first place if it's performance is bound like that. I'd like to
> understand that. If it's unavoidable, perhaps static linking is appropriate.
>
> (I didn't send my dynlink speedup patches yet; I intend to do that sometime
> soon)
That speedup will be very welcome, especially for clang! clang builds
are noticeably slower than gcc. Looking forward to it.
Yes, static linking sounds like a good idea for rrdtool; unfortunately
their build scripts are oriented towards dynamic linking. The other pain
is that statically linked versions of all the recursive dependencies are
not available from package repos, so you have to build the whole
dependency tree from scratch.
Thanks, AD
View attachment "callgrind.out.8354" of type "text/plain" (87491 bytes)
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
