Date: Mon, 29 Oct 2012 12:14:25 -0400 From: Rich Felker <dalias@...ifal.cx> To: musl@...ts.openwall.com Subject: Re: interesting discussion about static linking on luajit ML On Mon, Oct 29, 2012 at 10:50:03AM +0100, Szabolcs Nagy wrote: > * John Spencer <maillist-musl@...fooze.de> [2012-10-29 07:14:35 +0100]: > > especially the statement here about memory usage seems to be a > > common misconception: > > > > http://www.freelists.org/post/luajit/Creating-a-statically-linked-executable-for-a-LuaJITC-program,22 > > > > well obviously if the statically linked binary is smaller > then the dynamically linked one, then he cannot save > space the way he wants.. i'm not sure where he got that > size only matters on disk > > but he is right about that usually dynamically linked programs > are smaller than statically linked ones > (you should not compare to glibc binaries as they are huge no matter what) > > > rich, you recently said something about this topic on irc, iirc it > > was along the lines of: > > you'd need to link substantially huge parts of libc.a to use nearly > > as much memory, > > as is typically wasted with relocation overhead in dynamically > > linked apps, do i remember correctly ? If you use less than the "dynamic linking overhead" on the libc comparison page worth of code from the library, then static linking will use less memory at runtime than dynamic linking. Right now that threshold is at 20k but I hope to get it back down to 12k soon. > the point is that with dynamically linked binary you must > have the *entire* lib available in memory This is actually not true. Only the pages which are accessed must be resident in memory. If commonly-used and rarely-used functions are ordered randomly in the library, then this is likely to cause nearly the whole library to be paged in, except in cases where individual unused functions are larger than the page size. If the order of objects linked into the library is optimized to put all the rarely-used stuff together, then you can get reasonable memory usage with shared libraries. This is actually a direction of optimization we should explore in musl -- I think it should be fairly easy to make the build scripts group either a known set of commonly-used object files first, or a known set of rarely-used object files last. > (so even if you have that shared between different processes > you need multiple processes using the lib to gain anything) > (and of course not everything is shared: there are writable > data segments (.data, .bss) which cannot be shared between > processes once they are modified (although they are small in > case of musl), and pic code is a bit bigger as well) The sice increase of PIC is nontrivial; I'd say it's around 10-15% on average for i386. On x86_64 and ARM it should be less, but I have not measured. The worst case is that small functions can double or even triple in size. > with static linking you only link those object files that > are actually used by the binary > (eg. in case of musl the static lib is about 340k and about > 80k is just the math+complex part, if you don't use any of > that then it won't get linked in so you don't have to keep > that around in memory) Don't forget that iconv.o is 71k. So that's already a total of 150k that's unlikely to be used in most programs. Once you add in all the legacy cruft, I'd say easily over 50% of the library is unlikely to be used. > (this does not help when you link with xlib because the > simplest x client pulls in a very large part of xlib > and when you run x you surely run several x clients) Yes, that's a design flaw. > (btw even static linked programs can have in memory sharing: > when you run the same process image several times. Yes. This is especially worth noting for shells. > i think a serious script language should have a static > version optionally available, there are situations where > it makes sense and the size difference they are complaining > about rarely matters This is even more true for languages like lua whose primary use case is embedded in other programs. The people responsible for the interpreter/jit implementation have no idea what usage cases the embedding application will be dealing with and whether it might need static linking. Rich
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.