Date: Fri, 24 Aug 2012 15:54:25 +0200 From: musl <b.brezillon.musl@...il.com> To: musl@...ts.openwall.com Subject: Re: ldso: dlclose. On 24/08/2012 14:27, Rich Felker wrote: > On Fri, Aug 24, 2012 at 09:52:28AM +0200, musl wrote: >> On 23/08/2012 20:01, Rich Felker wrote: >>> On Fri, Aug 24, 2012 at 12:02:09AM +0800, orc wrote: >>>> On Thu, 23 Aug 2012 08:48:16 -0400 >>>> Rich Felker <dalias@...ifal.cx> wrote: >>>> >>>>> Anyway, unless the issue is fixed in binutils so that the vast >>>>> majority of libraries are marked non-unloadable, I don't see anything >>>>> we can do in musl. "glibc does it that way too" is not an excuse for >>>>> adding unsafe/non-robust behavior to musl. >>>>> >>>>> Rich >>>> The whole dlopen/dlclose/dlsym functions family are 'harmful': even if >>>> we want static linking, application will still rely on them and fail >>>> invisibly, creating more headaches. >>>> I think better leave dlclose() in it's current state now. It will always >>>> 'success', nobody will care. >>> In my view, there are only two downsides to the current behavior: >>> >>> 1. Some buggy plugin-based applications may expect dlclose(plugin) to >>> call the destructors in the plugin. This is of course an invalid >>> expectation per POSIX, but it may be the reality for some apps. >> Indeed, many plugins implem rely on constructors/destructors to >> allocate/free memory or intialize/cleanup context. >> This may lead to memory leaks or other issues if the plugin is >> loaded/unloaded multiple times. > A plugin cannot be loaded more than once. Subsequent calls to dlopen > use the existing loaded image. The only way it could be loaded again > is if the file were replaced by a new version. > > I think maybe you're not realizing that the "leak" can only happen if > a new version of the .so file is put in place of the old one... I was talking about this specific case : 1) unloding a plugin 2) updating the plugin (new plugin.so) 3) reloading the plugin During the whole sequence the application is up and running. Here is how I should do it if dlclose is implemented per posix : 1) stop the application 2) update the plugin 3) restart the application The application is not available during this sequence. > >>> 2. In an extremely long-lived app that loads and unloads plugins which >>> may be upgraded multiple times during the application's lifetime, each >>> new version of the plugin will consume additional virtual memory space >>> and commit charge, i.e. you have a memory leak. In the real world the >>> leak should be very slow, but it could become significant if the >>> plugins are very large and get reinstalled many times, perhaps if >>> someone is experimenting and running "make install" each time... >> It might be worst for long-lived apps running in a memory >> constrained environment (embedded systems). > Yes, but in this kind of system, ANY use of dynamic memory allocation > is frowned upon. Dynamic module loading even moreso. And of course I > don't think you'll be constantly replacing .so files on such a system > with new versions. > >>> In my view #2 is a very low-priority problem that's not worth caring >>> about on its own, but #1 may be relevant. If does become an important >>> issue that we can't get fixed at the application level, I think the >>> solution would be to add unloading, but have it only take effect for >>> the actual argument to dlopen/dlclose, never any libraries implicitly >>> loaded as dependencies (and of course to honor the flag that prevents >>> unloading). >> Does this mean you want to call plugin destructors in dlclose >> function and keep the plugin memory mapping ? > No. Calling dtors and unloading always come in a pair. You cannot call > dtors but keep and reuse the mapping because the static-storage > objects would retain their old values from the prior load, but a new > load would be visible to the code in the plugin. > > The potential design I'm talking about would have only the dlopen'd > library itself ever unloaded/unmapped. For example, if myplugin.so > depends on libfoo.so and libbar.so, libfoo.so and libbar.so, which > were implicitly loaded when loading myplugin.sh, will never be > unmappable. Only myplugin.so itself would be unmappable. On > unloading/unmapping dtors would be called as usual, and then the > reference would be removed entirely from the DSO chain, causing it to > be searched-out and loaded new next time dlopen is called. > > I do not want to do this except as a last resort, since as I've > already mentioned it's highly error-prone (see glibc) and fragile. I understand your concern and I'll modify my code to get rid of the dlclose function. I hope there's no other apps or libs relying on gnu dlclose specific implem. It should not if they've read carrefully the dlclose man page :-). BTW, thanks for taking the time to explain the dlclose implications. > > Rich
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.