Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Tue, 11 Jan 2022 21:30:10 +0000
From: Harmen Stoppels <me@...menstoppels.nl>
To: musl@...ts.openwall.com
Subject: Re: Cannot dlopen() an already loaded shared library by its SONAME name

> Given that this question already surfaced twice and was denied just as
> 

> often, I'm guessing no.

The fact that this is a common request could also imply it's behavior
people expect. So far I've seen:

1. Julia [1] splits binary dependencies into separate packages, so when
   liba.so depends on libb.so, they live in a different dir, where
   the absolute and relative paths are only known when the julia
   interpreter has started, so neither rpaths or LD_LIBRARY_PATH can
   be used.

   So they dlopen libb.so, and then dlopen liba.so in that
   order, and then assume liba.so does not have to locate libb.so
   again, because its soname is already seens before.

   The proposed workaround was: don't list libb.so in the
   DT_NEEDED of liba.so (that is, if you're already doing the work of
   the linker, you might as well not use the linker at all for locating
   libs). However, being able to run executables shipped with julia
   packages would still be nice (e.g. a subprocess with LD_LIBRARY_PATH
   set properly)
2. The Nix / Guix / Spack people are trying to reduce startup time of
   executables with many shared libraries (as well as fixing library
   paths once and for all to keep executables run deterministically).
   In Guix there's a blog post where they call this the "stat storm" [2],
   and they solve it in a glibc patch: using context dependent ld.so.cache,
   that is, a reverse mapping soname => library path.
   In Nix the proposal to fix the "stat storm" is to replace DT_NEEDED
   in executables with absolute paths of all required libs (also
   transient ones). This works fine, except on musl, where a dlopen by
   soname will still do a search.
3. Wine is reported to rely on this earlier in the mailing list, but that
   did not get any responses [3]. It reports the behavior is the same on
   glibc, FreeBSD, NetBSD, and OpenBSD dynamic loaders; and musl is the
   exception. 



[1] https://github.com/JuliaLang/julia/issues/40556
[2] https://guix.gnu.org/en/blog/2021/taming-the-stat-storm-with-a-loader-cache/
[3] https://www.openwall.com/lists/musl/2021/12/16/1


> The reason for the behavior is that loading a library with explicit path
> 

> name is what you do with a plugin, and you don't necessarily want a
> 

> plugin's symbols to be visible to everyone. That's why a library loaded
> 

> by path name does not get a shortname set, and a shortname is what you
> 

> need to be able to find a library by just its name.

To me it seems very uncommon that two libraries with the same soname are not
supposed to be the same library.
Download attachment "publickey - me@...menstoppels.nl - 0xFD537C88.asc" of type "application/pgp-keys" (1815 bytes)

Download attachment "signature.asc" of type "application/pgp-signature" (510 bytes)

Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.