Date: Tue, 5 Aug 2014 14:08:16 +0200 From: Szabolcs Nagy <nsz@...t70.net> To: musl@...ts.openwall.com Subject: Re: openmp/pthreads and fork... * Isaac Dunham <ibid.ag@...il.com> [2014-08-04 22:56:51 -0700]: > OpenBLAS (optionally) uses pthreads or OpenMP, and OpenMP uses pthreads. > OpenBLAS implements the BLAS (Basic Linear Algebra Subprograms) API/ABI, > like ATLAS, MKL, and so on. > Some programs that use BLAS will fork(); python is one of these. > With OpenBLAS, this had caused "random" segfaults due to use of threads > by the library both before and after the fork. > The OpenBLAS issue is here: > https://github.com/xianyi/OpenBLAS/issues/294 > > They worked around this using pthread_atfork() to cleanup before the fork(). > I see some claims that calling any pthread functions in the child would be > UB, so I'm wondering about a few things: > -Is the last-mentioned claim correct? the fix does make fork work with openmp in some cases: if the application calls fork when no openmp code is running then both the child and parent have the omp lib in consistent state if the application calls fork without synchronizing it with openmp calls then the application is not allowed to do much in the child anyway (it cannot call openmp code so inconsistent state there is not an issue) and the atfork handler is correct with some caveats: - gomp_we_are_forked var should be volatile sig_atomic_t in theory (but in practice their static int might work too) - if fork happens in an async signal handler that interrupted pthread_atfork, then the pthread_atfork global state may be inconsistent so fork can crash in libc (libc could probably guard against this but no libc does that afaik, howver other libcs dont have an async-signal-safe fork so this would be ub anyway) > -What is musl's behavior? > -How correct, and how likely to work with musl, is the fix for libgomp > mentioned here: > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=60035 > (and for that matter, the equivalent workaround referred to in OpenBLAS > issue 294)? the above workaround should work with musl (except when fork is used form a sig handler) with glibc there are other caveats: - glibc dlclose removes the shared library, so if openmp was dlopened and then dlclosed (may happen with python), then the next fork will crash (because of the dangling atfork handler) (the workaround is to use __register_atfork #ifdef __GLIBC__ which takes care of the dlclose issue) - pthread_atfork is only available in -lpthread, so a library that is itself not multi-threaded but tries to work around fork issues cannot use it without pulling in all the pthreads (__register_atfork does not have this issue) > -Is there a safe (non-crashing) way to use/write libraries that might or > might not be built with threading--whether POSIX-specified or just > "working with musl" ? threading is not the issue, forking is (more specifically doing library calls in the child after fork in a multi-threaded process) similar issue came up recently in libressl (see my notes starting at the "But.." section in http://port70.net/~nsz/47_arc4random.html ) my conclusion was that a library cannot reasonably do anything to guard against unsafe fork usage in the application so it shouldn't try to pthread_atfork is broken, makes fork non-async-signal-safe and cannot be used for the originally intended purposes the application may not be aware of underlying threads in various libraries, so fork should be avoided: use posix_spawn or fork+exec (without intervening library calls), other fork usage is just broken (unless the application has full control over its library dependencies)
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.