Date: Wed, 26 Sep 2012 14:15:33 -0400 From: Rich Felker <dalias@...ifal.cx> To: musl@...ts.openwall.com Subject: exit locking madness Some recent discussions regarding the way glibc and musl handle flushing/closing files at exit suggest that both are probably buggy. glibc basically ignores locking entirely, guaranteeing that exit will not deadlock but possibly giving incorrect/corrupt output, and in the worst cases even crashing with a race condition when a file is closed (in another thread) while exiting. musl does maximal locking: first the open file list is permanently locked (causing any further fopen/fclose attempts to deadlock) and then each file, one by one, is permanently locked and flushed. The permanent locking ensures that no further io can be performed (and thus lost) after flushing. Unfortunately, there are some potential problems with musl's approach: 1. If any thread is holding a file lock with flockfile, or by being blocked in a stdio function such as fprintf, fwrite, fread, etc. that's waiting for input, exit is blocked, perhaps indefinitely. This seems bad, but it might actually be the behavior POSIX mandates. I've opened a request for interpretation: http://austingroupbugs.net/view.php?id=611 2. Even if no thread is doing abusive long-term locking, a thread may be holding a lock on f1 while attempting to obtain a lock on f2, while exit is holding a lock on f2 and attempting to obtain a lock on f1. Should exit attempt to work around this? 3. Likewise, a thread might be holding a short-term lock on a file f while attempting to open or close another file. This will deadlock since the open file list lock is permanently held by exit. Here are some partial ideas for a fix: Instead of permanently locking files, lock them temporarily, flush them, then unbuffer them. In unbuffered mode, further io on the file will _mostly_ avoid messing up the state exit is supposed to end with, but it could become inconsistent (wrong final file offset) in the case where fscanf or ungetc is used. Switch to a special type of lock for the open file list, which has 3 modes: unlocked, locked, and perma-locked. When fopen/fclose detect the perma-locked mode, they can take special action. fopen would either always fail, or open an unbuffered file that's not linked in the open file list. fclose would do everything but unlinking the file from the open file list and freeing it, and would set the file descriptor to -1 before unlocking the file so that exit could not later attempt to operate on the fd. The only major issue I see which is not solved is #1, the long-term locks issue. In this case, we'd like to be able "steal" the lock out from under a thread to flush whatever's in the buffer, then let the thread go back to doing unbuffered io on its locked file. But I see no clean way to do this, and depending on the interpretation of the standard, it might not even be valid to do so (since presumably exit must behave "as if" it called flockfile, since it references the file). Ideas? Rich
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.