Date: Sun, 26 Jul 2015 17:17:11 +0300 From: Timo Teras <timo.teras@....fi> To: "musl@...ts.openwall.com" <musl@...ts.openwall.com> Subject: Multithreaded program VSZ increasing due to bad free() strategy Hi, It seems that the current free() strategy of "allocate all free neighbors" has results in huge VSZ sizes (while RSS size keeps small). The problem is that when the free() is made release most of the heap, and simultaneous malloc() occurs on another thread it is likely to extend heap. This is because free() has all memory allocated, and is executing madvise() (or several of them) which is slow. The result is that heap keeps getting extended all the time increasing VSZ, and the slowing down the program (madvise is slower, the larger area it's telling about). Though, due to madvise calls RSS stays low, and it's not actually using any of that memory. It would probably help a bit, to just move madvise() out from the for loop as one free() can result in multiple madvise() calls if other thread simultaneously free()'s neighboring areas and retried test of (self->psize & next->csize & C_INUSE) while locks held fails. It would probably make sense to move madvise() just out of loop when the final block size is known. But the strategy really needs to be better. When a multithreaded reactive program is mainly allocating temporary space and frees it soon after, it almost always end up in the race that free() holds locked most of the heap in preparation to merge it to one big continuous chunk. Causing the unwanted consequences. Ideas? Thanks, Timo
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.