Date: Thu, 11 Apr 2019 16:41:17 +0200 From: Markus Wichmann <nullplan@....net> To: musl@...ts.openwall.com Subject: Re: Thread-local memory for thread structures On Thu, Apr 11, 2019 at 12:12:46PM +0100, Raphael Cohn wrote: > Dear List, > > I'm playing around with allocating 100s of bytes of TLS memory for > various purposes. Something I noticed in the code for creating the > mmap'd memory for TLS is that it does not (quite reasonably) assign it > a NUMA memory policy. > > I'd like to assign a NUMA memory policy to the memory used for > managing a thread. Is there anything 'underhanded' I can do to find > out its location and size? I realize anything is likely to be brittle. Worse, it's going to be arch-specific. Location of the thread-local memory is easy, however, you just read the thread pointer. Basically, you follow your arch's ABI to find the TLS base pointer from the thread pointer, which must be in some kind of register. Finding the size is less easy. In the statically linked case, you can just read the program headers with getauxval(AT_PHDR) to find the size to the PT_TLS segment, but in the dynamically linked case, you would then have to add up all the sizes of the TLS segments of all the linked-in libraries. But you don't know their base pointers, only the libc does. Well, you /could/ parse /proc/self/maps. But at least once you add dlopen() into the mix, the task becomes next to entirely infeasible. You'd have to parse /proc/self/maps again after each dlopen(), find out the libs you missed earlier, and then add their TLS sizes to your total. And then ring up every thread to tell them the size has changed. > Ideally what I'd like is a 'set NUMA memory policy of this thead's > mmap'd management memory to the local NUMA node [once I've scheduled > it to run a particular set of CPUs]. > > Any suggestions? > > Raph If the manpage is to be trusted, then: | The thread memory policy is preserved across an execve(2), and is | inherited by child threads created using fork(2) or clone(2). So, you could use set_mempolicy() to set MPOL_LOCAL before spawning threads. Of course, that would be horrible for shared memory, but that is a cross you have to bear. Ciao, Markus
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.