Date: Sat, 16 Mar 2019 09:58:41 -0400 (EDT) From: Mathieu Desnoyers <mathieu.desnoyers@...icios.com> To: Szabolcs Nagy <nsz@...t70.net> Cc: musl@...ts.openwall.com, Michael Jeanson <mjeanson@...icios.com>, Richard Purdie <richard.purdie@...uxfoundation.org>, Jonathan Rajotte <jonathan.rajotte-julien@...icios.com> Subject: Re: sysconf(_SC_NPROCESSORS_CONF) returns the wrong value ----- On Mar 15, 2019, at 10:25 PM, Szabolcs Nagy nsz@...t70.net wrote: > * Jonathan Rajotte-Julien <jonathan.rajotte-julien@...icios.com> [2019-03-15 > 17:02:02 -0400]: >> We are currently in the process of making sure that lttng  (linux tracer) run >> smoothly on system using musl (Yocto, Alpine etc.). Most things work >> fine. Still, we currently have tests that are failing due to an issue regarding >> the reported number of configured processors on the system >> (__SC_NPROCESSORS_CONF). >> Note that users of LTTng are also affected by this if they chose to modify the >> sched affinity of their instrumented apps. This is relatively a big deal for us. [...] > i think we need to know why does a process care if musl returns > the wrong number? or what are the valid uses of such a number? > (there are heterogeous systems like arm big-little, numa systems > with many sockets, containers, virtualization,.. how deep may a > user process need to go down in this rabbit hole?) The use-case for LTTng-UST is as follows: it implements a ring buffer which is allocated (taking NUMA affinity into account) and owned by a centralized daemon (lttng-consumerd), passed to traced applications as file descriptors over unix sockets, and then memory-mapped into each application address space. This ring buffer is a per-cpu buffer for performance reasons. The way LTTng-UST handles cpu hotplug is to pre-allocate buffers for all possible CPUs, hand them over to applications, so those applications can always find an allocated buffer to write into for the current cpu reported by sched_getcpu(), even after a cpu hotplug. In comparison, the LTTng-modules Linux kernel tracer can do a much more clever use of memory resources, because it has access to cpu hotplug kernel modules APIs, which allows it to allocate buffers only for online CPUs. Unfortunately, there is no way to do this for user-space buffers that I'm aware of other than lazily allocating the resources on first use, and we cannot do this because allocation is performed by a centralized daemon (not by the application being traced and doing the actual "first use"). Moreover, as an overall design guide-line, we keep lazy allocation to a minimum in LTTng-UST, so tracing does not cause unexpected application latency. Thanks, Mathieu -- Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.