dwarf_loader: Multithreading with a job/worker model
Multithreading is now contained in dwarf_loader.c, and is implemented
using a jobs queue and a pool of worker threads. As a consequence,
multithreading-related code is removed from pahole.c.
A single-thread special case is removed: queueing setup works fine with
a single worker, which will switch between jobs as appropriate.
Code supporting previous version of the multithreading, such as
cu_state, thread_data and related functions, is also removed.
The reproducible_build flag is now moot: the BTF encoding is always
reproducible with these changes.
The goal outlined in the RFC [1] - making parallel reproducible BTF
generation as fast as non-reproducible - is achieved by implementing the
requirement of ordered CU encoding (stealing) directly in the job queue
in dwarf_loader.c
The synchronization in the queue is implemented by a mutex (which
ensures consistency of queue state) and a job_added condition variable.
Motivation behind using condition variables is a classic one: we want to
avoid the threads checking the state of the queue in a busy loop,
competing for a single mutex.
In comparison to the previous version of this patch [2], job_taken
condition variable is removed. The number of decoded CUs in memory is
now limited by initial JOB_DECODE jobs. The enqueue/dequeue interface is
changed aiming to reduce locking. See relevant discussion [3].
[1] https://lore.kernel.org/dwarves/20241128012341.4081072-1-ihor.solodrai@pm.me/
[2] https://lore.kernel.org/dwarves/20241213223641.564002-11-ihor.solodrai@pm.me/
[3] https://lore.kernel.org/dwarves/58dc053c9d47a18124d8711604b08acbc6400340.camel@gmail.com/
Co-developed-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Ihor Solodrai <ihor.solodrai@pm.me>
Cc: Alan Maguire <alan.maguire@oracle.com>
Cc: Andrii Nakryiko <andrii@kernel.org>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Mykola Lysenko <mykolal@fb.com>
Link: https://lore.kernel.org/r/20250109185950.653110-9-ihor.solodrai@pm.me
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
11 files changed
