tree c8646d00c0da74ab6520c7f114e022404320f785
parent 17c757110e5ff2f1f7c6b20d2b142cec056a6dd7
author Ihor Solodrai <ihor.solodrai@pm.me> 1736449241 +0000
committer Arnaldo Carvalho de Melo <acme@redhat.com> 1736458563 -0300

dwarf_loader: Multithreading with a job/worker model

Multithreading is now contained in dwarf_loader.c, and is implemented
using a jobs queue and a pool of worker threads. As a consequence,
multithreading-related code is removed from pahole.c.

A single-thread special case is removed: queueing setup works fine with
a single worker, which will switch between jobs as appropriate.

Code supporting previous version of the multithreading, such as
cu_state, thread_data and related functions, is also removed.

The reproducible_build flag is now moot: the BTF encoding is always
reproducible with these changes.

The goal outlined in the RFC [1] - making parallel reproducible BTF
generation as fast as non-reproducible - is achieved by implementing the
requirement of ordered CU encoding (stealing) directly in the job queue
in dwarf_loader.c

The synchronization in the queue is implemented by a mutex (which
ensures consistency of queue state) and a job_added condition variable.
Motivation behind using condition variables is a classic one: we want to
avoid the threads checking the state of the queue in a busy loop,
competing for a single mutex.

In comparison to the previous version of this patch [2], job_taken
condition variable is removed. The number of decoded CUs in memory is
now limited by initial JOB_DECODE jobs. The enqueue/dequeue interface is
changed aiming to reduce locking. See relevant discussion [3].

[1] https://lore.kernel.org/dwarves/20241128012341.4081072-1-ihor.solodrai@pm.me/
[2] https://lore.kernel.org/dwarves/20241213223641.564002-11-ihor.solodrai@pm.me/
[3] https://lore.kernel.org/dwarves/58dc053c9d47a18124d8711604b08acbc6400340.camel@gmail.com/

Co-developed-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Ihor Solodrai <ihor.solodrai@pm.me>
Cc: Alan Maguire <alan.maguire@oracle.com>
Cc: Andrii Nakryiko <andrii@kernel.org>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Mykola Lysenko <mykolal@fb.com>
Link: https://lore.kernel.org/r/20250109185950.653110-9-ihor.solodrai@pm.me
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>