tree 40f90f6a777a81de77325416209a035429a7a99d
parent f57805555834494e8cad729d01d86ba326d64959
parent 48a8f78c50bd6f7f08fd40daa62252fd043f2f18
author Palmer Dabbelt <palmer@rivosinc.com> 1694193482 -0700
committer Palmer Dabbelt <palmer@rivosinc.com> 1694197525 -0700
gpgsig -----BEGIN PGP SIGNATURE-----
 
 iQJHBAABCAAxFiEEKzw3R0RoQ7JKlDp6LhMZ81+7GIkFAmT7ZxUTHHBhbG1lckBk
 YWJiZWx0LmNvbQAKCRAuExnzX7sYiVDkD/9wlynmhL043PosQcftqh3fdmvNNHs3
 /kiY40L2DryWWXMzhZn6/iMykhjZO9A8PKXUssoJ+Y4dSQiENdYZHH3K9EOtLMxA
 JNAGxiqdsGk+vACz5GKFobamQkIKN03qm5Y7GCf6NdLbRF8Lu0UMGwM2IKU740Sv
 JQOl2prbtLmkJqS33pxIEj7sHTDKh7fMnwrhDonCF/oNOMZ2ZhFAC9cQSF7WwkKT
 mbtuLrMHYFZAMtv8fGDyiSrHRjhfA5/bFJoqOf/x6mr174Zls7uK07V3SSRiKAmI
 rSCVIkAFa6+tkEcOsPTv5cYNTyG4Eb63+aOTeF0oiA47BGUZta6uyt3DpV0Mq3pr
 SjUw8c/ey1VkXIAx+976wdN5x5/XXx7IvyJppePT6JhKPusP9WVTrJJmcwjJ3JKG
 yQXo47csHgEOqFfpZEAYnEsfl49x5Da3hUWTiG0TTRa0yHsffPsC5RpUyGAf1IsQ
 49R65ZueroabALRuahuuAfwPmIic72fzl329ERaRBS/wDuxe2xnSKGNF6me3wI9g
 ekKwQl1cSitDbn+3rsanej2+28e4FGDN4CEdOPAEumQEYRAK2V9mT7CqsL/3lE3w
 2DTrx7k+ePbpnIMHJFQNv8/FtwezChzgjrNT5NHoUJ3qCZH+AjBZ3rX8BZV0jmRb
 t02h8MWdHsqzpA==
 =8uah
 -----END PGP SIGNATURE-----

Merge patch series "bpf, riscv: use BPF prog pack allocator in BPF JIT"

Puranjay Mohan <puranjay12@gmail.com> says:

Here is some data to prove the V2 fixes the problem:

Without this series:
root@rv-selftester:~/src/kselftest/bpf# time ./test_tag
test_tag: OK (40945 tests)

real    7m47.562s
user    0m24.145s
sys     6m37.064s

With this series applied:
root@rv-selftester:~/src/selftest/bpf# time ./test_tag
test_tag: OK (40945 tests)

real    7m29.472s
user    0m25.865s
sys     6m18.401s

BPF programs currently consume a page each on RISCV. For systems with many BPF
programs, this adds significant pressure to instruction TLB. High iTLB pressure
usually causes slow down for the whole system.

Song Liu introduced the BPF prog pack allocator[1] to mitigate the above issue.
It packs multiple BPF programs into a single huge page. It is currently only
enabled for the x86_64 BPF JIT.

I enabled this allocator on the ARM64 BPF JIT[2]. It is being reviewed now.

This patch series enables the BPF prog pack allocator for the RISCV BPF JIT.

======================================================
Performance Analysis of prog pack allocator on RISCV64
======================================================

Test setup:
===========

Host machine: Debian GNU/Linux 11 (bullseye)
Qemu Version: QEMU emulator version 8.0.3 (Debian 1:8.0.3+dfsg-1)
u-boot-qemu Version: 2023.07+dfsg-1
opensbi Version: 1.3-1

To test the performance of the BPF prog pack allocator on RV, a stresser
tool[4] linked below was built. This tool loads 8 BPF programs on the system and
triggers 5 of them in an infinite loop by doing system calls.

The runner script starts 20 instances of the above which loads 8*20=160 BPF
programs on the system, 5*20=100 of which are being constantly triggered.
The script is passed a command which would be run in the above environment.

The script was run with following perf command:
./run.sh "perf stat -a \
        -e iTLB-load-misses \
        -e dTLB-load-misses  \
        -e dTLB-store-misses \
        -e instructions \
        --timeout 60000"

The output of the above command is discussed below before and after enabling the
BPF prog pack allocator.

The tests were run on qemu-system-riscv64 with 8 cpus, 16G memory. The rootfs
was created using Bjorn's riscv-cross-builder[5] docker container linked below.

Results
=======

Before enabling prog pack allocator:
------------------------------------

Performance counter stats for 'system wide':

           4939048      iTLB-load-misses
           5468689      dTLB-load-misses
            465234      dTLB-store-misses
     1441082097998      instructions

      60.045791200 seconds time elapsed

After enabling prog pack allocator:
-----------------------------------

Performance counter stats for 'system wide':

           3430035      iTLB-load-misses
           5008745      dTLB-load-misses
            409944      dTLB-store-misses
     1441535637988      instructions

      60.046296600 seconds time elapsed

Improvements in metrics
=======================

It was expected that the iTLB-load-misses would decrease as now a single huge
page is used to keep all the BPF programs compared to a single page for each
program earlier.

--------------------------------------------
The improvement in iTLB-load-misses: -30.5 %
--------------------------------------------

I repeated this expriment more than 100 times in different setups and the
improvement was always greater than 30%.

This patch series is boot tested on the Starfive VisionFive 2 board[6].
The performance analysis was not done on the board because it doesn't
expose iTLB-load-misses, etc. The stresser program was run on the board to test
the loading and unloading of BPF programs

[1] https://lore.kernel.org/bpf/20220204185742.271030-1-song@kernel.org/
[2] https://lore.kernel.org/all/20230626085811.3192402-1-puranjay12@gmail.com/
[3] https://lore.kernel.org/all/20230626085811.3192402-2-puranjay12@gmail.com/
[4] https://github.com/puranjaymohan/BPF-Allocator-Bench
[5] https://github.com/bjoto/riscv-cross-builder
[6] https://www.starfivetech.com/en/site/boards

* b4-shazam-merge:
  bpf, riscv: use prog pack allocator in the BPF JIT
  riscv: implement a memset like function for text
  riscv: extend patch_text_nosync() for multiple pages
  bpf: make bpf_prog_pack allocator portable

Link: https://lore.kernel.org/r/20230831131229.497941-1-puranjay12@gmail.com
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>