Helpful tools for debugging AMD Zen systems

This repository hosts open tools that are useful for debugging issues on AMD systems.

Installation

It is suggested to install tools in a virtual environment either using pipx or python3 -m venv.

From PyPI

amd-debug-tools is distributed as a python wheel, which is a binary package format for Python. To install from PyPI, run the following command:

pipx install amd-debug-tools

From source

To build the package from source, you will need to the python3-build package natively installed by your distribution package manager. Then you can generate and install a wheel by running the following commands:

python3 -m build
pipx install dist/amd-debug-tools-*.whl

Ensuring path

If you have not used a pipx environment before, you may need to run the following command to set up the environment:

pipx ensurepath

This will add the pipx environment to your path.

Running in tree

If you want to run the tools in tree, you need to make sure that distro dependencies that would normally install into a venv are installed. This can be done by running:

./install_deps.py

After dependencies are installed, you can run the tools by running:

./amd_s2idle.py
./amd_bios.py
./amd_pstate.py

amd-s2idle

amd-s2idle is a tool used for analyzing the entry and exit of the s2idle state of a Linux system.

It is intended to use with Linux kernel 6.1 or later and works by hooking into dynamic debugging messages and events that are generated by the kernel.

For analysis of power consumption issues it can be hooked into systemd to run a command to capture data right before and after the system enters and exits the s2idle state.

4 high level commands are supported.

amd-s2idle install

This will install the systemd hook so that data will be captured before and after the system enters and exits the s2idle state.

This will also install a bash completion script that can be used for other commands.

NOTE: This command is only supported when run from a venv.

amd-s2idle uninstall

This will uninstall the systemd hook and remove the bash completion script.

NOTE: This command is only supported when run from a venv.

amd-s2idle test

This will run a suspend cycle with a timer based wakeup and capture relevant data into a database and produce a report. This can also be used to run multiple cycles.

The following optional arguments are supported for this command:

    --count COUNT         Number of cycles to run
    --duration DURATION   Duration of the cycle in seconds
    --wait WAIT           Time to wait before starting the cycle in seconds
    --format FORMAT       Format of the report to produce (html, txt or md)
    --report-file         File to write the report to
    --force               Run a test cycle even if the system fails to pass prerequisite checks
    --random              Run sleep cycles for random durations and waits, using the --duration and --wait arguments as an upper bound
    --logind              Use logind to suspend the system
    --tool-debug          Enable debug logging
    --bios-debug          Enable BIOS debug logging instead of notify logging

If the tool is launched with an environment that can call xdg-open, the report will be opened in a browser.

amd-s2idle report

This will produce a report from the data captured by the test command and/or from the systemd hook. The report will default to 60 days of data.

The following optional arguments are supported for this command:

    --since SINCE         Date to start the report from
    --until UNTIL         Date to end the report at
    --format FORMAT       Format of the report to produce (html, txt or md)
    --report-file         File to write the report to
    --tool-debug          Enable tool debug logging
    --report-debug
    --no-report-debug
                          Include debug messages in report (WARNING: can significantly increase report size)

If the tool is launched with an environment that can call xdg-open, the report will be opened in a browser.

amd-s2idle --version

This will print the version of the tool and exit.

Debug output

All commands support the --tool-debug argument which will enable extra debug output. This is often needed for debugging issues with a particular cycle.

NOTE: enabling debug output significantly increases the size of the report. It's suggested that you use --since and --until to focus on the cycles that you are interested in.

amd-bios

amd-bios is a a tool that can be used to enable or disable BIOS AML debug logging -and to parse a kernel log that contains BIOS logs.

amd-bios trace

Modify BIOS AML trace debug logging.

One of the following arguments must be set for this command:

    --enable       Enable BIOS AML tracing
    --disable      Disable BIOS AML tracing

The following optional arguments are supported for this command:

    --tool-debug   Enable tool debug logging

amd-bios parse

Parses a kernel log that contains BIOS AML debug logging and produces a report.

The following optional arguments are supported for this command:

    --input INPUT  Optional input file to parse
    --tool-debug   Enable tool debug logging

amd-bios --version

This will print the version of the tool and exit.

amd-pstate

amd-pstate is a tool used for identification of issues with amd-pstate. It will capture some state from the system as well as from the machine specific registers that amd-pstate uses.

Compatibility scripts

Compatibility scripts are provided for the previous names the tools went by: amd_s2idle.py, amd_bios.py and amd_pstate.py. These allow cloning the repository and running the scripts without installing the package.