A testsuite covering functionality of btrfs-progs, i.e. the checker, image, mkfs and similar tools. There are no additional requirements on kernel features (other than CONFIG_BTRFS_FS
built-in or module), the tests build on top of the core functionality like snapshots and device management. In some cases optional features are turned on by mkfs and the filesystem image could be mounted, such tests might fail if there's lack of support.
Run the tests from the top directory:
$ make test $ make test-fsck $ make test-convert
or selectively from the tests/
directory:
$ ./fsck-tests.sh $ ./misc-tests.sh
The verbose output of the tests is logged into a file named after the test category, e.g. fsck-tests-results.txt
.
The tests are prefixed by a number for ordering and uniqueness. To run a particular test use:
$ make TEST=MASK test
where MASK
is a glob expression that will execute only tests that match the MASK. Here the test number comes handy:
$ make TEST=001\* test-fsck # in tests/ $ TEST=001\* ./fsck-tests.sh # in the top directory
will run the first test in fsck-tests subdirectory. If the test directories follow a good naming scheme, it‘s possible to select a subset e.g. like the convert tests for ext[234] filesystems using mask ‘TEST=’*ext[234]*’.
When running tests in a sequence and one fails, testing can be started from that test skipping all the ones that are supposed to pass, like:
$ make TEST_FROM=004\* test-fsck
This will skip any tests found in the directory that precede 004, assuming that there is such test.
tests/fsck-tests/
./test.sh
that will be run if presenttests/convert-tests/
tests/fuzz-tests/
tests/cli-tests/
./test.sh
in the test directorytests/misc-tests/
./test.sh
in the test directorytests/common, tests/common.convert
tests/test.img
TEST_DEV
variable, the file is never deleted by the scripts but truncated to 0 bytes, so it keeps it‘s permissions. It’s e.g. possible to host it on NFS, make it chmod a+w
for root.It's possible to wrap the tested commands to utilities that might do more checking or catch failures at runtime. This can be done by setting the INSTRUMENT
environment variable:
make INSTRUMENT=valgrind test-fuzz # in the top directory INSTRUMENT=valgrind ./fuzz-tests.sh # in tests/
The variable is prepended to the command unquoted, all sorts of shell tricks are possible.
Note: instrumentation is not applied to privileged commands (anything that uses the root helper), with exception of all commands built from git that will be instrumented even if run with the sudo helper.
run_check $SUDO_HELPER mount /dev/sdx /mnt # no instrumentation run_check $SUDO_HELPER "$TOP/btrfs" check /dev/sdx # with instrumentation
Instrumented commands: btrfs, btrfs-image, btrfs-convert, btrfs-tune, mkfs.btrfs, btrfs-select-super, btrfs-find-root, btrfs-corrupt-block.
As mentioned above, instrumentation tools are like valgrind
or potentially gdb
with some init script that will let the commands run until an exception occurs, possibly allowing to continue interactively debugging.
TEST_FROM=glob
-- start testing from the first test that matches the glob expression and skip anything before that, this can be combined with the TEST
variable that takes precedence
TEST_LOG=tty
-- setting the variable will print all commands executed by some of the wrappers (run_check
etc), other commands are not printed to the terminal (but the full output is in the log)
TEST_LOG=dump
-- dump the entire testing log when a test fails
TEST_FLAVOR
-- run binaries of specified flavor, which is dynamic build by default, or static using all the build binaries with .static suffix
TEST_ENABLE_OVERRIDE
-- defined either as make arguments or via tests/common.local
to enable additional arguments to some commands, using the variable(s) below (default: false, enable by setting to ‘true’)
TEST_ARGS_CHECK
-- user-defined arguments to btrfs check
, before the test-specific arguments
TEST_ARGS_MKFS
-- user-defined arguments to mkfs.btrfs
, before the test-specific arguments
TEST_ARGS_CONVERT
-- user-defined arguments to btrfs-convert
, before the test-specific arguments
Multiple values can be separated by ,
.
For example, running all fsck tests with the --mode=lowmem
option can be done as
$ make TEST_ENABLE_OVERRIDE=true TEST_ARGS_CHECK=--mode=lowmem test-check
Specifically, fsck-tests that are known to be able to repair images in the lowmem mode should be marked using a file .lowmem_repairable
in the test directory. Then the fsck-tests with the ‘mode=lowmem’ will continue when image repair is requested.
Some commands require root privileges (to mount/umount, access loop devices or call privileged ioctls). It is assumed that sudo
will work in some way (no password, password asked and cached). Note that instrumentation is not applied in this case, for safety reasons or because the tools refuse to run under root. You need to modify the test script instead.
The tests are supposed to cleanup after themselves if they pass. In case of failure, the rest of the tests are skipped and intermediate files, mounts and loop devices are kept. This should help to investigate the test failure but at least the mounts and loop devices need to be cleaned before the next run.
This is partially done by the script clean-tests.sh
, you may want to check the loop devices as they are managed on a per-test basis, see the output of command losetup
and eventually delete all existing loop devices with losetup -D
.
There's a script test-console.sh
that will run shell commands in a loop and logs the output with the testing environment set up. It sources the common helper scripts so the shell functions are available.
The tests use some common system utilities like find
, rm
, dd
. Additionally, specific tests need the following packages installed: acl
, attr
, e2fsprogs
, reiserfsprogs
.
Pick the category for the new test or fallback to misc-tests
if not sure. For an easy start copy an existing test.sh
script from some test that might be close to the purpose of your new test. The environment setup includes the common scripts and/or prepares the test devices. Other scripts contain examples how to do mkfs, mount, unmount, check, loop device management etc.
Use the highest unused number in the sequence, write a short descriptive title and join by dashes -
. This will become the directory name, e.g. 012-subvolume-sync-must-wait
.
Write a short description of the bug and how it‘s tested to the comment at the beginning of test.sh
. You don’t need to add the file to git yet. Don‘t forget to make the file executable, otherwise it’s not going to be executed by the infrastructure.
Write the test commands, comment anything that's not obvious.
Test your test. Use the TEST
variable to jump right to your test:
$ make TEST=012\* tests-misc # from top directory $ TEST=012\* ./misc-tests.sh # from tests/
The commit changelog should reference a commit that either introduced or fixed the bug (or both). Subject line of the shall mention the name of the new directory for ease of search, e.g. btrfs-progs: tests: add 012-subvolume-sync-must-wait
A commit that fixes a bug should be applied before the test that verifies the fix. This is to keep the git history bisectable.
Most tests should be able to create the test images from scratch, using regular commands and file operation. The commands also document the test case and use the test code and kernel of the environment.
In other cases, a pre-created image may be the right way if the above does not work (e.g. comparing output, requesting an exact layout or some intermediate state that would be hard to achieve otherwise).
images that don't need data and valid checksums can be created by btrfs-image
, the image can be compressed by the tool itself (file extension .img
) or compressed externally (recognized is .img.xz
)
raw images that are binary dump of an existing image, created e.g. from a sparse file (.raw
or .raw.xz
)
Use xz --best
and try to get the smallest size as the file is stored in git.
Images that are created by fuzzing or specially crafted to trigger some error conditions should be added to the directory fuzz-tests/images, accompanied by a textual description of the source (bugzilla, mail), the reporter, brief description of the problem or the stack trace.
If you have a fix for the problem, please submit it prior to the test image, so the fuzz tests always succeed when run on random checked out. This helps bisectability.
The tests are typically run from git on binaries built from the git sources. It is possible to extract only the testsuite files and run it independently. Use
$ make testsuite
This will gather scripts and generate tests/btrfs-progs-tests.tar.gz
. The files inside the tar are in the top level directory, make sure you extract the contents to an empty directory. From there you can start the tests as described above (the non-make variant).
By default the binaries found in $PATH
are used, this will normally mean the system binaries. You can also override the $TOP
shell variable and this path will be used as prefix for all btrfs binaries inside the tests.
There are some utilities that are not distributed but are necessary for the tests. They are in the top level directory of the testsuite and their path cannot be set.
The tests assume write access to their directories.
$SUDO_HELPER
as it might be intentionally unset#!/bin/bash
explicitlycheck_global_prereq
)check_prereq
), though the basic set is always built when the tests are started through makecommon
scriptxfs_io
that‘s conveniently used in fstests but we’d require xfsprogs
, so use dd
insteadThe file tests/common
provides shell functions to ease writing common things like setting up the test devices, making or mounting a filesystem, setting up loop devices.
#!/bin/bash # Simple test to create a new filesystem and test that it can be mounted source "$TEST_TOP/common" setup_root_helper prepare_test_dev run_check_mkfs_test_dev run_check_mount_test_dev run_check $SUDO_HELPER dd if=/dev/zero of="$TEST_MNT"/file bs=1M count=1 run_check_umount_test_dev
Each test should be briefly described, source the helpers like run_check
. The root helper is the sudo
wrapper that can be used as $SUDO_HELPER
variable. The implicit variables for testing device is $TEST_DEV
mounted at $TEST_MNT
. The mkfs and mount helpers take arguments that are then injected into the right place in the respective command.
Besides the setup and cleanup code, the main test in this example is dd
that writes 1MiB to a file in the newly created filesystem.
Tests that need more devices can utilize the loop devices, an example test of the above:
# Create a new multi-device filesystem and test that it can be mounted source "$TEST_TOP/common" setup_root_helper setup_loopdevs 4 prepare_loopdevs TEST_DEV=${loopdevs[1]} run_check $SUDO_HELPER "$TOP/mkfs.btrfs" -f -d raid1 -m raid1 "${loopdevs[@]}" run_check_mount_test_dev run_check $SUDO_HELPER dd if=/dev/zero of="$TEST_MNT"/file bs=1M count=1 run_check_umount_test_dev cleanup_loopdevs
The easiest way to test an image is to put it to the test directory, without the test.sh
script. All images found are simply processed by the shell function check_image
with default parameters.
Any tweaks to the ‘check’ subcommand can be done by redefining the function:
source "$TEST_TOP/common" check_image() { run_check "$TOP/btrfs" check --readonly "$1" } check_all_images
The images can be stored in various formats, see section ‘Test images’.
There are several helpers in tests/common
, it's recommended to read through that file or other tests to get the idea how easy writing a test really is.
_fail
- a failure condition has been found_not_run
- some prerequisite condition is not met, e.g. missing kernel functionality_log
- message printed to the result file (e.g. tests/mkfs-tests-results.txt
), and not printed to the terminal_log_stdout
- ditto but it is printed to the terminalrun_check
- should be used for basically all commands, the command and arguments are stored to the results log for debugging and the return value is checked so there are no silent failures even for the “unimportant” commandsrun_check_stdout
- like the above but the output can be processed further, e.g. filtering out some data or looking for some specific stringrun_mayfail
- the command is allowed to fail in a non-fatal way (e.g. no segfault), there's also the run_mayfail_stdout
variantrun_mustfail
- expected failure, note that the first argument is mandatory message describing unexpected pass condition