| _______________________ |
| BUILDING THE FSQA SUITE |
| _______________________ |
| |
| - cd into the xfstests directory |
| - install prerequisite packages |
| For example, for Ubuntu: |
| sudo apt-get install xfslibs-dev uuid-dev libtool-bin \ |
| e2fsprogs automake gcc libuuid1 quota attr make \ |
| libacl1-dev libaio-dev xfsprogs libgdbm-dev gawk fio dbench \ |
| uuid-runtime python sqlite3 liburing-dev libcap-dev |
| For Fedora, RHEL, or CentOS: |
| yum install acl attr automake bc dbench dump e2fsprogs fio \ |
| gawk gcc indent libtool lvm2 make psmisc quota sed \ |
| xfsdump xfsprogs \ |
| libacl-devel libaio-devel libuuid-devel \ |
| xfsprogs-devel btrfs-progs-devel python sqlite liburing-devel \ |
| libcap-devel |
| (Older distributions may require xfsprogs-qa-devel as well.) |
| (Note that for RHEL and CentOS, you may need the EPEL repo.) |
| - run make |
| - run make install |
| - create fsgqa test user ("sudo useradd -m fsgqa") |
| - create fsgqa group ("sudo groupadd fsgqa") |
| - create 123456-fsgqa test user ("sudo useradd 123456-fsgqa") |
| this 2nd user creation step can be safely skipped if your system |
| doesn't support names starting with digits, only a handful of tests |
| require it. |
| - create fsgqa2 test user ("sudo useradd fsgqa2") |
| |
| ______________________ |
| USING THE FSQA SUITE |
| ______________________ |
| |
| Preparing system for tests: |
| |
| - compile XFS into your kernel or load XFS modules |
| - install administrative tools specific to the filesystem you wish to test |
| - If you wish to run the udf components of the suite install |
| mkudffs. Also download and build the Philips UDF Verification Software |
| from https://www.lscdweb.com/registered/udf_verifier.html, then copy the udf_test |
| binary to xfstests/src/. If you wish to disable UDF verification test |
| set the environment variable DISABLE_UDF_TEST to 1. |
| |
| - create one or two partitions to use for testing |
| - one TEST partition |
| - format as XFS, mount & optionally populate with |
| NON-IMPORTANT stuff |
| - one SCRATCH partition (optional) |
| - leave empty and expect this partition to be clobbered |
| by some tests. If this is not provided, many tests will |
| not be run. |
| (SCRATCH and TEST must be two DIFFERENT partitions) |
| OR |
| - for btrfs only: some btrfs test cases will need 3 or more independent |
| SCRATCH disks which should be set using SCRATCH_DEV_POOL (for eg: |
| SCRATCH_DEV_POOL="/dev/sda /dev/sdb /dev/sdc") with which |
| SCRATCH_DEV should be unused by the tester, and for the legacy |
| support SCRATCH_DEV will be set to the first disk of the |
| SCRATCH_DEV_POOL by xfstests script. |
| |
| - setup your environment |
| Quick start: |
| - copy local.config.example to local.config and edit as needed |
| Or: |
| - setenv TEST_DEV "device containing TEST PARTITION" |
| - setenv TEST_DIR "mount point of TEST PARTITION" |
| - optionally: |
| - setenv SCRATCH_DEV "device containing SCRATCH PARTITION" OR |
| (btrfs only) setenv SCRATCH_DEV_POOL "to 3 or more SCRATCH disks for |
| testing btrfs raid concepts" |
| - setenv SCRATCH_MNT "mount point for SCRATCH PARTITION" |
| - setenv TAPE_DEV "tape device for testing xfsdump" |
| - setenv RMT_TAPE_DEV "remote tape device for testing xfsdump" |
| - setenv RMT_IRIXTAPE_DEV "remote IRIX tape device for testing xfsdump" |
| - setenv SCRATCH_LOGDEV "device for scratch-fs external log" |
| - setenv SCRATCH_RTDEV "device for scratch-fs realtime data" |
| - setenv TEST_LOGDEV "device for test-fs external log" |
| - setenv TEST_RTDEV "device for test-fs realtime data" |
| - if TEST_LOGDEV and/or TEST_RTDEV, these will always be used. |
| - set IDMAPPED_MOUNTS=true to run all tests on top of idmapped |
| mounts. While this option is supported for all filesystems |
| currently only -overlay is expected to run without issues. For |
| other filesystems additional patches and fixes to the test suite |
| might be needed. |
| - if SCRATCH_LOGDEV and/or SCRATCH_RTDEV, the USE_EXTERNAL |
| environment variable set to "yes" will enable their use. |
| - setenv DIFF_LENGTH "number of diff lines to print from a failed test", |
| by default 10, set to 0 to print the full diff |
| - setenv FSTYP "the filesystem you want to test", the filesystem |
| type is devised from the TEST_DEV device, but you may want to |
| override it; if unset, the default is 'xfs' |
| - setenv FSSTRESS_AVOID and/or FSX_AVOID, which contain options |
| added to the end of fsstresss and fsx invocations, respectively, |
| in case you wish to exclude certain operational modes from these |
| tests. |
| - set TEST_XFS_REPAIR_REBUILD=1 to have _check_xfs_filesystem |
| run xfs_repair -n to check the filesystem; xfs_repair to rebuild |
| metadata indexes; and xfs_repair -n (a third time) to check the |
| results of the rebuilding. |
| - xfs_scrub, if present, will always check the test and scratch |
| filesystems if they are still online at the end of the test. |
| It is no longer necessary to set TEST_XFS_SCRUB. |
| - setenv LOGWRITES_DEV to a block device to use for power fail |
| testing. |
| - setenv PERF_CONFIGNAME to a arbitrary string to be used for |
| identifying the test setup for running perf tests. This should |
| be different for each type of performance test you wish to run so |
| that relevant results are compared. For example 'spinningrust' |
| for configurations that use spinning disks and 'nvme' for tests |
| using nvme drives. |
| - set USE_KMEMLEAK=yes to scan for memory leaks in the kernel |
| after every test, if the kernel supports kmemleak. |
| - set KEEP_DMESG=yes to keep dmesg log after test |
| - Set TEST_FS_MODULE_RELOAD=1 to unload the module and reload |
| it between test invocations. This assumes that the name of |
| the module is the same as FSTYP. |
| - Set DUMP_CORRUPT_FS=1 to record metadata dumps of XFS, ext* or |
| btrfs filesystems if a filesystem check fails. |
| - Set DUMP_COMPRESSOR to a compression program to compress |
| metadumps of filesystems. This program must accept '-f' and the |
| name of a file to compress; and it must accept '-d -f -k' and |
| the name of a file to decompress. In other words, it must |
| emulate gzip. |
| - Set MIN_FSSIZE to specify the minimal size (bytes) of a |
| filesystem we can create. Setting this parameter will |
| skip the tests creating a filesystem less than |
| MIN_FSSIZE. |
| - Set MODPROBE_PATIENT_RM_TIMEOUT_SECONDS to specify the amount of |
| time we should try a patient module remove. The default is 50 |
| seconds. Set this to "forever" and we'll wait forever until the |
| module is gone. |
| - Set FORCE_XFS_CHECK_PROG=yes to have _check_xfs_filesystem run |
| xfs_check to check the filesystem. As of August 2021, |
| xfs_repair finds all filesystem corruptions found by xfs_check, |
| and more, which means that xfs_check is no longer run by default. |
| - Set KCONFIG_PATH to specify your preferred location of kernel |
| config file. |
| |
| - or add a case to the switch in common/config assigning |
| these variables based on the hostname of your test |
| machine |
| - or add these variables to a file called local.config and keep that |
| file in your workarea. |
| |
| - if testing xfsdump, make sure the tape devices have a |
| tape which can be overwritten. |
| |
| - make sure $TEST_DEV is a mounted XFS partition |
| - make sure that $SCRATCH_DEV or $SCRATCH_DEV_POOL contains nothing useful |
| |
| Running tests: |
| |
| - cd xfstests |
| - By default the tests suite will run all the tests in the auto group. These |
| are the tests that are expected to function correctly as regression tests, |
| and it excludes tests that exercise conditions known to cause machine |
| failures (i.e. the "dangerous" tests). |
| - ./check '*/001' '*/002' '*/003' |
| - ./check '*/06?' |
| - Groups of tests maybe ran by: ./check -g [group(s)] |
| See the tests/*/group.list files after building xfstests to learn about |
| each test's group memberships. |
| - If you want to run all tests regardless of what group they are in |
| (including dangerous tests), use the "all" group: ./check -g all |
| - To randomize test order: ./check -r [test(s)] |
| - You can explicitly specify NFS/CIFS/OVERLAY, otherwise |
| the filesystem type will be autodetected from $TEST_DEV: |
| - for running nfs tests: ./check -nfs [test(s)] |
| - for running cifs/smb3 tests: ./check -cifs [test(s)] |
| - for overlay tests: ./check -overlay [test(s)] |
| The TEST and SCRATCH partitions should be pre-formatted |
| with another base fs, where the overlay dirs will be created |
| |
| |
| The check script tests the return value of each script, and |
| compares the output against the expected output. If the output |
| is not as expected, a diff will be output and an .out.bad file |
| will be produced for the failing test. |
| |
| Unexpected console messages, crashes and hangs may be considered |
| to be failures but are not necessarily detected by the QA system. |
| |
| __________________________ |
| ADDING TO THE FSQA SUITE |
| __________________________ |
| |
| |
| Creating new tests scripts: |
| |
| Use the "new" script. |
| |
| Test script environment: |
| |
| When developing a new test script keep the following things in |
| mind. All of the environment variables and shell procedures are |
| available to the script once the "common/preamble" file has been |
| sourced and the "_begin_fstest" function has been called. |
| |
| 1. The tests are run from an arbitrary directory. If you want to |
| do operations on an XFS filesystem (good idea, eh?), then do |
| one of the following: |
| |
| (a) Create directories and files at will in the directory |
| $TEST_DIR ... this is within an XFS filesystem and world |
| writeable. You should cleanup when your test is done, |
| e.g. use a _cleanup shell procedure in the trap ... see |
| 001 for an example. If you need to know, the $TEST_DIR |
| directory is within the filesystem on the block device |
| $TEST_DEV. |
| |
| (b) mkfs a new XFS filesystem on $SCRATCH_DEV, and mount this |
| on $SCRATCH_MNT. Call the the _require_scratch function |
| on startup if you require use of the scratch partition. |
| _require_scratch does some checks on $SCRATCH_DEV & |
| $SCRATCH_MNT and makes sure they're unmounted. You should |
| cleanup when your test is done, and in particular unmount |
| $SCRATCH_MNT. |
| Tests can make use of $SCRATCH_LOGDEV and $SCRATCH_RTDEV |
| for testing external log and realtime volumes - however, |
| these tests need to simply "pass" (e.g. cat $seq.out; exit |
| - or default to an internal log) in the common case where |
| these variables are not set. |
| |
| 2. You can safely create temporary files that are not part of the |
| filesystem tests (e.g. to catch output, prepare lists of things |
| to do, etc.) in files named $tmp.<anything>. The standard test |
| script framework created by "new" will initialize $tmp and |
| cleanup on exit. |
| |
| 3. By default, tests are run as the same uid as the person |
| executing the control script "check" that runs the test scripts. |
| |
| 4. Some other useful shell procedures: |
| |
| _get_fqdn - echo the host's fully qualified |
| domain name |
| |
| _get_pids_by_name - one argument is a process name, and |
| return all of the matching pids on |
| standard output |
| |
| _within_tolerance - fancy numerical "close enough is good |
| enough" filter for deterministic |
| output ... see comments in |
| common/filter for an explanation |
| |
| _filter_date - turn ctime(3) format dates into the |
| string DATE for deterministic |
| output |
| |
| _cat_passwd, - dump the content of the password |
| _cat_group or group file (both the local file |
| and the content of the NIS database |
| if it is likely to be present) |
| |
| 5. General recommendations, usage conventions, etc.: |
| - When the content of the password or group file is |
| required, get it using the _cat_passwd and _cat_group |
| functions, to ensure NIS information is included if NIS |
| is active. |
| - When calling getfacl in a test, pass the "-n" argument so |
| that numeric rather than symbolic identifiers are used in |
| the output. |
| - When creating a new test, it is possible to enter a custom name |
| for the file. Filenames are in form NNN-custom-name, where NNN |
| is automatically added by the ./new script as an unique ID, |
| and "custom-name" is the optional string entered into a prompt |
| in the ./new script. It can contain only alphanumeric characters |
| and dash. Note the "NNN-" part is added automatically. |
| |
| 6. Test group membership: Each test can be associated with any number |
| of groups for convenient selection of subsets of tests. Group names |
| can be any sequence of non-whitespace characters. Test authors |
| associate a test with groups by passing the names of those groups as |
| arguments to the _begin_fstest function. For example, the code: |
| |
| _begin_fstest auto quick subvol snapshot |
| |
| associates the current test with the "auto", "quick", "subvol", and |
| "snapshot" groups. It is not necessary to specify the "all" group |
| in the list because that group is computed at run time. |
| |
| The build process scans test files for _begin_fstest invocations and |
| compiles the group list from that information. In other words, test |
| files must call _begin_fstest or they will not be run. |
| |
| Verified output: |
| |
| Each test script has a name, e.g. 007, and an associated |
| verified output, e.g. 007.out. |
| |
| It is important that the verified output is deterministic, and |
| part of the job of the test script is to filter the output to |
| make this so. Examples of the sort of things that need filtering: |
| |
| - dates |
| - pids |
| - hostnames |
| - filesystem names |
| - timezones |
| - variable directory contents |
| - imprecise numbers, especially sizes and times |
| |
| Pass/failure: |
| |
| The script "check" may be used to run one or more tests. |
| |
| Test number $seq is deemed to "pass" when: |
| (a) no "core" file is created, |
| (b) the file $seq.notrun is not created, |
| (c) the exit status is 0, and |
| (d) the output matches the verified output. |
| |
| In the "not run" case (b), the $seq.notrun file should contain a |
| short one-line summary of why the test was not run. The standard |
| output is not checked, so this can be used for a more verbose |
| explanation and to provide feedback when the QA test is run |
| interactively. |
| |
| |
| To force a non-zero exit status use: |
| status=1 |
| exit |
| |
| Note that: |
| exit 1 |
| won't have the desired effect because of the way the exit trap |
| works. |
| |
| The recent pass/fail history is maintained in the file "check.log". |
| The elapsed time for the most recent pass for each test is kept |
| in "check.time". |
| |
| The compare-failures script in tools/ may be used to compare failures |
| across multiple runs, given files containing stdout from those runs. |
| |
| __________________ |
| SUBMITTING PATCHES |
| __________________ |
| |
| Send patches to the fstests mailing list at fstests@vger.kernel.org. |