source/fs_mount_options.rst - pub/scm/docs/docsko/nvdimm - Git at Google

 ****************
 FS Mount options
 ****************

 Prior to kernel 5.8
 ===================

 Prior to kernel 5.8 enabling dax is done on an entire file system basis.
 It is done with a "-o dax" mount option.

 ::

    $ mount -o dax /dev/pmem0p1 /mnt/ext4-pmem0
    $ mount -o dax /dev/pmem0p2 /mnt/xfs-pmem0

 Kernels 5.8 or later
 ====================

 As of kernel 5.8 additional DAX enabling options are available on some
 file systems, specifically:

 ext2
 ----

 When mounting the filesystem, use the "-o dax" option on the command
 line or add 'dax' to the options in /etc/fstab. This works to enable DAX
 on all files within the filesystem. It is equivalent to the '-o
 dax=always' behavior below.

 ext4 and xfs
 ------------

 As of kernel 5.8 ext4 and xfs supports the new per-file dax
 configuration.

 Summary
 ~~~~~~~

 #. There exists an in-kernel file access mode flag S_DAX that
    corresponds to the statx flag STATX_ATTR_DAX. See the manpage for
    statx(2) for details about this access mode.
 #. There exists a persistent flag FS_XFLAG_DAX that can be applied to
    regular files and directories. This advisory flag can be set or
    cleared at any time, but doing so does not immediately affect the
    S_DAX state.
 #. If the persistent FS_XFLAG_DAX flag is set on a directory, this flag
    will be inherited by all regular files and subdirectories that are
    subsequently created in this directory. Files and subdirectories that
    exist at the time this flag is set or cleared on the parent directory
    are not modified by this modification of the parent directory.
 #. There exist dax mount options which can override FS_XFLAG_DAX in the
    setting of the S_DAX flag. Given underlying storage which supports
    DAX the following hold:

    -  "-o dax=inode" means "follow FS_XFLAG_DAX" and is the default.
    -  "-o dax=never" means "never set S_DAX, ignore FS_XFLAG_DAX."
    -  "-o dax=always" means "always set S_DAX ignore FS_XFLAG_DAX."
    -  "-o dax" is a legacy option which is an alias for "dax=always".

       -  *"-o dax" may be removed in the future so "-o dax=always" is
          the preferred method for specifying this behavior going
          forward.*

 #. The S_DAX policy can be changed via:

    #. Setting the parent directory FS_XFLAG_DAX as needed before files
       are created
    #. Setting the appropriate dax="foo" mount option
    #. Changing the FS_XFLAG_DAX flag on existing regular files and
       directories. This has runtime constraints and limitations that are
       described in 6) below.

 #. When changing the S_DAX policy via toggling the persistent
    FS_XFLAG_DAX flag, the change in behaviour for existing regular files
    may not occur immediately. If the change must take effect
    immediately, the administrator needs to:

    #. stop the application so there are no active references to the data
       set the policy change will affect
    #. evict the data set from kernel caches so it will be
       re-instantiated when the application is restarted. This can be
       achieved by:

 ::

         - drop-caches
         - a filesystem unmount and mount cycle
         - a system reboot

 ::

      NOTE: Modifications to and the inheritance behavior of FS_XFLAG_DAX remain
      the same even when the filesystem is mounted with a dax option.  However,
      in-core inode state (S_DAX) will be overridden until the filesystem is
      remounted with dax=inode and the inode is evicted from kernel memory.

 Examples and details
 ~~~~~~~~~~~~~~~~~~~~

 There are 2 per-file dax flags. One is a persistent inode setting
 (FS_XFLAG_DAX) and the other is a volatile flag indicating the active
 state of the feature (S_DAX).

 FS_XFLAG_DAX is preserved within the file system. This persistent config
 setting can be set, cleared and/or queried using the
 FS_IOC_FS[GS]ETXATTR ioctl (see ioctl_xfs_fsgetxattr(2)) or an utility
 such as 'xfs_io'.

 New files and directories automatically inherit FS_XFLAG_DAX from their
 parent directory \_when\_ \_created\_. Therefore, setting FS_XFLAG_DAX
 at directory creation time can be used to set a default behavior for an
 entire sub-tree.

 To clarify inheritance mount the file system with the inode option.

 ::

    $ mount /dev/pmem0p2 /mnt/xfs-pmem0

 Is equivalent to:

 ::

    $ mount -o dax=inode /dev/pmem0p2 /mnt/xfs-pmem0

 And here are 3 examples showing how to enable dax on individual files
 and/or directories.

 Inheritance Example A:
 ^^^^^^^^^^^^^^^^^^^^^^

 ::

    $ mkdir -p a/b/c
    $ xfs_io -c 'chattr +x' a
    $ xfs_io -c 'lsattr' a
    --------------x- a
    $ mkdir -p a/b/c/d
    $ mkdir -p a/e

 ::

          Results in:

          dax: a,e
          no dax: b,c,d

 Inheritance Example B:
 ^^^^^^^^^^^^^^^^^^^^^^

 ::

    $ mkdir a
    $ xfs_io -c 'chattr +x' a
    $ xfs_io -c 'lsattr' a
    --------------x- a
    $ mkdir -p a/b/c/d

 ::

          Results in:

          dax: a,b,c,d
          no dax:

 Inheritance Example C:
 ^^^^^^^^^^^^^^^^^^^^^^

 ::

    $ mkdir -p a/b/c
    $ xfs_io -c 'chattr +x' c
    $ xfs_io -c 'lsattr' c
    --------------x- c
    $ mkdir -p a/b/c/d

 ::

          Results in:

          dax: c,d
          no dax: a,b

 Detecting if a file is using dax
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

 The current enabled state (S_DAX) is set when a file inode is
 instantiated in memory by the kernel. It is set based on the underlying
 media support, the value of FS_XFLAG_DAX and the file system's dax mount
 option.

 statx can be used to query S_DAX. NOTE that only regular files will ever
 have S_DAX set and therefore statx will never indicate that S_DAX is set
 on directories.

 Continuing with Example C above we create a file foo in a dax enabled
 directory and it is enabled for dax with the FS_XFLAG_DAX set as well as
 S_DAX being set.

 ::

    $ touch a/b/c/foo
    $ xfs_io -c 'lsattr' a/b/c/foo                       # FS_XFLAG_DAX == true
    --------------x- a/b/c/foo
    $ xfs_io -c 'statx -r' a/b/c/foo | grep attributes   # S_DAX == true
    stat.attributes = 0x2000

 Setting the FS_XFLAG_DAX flag (specifically or through inheritance)
 occurs even if the underlying media does not support dax and/or the
 filesystem is overridden with a mount option.
	****************
	FS Mount options
	****************

	Prior to kernel 5.8
	===================

	Prior to kernel 5.8 enabling dax is done on an entire file system basis.
	It is done with a "-o dax" mount option.

	::

	$ mount -o dax /dev/pmem0p1 /mnt/ext4-pmem0
	$ mount -o dax /dev/pmem0p2 /mnt/xfs-pmem0

	Kernels 5.8 or later
	====================

	As of kernel 5.8 additional DAX enabling options are available on some
	file systems, specifically:

	ext2
	----

	When mounting the filesystem, use the "-o dax" option on the command
	line or add 'dax' to the options in /etc/fstab. This works to enable DAX
	on all files within the filesystem. It is equivalent to the '-o
	dax=always' behavior below.

	ext4 and xfs
	------------

	As of kernel 5.8 ext4 and xfs supports the new per-file dax
	configuration.

	Summary
	~~~~~~~

	#. There exists an in-kernel file access mode flag S_DAX that
	corresponds to the statx flag STATX_ATTR_DAX. See the manpage for
	statx(2) for details about this access mode.
	#. There exists a persistent flag FS_XFLAG_DAX that can be applied to
	regular files and directories. This advisory flag can be set or
	cleared at any time, but doing so does not immediately affect the
	S_DAX state.
	#. If the persistent FS_XFLAG_DAX flag is set on a directory, this flag
	will be inherited by all regular files and subdirectories that are
	subsequently created in this directory. Files and subdirectories that
	exist at the time this flag is set or cleared on the parent directory
	are not modified by this modification of the parent directory.
	#. There exist dax mount options which can override FS_XFLAG_DAX in the
	setting of the S_DAX flag. Given underlying storage which supports
	DAX the following hold:

	- "-o dax=inode" means "follow FS_XFLAG_DAX" and is the default.
	- "-o dax=never" means "never set S_DAX, ignore FS_XFLAG_DAX."
	- "-o dax=always" means "always set S_DAX ignore FS_XFLAG_DAX."
	- "-o dax" is a legacy option which is an alias for "dax=always".

	- *"-o dax" may be removed in the future so "-o dax=always" is
	the preferred method for specifying this behavior going
	forward.*

	#. The S_DAX policy can be changed via:

	#. Setting the parent directory FS_XFLAG_DAX as needed before files
	are created
	#. Setting the appropriate dax="foo" mount option
	#. Changing the FS_XFLAG_DAX flag on existing regular files and
	directories. This has runtime constraints and limitations that are
	described in 6) below.

	#. When changing the S_DAX policy via toggling the persistent
	FS_XFLAG_DAX flag, the change in behaviour for existing regular files
	may not occur immediately. If the change must take effect
	immediately, the administrator needs to:

	#. stop the application so there are no active references to the data
	set the policy change will affect
	#. evict the data set from kernel caches so it will be
	re-instantiated when the application is restarted. This can be
	achieved by:

	::

	- drop-caches
	- a filesystem unmount and mount cycle
	- a system reboot

	::

	NOTE: Modifications to and the inheritance behavior of FS_XFLAG_DAX remain
	the same even when the filesystem is mounted with a dax option. However,
	in-core inode state (S_DAX) will be overridden until the filesystem is
	remounted with dax=inode and the inode is evicted from kernel memory.

	Examples and details
	~~~~~~~~~~~~~~~~~~~~

	There are 2 per-file dax flags. One is a persistent inode setting
	(FS_XFLAG_DAX) and the other is a volatile flag indicating the active
	state of the feature (S_DAX).

	FS_XFLAG_DAX is preserved within the file system. This persistent config
	setting can be set, cleared and/or queried using the
	FS_IOC_FS[GS]ETXATTR ioctl (see ioctl_xfs_fsgetxattr(2)) or an utility
	such as 'xfs_io'.

	New files and directories automatically inherit FS_XFLAG_DAX from their
	parent directory \_when\_ \_created\_. Therefore, setting FS_XFLAG_DAX
	at directory creation time can be used to set a default behavior for an
	entire sub-tree.

	To clarify inheritance mount the file system with the inode option.

	::

	$ mount /dev/pmem0p2 /mnt/xfs-pmem0

	Is equivalent to:

	::

	$ mount -o dax=inode /dev/pmem0p2 /mnt/xfs-pmem0

	And here are 3 examples showing how to enable dax on individual files
	and/or directories.

	Inheritance Example A:
	^^^^^^^^^^^^^^^^^^^^^^

	::

	$ mkdir -p a/b/c
	$ xfs_io -c 'chattr +x' a
	$ xfs_io -c 'lsattr' a
	--------------x- a
	$ mkdir -p a/b/c/d
	$ mkdir -p a/e

	::

	Results in:

	dax: a,e
	no dax: b,c,d

	Inheritance Example B:
	^^^^^^^^^^^^^^^^^^^^^^

	::

	$ mkdir a
	$ xfs_io -c 'chattr +x' a
	$ xfs_io -c 'lsattr' a
	--------------x- a
	$ mkdir -p a/b/c/d

	::

	Results in:

	dax: a,b,c,d
	no dax:

	Inheritance Example C:
	^^^^^^^^^^^^^^^^^^^^^^

	::

	$ mkdir -p a/b/c
	$ xfs_io -c 'chattr +x' c
	$ xfs_io -c 'lsattr' c
	--------------x- c
	$ mkdir -p a/b/c/d

	::

	Results in:

	dax: c,d
	no dax: a,b

	Detecting if a file is using dax
	^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

	The current enabled state (S_DAX) is set when a file inode is
	instantiated in memory by the kernel. It is set based on the underlying
	media support, the value of FS_XFLAG_DAX and the file system's dax mount
	option.

	statx can be used to query S_DAX. NOTE that only regular files will ever
	have S_DAX set and therefore statx will never indicate that S_DAX is set
	on directories.

	Continuing with Example C above we create a file foo in a dax enabled
	directory and it is enabled for dax with the FS_XFLAG_DAX set as well as
	S_DAX being set.

	::

	$ touch a/b/c/foo
	$ xfs_io -c 'lsattr' a/b/c/foo # FS_XFLAG_DAX == true
	--------------x- a/b/c/foo
	$ xfs_io -c 'statx -r' a/b/c/foo \| grep attributes # S_DAX == true
	stat.attributes = 0x2000

	Setting the FS_XFLAG_DAX flag (specifically or through inheritance)
	occurs even if the underlying media does not support dax and/or the
	filesystem is overridden with a mount option.