| From hch@infradead.org Tue Nov 22 13:35:52 2011 |
| From: Dave Chinner <dchinner@redhat.com> |
| Date: Sat, 19 Nov 2011 13:13:40 -0500 |
| Subject: xfs: dont serialise direct IO reads on page cache |
| To: stable@vger.kernel.org |
| Cc: xfs@oss.sgi.com, Dave Chinner <dchinner@redhat.com>, Alex Elder <aelder@sgi.com> |
| Message-ID: <20111119181544.111984285@bombadil.infradead.org> |
| |
| From: Dave Chinner <dchinner@redhat.com> |
| |
| commit 0c38a2512df272b14ef4238b476a2e4f70da1479 upstream. |
| |
| There is no need to grab the i_mutex of the IO lock in exclusive |
| mode if we don't need to invalidate the page cache. Taking these |
| locks on every direct IO effective serialises them as taking the IO |
| lock in exclusive mode has to wait for all shared holders to drop |
| the lock. That only happens when IO is complete, so effective it |
| prevents dispatch of concurrent direct IO reads to the same inode. |
| |
| Fix this by taking the IO lock shared to check the page cache state, |
| and only then drop it and take the IO lock exclusively if there is |
| work to be done. Hence for the normal direct IO case, no exclusive |
| locking will occur. |
| |
| Signed-off-by: Dave Chinner <dchinner@redhat.com> |
| Tested-by: Joern Engel <joern@logfs.org> |
| Reviewed-by: Christoph Hellwig <hch@lst.de> |
| Signed-off-by: Alex Elder <aelder@sgi.com> |
| Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de> |
| |
| --- |
| fs/xfs/linux-2.6/xfs_file.c | 17 ++++++++++++++--- |
| 1 file changed, 14 insertions(+), 3 deletions(-) |
| |
| --- a/fs/xfs/linux-2.6/xfs_file.c |
| +++ b/fs/xfs/linux-2.6/xfs_file.c |
| @@ -309,7 +309,19 @@ xfs_file_aio_read( |
| if (XFS_FORCED_SHUTDOWN(mp)) |
| return -EIO; |
| |
| - if (unlikely(ioflags & IO_ISDIRECT)) { |
| + /* |
| + * Locking is a bit tricky here. If we take an exclusive lock |
| + * for direct IO, we effectively serialise all new concurrent |
| + * read IO to this file and block it behind IO that is currently in |
| + * progress because IO in progress holds the IO lock shared. We only |
| + * need to hold the lock exclusive to blow away the page cache, so |
| + * only take lock exclusively if the page cache needs invalidation. |
| + * This allows the normal direct IO case of no page cache pages to |
| + * proceeed concurrently without serialisation. |
| + */ |
| + xfs_rw_ilock(ip, XFS_IOLOCK_SHARED); |
| + if ((ioflags & IO_ISDIRECT) && inode->i_mapping->nrpages) { |
| + xfs_rw_iunlock(ip, XFS_IOLOCK_SHARED); |
| xfs_rw_ilock(ip, XFS_IOLOCK_EXCL); |
| |
| if (inode->i_mapping->nrpages) { |
| @@ -322,8 +334,7 @@ xfs_file_aio_read( |
| } |
| } |
| xfs_rw_ilock_demote(ip, XFS_IOLOCK_EXCL); |
| - } else |
| - xfs_rw_ilock(ip, XFS_IOLOCK_SHARED); |
| + } |
| |
| trace_xfs_file_read(ip, size, iocb->ki_pos, ioflags); |
| |