Asynchronous UVC

The Linux UVC driver has long provided adequate performance capabilities for
web-cams and low data rate video devices in Linux while resolutions were low.

Modern USB cameras are now capable of high data rates thanks to USB3 with
1080p, and even 4k capture resolutions supported.

Cameras such as the Stereolabs ZED (bulk transfers) or the Logitech BRIO
(isochronous transfers) can generate more data than an embedded ARM core is
able to process on a single core, resulting in frame loss.

A large part of this performance impact is from the requirement to
‘memcpy’ frames out from URB packets to destination frames. This unfortunate
requirement is due to the UVC protocol allowing a variable length header, and
thus it is not possible to provide the target frame buffers directly.

Extra throughput is possible by moving the actual memcpy actions to a work
queue, and moving the memcpy out of interrupt context thus allowing work tasks
to be scheduled across multiple cores.

This series has been tested on both the ZED and BRIO cameras on arm64
platforms, and with thanks to Randy Dunlap, a Dynex 1.3MP Webcam, a Sonix USB2
Camera, and a built in Toshiba Laptop camera, and with thanks to Philipp Zabel
for testing on a Lite-On internal Laptop Webcam, Logitech C910 (USB2 isoc),
Oculus Sensor (USB3 isoc), and Microsoft HoloLens Sensors (USB3 bulk).

As far as I am aware iSight devices, and devices which use UVC to encode data
(output device) have not yet been tested - but should find no ill effect (at
least not until they are tested of course :D )

Tested-by: Randy Dunlap <rdunlap@infradead.org>
Tested-by: Philipp Zabel <philipp.zabel@gmail.com>

v2:
 - Fix race reported by Guennadi

v3:
 - Fix similar race reported by Laurent
 - Only queue work if required (encode/isight do not queue work)
 - Refactor/Rename variables for clarity

v4:
 - (Yet another) Rework of the uninitialise path.
   This time to hopefully clean up the shutdown races for good.
   use usb_poison_urb() to halt all URBs, then flush the work queue
   before freeing.
 - Rebase to latest linux-media/master

v5:
 - Provide lockdep validation
 - rename uvc_queue_requeue -> uvc_queue_buffer_requeue()
 - Fix comments and periods throughout
 - Rebase to media/v4.20-2
 - Use GFP_KERNEL allocation in uvc_video_copy_data_work()
 - Fix function documentation for uvc_video_copy_data_work()
 - Add periods to the end of sentences
 - Rename 'decode' variable to 'op' in uvc_video_decode_data()
 - Move uvc_urb->async_operations initialisation to before use
 - Move async workqueue to match uvc_streaming lifetime instead of
   streamon/streamoff
 - bracket the for_each_uvc_urb() macro

 - New patches added to series:
    media: uvcvideo: Split uvc_video_enable into two
    media: uvcvideo: Rename uvc_{un,}init_video()
    media: uvcvideo: Utilise for_each_uvc_urb iterator

v6:
 - New patch added to series
    media: uvcvideo: Abstract streaming object lifetime

 - Utilise the new streaming object lifetime functions to perform
   allocation and destruction of the async workqueue.
 - Append _transfer to {_stop,_start} in uvc_video_{stop,start}_transfer
 - rename lone 'j' iterator to 'i'
 - Remove conversion which doesn't make sense due to needing the
   iterator value.
-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEkC3XmD+9KP3jctR6oR5GchCkYf0FAlvlvNsACgkQoR5GchCk
Yf0giA/9Ef7KVIw2xlHZ89bWdRRLD6odd8zW9UMdI9gRobtvW+NgPfuc/hcIjm2Q
uX1jT5fhZBogRH+RZ7yQ4IPcEyyZctKPA9uIbnawd4gU0ydUG/RKXZVduIfUHFZC
glvhO5bC3wcs70WXS/2trnKqQTnS+kpyRE3FASU9a8ZnnZii2QVZCiQxwp6uXxWR
018M85USBit7d6jq8nA5MBP7o8Q8ZicOSbS/i/fvRbCa4TDUCAebnBbUvua2+Isw
9L/ZIZfwQwYpGBKX18yltIEB9jWr1rEcjMbCQGNI4C3xhnKjqWHQ+L3J9h953Syx
KOjsB1KkD3kS6z8M1pCSnXGwdMlWhsy4BsqQ5Rdhc+5tjm6al3TMLLzwuNPmh7Eg
HYbA93BHXS92tDOKoqb6jb+VxMo+rFxHMbzHJAGBdbfdW9bOkZMayw/3+7fb7eAn
tDidoCyEcbNuC57c1cO+Rk/Zwa+unVAasfw3e4djLs65RF2k+nlgrUTM1pyef4lO
Q4Z1OnZjS3WlrrtSgbdQ37MlaLRFnVNaF1dVQ9Cb/Q4s6Z70h4T5HPXjGfHXZP28
VV7JzeZ03xCbtGIMsoUeWOBIVxGWorclWEHwaA+EGN8wfQpuosPE/2m0YVqeqzpo
sy5qGSQa80eIzCyhg1d9vf/FVkhJ/yH3VEWHIS2KzkUp/eik8fM=
=Nfip
-----END PGP SIGNATURE-----
media: uvcvideo: Utilise for_each_uvc_urb iterator

A new iterator is available for processing UVC URB structures. This
simplifies the processing of the internal stream data.

Convert the manual loop iterators to the new helper, adding an index
helper to keep the existing debug print.

Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>

---

v6:
 - rename lone 'j' iterator to 'i'
 - Remove conversion which doesn't make sense due to needing the
   iterator value.
2 files changed