upstream: smiapp: Add support for flash, lens and EEPROM devices
(3 patches merged together)
These types devices aren't directly related to the sensor, but are
nevertheless handled by the smiapp driver due to the relationship of
these component to the main part of the camera module --- the sensor.
Additionally, for the async sub-device registration to work, the
notifier containing matching fwnodes will need to be registered. This
is natural to perform in a sensor driver as well.
This does not yet address providing the user space with information on
how to associate the sensor, lens or EEPROM devices but the kernel now
has the necessary information to do that.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Add a check for v4l2_dev to v4l2_async_notifier_register() as to fail
as early as possible since this will fail later in
v4l2_async_test_notify().
Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@xxxxxxxxxxxx>
Acked-by: Sakari Ailus <sakari.ailus@xxxxxxxxxxxxxxx>
Acked-by: Hans Verkuil <hans.verkuil@xxxxxxxxx>
v4l: async: add subnotifier registration for subdevices
When the registered() callback of v4l2_subdev_internal_ops is called
the subdevice has access to the master devices v4l2_dev and it's
called with the async frameworks list_lock held. In this context the
subdevice can register its own notifiers to allow for incremental
discovery of subdevices.
The master device registers the subdevices closest to itself in its
notifier while the subdevice(s) register notifiers for their closest
neighboring devices when they are registered. Using this incremental
approach two problems can be solved:
1. The master device no longer has to care how many devices exist in
the pipeline. It only needs to care about its closest subdevice and
arbitrary long pipelines can be created without having to adapt the
master device for each case.
2. Subdevices which are represented as a single DT node but register
more than one subdevice can use this to improve the pipeline
discovery, since the subdevice driver is the only one who knows
which of its subdevices is linked with which subdevice of a
neighboring DT node.
To enable subdevices to register/unregister notifiers from the
registered()/unregistered() callback v4l2_async_subnotifier_register()
and v4l2_async_subnotifier_unregister() are added. These new notifier
register functions are similar to the master device equivalent
functions
but run without taking the v4l2-async list_lock which already is held
when the registered()/unregistered() callbacks are called.
Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@xxxxxxxxxxxx>
Acked-by: Hans Verkuil <hans.verkuil@xxxxxxxxx>
Acked-by: Sakari Ailus <sakari.ailus@xxxxxxxxxxxxxxx>
diff --git a/Documentation/media/kapi/v4l2-subdev.rst b/Documentation/media/kapi/v4l2-subdev.rst
index e1f0b72..e308f30 100644
--- a/Documentation/media/kapi/v4l2-subdev.rst
+++ b/Documentation/media/kapi/v4l2-subdev.rst
@@ -262,6 +262,26 @@
called. When a subdevice is removed from the system the .unbind() method is
called. All three callbacks are optional.
+Subdevice drivers might in turn register subnotifier objects with an
+array of other subdevice descriptors that the subdevice needs for its
+own operation. Subnotifiers are an extension of the bridge drivers
+notifier to allow for a incremental registering and matching of
+subdevices. This is useful when a driver only has information about
+which subdevice is closest to itself and would require knowledge from the
+driver of that subdevice to know which other subdevice(s) lie beyond.
+By registering subnotifiers drivers can incrementally move the subdevice
+matching down the chain of drivers. This is performed using the
+:c:func:`v4l2_async_subnotifier_register` call. To unregister the
+subnotifier the driver has to call
+:c:func:`v4l2_async_subnotifier_unregister`. These functions and their
+arguments behave almost the same as the bridge driver notifiers
+described above and are treated equally by the V4L2 core when matching
+asynchronously registered subdevices. The differences are that the
+subnotifier functions act on :c:type:`v4l2_subdev` instead of
+:c:type:`v4l2_device` and that they should be called from the subdevice's
+``.registered()`` and ``.unregistered()``
+:c:type:`v4l2_subdev_internal_ops` callbacks instead of at probe time.
+
V4L2 sub-device userspace API
-----------------------------
diff --git a/drivers/media/i2c/smiapp/smiapp-core.c b/drivers/media/i2c/smiapp/smiapp-core.c
index aff55e1..005257b3 100644
--- a/drivers/media/i2c/smiapp/smiapp-core.c
+++ b/drivers/media/i2c/smiapp/smiapp-core.c
@@ -2551,6 +2551,22 @@ static int smiapp_register_subdev(struct smiapp_sensor *sensor,
return 0;
}
+static int smiapp_subdev_notifier_bound(struct v4l2_async_notifier *notifier,
+ struct v4l2_subdev *sd,
+ struct v4l2_async_subdev *asd)
+{
+ return 0;
+}
+
+static int smiapp_subdev_notifier_complete(
+ struct v4l2_async_notifier *notifier)
+{
+ struct smiapp_sensor *sensor =
+ container_of(notifier, struct smiapp_sensor, notifier);
+
+ return v4l2_device_register_subdev_nodes(sensor->src->sd.v4l2_dev);
+}
+
static void smiapp_unregistered(struct v4l2_subdev *subdev)
{
struct smiapp_sensor *sensor = to_smiapp_sensor(subdev);
@@ -2558,6 +2574,8 @@ static void smiapp_unregistered(struct v4l2_subdev *subdev)
for (i = 1; i < sensor->ssds_used; i++)
v4l2_device_unregister_subdev(&sensor->ssds[i].sd);
+
+ v4l2_async_subnotifier_unregister(&sensor->notifier);
}
static int smiapp_registered(struct v4l2_subdev *subdev)
@@ -2581,6 +2599,15 @@ static int smiapp_registered(struct v4l2_subdev *subdev)
if (rval)
goto out_err;
+ if (!sensor->notifier.num_subdevs)
+ return 0;
+
+ sensor->notifier.bound = smiapp_subdev_notifier_bound;
+ sensor->notifier.complete = smiapp_subdev_notifier_complete;
+ rval = v4l2_async_subnotifier_register(subdev, &sensor->notifier);
+ if (rval)
+ goto out_err;
+
return 0;
out_err:
@@ -2782,13 +2809,15 @@ static int __maybe_unused smiapp_resume(struct device *dev)
return rval;
}
-static struct smiapp_hwconfig *smiapp_get_hwconfig(struct device *dev)
+static struct smiapp_hwconfig *smiapp_get_hwconfig(struct device *dev,
+ struct smiapp_sensor *sensor)
{
+ static const char *props[] = { "flash", "lens", "eeprom" };
struct smiapp_hwconfig *hwcfg;
struct v4l2_fwnode_endpoint *bus_cfg;
struct fwnode_handle *ep;
struct fwnode_handle *fwnode = dev_fwnode(dev);
- int i;
+ unsigned int i;
int rval;
if (!fwnode)
@@ -2855,6 +2884,45 @@ static struct smiapp_hwconfig *smiapp_get_hwconfig(struct device *dev)
v4l2_fwnode_endpoint_free(bus_cfg);
fwnode_handle_put(ep);
+
+ sensor->notifier.subdevs =
+ devm_kcalloc(dev, SMIAPP_MAX_ASYNC_SUBDEVS,
+ sizeof(struct v4l2_async_subdev *), GFP_KERNEL);
+ if (!sensor->notifier.subdevs)
+ goto out_err;
+
+ for (i = 0; i < ARRAY_SIZE(props); i++) {
+ struct device_node *node;
+ unsigned int j = 0;
+
+ while ((node = of_parse_phandle(dev->of_node, props[i], j++))) {
+ struct v4l2_async_subdev **asd =
+ &sensor->notifier.subdevs[
+ sensor->notifier.num_subdevs];
+
+ if (WARN_ON(sensor->notifier.num_subdevs >=
+ SMIAPP_MAX_ASYNC_SUBDEVS)) {
+ of_node_put(node);
+ goto out;
+ }
+
+ *asd = devm_kzalloc(
+ dev, sizeof(struct v4l2_async_subdev),
+ GFP_KERNEL);
+ if (!*asd) {
+ of_node_put(node);
+ goto out_err;
+ }
+
+ (*asd)->match.fwnode.fwnode = of_fwnode_handle(node);
+ (*asd)->match_type = V4L2_ASYNC_MATCH_FWNODE;
+ sensor->notifier.num_subdevs++;
+
+ of_node_put(node);
+ }
+ }
+
+out:
return hwcfg;
out_err:
@@ -2867,18 +2935,17 @@ static int smiapp_probe(struct i2c_client *client,
const struct i2c_device_id *devid)
{
struct smiapp_sensor *sensor;
- struct smiapp_hwconfig *hwcfg = smiapp_get_hwconfig(&client->dev);
unsigned int i;
int rval;
- if (hwcfg == NULL)
- return -ENODEV;
-
sensor = devm_kzalloc(&client->dev, sizeof(*sensor), GFP_KERNEL);
if (sensor == NULL)
return -ENOMEM;
- sensor->hwcfg = hwcfg;
+ sensor->hwcfg = smiapp_get_hwconfig(&client->dev, sensor);
+ if (sensor->hwcfg == NULL)
+ return -ENODEV;
+
mutex_init(&sensor->mutex);
sensor->src = &sensor->ssds[sensor->ssds_used];
diff --git a/drivers/media/i2c/smiapp/smiapp.h b/drivers/media/i2c/smiapp/smiapp.h
index f74d695..21a55de 100644
--- a/drivers/media/i2c/smiapp/smiapp.h
+++ b/drivers/media/i2c/smiapp/smiapp.h
@@ -20,6 +20,7 @@
#define __SMIAPP_PRIV_H_
#include <linux/mutex.h>
+#include <media/v4l2-async.h>
#include <media/v4l2-ctrls.h>
#include <media/v4l2-subdev.h>
#include <media/i2c/smiapp.h>
@@ -143,6 +144,9 @@ struct smiapp_csi_data_format {
u8 pixel_order;
};
+/* Lens, EEPROM and a flash LEDs? */
+#define SMIAPP_MAX_ASYNC_SUBDEVS 3
+
#define SMIAPP_SUBDEVS 3
#define SMIAPP_PA_PAD_SRC 0
@@ -189,6 +193,7 @@ struct smiapp_sensor {
struct regulator *vana;
struct clk *ext_clk;
struct gpio_desc *xshutdown;
+ struct v4l2_async_notifier notifier;
u32 limits[SMIAPP_LIMIT_LAST];
u8 nbinning_subtypes;
struct smiapp_binning_subtype binning_subtypes[SMIAPP_BINNING_SUBTYPES];
diff --git a/drivers/media/v4l2-core/v4l2-async.c b/drivers/media/v4l2-core/v4l2-async.c
index 851f128..2d166ed 100644
--- a/drivers/media/v4l2-core/v4l2-async.c
+++ b/drivers/media/v4l2-core/v4l2-async.c
@@ -142,8 +142,9 @@ static void v4l2_async_cleanup(struct v4l2_subdev *sd)
sd->dev = NULL;
}
-int v4l2_async_notifier_register(struct v4l2_device *v4l2_dev,
- struct v4l2_async_notifier *notifier)
+static int v4l2_async_do_notifier_register(struct v4l2_device *v4l2_dev,
+ struct v4l2_async_notifier *notifier,
+ bool subnotifier)
{
struct v4l2_subdev *sd, *tmp;
struct v4l2_async_subdev *asd;
@@ -175,8 +176,17 @@ int v4l2_async_notifier_register(struct v4l2_device *v4l2_dev,
list_add_tail(&asd->list, ¬ifier->waiting);
}
- mutex_lock(&list_lock);
+ if (subnotifier)
+ lockdep_assert_held(&list_lock);
+ else
+ mutex_lock(&list_lock);
+ /*
+ * This function can be called recursively so the list
+ * might be modified in a recursive call. Start from the
+ * top of the list each iteration.
+ */
+again:
list_for_each_entry_safe(sd, tmp, &subdev_list, async_list) {
int ret;
@@ -186,21 +196,39 @@ int v4l2_async_notifier_register(struct v4l2_device *v4l2_dev,
ret = v4l2_async_test_notify(notifier, sd, asd);
if (ret < 0) {
- mutex_unlock(&list_lock);
+ if (!subnotifier)
+ mutex_unlock(&list_lock);
return ret;
}
+ goto again;
}
/* Keep also completed notifiers on the list */
list_add(¬ifier->list, ¬ifier_list);
- mutex_unlock(&list_lock);
+ if (!subnotifier)
+ mutex_unlock(&list_lock);
return 0;
}
+
+int v4l2_async_subnotifier_register(struct v4l2_subdev *sd,
+ struct v4l2_async_notifier *notifier)
+{
+ return v4l2_async_do_notifier_register(sd->v4l2_dev, notifier, true);
+}
+EXPORT_SYMBOL(v4l2_async_subnotifier_register);
+
+int v4l2_async_notifier_register(struct v4l2_device *v4l2_dev,
+ struct v4l2_async_notifier *notifier)
+{
+ return v4l2_async_do_notifier_register(v4l2_dev, notifier, false);
+}
EXPORT_SYMBOL(v4l2_async_notifier_register);
-void v4l2_async_notifier_unregister(struct v4l2_async_notifier *notifier)
+static void
+v4l2_async_do_notifier_unregister(struct v4l2_async_notifier *notifier,
+ bool subnotifier)
{
struct v4l2_subdev *sd, *tmp;
unsigned int notif_n_subdev = notifier->num_subdevs;
@@ -217,7 +245,10 @@ void v4l2_async_notifier_unregister(struct v4l2_async_notifier *notifier)
"Failed to allocate device cache!\n");
}
- mutex_lock(&list_lock);
+ if (subnotifier)
+ lockdep_assert_held(&list_lock);
+ else
+ mutex_lock(&list_lock);
list_del(¬ifier->list);
@@ -244,15 +275,20 @@ void v4l2_async_notifier_unregister(struct v4l2_async_notifier *notifier)
put_device(d);
}
- mutex_unlock(&list_lock);
+ if (!subnotifier)
+ mutex_unlock(&list_lock);
/*
* Call device_attach() to reprobe devices
*
* NOTE: If dev allocation fails, i is 0, and the whole loop won't be
* executed.
+ * TODO: If we are unregistering a subdevice notifier we can't reprobe
+ * since the lock_list is held by the master device and attaching that
+ * device would call v4l2_async_register_subdev() and end in a deadlock
+ * on list_lock.
*/
- while (i--) {
+ while (i-- && !subnotifier) {
struct device *d = dev[i];
if (d && device_attach(d) < 0) {
@@ -276,6 +312,17 @@ void v4l2_async_notifier_unregister(struct v4l2_async_notifier *notifier)
* upon notifier registration.
*/
}
+
+void v4l2_async_subnotifier_unregister(struct v4l2_async_notifier *notifier)
+{
+ v4l2_async_do_notifier_unregister(notifier, true);
+}
+EXPORT_SYMBOL(v4l2_async_subnotifier_unregister);
+
+void v4l2_async_notifier_unregister(struct v4l2_async_notifier *notifier)
+{
+ v4l2_async_do_notifier_unregister(notifier, false);
+}
EXPORT_SYMBOL(v4l2_async_notifier_unregister);
int v4l2_async_register_subdev(struct v4l2_subdev *sd)
diff --git a/include/media/v4l2-async.h b/include/media/v4l2-async.h
index c69d8c8..f7e2a1a 100644
--- a/include/media/v4l2-async.h
+++ b/include/media/v4l2-async.h
@@ -105,6 +105,18 @@ struct v4l2_async_notifier {
};
/**
+ * v4l2_async_notifier_register - registers a subdevice asynchronous subnotifier
+ *
+ * @sd: pointer to &struct v4l2_subdev
+ * @notifier: pointer to &struct v4l2_async_notifier
+ *
+ * This function assumes the async list_lock is already locked, allowing it to
+ * be used from the struct v4l2_subdev_internal_ops registered() callback.
+ */
+int v4l2_async_subnotifier_register(struct v4l2_subdev *sd,
+ struct v4l2_async_notifier *notifier);
+
+/**
* v4l2_async_notifier_register - registers a subdevice asynchronous notifier
*
* @v4l2_dev: pointer to &struct v4l2_device
@@ -114,6 +126,16 @@ int v4l2_async_notifier_register(struct v4l2_device *v4l2_dev,
struct v4l2_async_notifier *notifier);
/**
+ * v4l2_async_subnotifier_unregister - unregisters a asynchronous subnotifier
+ *
+ * @notifier: pointer to &struct v4l2_async_notifier
+ *
+ * This function assumes the async list_lock is already locked, allowing it to
+ * be used from the struct v4l2_subdev_internal_ops unregistered() callback.
+ */
+void v4l2_async_subnotifier_unregister(struct v4l2_async_notifier *notifier);
+
+/**
* v4l2_async_notifier_unregister - unregisters a subdevice asynchronous notifier
*
* @notifier: pointer to &struct v4l2_async_notifier
