The logic is testing if num_links==0 at the wrong place. Due to
that, a backlink may be kept without removal, causing KASAN
to complain about usage after free during either entity or
link removal.
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
The media_entity_cleanup() function only cleans up the entity links list
but this operation is already made in media_device_unregister_entity().
In most cases this should be harmless (besides having duplicated code)
since the links list would be empty so the iteration would not happen
but the links list is initialized in media_device_register_entity() so
if a driver fails to register an entity with a media device and clean up
the entity in the error path, a NULL deference pointer error will happen.
So don't try to empty the links list in media_entity_cleanup() since
is either done already or haven't been initialized yet.
Signed-off-by: Javier Martinez Canillas <javier@osg.samsung.com>
Some parts of the media controller are using mutexes while
others are using spin locks in order to protect creation
and removal of elements in the graph. That's wrong!
Also, the V4L2 core can remove graph elements on non-interactive
context:
BUG: sleeping function called from invalid context at include/linux/sched.h:2776
Fix it by always using spin locks for graph element addition/removal,
just like entity creation/removal is protected at media-device.c
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Just like what's done with entities, when the media controller is
unregistered, release any interface and interface links that
might still be there.
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Add functions to explicitly unregister all entity links.
This function is called automatically when an entity
link is destroyed.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Every time a graph object is added or removed, the version
of the topology changes. That's a requirement for the new
MEDIA_IOC_G_TOPOLOGY, in order to allow userspace to know
that the topology has changed after a previous call to it.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
The MC next gen API sends objects to userspace grouped by
their types.
In the case of pads and links, in order to improve performance
and have a simpler code, the best is to store them also on
separate linked lists at MC.
If we don't do that, we would need this kind of interaction
to send data to userspace (code is in structured english):
for each entity:
for each pad:
store pads
for each entity:
for each link:
store link
for each interface:
for each link:
store link
With would require one nested loop for pads and two nested
loops for links. By using separate linked lists for them,
just one loop would be enough.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Let's control the topology changes inside the graph_object. So, move the
addition and removal of interfaces/entities from the mdev lists to
media_gobj_init() and media_gobj_remove().
The main reason is that mdev should have lists for all object types, as
the new MC api will require to store objects in separate places.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Sometimes, it is important to see if the created pad is
sink or source. Add info to track that.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
The media device should list the interface objects, so add a linked list
for those interfaces in struct media_device.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
As we'll be adding other interface types in the future, put the
common interface create code on a separate function.
Suggested-by: Hans Verkuil <hans.verkuil@cisco.com>
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Now that we have a new graph object called "interfaces", we
need to be able to link them to the entities.
Add a linked list to the interfaces to allow them to be
linked to the entities.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Remove entity name from the link as this exists only if the object
type is PAD on both link ends.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
The media_entity_add_link() function takes an entity
as an argument just to get the list head.
Make it more generic by changing the function argument
to list_head.
No functional changes.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
The entire logic that represent graph links were developed on a
time where there were no needs to dynamic remove links. So,
although links are created/removed one by one via some
functions, they're stored as an array inside the entity struct.
As the array may grow, there's a logic inside the code that
checks if the amount of space is not enough to store
the needed links. If it isn't the core uses krealloc()
to change the size of the link, with is bad, as it
leaves the memory fragmented.
So, convert links into a list.
Also, currently, both source and sink entities need the link
at the graph traversal logic inside media_entity. So there's
a logic duplicating all links. That makes it to spend
twice the memory needed. This is not a big deal for today's
usage, where the number of links are not big.
Yet, if during the MC workshop discussions, it was said that
IIO graphs could have up to 4,000 entities. So, we may
want to remove the duplication on some future. The problem
is that it would require a separate linked list to store
the backlinks inside the entity, or to use a more complex
algorithm to do graph backlink traversal, with is something
that the current graph traversal inside the core can't cope
with. So, let's postpone a such change if/when it is actually
needed.
It should also be noticed that the media_link structure uses
44 bytes on 32-bit architectures and 84 bytes on 64-bit
architecture. It will thus be allocated out of the 64-bytes and
96-bytes pools respectively. That's a 12.5% memory waste on
64-bit architectures and 31.25% on 32-bit architecture.
A linked list is less efficient than an array in this case, but
this could later be optimized if we can get rid of the reverse
links (with would reduce memory allocation by 50%).
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Links are graph objects that represent the links of two already
existing objects in the graph.
While with the current implementation, it is possible to create
the links earlier, It doesn't make any sense to allow linking
two objects when they are not both created.
So, remove the code that would be handling those early-created
links and add a BUG_ON() to ensure that.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Interfaces are different than entities: they represent a
Kernel<->userspace interaction, while entities represent a
piece of hardware/firmware/software that executes a function.
Let's distinguish them by creating a separate structure to
store the interfaces.
Later patches should change the existing drivers and logic
to split the current interface embedded inside the entity
structure (device nodes) into a separate object of the graph.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Reviewed-by: Javier Martinez Canillas <javier@osg.samsung.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
The struct media_entity has a .parent field that stores a pointer
to the parent struct media_device. But recently a media_gobj was
embedded into the entities and since struct media_gojb already has
a pointer to a struct media_device in the .mdev field, the .parent
field becomes redundant and can be removed.
This patch replaces all the usage of .parent by .graph_obj.mdev so
that field will become unused and can be removed on a later patch.
No functional changes.
The transformation was made using the following coccinelle spatch:
@@
struct media_entity *me;
@@
- me->parent
+ me->graph_obj.mdev
@@
struct media_entity *link;
@@
- link->source->entity->parent
+ link->source->entity->graph_obj.mdev
@@
struct exynos_video_entity *ve;
@@
- ve->vdev.entity.parent
+ ve->vdev.entity.graph_obj.mdev
Suggested-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Signed-off-by: Javier Martinez Canillas <javier@osg.samsung.com>
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
With the new API, a link can be either between two PADs or between an interface
and an entity. So, we need to use a better name for the function that create
links between two pads.
So, rename the such function to media_create_pad_link().
No functional changes.
This patch was created via this shell script:
for i in $(find drivers/media -name '*.[ch]' -type f) $(find drivers/staging/media -name '*.[ch]' -type f) $(find include/ -name '*.h' -type f) ; do sed s,media_entity_create_link,media_create_pad_link,g <$i >a && mv a $i; done
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Tested-by: Javier Martinez Canillas <javier@osg.samsung.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
It helps to check if the media controller is doing the
right thing with the object creation and removal.
No extra code/data will be produced if DEBUG or
CONFIG_DYNAMIC_DEBUG is not enabled.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Reviewed-by: Javier Martinez Canillas <javier@osg.samsung.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Just like entities and pads, links also need to have unique
Object IDs along a given media controller.
So, let's add a media_gobj inside it and initialize
the object then a new link is created.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Tested-by: Javier Martinez Canillas <javier@osg.samsung.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
PADs also need unique object IDs that won't conflict with
the entity object IDs.
The pad objects are currently created via media_entity_init()
and, once created, never change.
While this will likely change in the future in order to
support dynamic changes, for now we'll keep PADs as arrays
and initialize the media_gobj embedded structs when
registering the entity.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Tested-by: Javier Martinez Canillas <javier@osg.samsung.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
As entities are graph objects, let's embed media_gobj
on it. That ensures an unique ID for entities that can be
global along the entire media controller.
For now, we'll keep the already existing entity ID. Such
field need to be dropped at some point, but for now, let's
not do this, to avoid needing to review all drivers and
the userspace apps.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Tested-by: Javier Martinez Canillas <javier@osg.samsung.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Due to the MC API proposed changes, we'll need to have an unique
object ID for all graph objects, and have some shared fields
that will be common on all media graph objects.
Right now, the only common object is the object ID, but other
fields will be added later on.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Reviewed-by: Javier Martinez Canillas <javier@osg.samsung.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Instead of accessing directly entity.id, let's create a macro,
as this field will be moved into a common struct later on.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Reviewed-by: Javier Martinez Canillas <javier@osg.samsung.com>
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Currently, media_entity_init() creates an array with the links,
allocated at init time. It provides a parameter (extra_links)
that would allocate more links than the current needs, but this
is not used by any driver.
As we want to be able to do dynamic link allocation/removal,
we'll need to change the implementation of the links. So,
before doing that, let's first remove that extra unused
parameter, in order to cleanup the interface first.
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Acked-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Fix those sparse warnings:
drivers/media/media-entity.c:238:17: warning: Variable length array is used.
drivers/media/media-entity.c:239:17: warning: Variable length array is used.
That allows sparse and other code check tools to verify if the
function is using more stack than allowed.
It also solves a bad Kernel pratice of using var length arrays
at the stack.
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
On the place of the source entity name, the sink entity name was printed.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Patch "[media] media: Print information on failed link validation" cause a
harmless compiler warning in printing a debug message. Fix this.
The type casting is done do ensure the type really is suitable for printing
as %u, as find_first_zero_bit() does return int on some architectures and
unsigned long on others.
Signed-off-by: Sakari Ailus <sakari.ailus@iki.fi>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
The Media controller doesn't tell much to the user in cases such as pipeline
startup failure. The link validation is the most common media graph (or in
V4L2's case, format) related reason for the failure. In more complex
pipelines the reason may not always be obvious to the user, so point them to
look at the right direction.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Do not allow streaming if a pad with MEDIA_PAD_FL_MUST_CONNECT flag is not
connected by an active link.
This patch makes it possible to avoid drivers having to check for the most
common case of link state validation: a sink pad that must be connected.
Signed-off-by: Sakari Ailus <sakari.ailus@iki.fi>
Tested-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@redhat.com>
The graph traversal API (media_entity_graph_walk_*) doesn't support
cyclic graphs and will fail to correctly walk a graph when circular
links exist. Support circular graph traversal by checking whether an
entity has already been visited before pushing it to the stack.
Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Acked-by: Sakari Ailus <sakari.ailus@iki.fi>
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <m.chehab@samsung.com>
This function allows to remove all media entity's links to other
entities, leaving no references to a media entity's links array
at its remote entities.
Currently, when a driver of some entity is removed it will free its
media entities links[] array, leaving dangling pointers at other
entities that are part of same media graph. This is troublesome when
drivers of a media device entities are in separate kernel modules,
removing only some modules will leave others in an incorrect state.
This function is intended to be used when an entity is being
unregistered from a media device.
With an assumption that normally the media links should be created
between media entities registered to a media device, with the graph
mutex held.
Signed-off-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
Reviewed-by: Andrzej Hajda <a.hajda@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Sakari Ailus <sakari.ailus@iki.fi>
Signed-off-by: Mauro Carvalho Chehab <mchehab@redhat.com>
Currently the media device link_notify callback is invoked before the
actual change of state of a link when the link is being enabled, and
after the actual change of state when the link is being disabled.
This doesn't allow a media device driver to perform any operations
on a full graph before a link is disabled, as well as performing
any tasks on a modified graph right after a link's state is changed.
This patch modifies signature of the link_notify callback. This
callback is now called always before and after a link's state change.
To distinguish the notifications a 'notification' argument is added
to the link_notify callback: MEDIA_DEV_NOTIFY_PRE_LINK_CH indicates
notification before link's state change and
MEDIA_DEV_NOTIFY_POST_LINK_CH corresponds to a notification after
link flags change.
[mchehab@redhat.com: whitespace cleanups]
Signed-off-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Sakari Ailus <sakari.ailus@iki.fi>
Signed-off-by: Mauro Carvalho Chehab <mchehab@redhat.com>
Function media_entity_remote_source actually returns the remote pad to
the given one, regardless if this is the source or the sink pad.
Name media_entity_remote_pad is more adequate for this function.
Signed-off-by: Andrzej Hajda <a.hajda@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
Acked-by: Sakari Ailus <sakari.ailus@iki.fi>
Acked-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@redhat.com>
The purpose of the link_validate() op is to allow an entity driver to ensure
that the properties of the pads at the both ends of the link are suitable
for starting the pipeline. link_validate is called on sink pads on active
links which belong to the active part of the graph.
Signed-off-by: Sakari Ailus <sakari.ailus@iki.fi>
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@redhat.com>
The link flags were not properly handled in __media_entity_setup_link
which could lead to link_notify callback to be called when the flags are
not modified. Fix this.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@redhat.com>
Drivers often need to associate pipeline objects to entities, and to
take stream state into account when configuring entities and links. The
pipeline API helps drivers manage that information.
When starting streaming, drivers call media_entity_pipeline_start(). The
function marks all entities connected to the given entity through
enabled links, either directly or indirectly, as streaming. Similarly,
when stopping the stream, drivers call media_entity_pipeline_stop().
The media_entity_pipeline_start() function takes a pointer to a media
pipeline and stores it in every entity in the graph. Drivers should
embed the media_pipeline structure in higher-level pipeline structures
and can then access the pipeline through the media_entity structure.
Link configuration will fail with -EBUSY by default if either end of the
link is a streaming entity, unless the link is marked with the
MEDIA_LNK_FL_DYNAMIC flag.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Hans Verkuil <hverkuil@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@redhat.com>
Create the following ioctl and implement it at the media device level to
setup links.
- MEDIA_IOC_SETUP_LINK: Modify the properties of a given link
The only property that can currently be modified is the ENABLED link
flag to enable/disable a link. Links marked with the IMMUTABLE link flag
can not be enabled or disabled.
Enabling or disabling a link has effects on entities' use count. Those
changes are automatically propagated through the graph.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Stanimir Varbanov <svarbanov@mm-sol.com>
Signed-off-by: Sakari Ailus <sakari.ailus@iki.fi>
Acked-by: Hans Verkuil <hverkuil@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@redhat.com>
Due to the wide differences between drivers regarding power management
needs, the media controller does not implement power management.
However, the media_entity structure includes a use_count field that
media drivers can use to track the number of users of every entity for
power management needs.
The use_count field is owned by media drivers and must not be touched by
entity drivers. Access to the field must be protected by the media
device graph_mutex lock.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Hans Verkuil <hverkuil@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@redhat.com>
Add media entity graph traversal. The traversal follows enabled links by
depth first. Traversing graph backwards is prevented by comparing the next
possible entity in the graph with the previous one. Multiply connected
graphs are thus not supported.
Signed-off-by: Sakari Ailus <sakari.ailus@iki.fi>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Vimarsh Zutshi <vimarsh.zutshi@gmail.com>
Acked-by: Hans Verkuil <hverkuil@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@redhat.com>
As video hardware pipelines become increasingly complex and
configurable, the current hardware description through v4l2 subdevices
reaches its limits. In addition to enumerating and configuring
subdevices, video camera drivers need a way to discover and modify at
runtime how those subdevices are connected. This is done through new
elements called entities, pads and links.
An entity is a basic media hardware building block. It can correspond to
a large variety of logical blocks such as physical hardware devices
(CMOS sensor for instance), logical hardware devices (a building block
in a System-on-Chip image processing pipeline), DMA channels or physical
connectors.
A pad is a connection endpoint through which an entity can interact with
other entities. Data (not restricted to video) produced by an entity
flows from the entity's output to one or more entity inputs. Pads should
not be confused with physical pins at chip boundaries.
A link is a point-to-point oriented connection between two pads, either
on the same entity or on different entities. Data flows from a source
pad to a sink pad.
Links are stored in the source entity. To make backwards graph walk
faster, a copy of all links is also stored in the sink entity. The copy
is known as a backlink and is only used to help graph traversal.
The entity API is made of three functions:
- media_entity_init() initializes an entity. The caller must provide an
array of pads as well as an estimated number of links. The links array
is allocated dynamically and will be reallocated if it grows beyond the
initial estimate.
- media_entity_cleanup() frees resources allocated for an entity. It
must be called during the cleanup phase after unregistering the entity
and before freeing it.
- media_entity_create_link() creates a link between two entities. An
entry in the link array of each entity is allocated and stores pointers
to source and sink pads.
When a media device is unregistered, all its entities are unregistered
automatically.
The code is based on Hans Verkuil <hverkuil@xs4all.nl> initial work.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Sakari Ailus <sakari.ailus@iki.fi>
Acked-by: Hans Verkuil <hverkuil@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@redhat.com>
