media: camss: Fix signedness bug in video_enum_fmt()

This test has a problem because we want to know if "k" is -1 or a
positive value less than "f->index".  But the "f->index" variable is a
u32 so if "k == -1" then -1 gets type promoted to UINT_MAX which is
larger than "f->index".  I've added an explicit test to check for -1.

Fixes: a3d412d4b9 ("media: Revert "media: camss: Make use of V4L2_CAP_IO_MC"")
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Reviewed-by: Robert Foss <robert.foss@linaro.org>
Reviewed-by: Andrey Konovalov <andrey.konovalov@linaro.org>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
This commit is contained in:
Dan Carpenter 2020-12-09 07:50:34 +01:00 committed by Mauro Carvalho Chehab
parent a819678d0d
commit b00481bdca

View File

@ -579,7 +579,7 @@ static int video_enum_fmt(struct file *file, void *fh, struct v4l2_fmtdesc *f)
break;
}
if (k < f->index)
if (k == -1 || k < f->index)
/*
* All the unique pixel formats matching the arguments
* have been enumerated (k >= 0 and f->index > 0), or