Commit Graph

15 Commits

Author SHA1 Message Date
Xu Yilun
b6862193ca fpga: dfl: support multiple opens on feature device node.
Each DFL functional block, e.g. AFU (Accelerated Function Unit) and FME
(FPGA Management Engine), could implement more than one function within
its region, but current driver only allows one user application to access
it by exclusive open on device node. So this is not convenient and
flexible for userspace applications, as they have to combine lots of
different functions into one single application.

This patch removes the limitation here to allow multiple opens to each
feature device node for AFU and FME from userspace applications. If user
still needs exclusive access to these device node, O_EXCL flag must be
issued together with open.

Signed-off-by: Wu Hao <hao.wu@intel.com>
Signed-off-by: Xu Yilun <yilun.xu@intel.com>
Signed-off-by: Moritz Fischer <mdf@kernel.org>
2020-04-29 20:37:07 -07:00
Wu Hao
84b693e378 fpga: dfl: make init callback optional
This patch makes init callback of sub features optional. With
this change, people don't need to prepare any empty init callback.

Signed-off-by: Wu Hao <hao.wu@intel.com>
Acked-by: Moritz Fischer <mdf@kernel.org>
Signed-off-by: Moritz Fischer <mdf@kernel.org>
2019-09-03 19:35:40 -07:00
Wu Hao
3c51ff7722 fpga: dfl: make uinit callback optional
This patch makes uinit callback of sub features optional. With
this change, people don't need to prepare any empty uinit callback.

Signed-off-by: Wu Hao <hao.wu@intel.com>
Link: https://lore.kernel.org/r/1564914022-3710-9-git-send-email-hao.wu@intel.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2019-08-05 18:02:18 +02:00
Wu Hao
15bbb300fc fpga: dfl: add id_table for dfl private feature driver
This patch adds id_table for each dfl private feature driver,
it allows to reuse same private feature driver to match and support
multiple dfl private features.

Signed-off-by: Xu Yilun <yilun.xu@intel.com>
Signed-off-by: Wu Hao <hao.wu@intel.com>
Acked-by: Moritz Fischer <mdf@kernel.org>
Acked-by: Alan Tull <atull@kernel.org>
Signed-off-by: Moritz Fischer <mdf@kernel.org>
Link: https://lore.kernel.org/r/1564914022-3710-6-git-send-email-hao.wu@intel.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2019-08-05 18:01:24 +02:00
Wu Hao
bdd4f30795 fpga: dfl: pci: enable SRIOV support.
This patch enables the standard sriov support. It allows user to
enable SRIOV (and VFs), then user could pass through accelerators
(VFs) into virtual machine or use VFs directly in host.

Signed-off-by: Zhang Yi Z <yi.z.zhang@intel.com>
Signed-off-by: Xu Yilun <yilun.xu@intel.com>
Signed-off-by: Wu Hao <hao.wu@intel.com>
Acked-by: Alan Tull <atull@kernel.org>
Acked-by: Moritz Fischer <mdf@kernel.org>
Signed-off-by: Moritz Fischer <mdf@kernel.org>
Link: https://lore.kernel.org/r/1564914022-3710-3-git-send-email-hao.wu@intel.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2019-08-05 17:56:46 +02:00
Wu Hao
69bb18ddfc fpga: dfl: fme: add DFL_FPGA_FME_PORT_RELEASE/ASSIGN ioctl support.
In order to support virtualization usage via PCIe SRIOV, this patch
adds two ioctls under FPGA Management Engine (FME) to release and
assign back the port device. In order to safely turn Port from PF
into VF and enable PCIe SRIOV, it requires user to invoke this
PORT_RELEASE ioctl to release port firstly to remove userspace
interfaces, and then configure the PF/VF access register in FME.
After disable SRIOV, it requires user to invoke this PORT_ASSIGN
ioctl to attach the port back to PF.

 Ioctl interfaces:
 * DFL_FPGA_FME_PORT_RELEASE
   Release platform device of given port, it deletes port platform
   device to remove related userspace interfaces on PF. After this
   function, then it's safe to configure PF/VF access mode to VF,
   and enable VFs via SRIOV.

 * DFL_FPGA_FME_PORT_ASSIGN
   Assign platform device of given port back to PF. After configure
   PF/VF access mode to PF, this ioctl adds port platform device
   back to re-enable related userspace interfaces on PF.

Signed-off-by: Zhang Yi Z <yi.z.zhang@intel.com>
Signed-off-by: Xu Yilun <yilun.xu@intel.com>
Signed-off-by: Wu Hao <hao.wu@intel.com>
Acked-by: Alan Tull <atull@kernel.org>
Acked-by: Moritz Fischer <mdf@kernel.org>
Signed-off-by: Moritz Fischer <mdf@kernel.org>
Link: https://lore.kernel.org/r/1564914022-3710-2-git-send-email-hao.wu@intel.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2019-08-05 17:56:46 +02:00
Chengguang Xu
de9a7f6f5f fpga: dfl: expand minor range when registering chrdev region
Actually, total amount of available minor number
for a single major is MINORMASK + 1. So expand
minor range when registering chrdev region.

Signed-off-by: Chengguang Xu <cgxu519@gmx.com>
Acked-by: Wu Hao <hao.wu@intel.com>
Acked-by: Alan Tull <atull@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2019-05-24 20:32:12 +02:00
Scott Wood
dfe3de8d39 fpga: dfl: Add lockdep classes for pdata->lock
struct dfl_feature_platform_data (and it's mutex) is used
by both fme and port devices, and when lockdep is enabled it
complains about nesting between these locks.  Tell lockdep about
the difference so it can track each class separately.

Here's the lockdep complaint:
[  409.680668] WARNING: possible recursive locking detected
[  409.685983] 5.1.0-rc3.fpga+ #1 Tainted: G            E
[  409.691469] --------------------------------------------
[  409.696779] fpgaconf/9348 is trying to acquire lock:
[  409.701746] 00000000a443fe2e (&pdata->lock){+.+.}, at: port_enable_set+0x24/0x60 [dfl_afu]
[  409.710006]
[  409.710006] but task is already holding lock:
[  409.715837] 0000000063b78782 (&pdata->lock){+.+.}, at: fme_pr_ioctl+0x21d/0x330 [dfl_fme]
[  409.724012]
[  409.724012] other info that might help us debug this:
[  409.730535]  Possible unsafe locking scenario:
[  409.730535]
[  409.736457]        CPU0
[  409.738910]        ----
[  409.741360]   lock(&pdata->lock);
[  409.744679]   lock(&pdata->lock);
[  409.747999]
[  409.747999]  *** DEADLOCK ***
[  409.747999]
[  409.753920]  May be due to missing lock nesting notation
[  409.753920]
[  409.760704] 4 locks held by fpgaconf/9348:
[  409.764805]  #0: 0000000063b78782 (&pdata->lock){+.+.}, at: fme_pr_ioctl+0x21d/0x330 [dfl_fme]
[  409.773408]  #1: 00000000213c8a66 (&region->mutex){+.+.}, at: fpga_region_program_fpga+0x24/0x200 [fpga_region]
[  409.783489]  #2: 00000000fe63afb9 (&mgr->ref_mutex){+.+.}, at: fpga_mgr_lock+0x15/0x40 [fpga_mgr]
[  409.792354]  #3: 000000000b2285c5 (&bridge->mutex){+.+.}, at: __fpga_bridge_get+0x26/0xa0 [fpga_bridge]
[  409.801740]
[  409.801740] stack backtrace:
[  409.806102] CPU: 45 PID: 9348 Comm: fpgaconf Kdump: loaded Tainted: G            E     5.1.0-rc3.fpga+ #1
[  409.815658] Hardware name: Intel Corporation S2600BT/S2600BT, BIOS SE5C620.86B.01.00.0763.022420181017 02/24/2018
[  409.825911] Call Trace:
[  409.828369]  dump_stack+0x5e/0x8b
[  409.831686]  __lock_acquire+0xf3d/0x10e0
[  409.835612]  ? find_held_lock+0x3c/0xa0
[  409.839451]  lock_acquire+0xbc/0x1d0
[  409.843030]  ? port_enable_set+0x24/0x60 [dfl_afu]
[  409.847823]  ? port_enable_set+0x24/0x60 [dfl_afu]
[  409.852616]  __mutex_lock+0x86/0x970
[  409.856195]  ? port_enable_set+0x24/0x60 [dfl_afu]
[  409.860989]  ? port_enable_set+0x24/0x60 [dfl_afu]
[  409.865777]  ? __mutex_unlock_slowpath+0x4b/0x290
[  409.870486]  port_enable_set+0x24/0x60 [dfl_afu]
[  409.875106]  fpga_bridges_disable+0x36/0x50 [fpga_bridge]
[  409.880502]  fpga_region_program_fpga+0xea/0x200 [fpga_region]
[  409.886338]  fme_pr_ioctl+0x13e/0x330 [dfl_fme]
[  409.890870]  fme_ioctl+0x66/0xe0 [dfl_fme]
[  409.894973]  do_vfs_ioctl+0xa9/0x720
[  409.898548]  ? lockdep_hardirqs_on+0xf0/0x1a0
[  409.902907]  ksys_ioctl+0x60/0x90
[  409.906225]  __x64_sys_ioctl+0x16/0x20
[  409.909981]  do_syscall_64+0x5a/0x220
[  409.913644]  entry_SYSCALL_64_after_hwframe+0x49/0xbe
[  409.918698] RIP: 0033:0x7f9d31b9b8d7
[  409.922276] Code: 44 00 00 48 8b 05 b9 15 2d 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 89 15 2d 00 f7 d8 64 89 01 48
[  409.941020] RSP: 002b:00007ffe4cae0d68 EFLAGS: 00000202 ORIG_RAX: 0000000000000010
[  409.948588] RAX: ffffffffffffffda RBX: 00007f9d32ade6a0 RCX: 00007f9d31b9b8d7
[  409.955719] RDX: 00007ffe4cae0df0 RSI: 000000000000b680 RDI: 0000000000000003
[  409.962852] RBP: 0000000000000003 R08: 00007f9d2b70a177 R09: 00007ffe4cae0e40
[  409.969984] R10: 00007ffe4cae0160 R11: 0000000000000202 R12: 00007ffe4cae0df0
[  409.977115] R13: 000000000000b680 R14: 0000000000000000 R15: 00007ffe4cae0f60

Signed-off-by: Scott Wood <swood@redhat.com>
Acked-by: Wu Hao <hao.wu@intel.com>
Acked-by: Alan Tull <atull@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2019-05-24 20:32:12 +02:00
Alan Tull
fea82b7f66 fpga: add devm_fpga_region_create
Add devm_fpga_region_create() which is the
managed version of fpga_region_create().

Change current region drivers to use
devm_fpga_region_create().

Signed-off-by: Alan Tull <atull@kernel.org>
Suggested-by: Federico Vaga <federico.vaga@cern.ch>
Acked-by: Moritz Fischer <mdf@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2018-10-16 11:13:50 +02:00
Wu Hao
d06b004b99 fpga: dfl: add dfl_fpga_check_port_id function.
This patch adds one common function in DFL framework. It uses
port_ops get_id callback to get port id and compare it with given
value. This function could be used as match function of the
dfl_fpga_cdev_find_port function.

Signed-off-by: Wu Hao <hao.wu@intel.com>
Acked-by: Alan Tull <atull@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2018-07-15 13:55:45 +02:00
Wu Hao
6e8fd6e493 fpga: dfl: add dfl_fpga_port_ops support.
In some cases, other DFL driver modules may need to access some port
operations, e.g. disable / enable port for partial reconfiguration in
FME module. In order to avoid dependency between port and FME modules,
this patch introduces the dfl_fpga_port_ops support in DFL framework.
A global dfl_fpga_port_ops list is added in the DFL framework, and
it allows other DFL modules to use these port operations registered
to this list, even in virtualization case, the port platform device
is turned into VF / guest VM and hidden in host, the registered
port_ops is still usable. It resolves the dependency issues between
modules, but once get port ops API returns a valid port ops, that
means related port driver module has been module_get to prevent from
unexpected unload, and put port ops API must be invoked after use.

These APIs introduced by this patch is listed below:
 * dfl_fpga_port_ops_add
   add one port ops to the global list.

 * dfl_fpga_port_ops_del
   del one port ops from the global list.

 * dfl_fpga_port_ops_get / dfl_fpga_port_ops_put
   get/put the port ops before/after use.

Signed-off-by: Wu Hao <hao.wu@intel.com>
Acked-by: Alan Tull <atull@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2018-07-15 13:55:45 +02:00
Xiao Guangrong
5b57d02a2f fpga: dfl: add feature device infrastructure
This patch abstracts the common operations of the sub features and defines
the feature_ops data structure, including init, uinit and ioctl function
pointers. And this patch adds some common helper functions for FME and AFU
drivers, e.g. dfl_feature_dev_use_begin/end which are used to ensure
exclusive usage of the feature device file.

Signed-off-by: Tim Whisonant <tim.whisonant@intel.com>
Signed-off-by: Enno Luebbers <enno.luebbers@intel.com>
Signed-off-by: Shiva Rao <shiva.rao@intel.com>
Signed-off-by: Christopher Rauer <christopher.rauer@intel.com>
Signed-off-by: Kang Luwei <luwei.kang@intel.com>
Signed-off-by: Zhang Yi <yi.z.zhang@intel.com>
Signed-off-by: Xiao Guangrong <guangrong.xiao@linux.intel.com>
Signed-off-by: Wu Hao <hao.wu@intel.com>
Acked-by: Alan Tull <atull@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2018-07-15 13:55:45 +02:00
Wu Hao
5d56e11700 fpga: dfl: add dfl_fpga_cdev_find_port
For feature devices, we need a method to find the port dedicated
to the device. This patch adds a function dfl_fpga_cdev_find_port
for this purpose. e.g. FPGA Management Engine (FME) Partial
Reconfiguration sub feature, it uses this function to find
dedicated port on the device for PR function implementation.

Signed-off-by: Tim Whisonant <tim.whisonant@intel.com>
Signed-off-by: Enno Luebbers <enno.luebbers@intel.com>
Signed-off-by: Shiva Rao <shiva.rao@intel.com>
Signed-off-by: Christopher Rauer <christopher.rauer@intel.com>
Signed-off-by: Xiao Guangrong <guangrong.xiao@linux.intel.com>
Signed-off-by: Wu Hao <hao.wu@intel.com>
Acked-by: Alan Tull <atull@kernel.org>
Acked-by: Moritz Fischer <mdf@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2018-07-15 13:55:45 +02:00
Wu Hao
b16c5147dc fpga: dfl: add chardev support for feature devices
For feature devices drivers, both the FPGA Management Engine (FME) and
Accelerated Function Unit (AFU) driver need to expose user interfaces via
the device file, for example, mmap and ioctls.

This patch adds chardev support in the dfl driver for feature devices,
FME and AFU. It reserves the chardev regions for FME and AFU and provide
interfaces for FME and AFU driver to register their device file operations.

Signed-off-by: Tim Whisonant <tim.whisonant@intel.com>
Signed-off-by: Enno Luebbers <enno.luebbers@intel.com>
Signed-off-by: Shiva Rao <shiva.rao@intel.com>
Signed-off-by: Christopher Rauer <christopher.rauer@intel.com>
Signed-off-by: Zhang Yi <yi.z.zhang@intel.com>
Signed-off-by: Xiao Guangrong <guangrong.xiao@linux.intel.com>
Signed-off-by: Wu Hao <hao.wu@intel.com>
Acked-by: Alan Tull <atull@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2018-07-15 13:55:45 +02:00
Wu Hao
543be3d8c9 fpga: add device feature list support
Device Feature List (DFL) defines a feature list structure that creates
a linked list of feature headers within the MMIO space to provide an
extensible way of adding features. This patch introduces a kernel module
to provide basic infrastructure to support FPGA devices which implement
the Device Feature List.

Usually there will be different features and their sub features linked into
the DFL. This code provides common APIs for feature enumeration, it creates
a container device (FPGA base region), walks through the DFLs and creates
platform devices for feature devices (Currently it only supports two
different feature devices, FPGA Management Engine (FME) and Port which
the Accelerator Function Unit (AFU) connected to). In order to enumerate
the DFLs, the common APIs required low level driver to provide necessary
enumeration information (e.g. address for each device feature list for
given device) and fill it to the dfl_fpga_enum_info data structure. Please
refer to below description for APIs added for enumeration.

Functions for enumeration information preparation:
 *dfl_fpga_enum_info_alloc
   allocate enumeration information data structure.

 *dfl_fpga_enum_info_add_dfl
   add a device feature list to dfl_fpga_enum_info data structure.

 *dfl_fpga_enum_info_free
   free dfl_fpga_enum_info data structure and related resources.

Functions for feature device enumeration:
 *dfl_fpga_feature_devs_enumerate
   enumerate feature devices and return container device.

 *dfl_fpga_feature_devs_remove
   remove feature devices under given container device.

Signed-off-by: Tim Whisonant <tim.whisonant@intel.com>
Signed-off-by: Enno Luebbers <enno.luebbers@intel.com>
Signed-off-by: Shiva Rao <shiva.rao@intel.com>
Signed-off-by: Christopher Rauer <christopher.rauer@intel.com>
Signed-off-by: Zhang Yi <yi.z.zhang@intel.com>
Signed-off-by: Xiao Guangrong <guangrong.xiao@linux.intel.com>
Signed-off-by: Wu Hao <hao.wu@intel.com>
Acked-by: Alan Tull <atull@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2018-07-15 13:55:45 +02:00