linux/drivers
Bart Van Assche 65a558f66c block: Improve performance for BLK_MQ_F_BLOCKING drivers
blk_mq_run_queue() runs the queue asynchronously if BLK_MQ_F_BLOCKING
has been set. This is suboptimal since running the queue asynchronously
is slower than running the queue synchronously. This patch modifies
blk_mq_run_queue() as follows if BLK_MQ_F_BLOCKING has been set:
- Run the queue synchronously if it is allowed to sleep.
- Run the queue asynchronously if it is not allowed to sleep.
Additionally, blk_mq_run_hw_queue(hctx, false) calls are modified into
blk_mq_run_hw_queue(hctx, hctx->flags & BLK_MQ_F_BLOCKING) if the caller
may be invoked from atomic context.

The following caller chains have been reviewed:

blk_mq_run_hw_queue(hctx, false)
  blk_mq_get_tag()      /* may sleep, hence the functions it calls may also sleep */
  blk_execute_rq()             /* may sleep */
  blk_mq_run_hw_queues(q, async=false)
    blk_freeze_queue_start()   /* may sleep */
    blk_mq_requeue_work()      /* may sleep */
    scsi_kick_queue()
      scsi_requeue_run_queue() /* may sleep */
      scsi_run_host_queues()
        scsi_ioctl_reset()     /* may sleep */
  blk_mq_insert_requests(hctx, ctx, list, run_queue_async=false)
    blk_mq_dispatch_plug_list(plug, from_sched=false)
      blk_mq_flush_plug_list(plug, from_schedule=false)
        __blk_flush_plug(plug, from_schedule=false)
	blk_add_rq_to_plug()
	  blk_mq_submit_bio()  /* may sleep if REQ_NOWAIT has not been set */
  blk_mq_plug_issue_direct()
    blk_mq_flush_plug_list()   /* see above */
  blk_mq_dispatch_plug_list(plug, from_sched=false)
    blk_mq_flush_plug_list()   /* see above */
  blk_mq_try_issue_directly()
    blk_mq_submit_bio()        /* may sleep if REQ_NOWAIT has not been set */
  blk_mq_try_issue_list_directly(hctx, list)
    blk_mq_insert_requests() /* see above */

Cc: Christoph Hellwig <hch@lst.de>
Cc: Ming Lei <ming.lei@redhat.com>
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Link: https://lore.kernel.org/r/20230721172731.955724-4-bvanassche@acm.org
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2023-07-24 20:13:12 -06:00
..
accel Linux 6.5-rc1 2023-07-11 09:23:20 +02:00
accessibility
acpi More ACPI updates for 6.5-rc1 2023-07-06 22:25:06 -07:00
amba
android
ata driver core changes for 6.5-rc1 2023-07-03 12:56:23 -07:00
atm
auxdisplay drm changes for 6.5-rc1: 2023-06-29 11:00:17 -07:00
base regmap-irq: Fix out-of-bounds access when allocating config buffers 2023-07-12 12:05:10 +01:00
bcma
block nbd: automatically load module on genl access 2023-07-17 08:18:17 -06:00
bluetooth Including fixes from bluetooth, bpf and wireguard. 2023-07-05 15:44:45 -07:00
bus SoC fixes for 6.5, part 1 2023-07-06 09:56:53 -07:00
cdrom cdrom/gdrom: Fix build error 2023-06-29 08:09:31 -06:00
cdx
char virtio: features, fixes, cleanups 2023-07-03 15:38:26 -07:00
clk Another set of clk driver updates and fixes for the merge window. The 2023-07-04 11:07:45 -07:00
clocksource RISC-V Patches for the 6.5 Merge Window, Part 1 2023-06-30 09:37:26 -07:00
comedi comedi: make all 'class' structures const 2023-06-23 10:29:02 +02:00
connector
counter
cpufreq cpufreq: sparc: Don't mark cpufreq callbacks with __init 2023-07-12 12:45:18 +02:00
cpuidle
crypto s390 updates for 6.5 merge window part 2 2023-07-06 13:18:30 -07:00
cxl cxl: Fix one kernel-doc comment 2023-06-29 16:03:58 -07:00
dax dax: enable dax fault handler to report VM_FAULT_HWPOISON 2023-06-26 07:54:23 -06:00
dca
devfreq
dio
dma dmaengine updates for v6.5 2023-07-06 09:48:31 -07:00
dma-buf Linux 6.5-rc1 2023-07-11 09:23:20 +02:00
edac - Add initial support for RAS hardware found on AMD server GPUs (MI200). 2023-06-26 15:09:18 -07:00
eisa
extcon
firewire firewire: net: fix use after free in fwnet_finish_incoming_packet() 2023-06-24 14:03:46 +09:00
firmware Char/Misc and other driver subsystem updates for 6.5-rc1 2023-07-03 12:46:47 -07:00
fpga Char/Misc and other driver subsystem updates for 6.5-rc1 2023-07-03 12:46:47 -07:00
fsi
gnss
gpio Pin control changes for the v6.5 kernel cycle: 2023-06-30 14:57:19 -07:00
gpu amd-drm-fixes-6.5-2023-07-12: 2023-07-14 13:19:54 +10:00
greybus
hid for-linus-2023071101 2023-07-12 11:56:22 -07:00
hsi
hte
hv x86/hyperv: Fix hyperv_pcpu_input_arg handling when CPUs go online/offline 2023-06-17 23:09:47 +00:00
hwmon driver core changes for 6.5-rc1 2023-07-03 12:56:23 -07:00
hwspinlock
hwtracing Char/Misc and other driver subsystem updates for 6.5-rc1 2023-07-03 12:46:47 -07:00
i2c Part 2 of I2C patches for 6.5 2023-07-08 12:28:00 -07:00
i3c
idle intel_idle: Add __init annotation to matchup_vm_state_with_baremetal() 2023-06-28 19:09:55 +02:00
iio driver core changes for 6.5-rc1 2023-07-03 12:56:23 -07:00
infiniband v6.5 merge window RDMA pull request 2023-06-29 21:01:17 -07:00
input ARM: SoC changes for 6.5 2023-06-29 15:28:33 -07:00
interconnect
iommu iommu/sva: Fix signedness bug in iommu_sva_alloc_pasid() 2023-07-14 14:53:19 +02:00
ipack
irqchip RISC-V Patches for the 6.5 Merge Window, Part 1 2023-06-30 09:37:26 -07:00
isdn
leds - New Drivers 2023-07-03 11:26:05 -07:00
macintosh macintosh: Use of_property_read_reg() to parse "reg" 2023-06-21 14:08:54 +10:00
mailbox mailbox: ti-msgmgr: Fill non-message tx data fields with 0x0 2023-06-30 17:35:45 -05:00
mcb
md hardening fixes for v6.5-rc1 2023-07-08 12:08:39 -07:00
media media updates for v6.5-rc1 2023-07-05 10:42:32 -07:00
memory asm-generic updates for 6.5 2023-07-06 10:06:04 -07:00
memstick
message
mfd - New Drivers 2023-07-03 10:55:04 -07:00
misc Char/Misc and other driver subsystem updates for 6.5-rc1 2023-07-03 12:46:47 -07:00
mmc mmc: Revert "mmc: core: Allow mmc_start_host() synchronously detect a card" 2023-06-30 14:06:51 +02:00
most
mtd Core MTD changes: 2023-06-28 14:02:03 -07:00
mux
net Networking fixes for 6.5-rc2, including fixes from netfilter, 2023-07-13 14:21:22 -07:00
nfc Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2023-06-22 18:40:38 -07:00
ntb ntb: hw: amd: Fix debugfs_create_dir error checking 2023-07-08 13:55:44 -04:00
nubus
nvdimm dax: enable dax fault handler to report VM_FAULT_HWPOISON 2023-06-26 07:54:23 -06:00
nvme SCSI fixes on 20230714 2023-07-14 19:57:29 -07:00
nvmem
of Devicetree updates for v6.5: 2023-06-29 14:58:26 -07:00
opp OPP: Properly propagate error along when failing to get icc_path 2023-06-27 07:35:14 +05:30
parisc parisc: Move init function declarations into header file 2023-06-30 17:14:15 +02:00
parport Char/Misc and other driver subsystem updates for 6.5-rc1 2023-07-03 12:46:47 -07:00
pci Fixes for pci_clean_master, error handling in driver inits, and various 2023-07-09 09:35:51 -07:00
pcmcia
peci
perf perf: RISC-V: Remove PERF_HES_STOPPED flag checking in riscv_pmu_start() 2023-07-12 07:41:23 -07:00
phy phy-for-6.5-v2 2023-07-05 21:38:13 -07:00
pinctrl pinctrl: renesas: Fixes for v6.5 2023-07-13 00:05:52 +02:00
platform platform/x86: touchscreen_dmi: Add info for the Archos 101 Cesium Educ tablet 2023-07-11 12:15:30 +02:00
pnp
power power supply and reset changes for the 6.5 series 2023-07-03 17:23:16 -07:00
powercap ARM: SoC drivers for 6.5 2023-06-29 15:22:19 -07:00
pps
ps3
ptp ptp: Make max_phase_adjustment sysfs device attribute invisible when not supported 2023-07-03 13:17:25 -07:00
pwm pwm: Changes for v6.5-rc1 2023-07-05 12:55:06 -07:00
rapidio
ras
regulator regulator: Fix for v6.5 2023-07-06 19:20:23 -07:00
remoteproc remoteproc updates for v6.5 2023-07-03 17:03:05 -07:00
reset ARM: SoC drivers for 6.5 2023-06-29 15:22:19 -07:00
rpmsg
rtc RTC for 6.5 2023-07-03 10:43:10 -07:00
s390 Networking fixes for 6.5-rc2, including fixes from netfilter, 2023-07-13 14:21:22 -07:00
sbus oradax: make 'cl' a static const structure 2023-06-23 10:27:02 +02:00
scsi block: Improve performance for BLK_MQ_F_BLOCKING drivers 2023-07-24 20:13:12 -06:00
sh
siox
slimbus
soc USB / Thunderbolt driver updates for 6.5-rc1 2023-07-03 13:23:10 -07:00
soundwire soundwire: stream: Make master_list ordered to prevent deadlocks 2023-06-21 16:31:45 +05:30
spi spi: Fixes for v6.5 2023-07-15 08:51:02 -07:00
spmi
ssb
staging media updates for v6.5-rc1 2023-07-05 10:42:32 -07:00
target SCSI misc on 20230708 2023-07-08 12:35:18 -07:00
tc
tee ARM: SoC drivers for 6.5 2023-06-29 15:22:19 -07:00
thermal Thermal control updates for 6.5-rc1 2023-06-26 19:41:26 -07:00
thunderbolt
tty TTY/Serial driver updates for 6.5-rc1. 2023-07-03 13:14:58 -07:00
ufs Merge branch '6.5/scsi-staging' into 6.5/scsi-fixes 2023-07-11 12:15:15 -04:00
uio
usb USB / Thunderbolt driver updates for 6.5-rc1 2023-07-03 13:23:10 -07:00
vdpa virtio: features, fixes, cleanups 2023-07-03 15:38:26 -07:00
vfio VFIO updates for v6.5-rc1 2023-06-30 15:22:09 -07:00
vhost virtio: features, fixes, cleanups 2023-07-03 15:38:26 -07:00
video - New Functionality 2023-07-03 11:31:01 -07:00
virt workqueue: Ordered workqueue creation cleanups 2023-06-27 16:46:06 -07:00
virtio virtio: allow caller to override device DMA mask in vp_modern 2023-06-27 10:47:08 -04:00
vlynq
w1
watchdog watchdog: sp5100_tco: support Hygon FCH/SCH (Server Controller Hub) 2023-06-26 14:30:07 +02:00
xen xen: branch for v6.5-rc2 2023-07-13 13:39:36 -07:00
zorro
Kconfig
Makefile