leandrof/linux
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Apple SoC NVMe driver and dependencies:

Browse Source 
- RTKit IPC library required to boot and communicate with
     co-processors embedded inside Apple SoCs
   - SART DMA address filter required to allow some DMA transactions for
     the NVMe co-processor
   - NVMe platform driver
 
 The following minor changes since v3 on the mailing list have been
 folded in:
 
   - sart: %llx -> %pa for a phys_addr_t
   - rtkit:/sart: Drop IS_ENABLED inside headers
   - rtkit: Use EXPORT_SYMBOL_GPL instead of EXPORT_SYMBOL
   - nvme: Set NVME_REQ_CANCELLED in the timeout handler
   - nvme: Use DEFINE_SIMPLE_DEV_PM_OPS instead of #ifdef CONFIG_PM_SLEEP
 -----BEGIN PGP SIGNATURE-----
 
 iIkEABYKADEWIQRUAfN3OZ14tapALxsd6alOrecRcQUCYm/5lhMcc3ZlbkBzdmVu
 cGV0ZXIuZGV2AAoJEB3pqU6t5xFxITsBAI02bdBvHQAFouu1WzhLzlAPNqjGv3Th
 YFOJL4y2oyigAP9zkIGP0e1JcpApPxvlmQnM340ZY+TCth+ahoCWlS4VAg==
 =b0QX
 -----END PGP SIGNATURE-----
gpgsig -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEEo6/YBQwIrVS28WGKmmx57+YAGNkFAmJ0IIIACgkQmmx57+YA
 GNnxlg/+M0d7awpSyiN2g2aoPN9PzaDiWgjSDJ30THfY+JodFmK/Uw1L2dO32+md
 0UMuMH9Vtiy3NXf/dDoLU+kiNQgT1ZY7Iw2220GZyHxRUC/7RZ/zGNvAb1FclDhK
 YEYtlA2fK1MvlhEcriD8ytP78R80UA7Cx4o5T8CkgfN2OtjLip72aMCFtzRfxMk6
 W0GQvkTpDtp3qn59Wlz/JEJKMT+q0txxq1QkPW93Su190S/7Lk/nd+CkmFDHx54n
 2LFa33JuZZfNbzPMN9emTl7G5Z7Y241zYBMHvtAzd3WdaDZ6I1apeVnDXrNRhysC
 4B8si4qbo9NORhWR96okll0MCiBNZ1UfvS4U+/8A1b0CHwX5rEUKLNOXIfO8xApi
 L3HvVUpYPJw8w3aK2SKGVt2aQcnbluy9+LJQgHByPFbeIV80OjWoR4xSSZi2XErl
 tp4vmyUBTR8g85EnCqcc8OWCdl+yMjX+NmftLk/XB7/xD+HtqKocqizOVho/9u+K
 rv6Ivx6JHk33LFG/jJkN6q8OncgfSRirg0oy6IF/dy2xS2U3OtfaIXHIy9NosC36
 B3bm8mb3CVQJxazMYl3rstBGQoN1YpuwOsHRUWiWG1jDnXFwCSck9b5OH+y64PME
 45V27Rx1ftTgrOWgsllIqsviWADO4Qj7n5YXYvC+hGrBeDnM8pQ=
 =gi38
 -----END PGP SIGNATURE-----

Merge tag 'asahi-soc-rtkit-sart-nvme-for-5.19' of https://github.com/AsahiLinux/linux into arm/drivers

Apple SoC NVMe driver and dependencies:

  - RTKit IPC library required to boot and communicate with
    co-processors embedded inside Apple SoCs
  - SART DMA address filter required to allow some DMA transactions for
    the NVMe co-processor
  - NVMe platform driver

The following minor changes since v3 on the mailing list have been
folded in:

  - sart: %llx -> %pa for a phys_addr_t
  - rtkit:/sart: Drop IS_ENABLED inside headers
  - rtkit: Use EXPORT_SYMBOL_GPL instead of EXPORT_SYMBOL
  - nvme: Set NVME_REQ_CANCELLED in the timeout handler
  - nvme: Use DEFINE_SIMPLE_DEV_PM_OPS instead of #ifdef CONFIG_PM_SLEEP

* tag 'asahi-soc-rtkit-sart-nvme-for-5.19' of https://github.com/AsahiLinux/linux:
  nvme-apple: Add initial Apple SoC NVMe driver
  dt-bindings: nvme: Add Apple ANS NVMe
  soc: apple: Add SART driver
  dt-bindings: iommu: Add Apple SART DMA address filter
  soc: apple: Add RTKit IPC library
  soc: apple: Always include Makefile

Link: https://lore.kernel.org/r/20220505154020.84638-1-sven@svenpeter.dev
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
This commit is contained in:
Arnd Bergmann 2022-05-05 21:07:45 +02:00
commit bdc33dad34
15 changed files with 3517 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

52
Documentation/devicetree/bindings/iommu/apple,sart.yaml Normal file
View File

@ -0,0 +1,52 @@
# SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
%YAML 1.2
---
$id: http://devicetree.org/schemas/iommu/apple,sart.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
title: Apple SART DMA address filter
maintainers:
- Sven Peter <sven@svenpeter.dev>
description:
Apple SART is a simple address filter for DMA transactions. Regions of
physical memory must be added to the SART's allow list before any
DMA can target these. Unlike a proper IOMMU no remapping can be done and
special support in the consumer driver is required since not all DMA
transactions of a single device are subject to SART filtering.
SART1 has first been used since at least the A11 (iPhone 8 and iPhone X)
and allows 36 bit of physical address space and filter entries with sizes
up to 24 bit.
SART2, first seen in A14 and M1, allows 36 bit of physical address space
and filter entry size up to 36 bit.
SART3, first seen in M1 Pro/Max, extends both the address space and filter
entry size to 42 bit.
properties:
compatible:
enum:
- apple,t6000-sart
- apple,t8103-sart
reg:
maxItems: 1
power-domains:
maxItems: 1
required:
- compatible
- reg
additionalProperties: false
examples:
- |
iommu@7bc50000 {
compatible = "apple,t8103-sart";
reg = <0x7bc50000 0x4000>;
};

111
Documentation/devicetree/bindings/nvme/apple,nvme-ans.yaml Normal file
View File

@ -0,0 +1,111 @@
# SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
%YAML 1.2
---
$id: http://devicetree.org/schemas/nvme/apple,nvme-ans.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
title: Apple ANS NVM Express host controller
maintainers:
- Sven Peter <sven@svenpeter.dev>
properties:
compatible:
items:
- enum:
- apple,t8103-nvme-ans2
- apple,t6000-nvme-ans2
- const: apple,nvme-ans2
reg:
items:
- description: NVMe and NVMMU registers
- description: ANS2 co-processor control registers
reg-names:
items:
- const: nvme
- const: ans
resets:
maxItems: 1
power-domains:
# two domains for t8103, three for t6000
minItems: 2
items:
- description: power domain for the NVMe controller.
- description: power domain for the first PCIe bus connecting the NVMe
controller to the storage modules.
- description: optional power domain for the second PCIe bus
connecting the NVMe controller to the storage modules.
power-domain-names:
minItems: 2
items:
- const: ans
- const: apcie0
- const: apcie1
mboxes:
maxItems: 1
description: Mailbox of the ANS2 co-processor
interrupts:
maxItems: 1
apple,sart:
maxItems: 1
$ref: /schemas/types.yaml#/definitions/phandle
description: |
Reference to the SART address filter.
The SART address filter is documented in iommu/apple,sart.yaml.
if:
properties:
compatible:
contains:
const: apple,t8103-nvme-ans2
then:
properties:
power-domains:
maxItems: 2
power-domain-names:
maxItems: 2
else:
properties:
power-domains:
minItems: 3
power-domain-names:
minItems: 3
required:
- compatible
- reg
- reg-names
- resets
- power-domains
- power-domain-names
- mboxes
- interrupts
- apple,sart
additionalProperties: false
examples:
- |
#include <dt-bindings/interrupt-controller/apple-aic.h>
#include <dt-bindings/interrupt-controller/irq.h>
nvme@7bcc0000 {
compatible = "apple,t8103-nvme-ans2", "apple,nvme-ans2";
reg = <0x7bcc0000 0x40000>, <0x77400000 0x4000>;
reg-names = "nvme", "ans";
interrupts = <AIC_IRQ 590 IRQ_TYPE_LEVEL_HIGH>;
mboxes = <&ans>;
apple,sart = <&sart>;
power-domains = <&ps_ans2>, <&ps_apcie_st>;
power-domain-names = "ans", "apcie0";
resets = <&ps_ans2>;
};

4
MAINTAINERS
View File

@ -1837,7 +1837,9 @@ F: Documentation/devicetree/bindings/arm/apple/*
F: Documentation/devicetree/bindings/clock/apple,nco.yaml
F: Documentation/devicetree/bindings/i2c/apple,i2c.yaml
F: Documentation/devicetree/bindings/interrupt-controller/apple,*
F: Documentation/devicetree/bindings/iommu/apple,sart.yaml
F: Documentation/devicetree/bindings/mailbox/apple,mailbox.yaml
F: Documentation/devicetree/bindings/nvme/apple,nvme-ans.yaml
F: Documentation/devicetree/bindings/pci/apple,pcie.yaml
F: Documentation/devicetree/bindings/pinctrl/apple,pinctrl.yaml
F: Documentation/devicetree/bindings/power/apple*
@ -1848,12 +1850,14 @@ F: drivers/i2c/busses/i2c-pasemi-core.c
F: drivers/i2c/busses/i2c-pasemi-platform.c
F: drivers/irqchip/irq-apple-aic.c
F: drivers/mailbox/apple-mailbox.c
F: drivers/nvme/host/apple.c
F: drivers/pinctrl/pinctrl-apple-gpio.c
F: drivers/soc/apple/*
F: drivers/watchdog/apple_wdt.c
F: include/dt-bindings/interrupt-controller/apple-aic.h
F: include/dt-bindings/pinctrl/apple.h
F: include/linux/apple-mailbox.h
F: include/linux/soc/apple/*
ARM/ARTPEC MACHINE SUPPORT
M: Jesper Nilsson <jesper.nilsson@axis.com>

13
drivers/nvme/host/Kconfig
View File

@ -91,3 +91,16 @@ config NVME_TCP
from https://github.com/linux-nvme/nvme-cli.
If unsure, say N.
config NVME_APPLE
tristate "Apple ANS2 NVM Express host driver"
depends on OF && BLOCK
depends on APPLE_RTKIT && APPLE_SART
depends on ARCH_APPLE || COMPILE_TEST
select NVME_CORE
help
This provides support for the NVMe controller embedded in Apple SoCs
such as the M1.
To compile this driver as a module, choose M here: the
module will be called nvme-apple.

3
drivers/nvme/host/Makefile
View File

@ -8,6 +8,7 @@ obj-$(CONFIG_NVME_FABRICS) += nvme-fabrics.o
obj-$(CONFIG_NVME_RDMA) += nvme-rdma.o
obj-$(CONFIG_NVME_FC) += nvme-fc.o
obj-$(CONFIG_NVME_TCP) += nvme-tcp.o
obj-$(CONFIG_NVME_APPLE) += nvme-apple.o
nvme-core-y := core.o ioctl.o constants.o
nvme-core-$(CONFIG_TRACING) += trace.o
@ -25,3 +26,5 @@ nvme-rdma-y += rdma.o
nvme-fc-y += fc.o
nvme-tcp-y += tcp.o
nvme-apple-y += apple.o

1593
drivers/nvme/host/apple.c Normal file
View File

File diff suppressed because it is too large Load Diff

2
drivers/soc/Makefile
View File

@ -4,7 +4,7 @@
#
obj-$(CONFIG_ARCH_ACTIONS) += actions/
obj-$(CONFIG_ARCH_APPLE) += apple/
obj-y += apple/
obj-y += aspeed/
obj-$(CONFIG_ARCH_AT91) += atmel/
obj-y += bcm/

24
drivers/soc/apple/Kconfig
View File

@ -17,6 +17,30 @@ config APPLE_PMGR_PWRSTATE
controls for SoC devices. This driver manages them through the
generic power domain framework, and also provides reset support.
config APPLE_RTKIT
tristate "Apple RTKit co-processor IPC protocol"
depends on MAILBOX
depends on ARCH_APPLE || COMPILE_TEST
default ARCH_APPLE
help
Apple SoCs such as the M1 come with various co-processors running
their proprietary RTKit operating system. This option enables support
for the protocol library used to communicate with those. It is used
by various client drivers.
Say 'y' here if you have an Apple SoC.
config APPLE_SART
tristate "Apple SART DMA address filter"
depends on ARCH_APPLE || COMPILE_TEST
default ARCH_APPLE
help
Apple SART is a simple DMA address filter used on Apple SoCs such
as the M1. It is usually required for the NVMe coprocessor which does
not use a proper IOMMU.
Say 'y' here if you have an Apple SoC.
endmenu
endif

6
drivers/soc/apple/Makefile
View File

@ -1,2 +1,8 @@
# SPDX-License-Identifier: GPL-2.0-only
obj-$(CONFIG_APPLE_PMGR_PWRSTATE) += apple-pmgr-pwrstate.o
obj-$(CONFIG_APPLE_RTKIT) += apple-rtkit.o
apple-rtkit-y = rtkit.o rtkit-crashlog.o
obj-$(CONFIG_APPLE_SART) += apple-sart.o
apple-sart-y = sart.o

154
drivers/soc/apple/rtkit-crashlog.c Normal file
View File

@ -0,0 +1,154 @@
// SPDX-License-Identifier: GPL-2.0-only OR MIT
/*
* Apple RTKit IPC library
* Copyright (C) The Asahi Linux Contributors
*/
#include "rtkit-internal.h"
#define FOURCC(a, b, c, d) \
(((u32)(a) << 24) | ((u32)(b) << 16) | ((u32)(c) << 8) | ((u32)(d)))
#define APPLE_RTKIT_CRASHLOG_HEADER FOURCC('C', 'L', 'H', 'E')
#define APPLE_RTKIT_CRASHLOG_STR FOURCC('C', 's', 't', 'r')
#define APPLE_RTKIT_CRASHLOG_VERSION FOURCC('C', 'v', 'e', 'r')
#define APPLE_RTKIT_CRASHLOG_MBOX FOURCC('C', 'm', 'b', 'x')
#define APPLE_RTKIT_CRASHLOG_TIME FOURCC('C', 't', 'i', 'm')
struct apple_rtkit_crashlog_header {
u32 fourcc;
u32 version;
u32 size;
u32 flags;
u8 _unk[16];
};
static_assert(sizeof(struct apple_rtkit_crashlog_header) == 0x20);
struct apple_rtkit_crashlog_mbox_entry {
u64 msg0;
u64 msg1;
u32 timestamp;
u8 _unk[4];
};
static_assert(sizeof(struct apple_rtkit_crashlog_mbox_entry) == 0x18);
static void apple_rtkit_crashlog_dump_str(struct apple_rtkit *rtk, u8 *bfr,
size_t size)
{
u32 idx;
u8 *ptr, *end;
memcpy(&idx, bfr, 4);
ptr = bfr + 4;
end = bfr + size;
while (ptr < end) {
u8 *newline = memchr(ptr, '\n', end - ptr);
if (newline) {
u8 tmp = *newline;
*newline = '\0';
dev_warn(rtk->dev, "RTKit: Message (id=%x): %s\n", idx,
ptr);
*newline = tmp;
ptr = newline + 1;
} else {
dev_warn(rtk->dev, "RTKit: Message (id=%x): %s", idx,
ptr);
break;
}
}
}
static void apple_rtkit_crashlog_dump_version(struct apple_rtkit *rtk, u8 *bfr,
size_t size)
{
dev_warn(rtk->dev, "RTKit: Version: %s", bfr + 16);
}
static void apple_rtkit_crashlog_dump_time(struct apple_rtkit *rtk, u8 *bfr,
size_t size)
{
u64 crash_time;
memcpy(&crash_time, bfr, 8);
dev_warn(rtk->dev, "RTKit: Crash time: %lld", crash_time);
}
static void apple_rtkit_crashlog_dump_mailbox(struct apple_rtkit *rtk, u8 *bfr,
size_t size)
{
u32 type, index, i;
size_t n_messages;
struct apple_rtkit_crashlog_mbox_entry entry;
memcpy(&type, bfr + 16, 4);
memcpy(&index, bfr + 24, 4);
n_messages = (size - 28) / sizeof(entry);
dev_warn(rtk->dev, "RTKit: Mailbox history (type = %d, index = %d)",
type, index);
for (i = 0; i < n_messages; ++i) {
memcpy(&entry, bfr + 28 + i * sizeof(entry), sizeof(entry));
dev_warn(rtk->dev, "RTKit: #%03d@%08x: %016llx %016llx", i,
entry.timestamp, entry.msg0, entry.msg1);
}
}
void apple_rtkit_crashlog_dump(struct apple_rtkit *rtk, u8 *bfr, size_t size)
{
size_t offset;
u32 section_fourcc, section_size;
struct apple_rtkit_crashlog_header header;
memcpy(&header, bfr, sizeof(header));
if (header.fourcc != APPLE_RTKIT_CRASHLOG_HEADER) {
dev_warn(rtk->dev, "RTKit: Expected crashlog header but got %x",
header.fourcc);
return;
}
if (header.size > size) {
dev_warn(rtk->dev, "RTKit: Crashlog size (%x) is too large",
header.size);
return;
}
size = header.size;
offset = sizeof(header);
while (offset < size) {
memcpy(&section_fourcc, bfr + offset, 4);
memcpy(&section_size, bfr + offset + 12, 4);
switch (section_fourcc) {
case APPLE_RTKIT_CRASHLOG_HEADER:
dev_dbg(rtk->dev, "RTKit: End of crashlog reached");
return;
case APPLE_RTKIT_CRASHLOG_STR:
apple_rtkit_crashlog_dump_str(rtk, bfr + offset + 16,
section_size);
break;
case APPLE_RTKIT_CRASHLOG_VERSION:
apple_rtkit_crashlog_dump_version(
rtk, bfr + offset + 16, section_size);
break;
case APPLE_RTKIT_CRASHLOG_MBOX:
apple_rtkit_crashlog_dump_mailbox(
rtk, bfr + offset + 16, section_size);
break;
case APPLE_RTKIT_CRASHLOG_TIME:
apple_rtkit_crashlog_dump_time(rtk, bfr + offset + 16,
section_size);
break;
default:
dev_warn(rtk->dev,
"RTKit: Unknown crashlog section: %x",
section_fourcc);
}
offset += section_size;
}
dev_warn(rtk->dev,
"RTKit: End of crashlog reached but no footer present");
}

62
drivers/soc/apple/rtkit-internal.h Normal file
View File

@ -0,0 +1,62 @@
/* SPDX-License-Identifier: GPL-2.0-only OR MIT */
/*
* Apple RTKit IPC library
* Copyright (C) The Asahi Linux Contributors
*/
#ifndef _APPLE_RTKIT_INTERAL_H
#define _APPLE_RTKIT_INTERAL_H
#include <linux/apple-mailbox.h>
#include <linux/bitfield.h>
#include <linux/bitmap.h>
#include <linux/completion.h>
#include <linux/dma-mapping.h>
#include <linux/io.h>
#include <linux/kernel.h>
#include <linux/mailbox_client.h>
#include <linux/module.h>
#include <linux/slab.h>
#include <linux/soc/apple/rtkit.h>
#include <linux/workqueue.h>
#define APPLE_RTKIT_APP_ENDPOINT_START 0x20
#define APPLE_RTKIT_MAX_ENDPOINTS 0x100
struct apple_rtkit {
void *cookie;
const struct apple_rtkit_ops *ops;
struct device *dev;
const char *mbox_name;
int mbox_idx;
struct mbox_client mbox_cl;
struct mbox_chan *mbox_chan;
struct completion epmap_completion;
struct completion iop_pwr_ack_completion;
struct completion ap_pwr_ack_completion;
int boot_result;
int version;
unsigned int iop_power_state;
unsigned int ap_power_state;
bool crashed;
DECLARE_BITMAP(endpoints, APPLE_RTKIT_MAX_ENDPOINTS);
struct apple_rtkit_shmem ioreport_buffer;
struct apple_rtkit_shmem crashlog_buffer;
struct apple_rtkit_shmem syslog_buffer;
char *syslog_msg_buffer;
size_t syslog_n_entries;
size_t syslog_msg_size;
struct workqueue_struct *wq;
};
void apple_rtkit_crashlog_dump(struct apple_rtkit *rtk, u8 *bfr, size_t size);
#endif

958
drivers/soc/apple/rtkit.c Normal file
View File

@ -0,0 +1,958 @@
// SPDX-License-Identifier: GPL-2.0-only OR MIT
/*
* Apple RTKit IPC library
* Copyright (C) The Asahi Linux Contributors
*/
#include "rtkit-internal.h"
enum {
APPLE_RTKIT_PWR_STATE_OFF = 0x00, /* power off, cannot be restarted */
APPLE_RTKIT_PWR_STATE_SLEEP = 0x01, /* sleeping, can be restarted */
APPLE_RTKIT_PWR_STATE_QUIESCED = 0x10, /* running but no communication */
APPLE_RTKIT_PWR_STATE_ON = 0x20, /* normal operating state */
};
enum {
APPLE_RTKIT_EP_MGMT = 0,
APPLE_RTKIT_EP_CRASHLOG = 1,
APPLE_RTKIT_EP_SYSLOG = 2,
APPLE_RTKIT_EP_DEBUG = 3,
APPLE_RTKIT_EP_IOREPORT = 4,
APPLE_RTKIT_EP_OSLOG = 8,
};
#define APPLE_RTKIT_MGMT_TYPE GENMASK_ULL(59, 52)
enum {
APPLE_RTKIT_MGMT_HELLO = 1,
APPLE_RTKIT_MGMT_HELLO_REPLY = 2,
APPLE_RTKIT_MGMT_STARTEP = 5,
APPLE_RTKIT_MGMT_SET_IOP_PWR_STATE = 6,
APPLE_RTKIT_MGMT_SET_IOP_PWR_STATE_ACK = 7,
APPLE_RTKIT_MGMT_EPMAP = 8,
APPLE_RTKIT_MGMT_EPMAP_REPLY = 8,
APPLE_RTKIT_MGMT_SET_AP_PWR_STATE = 0xb,
APPLE_RTKIT_MGMT_SET_AP_PWR_STATE_ACK = 0xb,
};
#define APPLE_RTKIT_MGMT_HELLO_MINVER GENMASK_ULL(15, 0)
#define APPLE_RTKIT_MGMT_HELLO_MAXVER GENMASK_ULL(31, 16)
#define APPLE_RTKIT_MGMT_EPMAP_LAST BIT_ULL(51)
#define APPLE_RTKIT_MGMT_EPMAP_BASE GENMASK_ULL(34, 32)
#define APPLE_RTKIT_MGMT_EPMAP_BITMAP GENMASK_ULL(31, 0)
#define APPLE_RTKIT_MGMT_EPMAP_REPLY_MORE BIT_ULL(0)
#define APPLE_RTKIT_MGMT_STARTEP_EP GENMASK_ULL(39, 32)
#define APPLE_RTKIT_MGMT_STARTEP_FLAG BIT_ULL(1)
#define APPLE_RTKIT_MGMT_PWR_STATE GENMASK_ULL(15, 0)
#define APPLE_RTKIT_CRASHLOG_CRASH 1
#define APPLE_RTKIT_BUFFER_REQUEST 1
#define APPLE_RTKIT_BUFFER_REQUEST_SIZE GENMASK_ULL(51, 44)
#define APPLE_RTKIT_BUFFER_REQUEST_IOVA GENMASK_ULL(41, 0)
#define APPLE_RTKIT_SYSLOG_TYPE GENMASK_ULL(59, 52)
#define APPLE_RTKIT_SYSLOG_LOG 5
#define APPLE_RTKIT_SYSLOG_INIT 8
#define APPLE_RTKIT_SYSLOG_N_ENTRIES GENMASK_ULL(7, 0)
#define APPLE_RTKIT_SYSLOG_MSG_SIZE GENMASK_ULL(31, 24)
#define APPLE_RTKIT_OSLOG_TYPE GENMASK_ULL(63, 56)
#define APPLE_RTKIT_OSLOG_INIT 1
#define APPLE_RTKIT_OSLOG_ACK 3
#define APPLE_RTKIT_MIN_SUPPORTED_VERSION 11
#define APPLE_RTKIT_MAX_SUPPORTED_VERSION 12
struct apple_rtkit_msg {
struct completion *completion;
struct apple_mbox_msg mbox_msg;
};
struct apple_rtkit_rx_work {
struct apple_rtkit *rtk;
u8 ep;
u64 msg;
struct work_struct work;
};
bool apple_rtkit_is_running(struct apple_rtkit *rtk)
{
if (rtk->crashed)
return false;
if ((rtk->iop_power_state & 0xff) != APPLE_RTKIT_PWR_STATE_ON)
return false;
if ((rtk->ap_power_state & 0xff) != APPLE_RTKIT_PWR_STATE_ON)
return false;
return true;
}
EXPORT_SYMBOL_GPL(apple_rtkit_is_running);
bool apple_rtkit_is_crashed(struct apple_rtkit *rtk)
{
return rtk->crashed;
}
EXPORT_SYMBOL_GPL(apple_rtkit_is_crashed);
static void apple_rtkit_management_send(struct apple_rtkit *rtk, u8 type,
u64 msg)
{
msg &= ~APPLE_RTKIT_MGMT_TYPE;
msg |= FIELD_PREP(APPLE_RTKIT_MGMT_TYPE, type);
apple_rtkit_send_message(rtk, APPLE_RTKIT_EP_MGMT, msg, NULL, false);
}
static void apple_rtkit_management_rx_hello(struct apple_rtkit *rtk, u64 msg)
{
u64 reply;
int min_ver = FIELD_GET(APPLE_RTKIT_MGMT_HELLO_MINVER, msg);
int max_ver = FIELD_GET(APPLE_RTKIT_MGMT_HELLO_MAXVER, msg);
int want_ver = min(APPLE_RTKIT_MAX_SUPPORTED_VERSION, max_ver);
dev_dbg(rtk->dev, "RTKit: Min ver %d, max ver %d\n", min_ver, max_ver);
if (min_ver > APPLE_RTKIT_MAX_SUPPORTED_VERSION) {
dev_err(rtk->dev, "RTKit: Firmware min version %d is too new\n",
min_ver);
goto abort_boot;
}
if (max_ver < APPLE_RTKIT_MIN_SUPPORTED_VERSION) {
dev_err(rtk->dev, "RTKit: Firmware max version %d is too old\n",
max_ver);
goto abort_boot;
}
dev_info(rtk->dev, "RTKit: Initializing (protocol version %d)\n",
want_ver);
rtk->version = want_ver;
reply = FIELD_PREP(APPLE_RTKIT_MGMT_HELLO_MINVER, want_ver);
reply |= FIELD_PREP(APPLE_RTKIT_MGMT_HELLO_MAXVER, want_ver);
apple_rtkit_management_send(rtk, APPLE_RTKIT_MGMT_HELLO_REPLY, reply);
return;
abort_boot:
rtk->boot_result = -EINVAL;
complete_all(&rtk->epmap_completion);
}
static void apple_rtkit_management_rx_epmap(struct apple_rtkit *rtk, u64 msg)
{
int i, ep;
u64 reply;
unsigned long bitmap = FIELD_GET(APPLE_RTKIT_MGMT_EPMAP_BITMAP, msg);
u32 base = FIELD_GET(APPLE_RTKIT_MGMT_EPMAP_BASE, msg);
dev_dbg(rtk->dev,
"RTKit: received endpoint bitmap 0x%lx with base 0x%x\n",
bitmap, base);
for_each_set_bit(i, &bitmap, 32) {
ep = 32 * base + i;
dev_dbg(rtk->dev, "RTKit: Discovered endpoint 0x%02x\n", ep);
set_bit(ep, rtk->endpoints);
}
reply = FIELD_PREP(APPLE_RTKIT_MGMT_EPMAP_BASE, base);
if (msg & APPLE_RTKIT_MGMT_EPMAP_LAST)
reply |= APPLE_RTKIT_MGMT_EPMAP_LAST;
else
reply |= APPLE_RTKIT_MGMT_EPMAP_REPLY_MORE;
apple_rtkit_management_send(rtk, APPLE_RTKIT_MGMT_EPMAP_REPLY, reply);
if (!(msg & APPLE_RTKIT_MGMT_EPMAP_LAST))
return;
for_each_set_bit(ep, rtk->endpoints, APPLE_RTKIT_APP_ENDPOINT_START) {
switch (ep) {
/* the management endpoint is started by default */
case APPLE_RTKIT_EP_MGMT:
break;
/* without starting these RTKit refuses to boot */
case APPLE_RTKIT_EP_SYSLOG:
case APPLE_RTKIT_EP_CRASHLOG:
case APPLE_RTKIT_EP_DEBUG:
case APPLE_RTKIT_EP_IOREPORT:
case APPLE_RTKIT_EP_OSLOG:
dev_dbg(rtk->dev,
"RTKit: Starting system endpoint 0x%02x\n", ep);
apple_rtkit_start_ep(rtk, ep);
break;
default:
dev_warn(rtk->dev,
"RTKit: Unknown system endpoint: 0x%02x\n",
ep);
}
}
rtk->boot_result = 0;
complete_all(&rtk->epmap_completion);
}
static void apple_rtkit_management_rx_iop_pwr_ack(struct apple_rtkit *rtk,
u64 msg)
{
unsigned int new_state = FIELD_GET(APPLE_RTKIT_MGMT_PWR_STATE, msg);
dev_dbg(rtk->dev, "RTKit: IOP power state transition: 0x%x -> 0x%x\n",
rtk->iop_power_state, new_state);
rtk->iop_power_state = new_state;
complete_all(&rtk->iop_pwr_ack_completion);
}
static void apple_rtkit_management_rx_ap_pwr_ack(struct apple_rtkit *rtk,
u64 msg)
{
unsigned int new_state = FIELD_GET(APPLE_RTKIT_MGMT_PWR_STATE, msg);
dev_dbg(rtk->dev, "RTKit: AP power state transition: 0x%x -> 0x%x\n",
rtk->ap_power_state, new_state);
rtk->ap_power_state = new_state;
complete_all(&rtk->ap_pwr_ack_completion);
}
static void apple_rtkit_management_rx(struct apple_rtkit *rtk, u64 msg)
{
u8 type = FIELD_GET(APPLE_RTKIT_MGMT_TYPE, msg);
switch (type) {
case APPLE_RTKIT_MGMT_HELLO:
apple_rtkit_management_rx_hello(rtk, msg);
break;
case APPLE_RTKIT_MGMT_EPMAP:
apple_rtkit_management_rx_epmap(rtk, msg);
break;
case APPLE_RTKIT_MGMT_SET_IOP_PWR_STATE_ACK:
apple_rtkit_management_rx_iop_pwr_ack(rtk, msg);
break;
case APPLE_RTKIT_MGMT_SET_AP_PWR_STATE_ACK:
apple_rtkit_management_rx_ap_pwr_ack(rtk, msg);
break;
default:
dev_warn(
rtk->dev,
"RTKit: unknown management message: 0x%llx (type: 0x%02x)\n",
msg, type);
}
}
static int apple_rtkit_common_rx_get_buffer(struct apple_rtkit *rtk,
struct apple_rtkit_shmem *buffer,
u8 ep, u64 msg)
{
size_t n_4kpages = FIELD_GET(APPLE_RTKIT_BUFFER_REQUEST_SIZE, msg);
u64 reply;
int err;
buffer->buffer = NULL;
buffer->iomem = NULL;
buffer->is_mapped = false;
buffer->iova = FIELD_GET(APPLE_RTKIT_BUFFER_REQUEST_IOVA, msg);
buffer->size = n_4kpages << 12;
dev_dbg(rtk->dev, "RTKit: buffer request for 0x%zx bytes at %pad\n",
buffer->size, &buffer->iova);
if (buffer->iova &&
(!rtk->ops->shmem_setup || !rtk->ops->shmem_destroy)) {
err = -EINVAL;
goto error;
}
if (rtk->ops->shmem_setup) {
err = rtk->ops->shmem_setup(rtk->cookie, buffer);
if (err)
goto error;
} else {
buffer->buffer = dma_alloc_coherent(rtk->dev, buffer->size,
&buffer->iova, GFP_KERNEL);
if (!buffer->buffer) {
err = -ENOMEM;
goto error;
}
}
if (!buffer->is_mapped) {
reply = FIELD_PREP(APPLE_RTKIT_SYSLOG_TYPE,
APPLE_RTKIT_BUFFER_REQUEST);
reply |= FIELD_PREP(APPLE_RTKIT_BUFFER_REQUEST_SIZE, n_4kpages);
reply |= FIELD_PREP(APPLE_RTKIT_BUFFER_REQUEST_IOVA,
buffer->iova);
apple_rtkit_send_message(rtk, ep, reply, NULL, false);
}
return 0;
error:
buffer->buffer = NULL;
buffer->iomem = NULL;
buffer->iova = 0;
buffer->size = 0;
buffer->is_mapped = false;
return err;
}
static void apple_rtkit_free_buffer(struct apple_rtkit *rtk,
struct apple_rtkit_shmem *bfr)
{
if (bfr->size == 0)
return;
if (rtk->ops->shmem_destroy)
rtk->ops->shmem_destroy(rtk->cookie, bfr);
else if (bfr->buffer)
dma_free_coherent(rtk->dev, bfr->size, bfr->buffer, bfr->iova);
bfr->buffer = NULL;
bfr->iomem = NULL;
bfr->iova = 0;
bfr->size = 0;
bfr->is_mapped = false;
}
static void apple_rtkit_memcpy(struct apple_rtkit *rtk, void *dst,
struct apple_rtkit_shmem *bfr, size_t offset,
size_t len)
{
if (bfr->iomem)
memcpy_fromio(dst, bfr->iomem + offset, len);
else
memcpy(dst, bfr->buffer + offset, len);
}
static void apple_rtkit_crashlog_rx(struct apple_rtkit *rtk, u64 msg)
{
u8 type = FIELD_GET(APPLE_RTKIT_SYSLOG_TYPE, msg);
u8 *bfr;
if (type != APPLE_RTKIT_CRASHLOG_CRASH) {
dev_warn(rtk->dev, "RTKit: Unknown crashlog message: %llx\n",
msg);
return;
}
if (!rtk->crashlog_buffer.size) {
apple_rtkit_common_rx_get_buffer(rtk, &rtk->crashlog_buffer,
APPLE_RTKIT_EP_CRASHLOG, msg);
return;
}
dev_err(rtk->dev, "RTKit: co-processor has crashed\n");
/*
* create a shadow copy here to make sure the co-processor isn't able
* to change the log while we're dumping it. this also ensures
* the buffer is in normal memory and not iomem for e.g. the SMC
*/
bfr = kzalloc(rtk->crashlog_buffer.size, GFP_KERNEL);
if (bfr) {
apple_rtkit_memcpy(rtk, bfr, &rtk->crashlog_buffer, 0,
rtk->crashlog_buffer.size);
apple_rtkit_crashlog_dump(rtk, bfr, rtk->crashlog_buffer.size);
kfree(bfr);
} else {
dev_err(rtk->dev,
"RTKit: Couldn't allocate crashlog shadow buffer\n");
}
rtk->crashed = true;
if (rtk->ops->crashed)
rtk->ops->crashed(rtk->cookie);
}
static void apple_rtkit_ioreport_rx(struct apple_rtkit *rtk, u64 msg)
{
u8 type = FIELD_GET(APPLE_RTKIT_SYSLOG_TYPE, msg);
switch (type) {
case APPLE_RTKIT_BUFFER_REQUEST:
apple_rtkit_common_rx_get_buffer(rtk, &rtk->ioreport_buffer,
APPLE_RTKIT_EP_IOREPORT, msg);
break;
/* unknown, must be ACKed or the co-processor will hang */
case 0x8:
case 0xc:
apple_rtkit_send_message(rtk, APPLE_RTKIT_EP_IOREPORT, msg,
NULL, false);
break;
default:
dev_warn(rtk->dev, "RTKit: Unknown ioreport message: %llx\n",
msg);
}
}
static void apple_rtkit_syslog_rx_init(struct apple_rtkit *rtk, u64 msg)
{
rtk->syslog_n_entries = FIELD_GET(APPLE_RTKIT_SYSLOG_N_ENTRIES, msg);
rtk->syslog_msg_size = FIELD_GET(APPLE_RTKIT_SYSLOG_MSG_SIZE, msg);
rtk->syslog_msg_buffer = kzalloc(rtk->syslog_msg_size, GFP_KERNEL);
dev_dbg(rtk->dev,
"RTKit: syslog initialized: entries: %zd, msg_size: %zd\n",
rtk->syslog_n_entries, rtk->syslog_msg_size);
}
static void apple_rtkit_syslog_rx_log(struct apple_rtkit *rtk, u64 msg)
{
u8 idx = msg & 0xff;
char log_context[24];
size_t entry_size = 0x20 + rtk->syslog_msg_size;
if (!rtk->syslog_msg_buffer) {
dev_warn(
rtk->dev,
"RTKit: received syslog message but no syslog_msg_buffer\n");
goto done;
}
if (!rtk->syslog_buffer.size) {
dev_warn(
rtk->dev,
"RTKit: received syslog message but syslog_buffer.size is zero\n");
goto done;
}
if (!rtk->syslog_buffer.buffer && !rtk->syslog_buffer.iomem) {
dev_warn(
rtk->dev,
"RTKit: received syslog message but no syslog_buffer.buffer or syslog_buffer.iomem\n");
goto done;
}
if (idx > rtk->syslog_n_entries) {
dev_warn(rtk->dev, "RTKit: syslog index %d out of range\n",
idx);
goto done;
}
apple_rtkit_memcpy(rtk, log_context, &rtk->syslog_buffer,
idx * entry_size + 8, sizeof(log_context));
apple_rtkit_memcpy(rtk, rtk->syslog_msg_buffer, &rtk->syslog_buffer,
idx * entry_size + 8 + sizeof(log_context),
rtk->syslog_msg_size);
log_context[sizeof(log_context) - 1] = 0;
rtk->syslog_msg_buffer[rtk->syslog_msg_size - 1] = 0;
dev_info(rtk->dev, "RTKit: syslog message: %s: %s\n", log_context,
rtk->syslog_msg_buffer);
done:
apple_rtkit_send_message(rtk, APPLE_RTKIT_EP_SYSLOG, msg, NULL, false);
}
static void apple_rtkit_syslog_rx(struct apple_rtkit *rtk, u64 msg)
{
u8 type = FIELD_GET(APPLE_RTKIT_SYSLOG_TYPE, msg);
switch (type) {
case APPLE_RTKIT_BUFFER_REQUEST:
apple_rtkit_common_rx_get_buffer(rtk, &rtk->syslog_buffer,
APPLE_RTKIT_EP_SYSLOG, msg);
break;
case APPLE_RTKIT_SYSLOG_INIT:
apple_rtkit_syslog_rx_init(rtk, msg);
break;
case APPLE_RTKIT_SYSLOG_LOG:
apple_rtkit_syslog_rx_log(rtk, msg);
break;
default:
dev_warn(rtk->dev, "RTKit: Unknown syslog message: %llx\n",
msg);
}
}
static void apple_rtkit_oslog_rx_init(struct apple_rtkit *rtk, u64 msg)
{
u64 ack;
dev_dbg(rtk->dev, "RTKit: oslog init: msg: 0x%llx\n", msg);
ack = FIELD_PREP(APPLE_RTKIT_OSLOG_TYPE, APPLE_RTKIT_OSLOG_ACK);
apple_rtkit_send_message(rtk, APPLE_RTKIT_EP_OSLOG, ack, NULL, false);
}
static void apple_rtkit_oslog_rx(struct apple_rtkit *rtk, u64 msg)
{
u8 type = FIELD_GET(APPLE_RTKIT_OSLOG_TYPE, msg);
switch (type) {
case APPLE_RTKIT_OSLOG_INIT:
apple_rtkit_oslog_rx_init(rtk, msg);
break;
default:
dev_warn(rtk->dev, "RTKit: Unknown oslog message: %llx\n", msg);
}
}
static void apple_rtkit_rx_work(struct work_struct *work)
{
struct apple_rtkit_rx_work *rtk_work =
container_of(work, struct apple_rtkit_rx_work, work);
struct apple_rtkit *rtk = rtk_work->rtk;
switch (rtk_work->ep) {
case APPLE_RTKIT_EP_MGMT:
apple_rtkit_management_rx(rtk, rtk_work->msg);
break;
case APPLE_RTKIT_EP_CRASHLOG:
apple_rtkit_crashlog_rx(rtk, rtk_work->msg);
break;
case APPLE_RTKIT_EP_SYSLOG:
apple_rtkit_syslog_rx(rtk, rtk_work->msg);
break;
case APPLE_RTKIT_EP_IOREPORT:
apple_rtkit_ioreport_rx(rtk, rtk_work->msg);
break;
case APPLE_RTKIT_EP_OSLOG:
apple_rtkit_oslog_rx(rtk, rtk_work->msg);
break;
case APPLE_RTKIT_APP_ENDPOINT_START ... 0xff:
if (rtk->ops->recv_message)
rtk->ops->recv_message(rtk->cookie, rtk_work->ep,
rtk_work->msg);
else
dev_warn(
rtk->dev,
"Received unexpected message to EP%02d: %llx\n",
rtk_work->ep, rtk_work->msg);
break;
default:
dev_warn(rtk->dev,
"RTKit: message to unknown endpoint %02x: %llx\n",
rtk_work->ep, rtk_work->msg);
}
kfree(rtk_work);
}
static void apple_rtkit_rx(struct mbox_client *cl, void *mssg)
{
struct apple_rtkit *rtk = container_of(cl, struct apple_rtkit, mbox_cl);
struct apple_mbox_msg *msg = mssg;
struct apple_rtkit_rx_work *work;
u8 ep = msg->msg1;
/*
* The message was read from a MMIO FIFO and we have to make
* sure all reads from buffers sent with that message happen
* afterwards.
*/
dma_rmb();
if (!test_bit(ep, rtk->endpoints))
dev_warn(rtk->dev,
"RTKit: Message to undiscovered endpoint 0x%02x\n",
ep);
if (ep >= APPLE_RTKIT_APP_ENDPOINT_START &&
rtk->ops->recv_message_early &&
rtk->ops->recv_message_early(rtk->cookie, ep, msg->msg0))
return;
work = kzalloc(sizeof(*work), GFP_ATOMIC);
if (!work)
return;
work->rtk = rtk;
work->ep = ep;
work->msg = msg->msg0;
INIT_WORK(&work->work, apple_rtkit_rx_work);
queue_work(rtk->wq, &work->work);
}
static void apple_rtkit_tx_done(struct mbox_client *cl, void *mssg, int r)
{
struct apple_rtkit_msg *msg =
container_of(mssg, struct apple_rtkit_msg, mbox_msg);
if (r == -ETIME)
return;
if (msg->completion)
complete(msg->completion);
kfree(msg);
}
int apple_rtkit_send_message(struct apple_rtkit *rtk, u8 ep, u64 message,
struct completion *completion, bool atomic)
{
struct apple_rtkit_msg *msg;
int ret;
gfp_t flags;
if (rtk->crashed)
return -EINVAL;
if (ep >= APPLE_RTKIT_APP_ENDPOINT_START &&
!apple_rtkit_is_running(rtk))
return -EINVAL;
if (atomic)
flags = GFP_ATOMIC;
else
flags = GFP_KERNEL;
msg = kzalloc(sizeof(*msg), flags);
if (!msg)
return -ENOMEM;
msg->mbox_msg.msg0 = message;
msg->mbox_msg.msg1 = ep;
msg->completion = completion;
/*
* The message will be sent with a MMIO write. We need the barrier
* here to ensure any previous writes to buffers are visible to the
* device before that MMIO write happens.
*/
dma_wmb();
ret = mbox_send_message(rtk->mbox_chan, &msg->mbox_msg);
if (ret < 0) {
kfree(msg);
return ret;
}
return 0;
}
EXPORT_SYMBOL_GPL(apple_rtkit_send_message);
int apple_rtkit_send_message_wait(struct apple_rtkit *rtk, u8 ep, u64 message,
unsigned long timeout, bool atomic)
{
DECLARE_COMPLETION_ONSTACK(completion);
int ret;
long t;
ret = apple_rtkit_send_message(rtk, ep, message, &completion, atomic);
if (ret < 0)
return ret;
if (atomic) {
ret = mbox_flush(rtk->mbox_chan, timeout);
if (ret < 0)
return ret;
if (try_wait_for_completion(&completion))
return 0;
return -ETIME;
} else {
t = wait_for_completion_interruptible_timeout(
&completion, msecs_to_jiffies(timeout));
if (t < 0)
return t;
else if (t == 0)
return -ETIME;
return 0;
}
}
EXPORT_SYMBOL_GPL(apple_rtkit_send_message_wait);
int apple_rtkit_start_ep(struct apple_rtkit *rtk, u8 endpoint)
{
u64 msg;
if (!test_bit(endpoint, rtk->endpoints))
return -EINVAL;
if (endpoint >= APPLE_RTKIT_APP_ENDPOINT_START &&
!apple_rtkit_is_running(rtk))
return -EINVAL;
msg = FIELD_PREP(APPLE_RTKIT_MGMT_STARTEP_EP, endpoint);
msg |= APPLE_RTKIT_MGMT_STARTEP_FLAG;
apple_rtkit_management_send(rtk, APPLE_RTKIT_MGMT_STARTEP, msg);
return 0;
}
EXPORT_SYMBOL_GPL(apple_rtkit_start_ep);
static int apple_rtkit_request_mbox_chan(struct apple_rtkit *rtk)
{
if (rtk->mbox_name)
rtk->mbox_chan = mbox_request_channel_byname(&rtk->mbox_cl,
rtk->mbox_name);
else
rtk->mbox_chan =
mbox_request_channel(&rtk->mbox_cl, rtk->mbox_idx);
if (IS_ERR(rtk->mbox_chan))
return PTR_ERR(rtk->mbox_chan);
return 0;
}
static struct apple_rtkit *apple_rtkit_init(struct device *dev, void *cookie,
const char *mbox_name, int mbox_idx,
const struct apple_rtkit_ops *ops)
{
struct apple_rtkit *rtk;
int ret;
if (!ops)
return ERR_PTR(-EINVAL);
rtk = kzalloc(sizeof(*rtk), GFP_KERNEL);
if (!rtk)
return ERR_PTR(-ENOMEM);
rtk->dev = dev;
rtk->cookie = cookie;
rtk->ops = ops;
init_completion(&rtk->epmap_completion);
init_completion(&rtk->iop_pwr_ack_completion);
init_completion(&rtk->ap_pwr_ack_completion);
bitmap_zero(rtk->endpoints, APPLE_RTKIT_MAX_ENDPOINTS);
set_bit(APPLE_RTKIT_EP_MGMT, rtk->endpoints);
rtk->mbox_name = mbox_name;
rtk->mbox_idx = mbox_idx;
rtk->mbox_cl.dev = dev;
rtk->mbox_cl.tx_block = false;
rtk->mbox_cl.knows_txdone = false;
rtk->mbox_cl.rx_callback = &apple_rtkit_rx;
rtk->mbox_cl.tx_done = &apple_rtkit_tx_done;
rtk->wq = alloc_ordered_workqueue("rtkit-%s", WQ_MEM_RECLAIM,
dev_name(rtk->dev));
if (!rtk->wq) {
ret = -ENOMEM;
goto free_rtk;
}
ret = apple_rtkit_request_mbox_chan(rtk);
if (ret)
goto destroy_wq;
return rtk;
destroy_wq:
destroy_workqueue(rtk->wq);
free_rtk:
kfree(rtk);
return ERR_PTR(ret);
}
static int apple_rtkit_wait_for_completion(struct completion *c)
{
long t;
t = wait_for_completion_interruptible_timeout(c,
msecs_to_jiffies(1000));
if (t < 0)
return t;
else if (t == 0)
return -ETIME;
else
return 0;
}
int apple_rtkit_reinit(struct apple_rtkit *rtk)
{
/* make sure we don't handle any messages while reinitializing */
mbox_free_channel(rtk->mbox_chan);
flush_workqueue(rtk->wq);
apple_rtkit_free_buffer(rtk, &rtk->ioreport_buffer);
apple_rtkit_free_buffer(rtk, &rtk->crashlog_buffer);
apple_rtkit_free_buffer(rtk, &rtk->syslog_buffer);
kfree(rtk->syslog_msg_buffer);
rtk->syslog_msg_buffer = NULL;
rtk->syslog_n_entries = 0;
rtk->syslog_msg_size = 0;
bitmap_zero(rtk->endpoints, APPLE_RTKIT_MAX_ENDPOINTS);
set_bit(APPLE_RTKIT_EP_MGMT, rtk->endpoints);
reinit_completion(&rtk->epmap_completion);
reinit_completion(&rtk->iop_pwr_ack_completion);
reinit_completion(&rtk->ap_pwr_ack_completion);
rtk->crashed = false;
rtk->iop_power_state = APPLE_RTKIT_PWR_STATE_OFF;
rtk->ap_power_state = APPLE_RTKIT_PWR_STATE_OFF;
return apple_rtkit_request_mbox_chan(rtk);
}
EXPORT_SYMBOL_GPL(apple_rtkit_reinit);
static int apple_rtkit_set_ap_power_state(struct apple_rtkit *rtk,
unsigned int state)
{
u64 msg;
int ret;
reinit_completion(&rtk->ap_pwr_ack_completion);
msg = FIELD_PREP(APPLE_RTKIT_MGMT_PWR_STATE, state);
apple_rtkit_management_send(rtk, APPLE_RTKIT_MGMT_SET_AP_PWR_STATE,
msg);
ret = apple_rtkit_wait_for_completion(&rtk->ap_pwr_ack_completion);
if (ret)
return ret;
if (rtk->ap_power_state != state)
return -EINVAL;
return 0;
}
static int apple_rtkit_set_iop_power_state(struct apple_rtkit *rtk,
unsigned int state)
{
u64 msg;
int ret;
reinit_completion(&rtk->iop_pwr_ack_completion);
msg = FIELD_PREP(APPLE_RTKIT_MGMT_PWR_STATE, state);
apple_rtkit_management_send(rtk, APPLE_RTKIT_MGMT_SET_IOP_PWR_STATE,
msg);
ret = apple_rtkit_wait_for_completion(&rtk->iop_pwr_ack_completion);
if (ret)
return ret;
if (rtk->iop_power_state != state)
return -EINVAL;
return 0;
}
int apple_rtkit_boot(struct apple_rtkit *rtk)
{
int ret;
if (apple_rtkit_is_running(rtk))
return 0;
if (rtk->crashed)
return -EINVAL;
dev_dbg(rtk->dev, "RTKit: waiting for boot to finish\n");
ret = apple_rtkit_wait_for_completion(&rtk->epmap_completion);
if (ret)
return ret;
if (rtk->boot_result)
return rtk->boot_result;
dev_dbg(rtk->dev, "RTKit: waiting for IOP power state ACK\n");
ret = apple_rtkit_wait_for_completion(&rtk->iop_pwr_ack_completion);
if (ret)
return ret;
return apple_rtkit_set_ap_power_state(rtk, APPLE_RTKIT_PWR_STATE_ON);
}
EXPORT_SYMBOL_GPL(apple_rtkit_boot);
int apple_rtkit_shutdown(struct apple_rtkit *rtk)
{
int ret;
/* if OFF is used here the co-processor will not wake up again */
ret = apple_rtkit_set_ap_power_state(rtk,
APPLE_RTKIT_PWR_STATE_QUIESCED);
if (ret)
return ret;
ret = apple_rtkit_set_iop_power_state(rtk, APPLE_RTKIT_PWR_STATE_SLEEP);
if (ret)
return ret;
return apple_rtkit_reinit(rtk);
}
EXPORT_SYMBOL_GPL(apple_rtkit_shutdown);
int apple_rtkit_quiesce(struct apple_rtkit *rtk)
{
int ret;
ret = apple_rtkit_set_ap_power_state(rtk,
APPLE_RTKIT_PWR_STATE_QUIESCED);
if (ret)
return ret;
ret = apple_rtkit_set_iop_power_state(rtk,
APPLE_RTKIT_PWR_STATE_QUIESCED);
if (ret)
return ret;
ret = apple_rtkit_reinit(rtk);
if (ret)
return ret;
rtk->iop_power_state = APPLE_RTKIT_PWR_STATE_QUIESCED;
rtk->ap_power_state = APPLE_RTKIT_PWR_STATE_QUIESCED;
return 0;
}
EXPORT_SYMBOL_GPL(apple_rtkit_quiesce);
int apple_rtkit_wake(struct apple_rtkit *rtk)
{
u64 msg;
if (apple_rtkit_is_running(rtk))
return -EINVAL;
reinit_completion(&rtk->iop_pwr_ack_completion);
/*
* Use open-coded apple_rtkit_set_iop_power_state since apple_rtkit_boot
* will wait for the completion anyway.
*/
msg = FIELD_PREP(APPLE_RTKIT_MGMT_PWR_STATE, APPLE_RTKIT_PWR_STATE_ON);
apple_rtkit_management_send(rtk, APPLE_RTKIT_MGMT_SET_IOP_PWR_STATE,
msg);
return apple_rtkit_boot(rtk);
}
EXPORT_SYMBOL_GPL(apple_rtkit_wake);
static void apple_rtkit_free(struct apple_rtkit *rtk)
{
mbox_free_channel(rtk->mbox_chan);
destroy_workqueue(rtk->wq);
apple_rtkit_free_buffer(rtk, &rtk->ioreport_buffer);
apple_rtkit_free_buffer(rtk, &rtk->crashlog_buffer);
apple_rtkit_free_buffer(rtk, &rtk->syslog_buffer);
kfree(rtk->syslog_msg_buffer);
kfree(rtk);
}
struct apple_rtkit *devm_apple_rtkit_init(struct device *dev, void *cookie,
const char *mbox_name, int mbox_idx,
const struct apple_rtkit_ops *ops)
{
struct apple_rtkit *rtk;
int ret;
rtk = apple_rtkit_init(dev, cookie, mbox_name, mbox_idx, ops);
if (IS_ERR(rtk))
return rtk;
ret = devm_add_action_or_reset(dev, (void (*)(void *))apple_rtkit_free,
rtk);
if (ret)
return ERR_PTR(ret);
return rtk;
}
EXPORT_SYMBOL_GPL(devm_apple_rtkit_init);
MODULE_LICENSE("Dual MIT/GPL");
MODULE_AUTHOR("Sven Peter <sven@svenpeter.dev>");
MODULE_DESCRIPTION("Apple RTKit driver");

328
drivers/soc/apple/sart.c Normal file
View File

@ -0,0 +1,328 @@
// SPDX-License-Identifier: GPL-2.0-only OR MIT
/*
* Apple SART device driver
* Copyright (C) The Asahi Linux Contributors
*
* Apple SART is a simple address filter for some DMA transactions.
* Regions of physical memory must be added to the SART's allow
* list before any DMA can target these. Unlike a proper
* IOMMU no remapping can be done and special support in the
* consumer driver is required since not all DMA transactions of
* a single device are subject to SART filtering.
*/
#include <linux/soc/apple/sart.h>
#include <linux/atomic.h>
#include <linux/bits.h>
#include <linux/bitfield.h>
#include <linux/device.h>
#include <linux/io.h>
#include <linux/module.h>
#include <linux/of.h>
#include <linux/of_platform.h>
#include <linux/platform_device.h>
#include <linux/types.h>
#define APPLE_SART_MAX_ENTRIES 16
/* This is probably a bitfield but the exact meaning of each bit is unknown. */
#define APPLE_SART_FLAGS_ALLOW 0xff
/* SARTv2 registers */
#define APPLE_SART2_CONFIG(idx) (0x00 + 4 * (idx))
#define APPLE_SART2_CONFIG_FLAGS GENMASK(31, 24)
#define APPLE_SART2_CONFIG_SIZE GENMASK(23, 0)
#define APPLE_SART2_CONFIG_SIZE_SHIFT 12
#define APPLE_SART2_CONFIG_SIZE_MAX GENMASK(23, 0)
#define APPLE_SART2_PADDR(idx) (0x40 + 4 * (idx))
#define APPLE_SART2_PADDR_SHIFT 12
/* SARTv3 registers */
#define APPLE_SART3_CONFIG(idx) (0x00 + 4 * (idx))
#define APPLE_SART3_PADDR(idx) (0x40 + 4 * (idx))
#define APPLE_SART3_PADDR_SHIFT 12
#define APPLE_SART3_SIZE(idx) (0x80 + 4 * (idx))
#define APPLE_SART3_SIZE_SHIFT 12
#define APPLE_SART3_SIZE_MAX GENMASK(29, 0)
struct apple_sart_ops {
void (*get_entry)(struct apple_sart *sart, int index, u8 *flags,
phys_addr_t *paddr, size_t *size);
void (*set_entry)(struct apple_sart *sart, int index, u8 flags,
phys_addr_t paddr_shifted, size_t size_shifted);
unsigned int size_shift;
unsigned int paddr_shift;
size_t size_max;
};
struct apple_sart {
struct device *dev;
void __iomem *regs;
const struct apple_sart_ops *ops;
unsigned long protected_entries;
unsigned long used_entries;
};
static void sart2_get_entry(struct apple_sart *sart, int index, u8 *flags,
phys_addr_t *paddr, size_t *size)
{
u32 cfg = readl(sart->regs + APPLE_SART2_CONFIG(index));
phys_addr_t paddr_ = readl(sart->regs + APPLE_SART2_PADDR(index));
size_t size_ = FIELD_GET(APPLE_SART2_CONFIG_SIZE, cfg);
*flags = FIELD_GET(APPLE_SART2_CONFIG_FLAGS, cfg);
*size = size_ << APPLE_SART2_CONFIG_SIZE_SHIFT;
*paddr = paddr_ << APPLE_SART2_PADDR_SHIFT;
}
static void sart2_set_entry(struct apple_sart *sart, int index, u8 flags,
phys_addr_t paddr_shifted, size_t size_shifted)
{
u32 cfg;
cfg = FIELD_PREP(APPLE_SART2_CONFIG_FLAGS, flags);
cfg |= FIELD_PREP(APPLE_SART2_CONFIG_SIZE, size_shifted);
writel(paddr_shifted, sart->regs + APPLE_SART2_PADDR(index));
writel(cfg, sart->regs + APPLE_SART2_CONFIG(index));
}
static struct apple_sart_ops sart_ops_v2 = {
.get_entry = sart2_get_entry,
.set_entry = sart2_set_entry,
.size_shift = APPLE_SART2_CONFIG_SIZE_SHIFT,
.paddr_shift = APPLE_SART2_PADDR_SHIFT,
.size_max = APPLE_SART2_CONFIG_SIZE_MAX,
};
static void sart3_get_entry(struct apple_sart *sart, int index, u8 *flags,
phys_addr_t *paddr, size_t *size)
{
phys_addr_t paddr_ = readl(sart->regs + APPLE_SART3_PADDR(index));
size_t size_ = readl(sart->regs + APPLE_SART3_SIZE(index));
*flags = readl(sart->regs + APPLE_SART3_CONFIG(index));
*size = size_ << APPLE_SART3_SIZE_SHIFT;
*paddr = paddr_ << APPLE_SART3_PADDR_SHIFT;
}
static void sart3_set_entry(struct apple_sart *sart, int index, u8 flags,
phys_addr_t paddr_shifted, size_t size_shifted)
{
writel(paddr_shifted, sart->regs + APPLE_SART3_PADDR(index));
writel(size_shifted, sart->regs + APPLE_SART3_SIZE(index));
writel(flags, sart->regs + APPLE_SART3_CONFIG(index));
}
static struct apple_sart_ops sart_ops_v3 = {
.get_entry = sart3_get_entry,
.set_entry = sart3_set_entry,
.size_shift = APPLE_SART3_SIZE_SHIFT,
.paddr_shift = APPLE_SART3_PADDR_SHIFT,
.size_max = APPLE_SART3_SIZE_MAX,
};
static int apple_sart_probe(struct platform_device *pdev)
{
int i;
struct apple_sart *sart;
struct device *dev = &pdev->dev;
sart = devm_kzalloc(dev, sizeof(*sart), GFP_KERNEL);
if (!sart)
return -ENOMEM;
sart->dev = dev;
sart->ops = of_device_get_match_data(dev);
sart->regs = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(sart->regs))
return PTR_ERR(sart->regs);
for (i = 0; i < APPLE_SART_MAX_ENTRIES; ++i) {
u8 flags;
size_t size;
phys_addr_t paddr;
sart->ops->get_entry(sart, i, &flags, &paddr, &size);
if (!flags)
continue;
dev_dbg(sart->dev,
"SART bootloader entry: index %02d; flags: 0x%02x; paddr: %pa; size: 0x%zx\n",
i, flags, &paddr, size);
set_bit(i, &sart->protected_entries);
}
platform_set_drvdata(pdev, sart);
return 0;
}
struct apple_sart *devm_apple_sart_get(struct device *dev)
{
struct device_node *sart_node;
struct platform_device *sart_pdev;
struct apple_sart *sart;
int ret;
sart_node = of_parse_phandle(dev->of_node, "apple,sart", 0);
if (!sart_node)
return ERR_PTR(-ENODEV);
sart_pdev = of_find_device_by_node(sart_node);
of_node_put(sart_node);
if (!sart_pdev)
return ERR_PTR(-ENODEV);
sart = dev_get_drvdata(&sart_pdev->dev);
if (!sart) {
put_device(&sart_pdev->dev);
return ERR_PTR(-EPROBE_DEFER);
}
ret = devm_add_action_or_reset(dev, (void (*)(void *))put_device,
&sart_pdev->dev);
if (ret)
return ERR_PTR(ret);
device_link_add(dev, &sart_pdev->dev,
DL_FLAG_PM_RUNTIME | DL_FLAG_AUTOREMOVE_SUPPLIER);
return sart;
}
EXPORT_SYMBOL_GPL(devm_apple_sart_get);
static int sart_set_entry(struct apple_sart *sart, int index, u8 flags,
phys_addr_t paddr, size_t size)
{
if (size & ((1 << sart->ops->size_shift) - 1))
return -EINVAL;
if (paddr & ((1 << sart->ops->paddr_shift) - 1))
return -EINVAL;
paddr >>= sart->ops->size_shift;
size >>= sart->ops->paddr_shift;
if (size > sart->ops->size_max)
return -EINVAL;
sart->ops->set_entry(sart, index, flags, paddr, size);
return 0;
}
int apple_sart_add_allowed_region(struct apple_sart *sart, phys_addr_t paddr,
size_t size)
{
int i, ret;
for (i = 0; i < APPLE_SART_MAX_ENTRIES; ++i) {
if (test_bit(i, &sart->protected_entries))
continue;
if (test_and_set_bit(i, &sart->used_entries))
continue;
ret = sart_set_entry(sart, i, APPLE_SART_FLAGS_ALLOW, paddr,
size);
if (ret) {
dev_dbg(sart->dev,
"unable to set entry %d to [%pa, 0x%zx]\n",
i, &paddr, size);
clear_bit(i, &sart->used_entries);
return ret;
}
dev_dbg(sart->dev, "wrote [%pa, 0x%zx] to %d\n", &paddr, size,
i);
return 0;
}
dev_warn(sart->dev,
"no free entries left to add [paddr: 0x%pa, size: 0x%zx]\n",
&paddr, size);
return -EBUSY;
}
EXPORT_SYMBOL_GPL(apple_sart_add_allowed_region);
int apple_sart_remove_allowed_region(struct apple_sart *sart, phys_addr_t paddr,
size_t size)
{
int i;
dev_dbg(sart->dev,
"will remove [paddr: %pa, size: 0x%zx] from allowed regions\n",
&paddr, size);
for (i = 0; i < APPLE_SART_MAX_ENTRIES; ++i) {
u8 eflags;
size_t esize;
phys_addr_t epaddr;
if (test_bit(i, &sart->protected_entries))
continue;
sart->ops->get_entry(sart, i, &eflags, &epaddr, &esize);
if (epaddr != paddr || esize != size)
continue;
sart->ops->set_entry(sart, i, 0, 0, 0);
clear_bit(i, &sart->used_entries);
dev_dbg(sart->dev, "cleared entry %d\n", i);
return 0;
}
dev_warn(sart->dev, "entry [paddr: 0x%pa, size: 0x%zx] not found\n",
&paddr, size);
return -EINVAL;
}
EXPORT_SYMBOL_GPL(apple_sart_remove_allowed_region);
static void apple_sart_shutdown(struct platform_device *pdev)
{
struct apple_sart *sart = dev_get_drvdata(&pdev->dev);
int i;
for (i = 0; i < APPLE_SART_MAX_ENTRIES; ++i) {
if (test_bit(i, &sart->protected_entries))
continue;
sart->ops->set_entry(sart, i, 0, 0, 0);
}
}
static const struct of_device_id apple_sart_of_match[] = {
{
.compatible = "apple,t6000-sart",
.data = &sart_ops_v3,
},
{
.compatible = "apple,t8103-sart",
.data = &sart_ops_v2,
},
{}
};
MODULE_DEVICE_TABLE(of, apple_sart_of_match);
static struct platform_driver apple_sart_driver = {
.driver = {
.name = "apple-sart",
.of_match_table = apple_sart_of_match,
},
.probe = apple_sart_probe,
.shutdown = apple_sart_shutdown,
};
module_platform_driver(apple_sart_driver);
MODULE_LICENSE("Dual MIT/GPL");
MODULE_AUTHOR("Sven Peter <sven@svenpeter.dev>");
MODULE_DESCRIPTION("Apple SART driver");

155
include/linux/soc/apple/rtkit.h Normal file
View File

@ -0,0 +1,155 @@
/* SPDX-License-Identifier: GPL-2.0-only OR MIT */
/*
* Apple RTKit IPC Library
* Copyright (C) The Asahi Linux Contributors
*
* Apple's SoCs come with various co-processors running their RTKit operating
* system. This protocol library is used by client drivers to use the
* features provided by them.
*/
#ifndef _LINUX_APPLE_RTKIT_H_
#define _LINUX_APPLE_RTKIT_H_
#include <linux/device.h>
#include <linux/types.h>
#include <linux/mailbox_client.h>
/*
* Struct to represent implementation-specific RTKit operations.
*
* @buffer: Shared memory buffer allocated inside normal RAM.
* @iomem: Shared memory buffer controlled by the co-processors.
* @size: Size of the shared memory buffer.
* @iova: Device VA of shared memory buffer.
* @is_mapped: Shared memory buffer is managed by the co-processor.
*/
struct apple_rtkit_shmem {
void *buffer;
void __iomem *iomem;
size_t size;
dma_addr_t iova;
bool is_mapped;
};
/*
* Struct to represent implementation-specific RTKit operations.
*
* @crashed: Called when the co-processor has crashed. Runs in process
* context.
* @recv_message: Function called when a message from RTKit is received
* on a non-system endpoint. Called from a worker thread.
* @recv_message_early:
* Like recv_message, but called from atomic context. It
* should return true if it handled the message. If it
* returns false, the message will be passed on to the
* worker thread.
* @shmem_setup: Setup shared memory buffer. If bfr.is_iomem is true the
* buffer is managed by the co-processor and needs to be mapped.
* Otherwise the buffer is managed by Linux and needs to be
* allocated. If not specified dma_alloc_coherent is used.
* Called in process context.
* @shmem_destroy: Undo the shared memory buffer setup in shmem_setup. If not
* specified dma_free_coherent is used. Called in process
* context.
*/
struct apple_rtkit_ops {
void (*crashed)(void *cookie);
void (*recv_message)(void *cookie, u8 endpoint, u64 message);
bool (*recv_message_early)(void *cookie, u8 endpoint, u64 message);
int (*shmem_setup)(void *cookie, struct apple_rtkit_shmem *bfr);
void (*shmem_destroy)(void *cookie, struct apple_rtkit_shmem *bfr);
};
struct apple_rtkit;
/*
* Initializes the internal state required to handle RTKit. This
* should usually be called within _probe.
*
* @dev: Pointer to the device node this coprocessor is assocated with
* @cookie: opaque cookie passed to all functions defined in rtkit_ops
* @mbox_name: mailbox name used to communicate with the co-processor
* @mbox_idx: mailbox index to be used if mbox_name is NULL
* @ops: pointer to rtkit_ops to be used for this co-processor
*/
struct apple_rtkit *devm_apple_rtkit_init(struct device *dev, void *cookie,
const char *mbox_name, int mbox_idx,
const struct apple_rtkit_ops *ops);
/*
* Reinitialize internal structures. Must only be called with the co-processor
* is held in reset.
*/
int apple_rtkit_reinit(struct apple_rtkit *rtk);
/*
* Handle RTKit's boot process. Should be called after the CPU of the
* co-processor has been started.
*/
int apple_rtkit_boot(struct apple_rtkit *rtk);
/*
* Quiesce the co-processor.
*/
int apple_rtkit_quiesce(struct apple_rtkit *rtk);
/*
* Wake the co-processor up from hibernation mode.
*/
int apple_rtkit_wake(struct apple_rtkit *rtk);
/*
* Shutdown the co-processor
*/
int apple_rtkit_shutdown(struct apple_rtkit *rtk);
/*
* Checks if RTKit is running and ready to handle messages.
*/
bool apple_rtkit_is_running(struct apple_rtkit *rtk);
/*
* Checks if RTKit has crashed.
*/
bool apple_rtkit_is_crashed(struct apple_rtkit *rtk);
/*
* Starts an endpoint. Must be called after boot but before any messages can be
* sent or received from that endpoint.
*/
int apple_rtkit_start_ep(struct apple_rtkit *rtk, u8 endpoint);
/*
* Send a message to the given endpoint.
*
* @rtk: RTKit reference
* @ep: target endpoint
* @message: message to be sent
* @completeion: will be completed once the message has been submitted
* to the hardware FIFO. Can be NULL.
* @atomic: if set to true this function can be called from atomic
* context.
*/
int apple_rtkit_send_message(struct apple_rtkit *rtk, u8 ep, u64 message,
struct completion *completion, bool atomic);
/*
* Send a message to the given endpoint and wait until it has been submitted
* to the hardware FIFO.
* Will return zero on success and a negative error code on failure
* (e.g. -ETIME when the message couldn't be written within the given
* timeout)
*
* @rtk: RTKit reference
* @ep: target endpoint
* @message: message to be sent
* @timeout: timeout in milliseconds to allow the message transmission
* to be completed
* @atomic: if set to true this function can be called from atomic
* context.
*/
int apple_rtkit_send_message_wait(struct apple_rtkit *rtk, u8 ep, u64 message,
unsigned long timeout, bool atomic);
#endif /* _LINUX_APPLE_RTKIT_H_ */

53
include/linux/soc/apple/sart.h Normal file
View File

@ -0,0 +1,53 @@
/* SPDX-License-Identifier: GPL-2.0-only OR MIT */
/*
* Apple SART device driver
* Copyright (C) The Asahi Linux Contributors
*
* Apple SART is a simple address filter for DMA transactions.
* Regions of physical memory must be added to the SART's allow
* list before any DMA can target these. Unlike a proper
* IOMMU no remapping can be done.
*/
#ifndef _LINUX_SOC_APPLE_SART_H_
#define _LINUX_SOC_APPLE_SART_H_
#include <linux/device.h>
#include <linux/err.h>
#include <linux/types.h>
struct apple_sart;
/*
* Get a reference to the SART attached to dev.
*
* Looks for the phandle reference in apple,sart and returns a pointer
* to the corresponding apple_sart struct to be used with
* apple_sart_add_allowed_region and apple_sart_remove_allowed_region.
*/
struct apple_sart *devm_apple_sart_get(struct device *dev);
/*
* Adds the region [paddr, paddr+size] to the DMA allow list.
*
* @sart: SART reference
* @paddr: Start address of the region to be used for DMA
* @size: Size of the region to be used for DMA.
*/
int apple_sart_add_allowed_region(struct apple_sart *sart, phys_addr_t paddr,
size_t size);
/*
* Removes the region [paddr, paddr+size] from the DMA allow list.
*
* Note that exact same paddr and size used for apple_sart_add_allowed_region
* have to be passed.
*
* @sart: SART reference
* @paddr: Start address of the region no longer used for DMA
* @size: Size of the region no longer used for DMA.
*/
int apple_sart_remove_allowed_region(struct apple_sart *sart, phys_addr_t paddr,
size_t size);
#endif /* _LINUX_SOC_APPLE_SART_H_ */