Add support for the RK3588 to the driver. The RK3588 has four DDR
channels with a register stride of 0x4000 between the channel
registers, also it has a DDRMON_CTRL register per channel.
Link: https://lore.kernel.org/all/20231018061714.3553817-20-s.hauer@pengutronix.de/
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: Sebastian Reichel <sebastian.reichel@collabora.com>
Acked-by: Chanwoo Choi <cw00.choi@samsung.com>
Acked-by: Heiko Stuebner <heiko@sntech.de>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: Chanwoo Choi <cw00.choi@samsung.com>
The DFI is a unit which is suitable for measuring DDR utilization, but
so far it could only be used as an event driver for the DDR frequency
scaling driver. This adds perf support to the DFI driver.
Usage with the 'perf' tool can look like:
perf stat -a -e rockchip_ddr/cycles/,\
rockchip_ddr/read-bytes/,\
rockchip_ddr/write-bytes/,\
rockchip_ddr/bytes/ sleep 1
Performance counter stats for 'system wide':
1582524826 rockchip_ddr/cycles/
1802.25 MB rockchip_ddr/read-bytes/
1793.72 MB rockchip_ddr/write-bytes/
3595.90 MB rockchip_ddr/bytes/
1.014369709 seconds time elapsed
perf support has been tested on a RK3568 and a RK3399, the latter with
dual channel DDR.
Link: https://lore.kernel.org/all/20231019064819.3496740-1-s.hauer@pengutronix.de/
Reviewed-by: Sebastian Reichel <sebastian.reichel@collabora.com>
Acked-by: Chanwoo Choi <cw00.choi@samsung.com>
Acked-by: Heiko Stuebner <heiko@sntech.de>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
[cw00.choi: Fix typo from 'write_acccess' to 'write_access']
Signed-off-by: Chanwoo Choi <cw00.choi@samsung.com>
In the DFI driver LPDDR4X can be handled in the same way as LPDDR4. Add
the missing case.
Link: https://lore.kernel.org/all/20231018061714.3553817-13-s.hauer@pengutronix.de/
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: Sebastian Reichel <sebastian.reichel@collabora.com>
Acked-by: Chanwoo Choi <cw00.choi@samsung.com>
Acked-by: Heiko Stuebner <heiko@sntech.de>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: Chanwoo Choi <cw00.choi@samsung.com>
This adds RK3568 support to the DFI driver. Only iniitialization
differs from the currently supported RK3399.
Link: https://lore.kernel.org/all/20231018061714.3553817-11-s.hauer@pengutronix.de/
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Acked-by: Heiko Stuebner <heiko@sntech.de>
Signed-off-by: Chanwoo Choi <cw00.choi@samsung.com>
The DDRTYPE defines are named to be RK3399 specific, but they can be
used for other Rockchip SoCs as well, so replace the RK3399_PMUGRF_
prefix with ROCKCHIP_. They are defined in a SoC specific header
file, so when generalizing the prefix also move the new defines to
a SoC agnostic header file. While at it use GENMASK to define the
DDRTYPE bitfield and give it a name including the full register name.
Link: https://lore.kernel.org/all/20231018061714.3553817-9-s.hauer@pengutronix.de/
Reviewed-by: Sebastian Reichel <sebastian.reichel@collabora.com>
Acked-by: Chanwoo Choi <cw00.choi@samsung.com>
Acked-by: Heiko Stuebner <heiko@sntech.de>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: Chanwoo Choi <cw00.choi@samsung.com>
On RK3399 platforms, power domains are managed mostly by the kernel
(drivers/soc/rockchip/pm_domains.c), but there are a few exceptions
where ARM Trusted Firmware has to be involved:
(1) system suspend/resume
(2) DRAM DVFS (a.k.a., "ddrfreq")
Exception (1) does not cause much conflict, since the kernel has
quiesced itself by the time we make the relevant PSCI call.
Exception (2) can cause conflict, because of two actions:
(a) ARM Trusted Firmware needs to read/modify/write the PMU_BUS_IDLE_REQ
register to idle the memory controller domain; the kernel driver
also has to touch this register for other domains.
(b) ARM Trusted Firmware needs to manage the clocks associated with
these domains.
To elaborate on (b): idling a power domain has always required ungating
an array of clocks; see this old explanation from Rockchip:
https://lore.kernel.org/linux-arm-kernel/54503C19.9060607@rock-chips.com/
Historically, ARM Trusted Firmware has avoided this issue by using a
special PMU_CRU_GATEDIS_CON0 register -- this register ungates all the
necessary clocks -- when idling the memory controller. Unfortunately,
we've found that this register is not 100% sufficient; it does not turn
the relevant PLLs on [0].
So it's possible to trigger issues with something like the following:
1. enable a power domain (e.g., RK3399_PD_VDU) -- kernel will
temporarily enable relevant clocks/PLLs, then turn them back off
2. a PLL (e.g., PLL_NPLL) is part of the clock tree for
RK3399_PD_VDU's clocks but otherwise unused; NPLL is disabled
3. perform a ddrfreq transition (rk3399_dmcfreq_target() -> ...
drivers/clk/rockchip/clk-ddr.c / ROCKCHIP_SIP_DRAM_FREQ)
4. ARM Trusted Firmware unagates VDU clocks (via PMU_CRU_GATEDIS_CON0)
5. ARM Trusted firmware idles the memory controller domain
6. Step 5 waits on the VDU domain/clocks, but NPLL is still off
i.e., we hang the system.
So for (b), we need to at a minimum manage the relevant PLLs on behalf
of firmware. It's easier to simply manage the whole clock tree, in a
similar way we do in rockchip_pd_power().
For (a), we need to provide mutual exclusion betwen rockchip_pd_power()
and firmware. To resolve that, we simply grab the PMU mutex and release
it when ddrfreq is done.
The Chromium OS kernel has been carrying versions of part of this hack
for a while, based on some new custom notifiers [1]. I've rewritten as a
simple function call between the drivers, which is OK because:
* the PMU driver isn't enabled, and we don't have this problem at all
(the firmware should have left us in an OK state, and there are no
runtime conflicts); or
* the PMU driver is present, and is a single instance.
And the power-domain driver cannot be removed, so there's no lifetime
management to worry about.
For completeness, there's a 'dmc_pmu_mutex' to guard (likely
theoretical?) probe()-time races. It's OK for the memory controller
driver to start running before the PMU, because the PMU will avoid any
critical actions during the block() sequence.
[0] The RK3399 TRM for PMU_CRU_GATEDIS_CON0 only talks about ungating
clocks. Based on experimentation, we've found that it does not power
up the necessary PLLs.
[1] CHROMIUM: soc: rockchip: power-domain: Add notifier to dmc driver
https://chromium-review.googlesource.com/q/I242dbd706d352f74ff706f5cbf42ebb92f9bcc60
Notably, the Chromium solution only handled conflict (a), not (b).
In practice, item (b) wasn't a problem in many cases because we
never managed to fully power off PLLs. Now that the (upstream) video
decoder driver performs runtime clock management, we often power off
NPLL.
Signed-off-by: Brian Norris <briannorris@chromium.org>
Tested-by: Peter Geis <pgwipeout@gmail.com>
Reviewed-by: Heiko Stuebner <heiko@sntech.de>
Signed-off-by: Chanwoo Choi <cw00.choi@samsung.com>
Based on 1 normalized pattern(s):
this program is free software you can redistribute it and or modify
it under the terms and conditions of the gnu general public license
version 2 as published by the free software foundation this program
is distributed in the hope it will be useful but without any
warranty without even the implied warranty of merchantability or
fitness for a particular purpose see the gnu general public license
for more details
extracted by the scancode license scanner the SPDX license identifier
GPL-2.0-only
has been chosen to replace the boilerplate/reference in 263 file(s).
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Allison Randal <allison@lohutok.net>
Reviewed-by: Alexios Zavras <alexios.zavras@intel.com>
Cc: linux-spdx@vger.kernel.org
Link: https://lkml.kernel.org/r/20190529141901.208660670@linutronix.de
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Trusted Firmware-A (TF-A) for rk3399 implements a SiP call to get the
on-die termination (ODT) and auto power down parameters from kernel,
this patch adds the functionality to do this. Also, if DDR clock
frequency is lower than the on-die termination (ODT) disable frequency
this driver should disable the DDR ODT.
Signed-off-by: Enric Balletbo i Serra <enric.balletbo@collabora.com>
Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>
Signed-off-by: Gaël PORTAY <gael.portay@collabora.com>
Signed-off-by: MyungJoo Ham <myungjoo.ham@samsung.com>
Some rk3399 GRF (Generic Register Files) definitions can be used for
different drivers. Move these definitions to a common include so we
don't need to duplicate these definitions.
Signed-off-by: Enric Balletbo i Serra <enric.balletbo@collabora.com>
Acked-by: Chanwoo Choi <cw00.choi@samsung.com>
Signed-off-by: Gaël PORTAY <gael.portay@collabora.com>
Signed-off-by: MyungJoo Ham <myungjoo.ham@samsung.com>
Add a header for the SIP interface defined to access the dcf controller
handling ddr rate changes on rk3399 (and most likely later socs).
This interface is shared between the clock driver as well as the
devfreq driver.
The SIP interface counterpart was merged from pull-request #684 [0]
into the upstream arm-trusted-firmware codebase.
[0] https://github.com/ARM-software/arm-trusted-firmware/pull/684
Signed-off-by: Lin Huang <hl@rock-chips.com>
Signed-off-by: Heiko Stuebner <heiko@sntech.de>
