Merge tag 'pm-5.9-rc1-2' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm

Pull more power management updates from Rafael Wysocki: "These are mostly ARM cpufreq driver updates plus a cpufreq core cleanup, an ARM-wide change to make schedutil the default scaling governor, an intel_pstate driver fix and some runtime PM changes regarding kerneldoc comments. Specifics: - Add adaptive voltage scaling (AVS) support to the brcmstb cpufreq driver and clean it up (Florian Fainelli, Markus Mayer). - Add a new Tegra cpufreq driver and clean up the existing one (Jon Hunter, Sumit Gupta). - Add bandwidth level support to the Qcom cpufreq driver along with OPP changes (Sibi Sankar). - Clean up the sti, cpufreq-dt, ap806, CPPC cpufreq drivers (Viresh Kumar, Lee Jones, Ivan Kokshaysky, Sven Auhagen, Xin Hao). - Make schedutil the default governor for ARM (Valentin Schneider). - Fix dependency issues for the imx cpufreq driver (Walter Lozano). - Clean up cached_resolved_idx handlihng in the cpufreq core (Viresh Kumar). - Fix the intel_pstate driver to use the correct maximum frequency value when MSR_TURBO_RATIO_LIMIT is 0 (Srinivas Pandruvada). - Provide kenrneldoc comments for multiple runtime PM helpers and improve the pm_runtime_get_if_active() kerneldoc (Rafael Wysocki)" * tag 'pm-5.9-rc1-2' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm: (22 commits) cpufreq: intel_pstate: Fix cpuinfo_max_freq when MSR_TURBO_RATIO_LIMIT is 0 PM: runtime: Improve kerneldoc of pm_runtime_get_if_active() PM: runtime: Add kerneldoc comments to multiple helpers cpufreq: make schedutil the default for arm and arm64 cpufreq: cached_resolved_idx can not be negative cpufreq: Add Tegra194 cpufreq driver dt-bindings: arm: Add NVIDIA Tegra194 CPU Complex binding cpufreq: imx: Select NVMEM_IMX_OCOTP cpufreq: sti-cpufreq: Fix some formatting and misspelling issues cpufreq: tegra186: Simplify probe return path cpufreq: CPPC: Reuse caps variable in few routines cpufreq: ap806: fix cpufreq driver needs ap cpu clk cpufreq: cppc: Reorder code and remove apply_hisi_workaround variable cpufreq: dt: fix oops on armada37xx cpufreq: brcmstb-avs-cpufreq: send S2_ENTER / S2_EXIT commands to AVS cpufreq: brcmstb-avs-cpufreq: Support polling AVS firmware cpufreq: brcmstb-avs-cpufreq: more flexible interface for __issue_avs_command() cpufreq: qcom: Disable fast switch when scaling DDR/L3 cpufreq: qcom: Update the bandwidth levels on frequency change OPP: Add and export helper to set bandwidth ...

Linus Torvalds committed 6 years ago

Commit f6235eb2 parents 2f12d44 0873ad919 files · +963 −114Browse files

Unified · Split

19 files

Documentation/devicetree/bindings/arm/nvidia,tegra194-ccplex.yaml +69 −0
drivers/base/power/runtime.c +14 −12
drivers/cpufreq/Kconfig +1 −1
drivers/cpufreq/Kconfig.arm +9 −0
drivers/cpufreq/Makefile +1 −0
drivers/cpufreq/armada-37xx-cpufreq.c +1 −0
drivers/cpufreq/brcmstb-avs-cpufreq.c +59 −30
drivers/cpufreq/cppc_cpufreq.c +44 −51
drivers/cpufreq/cpufreq-dt-platdev.c +2 −0
drivers/cpufreq/cpufreq.c +1 −1
drivers/cpufreq/intel_pstate.c +1 −0
drivers/cpufreq/qcom-cpufreq-hw.c +83 −8
drivers/cpufreq/sti-cpufreq.c +4 −4
drivers/cpufreq/tegra186-cpufreq.c +0 −6
drivers/cpufreq/tegra194-cpufreq.c +390 −0
drivers/opp/core.c +31 −0
include/linux/cpufreq.h +1 −1
include/linux/pm_opp.h +6 −0
include/linux/pm_runtime.h +246 −0

Documentation/devicetree/bindings/arm/nvidia,tegra194-ccplex.yaml+69−0

@@ -0,0 +1,69 @@
	1	+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
	2	+%YAML 1.2
	3	+---
	4	+$id: "http://devicetree.org/schemas/arm/nvidia,tegra194-ccplex.yaml#"
	5	+$schema: "http://devicetree.org/meta-schemas/core.yaml#"
	6	+
	7	+title: NVIDIA Tegra194 CPU Complex device tree bindings
	8	+
	9	+maintainers:
	10	+ - Thierry Reding <thierry.reding@gmail.com>
	11	+ - Jonathan Hunter <jonathanh@nvidia.com>
	12	+ - Sumit Gupta <sumitg@nvidia.com>
	13	+
	14	+description: \|+
	15	+ Tegra194 SOC has homogeneous architecture where each cluster has two
	16	+ symmetric cores. Compatible string in "cpus" node represents the CPU
	17	+ Complex having all clusters.
	18	+
	19	+properties:
	20	+ $nodename:
	21	+ const: cpus
	22	+
	23	+ compatible:
	24	+ enum:
	25	+ - nvidia,tegra194-ccplex
	26	+
	27	+ nvidia,bpmp:
	28	+ $ref: '/schemas/types.yaml#/definitions/phandle'
	29	+ description: \|
	30	+ Specifies the bpmp node that needs to be queried to get
	31	+ operating point data for all CPUs.
	32	+
	33	+examples:
	34	+ - \|
	35	+ cpus {
	36	+ compatible = "nvidia,tegra194-ccplex";
	37	+ nvidia,bpmp = <&bpmp>;
	38	+ #address-cells = <1>;
	39	+ #size-cells = <0>;
	40	+
	41	+ cpu0_0: cpu@0 {
	42	+ compatible = "nvidia,tegra194-carmel";
	43	+ device_type = "cpu";
	44	+ reg = <0x0>;
	45	+ enable-method = "psci";
	46	+ };
	47	+
	48	+ cpu0_1: cpu@1 {
	49	+ compatible = "nvidia,tegra194-carmel";
	50	+ device_type = "cpu";
	51	+ reg = <0x001>;
	52	+ enable-method = "psci";
	53	+ };
	54	+
	55	+ cpu1_0: cpu@100 {
	56	+ compatible = "nvidia,tegra194-carmel";
	57	+ device_type = "cpu";
	58	+ reg = <0x100>;
	59	+ enable-method = "psci";
	60	+ };
	61	+
	62	+ cpu1_1: cpu@101 {
	63	+ compatible = "nvidia,tegra194-carmel";
	64	+ device_type = "cpu";
	65	+ reg = <0x101>;
	66	+ enable-method = "psci";
	67	+ };
	68	+ };
	69	+...

drivers/base/power/runtime.c+14−12

@@ -1085,24 +1085,26 @@ int __pm_runtime_resume(struct device *dev, int rpmflags) · 1084 unmodified lines
1085	1085	EXPORT_SYMBOL_GPL(__pm_runtime_resume);
1086	1086
1087	1087	/**
1088		− * pm_runtime_get_if_active - Conditionally bump up the device's usage counter.
	1088	+ * pm_runtime_get_if_active - Conditionally bump up device usage counter.
1089	1089	* @dev: Device to handle.
	1090	+ * @ign_usage_count: Whether or not to look at the current usage counter value.
1090	1091	*
1091		− * Return -EINVAL if runtime PM is disabled for the device.
	1092	+ * Return -EINVAL if runtime PM is disabled for @dev.
1092	1093	*
1093		− * Otherwise, if the device's runtime PM status is RPM_ACTIVE and either
1094		− * ign_usage_count is true or the device's usage_count is non-zero, increment
1095		− * the counter and return 1. Otherwise return 0 without changing the counter.
	1094	+ * Otherwise, if the runtime PM status of @dev is %RPM_ACTIVE and either
	1095	+ * @ign_usage_count is %true or the runtime PM usage counter of @dev is not
	1096	+ * zero, increment the usage counter of @dev and return 1. Otherwise, return 0
	1097	+ * without changing the usage counter.
1096	1098	*
1097		− * If ign_usage_count is true, the function can be used to prevent suspending
1098		− * the device when its runtime PM status is RPM_ACTIVE.
	1099	+ * If @ign_usage_count is %true, this function can be used to prevent suspending
	1100	+ * the device when its runtime PM status is %RPM_ACTIVE.
1099	1101	*
1100		− * If ign_usage_count is false, the function can be used to prevent suspending
1101		− * the device when both its runtime PM status is RPM_ACTIVE and its usage_count
1102		− * is non-zero.
	1102	+ * If @ign_usage_count is %false, this function can be used to prevent
	1103	+ * suspending the device when both its runtime PM status is %RPM_ACTIVE and its
	1104	+ * runtime PM usage counter is not zero.
1103	1105	*
1104		− * The caller is resposible for putting the device's usage count when ther
1105		− * return value is greater than zero.
	1106	+ * The caller is resposible for decrementing the runtime PM usage counter of
	1107	+ * @dev after this function has returned a positive value for it.
1106	1108	*/
1107	1109	int pm_runtime_get_if_active(struct device *dev, bool ign_usage_count)
1108	1110	{

drivers/cpufreq/Kconfig+1−1

@@ -37,7 +37,7 @@ config CPU_FREQ_STAT · 36 unmodified lines
37	37	choice
38	38	prompt "Default CPUFreq governor"
39	39	default CPU_FREQ_DEFAULT_GOV_USERSPACE if ARM_SA1100_CPUFREQ \|\| ARM_SA1110_CPUFREQ
40		− default CPU_FREQ_DEFAULT_GOV_SCHEDUTIL if BIG_LITTLE
	40	+ default CPU_FREQ_DEFAULT_GOV_SCHEDUTIL if ARM64 \|\| ARM
41	41	default CPU_FREQ_DEFAULT_GOV_SCHEDUTIL if X86_INTEL_PSTATE && SMP
42	42	default CPU_FREQ_DEFAULT_GOV_PERFORMANCE
43	43	help

drivers/cpufreq/Kconfig.arm+9−0

@@ -41,6 +41,7 @@ config ARM_ARMADA_37XX_CPUFREQ · 40 unmodified lines
41	41	config ARM_ARMADA_8K_CPUFREQ
42	42	tristate "Armada 8K CPUFreq driver"
43	43	depends on ARCH_MVEBU && CPUFREQ_DT
	44	+ select ARMADA_AP_CPU_CLK
44	45	help
45	46	This enables the CPUFreq driver support for Marvell
46	47	Armada8k SOCs.
@@ -93,6 +94,7 @@ config ARM_IMX6Q_CPUFREQ · 46 unmodified lines
93	94	tristate "Freescale i.MX6 cpufreq support"
94	95	depends on ARCH_MXC
95	96	depends on REGULATOR_ANATOP
	97	+ select NVMEM_IMX_OCOTP
96	98	select PM_OPP
97	99	help
98	100	This adds cpufreq driver support for Freescale i.MX6 series SoCs.
@@ -314,6 +316,13 @@ config ARM_TEGRA186_CPUFREQ · 215 unmodified lines
314	316	help
315	317	This adds the CPUFreq driver support for Tegra186 SOCs.
316	318
	319	+config ARM_TEGRA194_CPUFREQ
	320	+ tristate "Tegra194 CPUFreq support"
	321	+ depends on ARCH_TEGRA_194_SOC && TEGRA_BPMP
	322	+ default y
	323	+ help
	324	+ This adds CPU frequency driver support for Tegra194 SOCs.
	325	+
317	326	config ARM_TI_CPUFREQ
318	327	bool "Texas Instruments CPUFreq support"
319	328	depends on ARCH_OMAP2PLUS

drivers/cpufreq/Makefile+1−0

@@ -83,6 +83,7 @@ obj-$(CONFIG_ARM_TANGO_CPUFREQ) += tango-cpufreq.o · 82 unmodified lines
83	83	obj-$(CONFIG_ARM_TEGRA20_CPUFREQ) += tegra20-cpufreq.o
84	84	obj-$(CONFIG_ARM_TEGRA124_CPUFREQ) += tegra124-cpufreq.o
85	85	obj-$(CONFIG_ARM_TEGRA186_CPUFREQ) += tegra186-cpufreq.o
	86	+obj-$(CONFIG_ARM_TEGRA194_CPUFREQ) += tegra194-cpufreq.o
86	87	obj-$(CONFIG_ARM_TI_CPUFREQ) += ti-cpufreq.o
87	88	obj-$(CONFIG_ARM_VEXPRESS_SPC_CPUFREQ) += vexpress-spc-cpufreq.o
88	89

drivers/cpufreq/armada-37xx-cpufreq.c+1−0

@@ -456,6 +456,7 @@ static int __init armada37xx_cpufreq_driver_init(void) · 455 unmodified lines
456	456	/* Now that everything is setup, enable the DVFS at hardware level */
457	457	armada37xx_cpufreq_enable_dvfs(nb_pm_base);
458	458
	459	+ memset(&pdata, 0, sizeof(pdata));
459	460	pdata.suspend = armada37xx_cpufreq_suspend;
460	461	pdata.resume = armada37xx_cpufreq_resume;
461	462

drivers/cpufreq/brcmstb-avs-cpufreq.c+59−30

@@ -42,6 +42,7 @@ · 41 unmodified lines
42	42	*/
43	43
44	44	#include <linux/cpufreq.h>
	45	+#include <linux/delay.h>
45	46	#include <linux/interrupt.h>
46	47	#include <linux/io.h>
47	48	#include <linux/module.h>
@@ -178,6 +179,7 @@ struct private_data { · 130 unmodified lines
178	179	struct completion done;
179	180	struct semaphore sem;
180	181	struct pmap pmap;
	182	+ int host_irq;
181	183	};
182	184
183	185	static void __iomem __map_region(const char name)
@@ -195,11 +197,36 @@ static void __iomem __map_region(const char name) · 11 unmodified lines
195	197	return ptr;
196	198	}
197	199
198		−static int __issue_avs_command(struct private_data *priv, int cmd, bool is_send,
	200	+static unsigned long wait_for_avs_command(struct private_data *priv,
	201	+ unsigned long timeout)
	202	+{
	203	+ unsigned long time_left = 0;
	204	+ u32 val;
	205	+
	206	+ /* Event driven, wait for the command interrupt */
	207	+ if (priv->host_irq >= 0)
	208	+ return wait_for_completion_timeout(&priv->done,
	209	+ msecs_to_jiffies(timeout));
	210	+
	211	+ /* Polling for command completion */
	212	+ do {
	213	+ time_left = timeout;
	214	+ val = readl(priv->base + AVS_MBOX_STATUS);
	215	+ if (val)
	216	+ break;
	217	+
	218	+ usleep_range(1000, 2000);
	219	+ } while (--timeout);
	220	+
	221	+ return time_left;
	222	+}
	223	+
	224	+static int __issue_avs_command(struct private_data *priv, unsigned int cmd,
	225	+ unsigned int num_in, unsigned int num_out,
199	226	u32 args[])
200	227	{
201		− unsigned long time_left = msecs_to_jiffies(AVS_TIMEOUT);
202	228	void __iomem *base = priv->base;
	229	+ unsigned long time_left;
203	230	unsigned int i;
204	231	int ret;
205	232	u32 val;
@@ -225,11 +252,9 @@ static int __issue_avs_command(struct private_data *priv, int cmd, bool is_send, · 19 unmodified lines
225	252	/* Clear status before we begin. */
226	253	writel(AVS_STATUS_CLEAR, base + AVS_MBOX_STATUS);
227	254
228		− /* We need to send arguments for this command. */
229		− if (args && is_send) {
230		− for (i = 0; i < AVS_MAX_CMD_ARGS; i++)
231		− writel(args[i], base + AVS_MBOX_PARAM(i));
232		− }
	255	+ /* Provide input parameters */
	256	+ for (i = 0; i < num_in; i++)
	257	+ writel(args[i], base + AVS_MBOX_PARAM(i));
233	258
234	259	/* Protect from spurious interrupts. */
235	260	reinit_completion(&priv->done);
@@ -239,7 +264,7 @@ static int __issue_avs_command(struct private_data *priv, int cmd, bool is_send, · 3 unmodified lines
239	264	writel(AVS_CPU_L2_INT_MASK, priv->avs_intr_base + AVS_CPU_L2_SET0);
240	265
241	266	/* Wait for AVS co-processor to finish processing the command. */
242		− time_left = wait_for_completion_timeout(&priv->done, time_left);
	267	+ time_left = wait_for_avs_command(priv, AVS_TIMEOUT);
243	268
244	269	/*
245	270	* If the AVS status is not in the expected range, it means AVS didn't
@@ -256,11 +281,9 @@ static int __issue_avs_command(struct private_data *priv, int cmd, bool is_send, · 10 unmodified lines
256	281	goto out;
257	282	}
258	283
259		− /* This command returned arguments, so we read them back. */
260		− if (args && !is_send) {
261		− for (i = 0; i < AVS_MAX_CMD_ARGS; i++)
262		− args[i] = readl(base + AVS_MBOX_PARAM(i));
263		− }
	284	+ /* Process returned values */
	285	+ for (i = 0; i < num_out; i++)
	286	+ args[i] = readl(base + AVS_MBOX_PARAM(i));
264	287
265	288	/* Clear status to tell AVS co-processor we are done. */
266	289	writel(AVS_STATUS_CLEAR, base + AVS_MBOX_STATUS);
@@ -338,7 +361,7 @@ static int brcm_avs_get_pmap(struct private_data priv, struct pmap pmap) · 71 unmodified lines
338	361	u32 args[AVS_MAX_CMD_ARGS];
339	362	int ret;
340	363
341		− ret = __issue_avs_command(priv, AVS_CMD_GET_PMAP, false, args);
	364	+ ret = __issue_avs_command(priv, AVS_CMD_GET_PMAP, 0, 4, args);
342	365	if (ret \|\| !pmap)
343	366	return ret;
344	367
@@ -359,7 +382,7 @@ static int brcm_avs_set_pmap(struct private_data priv, struct pmap pmap) · 14 unmodified lines
359	382	args[2] = pmap->p2;
360	383	args[3] = pmap->state;
361	384
362		− return __issue_avs_command(priv, AVS_CMD_SET_PMAP, true, args);
	385	+ return __issue_avs_command(priv, AVS_CMD_SET_PMAP, 4, 0, args);
363	386	}
364	387
365	388	static int brcm_avs_get_pstate(struct private_data priv, unsigned int pstate)
@@ -367,7 +390,7 @@ static int brcm_avs_get_pstate(struct private_data priv, unsigned int pstate) · 1 unmodified line
367	390	u32 args[AVS_MAX_CMD_ARGS];
368	391	int ret;
369	392
370		− ret = __issue_avs_command(priv, AVS_CMD_GET_PSTATE, false, args);
	393	+ ret = __issue_avs_command(priv, AVS_CMD_GET_PSTATE, 0, 1, args);
371	394	if (ret)
372	395	return ret;
373	396	*pstate = args[0];
@@ -381,7 +404,8 @@ static int brcm_avs_set_pstate(struct private_data *priv, unsigned int pstate) · 7 unmodified lines
381	404
382	405	args[0] = pstate;
383	406
384		− return __issue_avs_command(priv, AVS_CMD_SET_PSTATE, true, args);
	407	+ return __issue_avs_command(priv, AVS_CMD_SET_PSTATE, 1, 0, args);
	408	+
385	409	}
386	410
387	411	static u32 brcm_avs_get_voltage(void __iomem *base)
@@ -482,7 +506,14 @@ static int brcm_avs_suspend(struct cpufreq_policy *policy) · 94 unmodified lines
482	506	* AVS co-processor, not necessarily the P-state we are running at now.
483	507	* So, we get the current P-state explicitly.
484	508	*/
485		− return brcm_avs_get_pstate(priv, &priv->pmap.state);
	509	+ ret = brcm_avs_get_pstate(priv, &priv->pmap.state);
	510	+ if (ret)
	511	+ return ret;
	512	+
	513	+ /* This is best effort. Nothing to do if it fails. */
	514	+ (void)__issue_avs_command(priv, AVS_CMD_S2_ENTER, 0, 0, NULL);
	515	+
	516	+ return 0;
486	517	}
487	518
488	519	static int brcm_avs_resume(struct cpufreq_policy *policy)
@@ -490,6 +521,9 @@ static int brcm_avs_resume(struct cpufreq_policy *policy) · 1 unmodified line
490	521	struct private_data *priv = policy->driver_data;
491	522	int ret;
492	523
	524	+ /* This is best effort. Nothing to do if it fails. */
	525	+ (void)__issue_avs_command(priv, AVS_CMD_S2_EXIT, 0, 0, NULL);
	526	+
493	527	ret = brcm_avs_set_pmap(priv, &priv->pmap);
494	528	if (ret == -EEXIST) {
495	529	struct platform_device *pdev = cpufreq_get_driver_data();
@@ -511,7 +545,7 @@ static int brcm_avs_prepare_init(struct platform_device *pdev) · 15 unmodified lines
511	545	{
512	546	struct private_data *priv;
513	547	struct device *dev;
514		− int host_irq, ret;
	548	+ int ret;
515	549
516	550	dev = &pdev->dev;
517	551	priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
@@ -538,19 +572,14 @@ static int brcm_avs_prepare_init(struct platform_device *pdev) · 20 unmodified lines
538	572	goto unmap_base;
539	573	}
540	574
541		− host_irq = platform_get_irq_byname(pdev, BRCM_AVS_HOST_INTR);
542		− if (host_irq < 0) {
543		− dev_err(dev, "Couldn't find interrupt %s -- %d\n",
544		− BRCM_AVS_HOST_INTR, host_irq);
545		− ret = host_irq;
546		− goto unmap_intr_base;
547		− }
	575	+ priv->host_irq = platform_get_irq_byname(pdev, BRCM_AVS_HOST_INTR);
548	576
549		− ret = devm_request_irq(dev, host_irq, irq_handler, IRQF_TRIGGER_RISING,
	577	+ ret = devm_request_irq(dev, priv->host_irq, irq_handler,
	578	+ IRQF_TRIGGER_RISING,
550	579	BRCM_AVS_HOST_INTR, priv);
551		− if (ret) {
	580	+ if (ret && priv->host_irq >= 0) {
552	581	dev_err(dev, "IRQ request failed: %s (%d) -- %d\n",
553		− BRCM_AVS_HOST_INTR, host_irq, ret);
	582	+ BRCM_AVS_HOST_INTR, priv->host_irq, ret);
554	583	goto unmap_intr_base;
555	584	}
556	585
@@ -593,7 +622,7 @@ static int brcm_avs_cpufreq_init(struct cpufreq_policy *policy) · 36 unmodified lines
593	622	/* All cores share the same clock and thus the same policy. */
594	623	cpumask_setall(policy->cpus);
595	624
596		− ret = __issue_avs_command(priv, AVS_CMD_ENABLE, false, NULL);
	625	+ ret = __issue_avs_command(priv, AVS_CMD_ENABLE, 0, 0, NULL);
597	626	if (!ret) {
598	627	unsigned int pstate;
599	628

drivers/cpufreq/cppc_cpufreq.c+44−51

@@ -45,8 +45,6 @@ struct cppc_workaround_oem_info { · 44 unmodified lines
45	45	u32 oem_revision;
46	46	};
47	47
48		−static bool apply_hisi_workaround;
49		−
50	48	static struct cppc_workaround_oem_info wa_info[] = {
51	49	{
52	50	.oem_id = "HISI ",
@@ -59,50 +57,6 @@ static struct cppc_workaround_oem_info wa_info[] = { · 6 unmodified lines
59	57	}
60	58	};
61	59
62		−static unsigned int cppc_cpufreq_perf_to_khz(struct cppc_cpudata *cpu,
63		− unsigned int perf);
64		−
65		−/*
66		− * HISI platform does not support delivered performance counter and
67		− * reference performance counter. It can calculate the performance using the
68		− * platform specific mechanism. We reuse the desired performance register to
69		− * store the real performance calculated by the platform.
70		− */
71		−static unsigned int hisi_cppc_cpufreq_get_rate(unsigned int cpunum)
72		−{
73		− struct cppc_cpudata *cpudata = all_cpu_data[cpunum];
74		− u64 desired_perf;
75		− int ret;
76		−
77		− ret = cppc_get_desired_perf(cpunum, &desired_perf);
78		− if (ret < 0)
79		− return -EIO;
80		−
81		− return cppc_cpufreq_perf_to_khz(cpudata, desired_perf);
82		−}
83		−
84		−static void cppc_check_hisi_workaround(void)
85		−{
86		− struct acpi_table_header *tbl;
87		− acpi_status status = AE_OK;
88		− int i;
89		−
90		− status = acpi_get_table(ACPI_SIG_PCCT, 0, &tbl);
91		− if (ACPI_FAILURE(status) \|\| !tbl)
92		− return;
93		−
94		− for (i = 0; i < ARRAY_SIZE(wa_info); i++) {
95		− if (!memcmp(wa_info[i].oem_id, tbl->oem_id, ACPI_OEM_ID_SIZE) &&
96		− !memcmp(wa_info[i].oem_table_id, tbl->oem_table_id, ACPI_OEM_TABLE_ID_SIZE) &&
97		− wa_info[i].oem_revision == tbl->oem_revision) {
98		− apply_hisi_workaround = true;
99		− break;
100		− }
101		− }
102		−
103		− acpi_put_table(tbl);
104		−}
105		−
106	60	/* Callback function used to retrieve the max frequency from DMI */
107	61	static void cppc_find_dmi_mhz(const struct dmi_header dm, void private)
108	62	{
@@ -161,7 +115,7 @@ static unsigned int cppc_cpufreq_perf_to_khz(struct cppc_cpudata *cpu, · 52 unmodified lines
161	115	if (!max_khz)
162	116	max_khz = cppc_get_dmi_max_khz();
163	117	mul = max_khz;
164		− div = cpu->perf_caps.highest_perf;
	118	+ div = caps->highest_perf;
165	119	}
166	120	return (u64)perf * mul / div;
167	121	}
@@ -184,7 +138,7 @@ static unsigned int cppc_cpufreq_khz_to_perf(struct cppc_cpudata *cpu, · 16 unmodified lines
184	138	} else {
185	139	if (!max_khz)
186	140	max_khz = cppc_get_dmi_max_khz();
187		− mul = cpu->perf_caps.highest_perf;
	141	+ mul = caps->highest_perf;
188	142	div = max_khz;
189	143	}
190	144
@@ -402,9 +356,6 @@ static unsigned int cppc_cpufreq_get_rate(unsigned int cpunum) · 211 unmodified lines
402	356	struct cppc_cpudata *cpu = all_cpu_data[cpunum];
403	357	int ret;
404	358
405		− if (apply_hisi_workaround)
406		− return hisi_cppc_cpufreq_get_rate(cpunum);
407		−
408	359	ret = cppc_get_perf_ctrs(cpunum, &fb_ctrs_t0);
409	360	if (ret)
410	361	return ret;
@@ -455,6 +406,48 @@ static struct cpufreq_driver cppc_cpufreq_driver = { · 44 unmodified lines
455	406	.name = "cppc_cpufreq",
456	407	};
457	408
	409	+/*
	410	+ * HISI platform does not support delivered performance counter and
	411	+ * reference performance counter. It can calculate the performance using the
	412	+ * platform specific mechanism. We reuse the desired performance register to
	413	+ * store the real performance calculated by the platform.
	414	+ */
	415	+static unsigned int hisi_cppc_cpufreq_get_rate(unsigned int cpunum)
	416	+{
	417	+ struct cppc_cpudata *cpudata = all_cpu_data[cpunum];
	418	+ u64 desired_perf;
	419	+ int ret;
	420	+
	421	+ ret = cppc_get_desired_perf(cpunum, &desired_perf);
	422	+ if (ret < 0)
	423	+ return -EIO;
	424	+
	425	+ return cppc_cpufreq_perf_to_khz(cpudata, desired_perf);
	426	+}
	427	+
	428	+static void cppc_check_hisi_workaround(void)
	429	+{
	430	+ struct acpi_table_header *tbl;
	431	+ acpi_status status = AE_OK;
	432	+ int i;
	433	+
	434	+ status = acpi_get_table(ACPI_SIG_PCCT, 0, &tbl);
	435	+ if (ACPI_FAILURE(status) \|\| !tbl)
	436	+ return;
	437	+
	438	+ for (i = 0; i < ARRAY_SIZE(wa_info); i++) {
	439	+ if (!memcmp(wa_info[i].oem_id, tbl->oem_id, ACPI_OEM_ID_SIZE) &&
	440	+ !memcmp(wa_info[i].oem_table_id, tbl->oem_table_id, ACPI_OEM_TABLE_ID_SIZE) &&
	441	+ wa_info[i].oem_revision == tbl->oem_revision) {
	442	+ /* Overwrite the get() callback */
	443	+ cppc_cpufreq_driver.get = hisi_cppc_cpufreq_get_rate;
	444	+ break;
	445	+ }
	446	+ }
	447	+
	448	+ acpi_put_table(tbl);
	449	+}
	450	+
458	451	static int __init cppc_cpufreq_init(void)
459	452	{
460	453	int i, ret = 0;

drivers/cpufreq/cpufreq-dt-platdev.c+2−0

@@ -132,6 +132,8 @@ static const struct of_device_id blacklist[] __initconst = { · 131 unmodified lines
132	132	{ .compatible = "qcom,apq8096", },
133	133	{ .compatible = "qcom,msm8996", },
134	134	{ .compatible = "qcom,qcs404", },
	135	+ { .compatible = "qcom,sc7180", },
	136	+ { .compatible = "qcom,sdm845", },
135	137
136	138	{ .compatible = "st,stih407", },
137	139	{ .compatible = "st,stih410", },

drivers/cpufreq/cpufreq.c+1−1

@@ -541,7 +541,7 @@ unsigned int cpufreq_driver_resolve_freq(struct cpufreq_policy *policy, · 540 unmodified lines
541	541	policy->cached_target_freq = target_freq;
542	542
543	543	if (cpufreq_driver->target_index) {
544		− int idx;
	544	+ unsigned int idx;
545	545
546	546	idx = cpufreq_frequency_table_target(policy, target_freq,
547	547	CPUFREQ_RELATION_L);

drivers/cpufreq/intel_pstate.c+1−0

@@ -1647,6 +1647,7 @@ static void intel_pstate_get_cpu_pstates(struct cpudata *cpu) · 1646 unmodified lines
1647	1647
1648	1648	intel_pstate_get_hwp_max(cpu->cpu, &phy_max, &current_max);
1649	1649	cpu->pstate.turbo_freq = phy_max * cpu->pstate.scaling;
	1650	+ cpu->pstate.turbo_pstate = phy_max;
1650	1651	} else {
1651	1652	cpu->pstate.turbo_freq = cpu->pstate.turbo_pstate * cpu->pstate.scaling;
1652	1653	}

drivers/cpufreq/qcom-cpufreq-hw.c+83−8

@@ -6,6 +6,7 @@ · 5 unmodified lines
6	6	#include <linux/bitfield.h>
7	7	#include <linux/cpufreq.h>
8	8	#include <linux/init.h>
	9	+#include <linux/interconnect.h>
9	10	#include <linux/kernel.h>
10	11	#include <linux/module.h>
11	12	#include <linux/of_address.h>
@@ -30,6 +31,48 @@ · 18 unmodified lines
30	31
31	32	static unsigned long cpu_hw_rate, xo_rate;
32	33	static struct platform_device *global_pdev;
	34	+static bool icc_scaling_enabled;
	35	+
	36	+static int qcom_cpufreq_set_bw(struct cpufreq_policy *policy,
	37	+ unsigned long freq_khz)
	38	+{
	39	+ unsigned long freq_hz = freq_khz * 1000;
	40	+ struct dev_pm_opp *opp;
	41	+ struct device *dev;
	42	+ int ret;
	43	+
	44	+ dev = get_cpu_device(policy->cpu);
	45	+ if (!dev)
	46	+ return -ENODEV;
	47	+
	48	+ opp = dev_pm_opp_find_freq_exact(dev, freq_hz, true);
	49	+ if (IS_ERR(opp))
	50	+ return PTR_ERR(opp);
	51	+
	52	+ ret = dev_pm_opp_set_bw(dev, opp);
	53	+ dev_pm_opp_put(opp);
	54	+ return ret;
	55	+}
	56	+
	57	+static int qcom_cpufreq_update_opp(struct device *cpu_dev,
	58	+ unsigned long freq_khz,
	59	+ unsigned long volt)
	60	+{
	61	+ unsigned long freq_hz = freq_khz * 1000;
	62	+ int ret;
	63	+
	64	+ /* Skip voltage update if the opp table is not available */
	65	+ if (!icc_scaling_enabled)
	66	+ return dev_pm_opp_add(cpu_dev, freq_hz, volt);
	67	+
	68	+ ret = dev_pm_opp_adjust_voltage(cpu_dev, freq_hz, volt, volt, volt);
	69	+ if (ret) {
	70	+ dev_err(cpu_dev, "Voltage update failed freq=%ld\n", freq_khz);
	71	+ return ret;
	72	+ }
	73	+
	74	+ return dev_pm_opp_enable(cpu_dev, freq_hz);
	75	+}
33	76
34	77	static int qcom_cpufreq_hw_target_index(struct cpufreq_policy *policy,
35	78	unsigned int index)
@@ -39,6 +82,9 @@ static int qcom_cpufreq_hw_target_index(struct cpufreq_policy *policy, · 3 unmodified lines
39	82
40	83	writel_relaxed(index, perf_state_reg);
41	84
	85	+ if (icc_scaling_enabled)
	86	+ qcom_cpufreq_set_bw(policy, freq);
	87	+
42	88	arch_set_freq_scale(policy->related_cpus, freq,
43	89	policy->cpuinfo.max_freq);
44	90	return 0;
@@ -66,13 +112,10 @@ static unsigned int qcom_cpufreq_hw_fast_switch(struct cpufreq_policy *policy, · 21 unmodified lines
66	112	unsigned int target_freq)
67	113	{
68	114	void __iomem *perf_state_reg = policy->driver_data;
69		− int index;
	115	+ unsigned int index;
70	116	unsigned long freq;
71	117
72	118	index = policy->cached_resolved_idx;
73		− if (index < 0)
74		− return 0;
75		−
76	119	writel_relaxed(index, perf_state_reg);
77	120
78	121	freq = policy->freq_table[index].frequency;
@@ -89,11 +132,34 @@ static int qcom_cpufreq_hw_read_lut(struct device *cpu_dev, · 10 unmodified lines
89	132	u32 data, src, lval, i, core_count, prev_freq = 0, freq;
90	133	u32 volt;
91	134	struct cpufreq_frequency_table *table;
	135	+ struct dev_pm_opp *opp;
	136	+ unsigned long rate;
	137	+ int ret;
92	138
93	139	table = kcalloc(LUT_MAX_ENTRIES + 1, sizeof(*table), GFP_KERNEL);
94	140	if (!table)
95	141	return -ENOMEM;
96	142
	143	+ ret = dev_pm_opp_of_add_table(cpu_dev);
	144	+ if (!ret) {
	145	+ /* Disable all opps and cross-validate against LUT later */
	146	+ icc_scaling_enabled = true;
	147	+ for (rate = 0; ; rate++) {
	148	+ opp = dev_pm_opp_find_freq_ceil(cpu_dev, &rate);
	149	+ if (IS_ERR(opp))
	150	+ break;
	151	+
	152	+ dev_pm_opp_put(opp);
	153	+ dev_pm_opp_disable(cpu_dev, rate);
	154	+ }
	155	+ } else if (ret != -ENODEV) {
	156	+ dev_err(cpu_dev, "Invalid opp table in device tree\n");
	157	+ return ret;
	158	+ } else {
	159	+ policy->fast_switch_possible = true;
	160	+ icc_scaling_enabled = false;
	161	+ }
	162	+
97	163	for (i = 0; i < LUT_MAX_ENTRIES; i++) {
98	164	data = readl_relaxed(base + REG_FREQ_LUT +
99	165	i * LUT_ROW_SIZE);
@@ -112,7 +178,7 @@ static int qcom_cpufreq_hw_read_lut(struct device *cpu_dev, · 12 unmodified lines
112	178
113	179	if (freq != prev_freq && core_count != LUT_TURBO_IND) {
114	180	table[i].frequency = freq;
115		− dev_pm_opp_add(cpu_dev, freq * 1000, volt);
	181	+ qcom_cpufreq_update_opp(cpu_dev, freq, volt);
116	182	dev_dbg(cpu_dev, "index=%d freq=%d, core_count %d\n", i,
117	183	freq, core_count);
118	184	} else if (core_count == LUT_TURBO_IND) {
@@ -133,7 +199,7 @@ static int qcom_cpufreq_hw_read_lut(struct device *cpu_dev, · 14 unmodified lines
133	199	if (prev->frequency == CPUFREQ_ENTRY_INVALID) {
134	200	prev->frequency = prev_freq;
135	201	prev->flags = CPUFREQ_BOOST_FREQ;
136		− dev_pm_opp_add(cpu_dev, prev_freq * 1000, volt);
	202	+ qcom_cpufreq_update_opp(cpu_dev, prev_freq, volt);
137	203	}
138	204
139	205	break;
@@ -240,8 +306,6 @@ static int qcom_cpufreq_hw_cpu_init(struct cpufreq_policy *policy) · 100 unmodified lines
240	306
241	307	dev_pm_opp_of_register_em(cpu_dev, policy->cpus);
242	308
243		− policy->fast_switch_possible = true;
244		−
245	309	return 0;
246	310	error:
247	311	devm_iounmap(dev, base);
@@ -254,6 +318,7 @@ static int qcom_cpufreq_hw_cpu_exit(struct cpufreq_policy *policy) · 6 unmodified lines
254	318	void __iomem *base = policy->driver_data - REG_PERF_STATE;
255	319
256	320	dev_pm_opp_remove_all_dynamic(cpu_dev);
	321	+ dev_pm_opp_of_cpumask_remove_table(policy->related_cpus);
257	322	kfree(policy->freq_table);
258	323	devm_iounmap(&global_pdev->dev, base);
259	324
@@ -282,6 +347,7 @@ static struct cpufreq_driver cpufreq_qcom_hw_driver = { · 22 unmodified lines
282	347
283	348	static int qcom_cpufreq_hw_driver_probe(struct platform_device *pdev)
284	349	{
	350	+ struct device *cpu_dev;
285	351	struct clk *clk;
286	352	int ret;
287	353
@@ -301,6 +367,15 @@ static int qcom_cpufreq_hw_driver_probe(struct platform_device *pdev) · 13 unmodified lines
301	367
302	368	global_pdev = pdev;
303	369
	370	+ /* Check for optional interconnect paths on CPU0 */
	371	+ cpu_dev = get_cpu_device(0);
	372	+ if (!cpu_dev)
	373	+ return -EPROBE_DEFER;
	374	+
	375	+ ret = dev_pm_opp_of_find_icc_paths(cpu_dev, NULL);
	376	+ if (ret)
	377	+ return ret;
	378	+
304	379	ret = cpufreq_register_driver(&cpufreq_qcom_hw_driver);
305	380	if (ret)
306	381	dev_err(&pdev->dev, "CPUFreq HW driver failed to register\n");

drivers/cpufreq/sti-cpufreq.c+4−4

@@ -40,11 +40,11 @@ enum { · 39 unmodified lines
40	40	};
41	41
42	42	/**
43		− * ST CPUFreq Driver Data
	43	+ * struct sti_cpufreq_ddata - ST CPUFreq Driver Data
44	44	*
45		− * @cpu_node CPU's OF node
46		− * @syscfg_eng Engineering Syscon register map
47		− * @regmap Syscon register map
	45	+ * @cpu: CPU's OF node
	46	+ * @syscfg_eng: Engineering Syscon register map
	47	+ * @syscfg: Syscon register map
48	48	*/
49	49	static struct sti_cpufreq_ddata {
50	50	struct device *cpu;

drivers/cpufreq/tegra186-cpufreq.c+0−6

@@ -223,15 +223,9 @@ static int tegra186_cpufreq_probe(struct platform_device *pdev) · 222 unmodified lines
223	223	}
224	224	}
225	225
226		− tegra_bpmp_put(bpmp);
227		−
228	226	tegra186_cpufreq_driver.driver_data = data;
229	227
230	228	err = cpufreq_register_driver(&tegra186_cpufreq_driver);
231		− if (err)
232		− return err;
233		−
234		− return 0;
235	229
236	230	put_bpmp:
237	231	tegra_bpmp_put(bpmp);

drivers/cpufreq/tegra194-cpufreq.c+390−0

@@ -0,0 +1,390 @@
	1	+// SPDX-License-Identifier: GPL-2.0
	2	+/*
	3	+ * Copyright (c) 2020, NVIDIA CORPORATION. All rights reserved
	4	+ */
	5	+
	6	+#include <linux/cpu.h>
	7	+#include <linux/cpufreq.h>
	8	+#include <linux/delay.h>
	9	+#include <linux/dma-mapping.h>
	10	+#include <linux/module.h>
	11	+#include <linux/of.h>
	12	+#include <linux/of_platform.h>
	13	+#include <linux/platform_device.h>
	14	+#include <linux/slab.h>
	15	+
	16	+#include <asm/smp_plat.h>
	17	+
	18	+#include <soc/tegra/bpmp.h>
	19	+#include <soc/tegra/bpmp-abi.h>
	20	+
	21	+#define KHZ 1000
	22	+#define REF_CLK_MHZ 408 /* 408 MHz */
	23	+#define US_DELAY 500
	24	+#define US_DELAY_MIN 2
	25	+#define CPUFREQ_TBL_STEP_HZ (50 * KHZ * KHZ)
	26	+#define MAX_CNT ~0U
	27	+
	28	+/* cpufreq transisition latency */
	29	+#define TEGRA_CPUFREQ_TRANSITION_LATENCY (300 * 1000) /* unit in nanoseconds */
	30	+
	31	+enum cluster {
	32	+ CLUSTER0,
	33	+ CLUSTER1,
	34	+ CLUSTER2,
	35	+ CLUSTER3,
	36	+ MAX_CLUSTERS,
	37	+};
	38	+
	39	+struct tegra194_cpufreq_data {
	40	+ void __iomem *regs;
	41	+ size_t num_clusters;
	42	+ struct cpufreq_frequency_table **tables;
	43	+};
	44	+
	45	+struct tegra_cpu_ctr {
	46	+ u32 cpu;
	47	+ u32 delay;
	48	+ u32 coreclk_cnt, last_coreclk_cnt;
	49	+ u32 refclk_cnt, last_refclk_cnt;
	50	+};
	51	+
	52	+struct read_counters_work {
	53	+ struct work_struct work;
	54	+ struct tegra_cpu_ctr c;
	55	+};
	56	+
	57	+static struct workqueue_struct *read_counters_wq;
	58	+
	59	+static enum cluster get_cpu_cluster(u8 cpu)
	60	+{
	61	+ return MPIDR_AFFINITY_LEVEL(cpu_logical_map(cpu), 1);
	62	+}
	63	+
	64	+/*
	65	+ * Read per-core Read-only system register NVFREQ_FEEDBACK_EL1.
	66	+ * The register provides frequency feedback information to
	67	+ * determine the average actual frequency a core has run at over
	68	+ * a period of time.
	69	+ * [31:0] PLLP counter: Counts at fixed frequency (408 MHz)
	70	+ * [63:32] Core clock counter: counts on every core clock cycle
	71	+ * where the core is architecturally clocking
	72	+ */
	73	+static u64 read_freq_feedback(void)
	74	+{
	75	+ u64 val = 0;
	76	+
	77	+ asm volatile("mrs %0, s3_0_c15_c0_5" : "=r" (val) : );
	78	+
	79	+ return val;
	80	+}
	81	+
	82	+static inline u32 map_ndiv_to_freq(struct mrq_cpu_ndiv_limits_response
	83	+ *nltbl, u16 ndiv)
	84	+{
	85	+ return nltbl->ref_clk_hz / KHZ * ndiv / (nltbl->pdiv * nltbl->mdiv);
	86	+}
	87	+
	88	+static void tegra_read_counters(struct work_struct *work)
	89	+{
	90	+ struct read_counters_work *read_counters_work;
	91	+ struct tegra_cpu_ctr *c;
	92	+ u64 val;
	93	+
	94	+ /*
	95	+ * ref_clk_counter(32 bit counter) runs on constant clk,
	96	+ * pll_p(408MHz).
	97	+ * It will take = 2 ^ 32 / 408 MHz to overflow ref clk counter
	98	+ * = 10526880 usec = 10.527 sec to overflow
	99	+ *
	100	+ * Like wise core_clk_counter(32 bit counter) runs on core clock.
	101	+ * It's synchronized to crab_clk (cpu_crab_clk) which runs at
	102	+ * freq of cluster. Assuming max cluster clock ~2000MHz,
	103	+ * It will take = 2 ^ 32 / 2000 MHz to overflow core clk counter
	104	+ * = ~2.147 sec to overflow
	105	+ */
	106	+ read_counters_work = container_of(work, struct read_counters_work,
	107	+ work);
	108	+ c = &read_counters_work->c;
	109	+
	110	+ val = read_freq_feedback();
	111	+ c->last_refclk_cnt = lower_32_bits(val);
	112	+ c->last_coreclk_cnt = upper_32_bits(val);
	113	+ udelay(c->delay);
	114	+ val = read_freq_feedback();
	115	+ c->refclk_cnt = lower_32_bits(val);
	116	+ c->coreclk_cnt = upper_32_bits(val);
	117	+}
	118	+
	119	+/*
	120	+ * Return instantaneous cpu speed
	121	+ * Instantaneous freq is calculated as -
	122	+ * -Takes sample on every query of getting the freq.
	123	+ * - Read core and ref clock counters;
	124	+ * - Delay for X us
	125	+ * - Read above cycle counters again
	126	+ * - Calculates freq by subtracting current and previous counters
	127	+ * divided by the delay time or eqv. of ref_clk_counter in delta time
	128	+ * - Return Kcycles/second, freq in KHz
	129	+ *
	130	+ * delta time period = x sec
	131	+ * = delta ref_clk_counter / (408 * 10^6) sec
	132	+ * freq in Hz = cycles/sec
	133	+ * = (delta cycles / x sec
	134	+ * = (delta cycles * 408 * 10^6) / delta ref_clk_counter
	135	+ * in KHz = (delta cycles * 408 * 10^3) / delta ref_clk_counter
	136	+ *
	137	+ * @cpu - logical cpu whose freq to be updated
	138	+ * Returns freq in KHz on success, 0 if cpu is offline
	139	+ */
	140	+static unsigned int tegra194_get_speed_common(u32 cpu, u32 delay)
	141	+{
	142	+ struct read_counters_work read_counters_work;
	143	+ struct tegra_cpu_ctr c;
	144	+ u32 delta_refcnt;
	145	+ u32 delta_ccnt;
	146	+ u32 rate_mhz;
	147	+
	148	+ /*
	149	+ * udelay() is required to reconstruct cpu frequency over an
	150	+ * observation window. Using workqueue to call udelay() with
	151	+ * interrupts enabled.
	152	+ */
	153	+ read_counters_work.c.cpu = cpu;
	154	+ read_counters_work.c.delay = delay;
	155	+ INIT_WORK_ONSTACK(&read_counters_work.work, tegra_read_counters);
	156	+ queue_work_on(cpu, read_counters_wq, &read_counters_work.work);
	157	+ flush_work(&read_counters_work.work);
	158	+ c = read_counters_work.c;
	159	+
	160	+ if (c.coreclk_cnt < c.last_coreclk_cnt)
	161	+ delta_ccnt = c.coreclk_cnt + (MAX_CNT - c.last_coreclk_cnt);
	162	+ else
	163	+ delta_ccnt = c.coreclk_cnt - c.last_coreclk_cnt;
	164	+ if (!delta_ccnt)
	165	+ return 0;
	166	+
	167	+ /* ref clock is 32 bits */
	168	+ if (c.refclk_cnt < c.last_refclk_cnt)
	169	+ delta_refcnt = c.refclk_cnt + (MAX_CNT - c.last_refclk_cnt);
	170	+ else
	171	+ delta_refcnt = c.refclk_cnt - c.last_refclk_cnt;
	172	+ if (!delta_refcnt) {
	173	+ pr_debug("cpufreq: %d is idle, delta_refcnt: 0\n", cpu);
	174	+ return 0;
	175	+ }
	176	+ rate_mhz = ((unsigned long)(delta_ccnt * REF_CLK_MHZ)) / delta_refcnt;
	177	+
	178	+ return (rate_mhz * KHZ); /* in KHz */
	179	+}
	180	+
	181	+static unsigned int tegra194_get_speed(u32 cpu)
	182	+{
	183	+ return tegra194_get_speed_common(cpu, US_DELAY);
	184	+}
	185	+
	186	+static int tegra194_cpufreq_init(struct cpufreq_policy *policy)
	187	+{
	188	+ struct tegra194_cpufreq_data *data = cpufreq_get_driver_data();
	189	+ int cl = get_cpu_cluster(policy->cpu);
	190	+ u32 cpu;
	191	+
	192	+ if (cl >= data->num_clusters)
	193	+ return -EINVAL;
	194	+
	195	+ /* boot freq */
	196	+ policy->cur = tegra194_get_speed_common(policy->cpu, US_DELAY_MIN);
	197	+
	198	+ /* set same policy for all cpus in a cluster */
	199	+ for (cpu = (cl * 2); cpu < ((cl + 1) * 2); cpu++)
	200	+ cpumask_set_cpu(cpu, policy->cpus);
	201	+
	202	+ policy->freq_table = data->tables[cl];
	203	+ policy->cpuinfo.transition_latency = TEGRA_CPUFREQ_TRANSITION_LATENCY;
	204	+
	205	+ return 0;
	206	+}
	207	+
	208	+static void set_cpu_ndiv(void *data)
	209	+{
	210	+ struct cpufreq_frequency_table *tbl = data;
	211	+ u64 ndiv_val = (u64)tbl->driver_data;
	212	+
	213	+ asm volatile("msr s3_0_c15_c0_4, %0" : : "r" (ndiv_val));
	214	+}
	215	+
	216	+static int tegra194_cpufreq_set_target(struct cpufreq_policy *policy,
	217	+ unsigned int index)
	218	+{
	219	+ struct cpufreq_frequency_table *tbl = policy->freq_table + index;
	220	+
	221	+ /*
	222	+ * Each core writes frequency in per core register. Then both cores
	223	+ * in a cluster run at same frequency which is the maximum frequency
	224	+ * request out of the values requested by both cores in that cluster.
	225	+ */
	226	+ on_each_cpu_mask(policy->cpus, set_cpu_ndiv, tbl, true);
	227	+
	228	+ return 0;
	229	+}
	230	+
	231	+static struct cpufreq_driver tegra194_cpufreq_driver = {
	232	+ .name = "tegra194",
	233	+ .flags = CPUFREQ_STICKY \| CPUFREQ_CONST_LOOPS \|
	234	+ CPUFREQ_NEED_INITIAL_FREQ_CHECK,
	235	+ .verify = cpufreq_generic_frequency_table_verify,
	236	+ .target_index = tegra194_cpufreq_set_target,
	237	+ .get = tegra194_get_speed,
	238	+ .init = tegra194_cpufreq_init,
	239	+ .attr = cpufreq_generic_attr,
	240	+};
	241	+
	242	+static void tegra194_cpufreq_free_resources(void)
	243	+{
	244	+ destroy_workqueue(read_counters_wq);
	245	+}
	246	+
	247	+static struct cpufreq_frequency_table *
	248	+init_freq_table(struct platform_device pdev, struct tegra_bpmp bpmp,
	249	+ unsigned int cluster_id)
	250	+{
	251	+ struct cpufreq_frequency_table *freq_table;
	252	+ struct mrq_cpu_ndiv_limits_response resp;
	253	+ unsigned int num_freqs, ndiv, delta_ndiv;
	254	+ struct mrq_cpu_ndiv_limits_request req;
	255	+ struct tegra_bpmp_message msg;
	256	+ u16 freq_table_step_size;
	257	+ int err, index;
	258	+
	259	+ memset(&req, 0, sizeof(req));
	260	+ req.cluster_id = cluster_id;
	261	+
	262	+ memset(&msg, 0, sizeof(msg));
	263	+ msg.mrq = MRQ_CPU_NDIV_LIMITS;
	264	+ msg.tx.data = &req;
	265	+ msg.tx.size = sizeof(req);
	266	+ msg.rx.data = &resp;
	267	+ msg.rx.size = sizeof(resp);
	268	+
	269	+ err = tegra_bpmp_transfer(bpmp, &msg);
	270	+ if (err)
	271	+ return ERR_PTR(err);
	272	+
	273	+ /*
	274	+ * Make sure frequency table step is a multiple of mdiv to match
	275	+ * vhint table granularity.
	276	+ */
	277	+ freq_table_step_size = resp.mdiv *
	278	+ DIV_ROUND_UP(CPUFREQ_TBL_STEP_HZ, resp.ref_clk_hz);
	279	+
	280	+ dev_dbg(&pdev->dev, "cluster %d: frequency table step size: %d\n",
	281	+ cluster_id, freq_table_step_size);
	282	+
	283	+ delta_ndiv = resp.ndiv_max - resp.ndiv_min;
	284	+
	285	+ if (unlikely(delta_ndiv == 0)) {
	286	+ num_freqs = 1;
	287	+ } else {
	288	+ /* We store both ndiv_min and ndiv_max hence the +1 */
	289	+ num_freqs = delta_ndiv / freq_table_step_size + 1;
	290	+ }
	291	+
	292	+ num_freqs += (delta_ndiv % freq_table_step_size) ? 1 : 0;
	293	+
	294	+ freq_table = devm_kcalloc(&pdev->dev, num_freqs + 1,
	295	+ sizeof(*freq_table), GFP_KERNEL);
	296	+ if (!freq_table)
	297	+ return ERR_PTR(-ENOMEM);
	298	+
	299	+ for (index = 0, ndiv = resp.ndiv_min;
	300	+ ndiv < resp.ndiv_max;
	301	+ index++, ndiv += freq_table_step_size) {
	302	+ freq_table[index].driver_data = ndiv;
	303	+ freq_table[index].frequency = map_ndiv_to_freq(&resp, ndiv);
	304	+ }
	305	+
	306	+ freq_table[index].driver_data = resp.ndiv_max;
	307	+ freq_table[index++].frequency = map_ndiv_to_freq(&resp, resp.ndiv_max);
	308	+ freq_table[index].frequency = CPUFREQ_TABLE_END;
	309	+
	310	+ return freq_table;
	311	+}
	312	+
	313	+static int tegra194_cpufreq_probe(struct platform_device *pdev)
	314	+{
	315	+ struct tegra194_cpufreq_data *data;
	316	+ struct tegra_bpmp *bpmp;
	317	+ int err, i;
	318	+
	319	+ data = devm_kzalloc(&pdev->dev, sizeof(*data), GFP_KERNEL);
	320	+ if (!data)
	321	+ return -ENOMEM;
	322	+
	323	+ data->num_clusters = MAX_CLUSTERS;
	324	+ data->tables = devm_kcalloc(&pdev->dev, data->num_clusters,
	325	+ sizeof(*data->tables), GFP_KERNEL);
	326	+ if (!data->tables)
	327	+ return -ENOMEM;
	328	+
	329	+ platform_set_drvdata(pdev, data);
	330	+
	331	+ bpmp = tegra_bpmp_get(&pdev->dev);
	332	+ if (IS_ERR(bpmp))
	333	+ return PTR_ERR(bpmp);
	334	+
	335	+ read_counters_wq = alloc_workqueue("read_counters_wq", __WQ_LEGACY, 1);
	336	+ if (!read_counters_wq) {
	337	+ dev_err(&pdev->dev, "fail to create_workqueue\n");
	338	+ err = -EINVAL;
	339	+ goto put_bpmp;
	340	+ }
	341	+
	342	+ for (i = 0; i < data->num_clusters; i++) {
	343	+ data->tables[i] = init_freq_table(pdev, bpmp, i);
	344	+ if (IS_ERR(data->tables[i])) {
	345	+ err = PTR_ERR(data->tables[i]);
	346	+ goto err_free_res;
	347	+ }
	348	+ }
	349	+
	350	+ tegra194_cpufreq_driver.driver_data = data;
	351	+
	352	+ err = cpufreq_register_driver(&tegra194_cpufreq_driver);
	353	+ if (!err)
	354	+ goto put_bpmp;
	355	+
	356	+err_free_res:
	357	+ tegra194_cpufreq_free_resources();
	358	+put_bpmp:
	359	+ tegra_bpmp_put(bpmp);
	360	+ return err;
	361	+}
	362	+
	363	+static int tegra194_cpufreq_remove(struct platform_device *pdev)
	364	+{
	365	+ cpufreq_unregister_driver(&tegra194_cpufreq_driver);
	366	+ tegra194_cpufreq_free_resources();
	367	+
	368	+ return 0;
	369	+}
	370	+
	371	+static const struct of_device_id tegra194_cpufreq_of_match[] = {
	372	+ { .compatible = "nvidia,tegra194-ccplex", },
	373	+ { /* sentinel */ }
	374	+};
	375	+MODULE_DEVICE_TABLE(of, tegra194_cpufreq_of_match);
	376	+
	377	+static struct platform_driver tegra194_ccplex_driver = {
	378	+ .driver = {
	379	+ .name = "tegra194-cpufreq",
	380	+ .of_match_table = tegra194_cpufreq_of_match,
	381	+ },
	382	+ .probe = tegra194_cpufreq_probe,
	383	+ .remove = tegra194_cpufreq_remove,
	384	+};
	385	+module_platform_driver(tegra194_ccplex_driver);
	386	+
	387	+MODULE_AUTHOR("Mikko Perttunen <mperttunen@nvidia.com>");
	388	+MODULE_AUTHOR("Sumit Gupta <sumitg@nvidia.com>");
	389	+MODULE_DESCRIPTION("NVIDIA Tegra194 cpufreq driver");
	390	+MODULE_LICENSE("GPL v2");

drivers/opp/core.c+31−0

@@ -831,6 +831,37 @@ static int _set_required_opps(struct device *dev, · 830 unmodified lines
831	831	return ret;
832	832	}
833	833
	834	+/**
	835	+ * dev_pm_opp_set_bw() - sets bandwidth levels corresponding to an opp
	836	+ * @dev: device for which we do this operation
	837	+ * @opp: opp based on which the bandwidth levels are to be configured
	838	+ *
	839	+ * This configures the bandwidth to the levels specified by the OPP. However
	840	+ * if the OPP specified is NULL the bandwidth levels are cleared out.
	841	+ *
	842	+ * Return: 0 on success or a negative error value.
	843	+ */
	844	+int dev_pm_opp_set_bw(struct device dev, struct dev_pm_opp opp)
	845	+{
	846	+ struct opp_table *opp_table;
	847	+ int ret;
	848	+
	849	+ opp_table = _find_opp_table(dev);
	850	+ if (IS_ERR(opp_table)) {
	851	+ dev_err(dev, "%s: device opp table doesn't exist\n", __func__);
	852	+ return PTR_ERR(opp_table);
	853	+ }
	854	+
	855	+ if (opp)
	856	+ ret = _set_opp_bw(opp_table, opp, dev, false);
	857	+ else
	858	+ ret = _set_opp_bw(opp_table, NULL, dev, true);
	859	+
	860	+ dev_pm_opp_put_opp_table(opp_table);
	861	+ return ret;
	862	+}
	863	+EXPORT_SYMBOL_GPL(dev_pm_opp_set_bw);
	864	+
834	865	/**
835	866	* dev_pm_opp_set_rate() - Configure new OPP based on frequency
836	867	* @dev: device for which we do this operation

include/linux/cpufreq.h+1−1

@@ -127,7 +127,7 @@ struct cpufreq_policy { · 126 unmodified lines
127	127
128	128	/* Cached frequency lookup from cpufreq_driver_resolve_freq. */
129	129	unsigned int cached_target_freq;
130		− int cached_resolved_idx;
	130	+ unsigned int cached_resolved_idx;
131	131
132	132	/* Synchronization for frequency transitions */
133	133	bool transition_ongoing; /* Tracks transition status */

include/linux/pm_opp.h+6−0

@@ -152,6 +152,7 @@ struct opp_table dev_pm_opp_attach_genpd(struct device dev, const char **names · 151 unmodified lines
152	152	void dev_pm_opp_detach_genpd(struct opp_table *opp_table);
153	153	int dev_pm_opp_xlate_performance_state(struct opp_table src_table, struct opp_table dst_table, unsigned int pstate);
154	154	int dev_pm_opp_set_rate(struct device *dev, unsigned long target_freq);
	155	+int dev_pm_opp_set_bw(struct device dev, struct dev_pm_opp opp);
155	156	int dev_pm_opp_set_sharing_cpus(struct device cpu_dev, const struct cpumask cpumask);
156	157	int dev_pm_opp_get_sharing_cpus(struct device cpu_dev, struct cpumask cpumask);
157	158	void dev_pm_opp_remove_table(struct device *dev);
@@ -343,6 +344,11 @@ static inline int dev_pm_opp_set_rate(struct device *dev, unsigned long target_f · 185 unmodified lines
343	344	return -ENOTSUPP;
344	345	}
345	346
	347	+static inline int dev_pm_opp_set_bw(struct device dev, struct dev_pm_opp opp)
	348	+{
	349	+ return -EOPNOTSUPP;
	350	+}
	351	+
346	352	static inline int dev_pm_opp_set_sharing_cpus(struct device cpu_dev, const struct cpumask cpumask)
347	353	{
348	354	return -ENOTSUPP;

include/linux/pm_runtime.h+246−0

@@ -60,58 +60,151 @@ extern void pm_runtime_put_suppliers(struct device *dev); · 59 unmodified lines
60	60	extern void pm_runtime_new_link(struct device *dev);
61	61	extern void pm_runtime_drop_link(struct device *dev);
62	62
	63	+/**
	64	+ * pm_runtime_get_if_in_use - Conditionally bump up runtime PM usage counter.
	65	+ * @dev: Target device.
	66	+ *
	67	+ * Increment the runtime PM usage counter of @dev if its runtime PM status is
	68	+ * %RPM_ACTIVE and its runtime PM usage counter is greater than 0.
	69	+ */
63	70	static inline int pm_runtime_get_if_in_use(struct device *dev)
64	71	{
65	72	return pm_runtime_get_if_active(dev, false);
66	73	}
67	74
	75	+/**
	76	+ * pm_suspend_ignore_children - Set runtime PM behavior regarding children.
	77	+ * @dev: Target device.
	78	+ * @enable: Whether or not to ignore possible dependencies on children.
	79	+ *
	80	+ * The dependencies of @dev on its children will not be taken into account by
	81	+ * the runtime PM framework going forward if @enable is %true, or they will
	82	+ * be taken into account otherwise.
	83	+ */
68	84	static inline void pm_suspend_ignore_children(struct device *dev, bool enable)
69	85	{
70	86	dev->power.ignore_children = enable;
71	87	}
72	88
	89	+/**
	90	+ * pm_runtime_get_noresume - Bump up runtime PM usage counter of a device.
	91	+ * @dev: Target device.
	92	+ */
73	93	static inline void pm_runtime_get_noresume(struct device *dev)
74	94	{
75	95	atomic_inc(&dev->power.usage_count);
76	96	}
77	97
	98	+/**
	99	+ * pm_runtime_put_noidle - Drop runtime PM usage counter of a device.
	100	+ * @dev: Target device.
	101	+ *
	102	+ * Decrement the runtime PM usage counter of @dev unless it is 0 already.
	103	+ */
78	104	static inline void pm_runtime_put_noidle(struct device *dev)
79	105	{
80	106	atomic_add_unless(&dev->power.usage_count, -1, 0);
81	107	}
82	108
	109	+/**
	110	+ * pm_runtime_suspended - Check whether or not a device is runtime-suspended.
	111	+ * @dev: Target device.
	112	+ *
	113	+ * Return %true if runtime PM is enabled for @dev and its runtime PM status is
	114	+ * %RPM_SUSPENDED, or %false otherwise.
	115	+ *
	116	+ * Note that the return value of this function can only be trusted if it is
	117	+ * called under the runtime PM lock of @dev or under conditions in which
	118	+ * runtime PM cannot be either disabled or enabled for @dev and its runtime PM
	119	+ * status cannot change.
	120	+ */
83	121	static inline bool pm_runtime_suspended(struct device *dev)
84	122	{
85	123	return dev->power.runtime_status == RPM_SUSPENDED
86	124	&& !dev->power.disable_depth;
87	125	}
88	126
	127	+/**
	128	+ * pm_runtime_active - Check whether or not a device is runtime-active.
	129	+ * @dev: Target device.
	130	+ *
	131	+ * Return %true if runtime PM is enabled for @dev and its runtime PM status is
	132	+ * %RPM_ACTIVE, or %false otherwise.
	133	+ *
	134	+ * Note that the return value of this function can only be trusted if it is
	135	+ * called under the runtime PM lock of @dev or under conditions in which
	136	+ * runtime PM cannot be either disabled or enabled for @dev and its runtime PM
	137	+ * status cannot change.
	138	+ */
89	139	static inline bool pm_runtime_active(struct device *dev)
90	140	{
91	141	return dev->power.runtime_status == RPM_ACTIVE
92	142	\|\| dev->power.disable_depth;
93	143	}
94	144
	145	+/**
	146	+ * pm_runtime_status_suspended - Check if runtime PM status is "suspended".
	147	+ * @dev: Target device.
	148	+ *
	149	+ * Return %true if the runtime PM status of @dev is %RPM_SUSPENDED, or %false
	150	+ * otherwise, regardless of whether or not runtime PM has been enabled for @dev.
	151	+ *
	152	+ * Note that the return value of this function can only be trusted if it is
	153	+ * called under the runtime PM lock of @dev or under conditions in which the
	154	+ * runtime PM status of @dev cannot change.
	155	+ */
95	156	static inline bool pm_runtime_status_suspended(struct device *dev)
96	157	{
97	158	return dev->power.runtime_status == RPM_SUSPENDED;
98	159	}
99	160
	161	+/**
	162	+ * pm_runtime_enabled - Check if runtime PM is enabled.
	163	+ * @dev: Target device.
	164	+ *
	165	+ * Return %true if runtime PM is enabled for @dev or %false otherwise.
	166	+ *
	167	+ * Note that the return value of this function can only be trusted if it is
	168	+ * called under the runtime PM lock of @dev or under conditions in which
	169	+ * runtime PM cannot be either disabled or enabled for @dev.
	170	+ */
100	171	static inline bool pm_runtime_enabled(struct device *dev)
101	172	{
102	173	return !dev->power.disable_depth;
103	174	}
104	175
	176	+/**
	177	+ * pm_runtime_has_no_callbacks - Check if runtime PM callbacks may be present.
	178	+ * @dev: Target device.
	179	+ *
	180	+ * Return %true if @dev is a special device without runtime PM callbacks or
	181	+ * %false otherwise.
	182	+ */
105	183	static inline bool pm_runtime_has_no_callbacks(struct device *dev)
106	184	{
107	185	return dev->power.no_callbacks;
108	186	}
109	187
	188	+/**
	189	+ * pm_runtime_mark_last_busy - Update the last access time of a device.
	190	+ * @dev: Target device.
	191	+ *
	192	+ * Update the last access time of @dev used by the runtime PM autosuspend
	193	+ * mechanism to the current time as returned by ktime_get_mono_fast_ns().
	194	+ */
110	195	static inline void pm_runtime_mark_last_busy(struct device *dev)
111	196	{
112	197	WRITE_ONCE(dev->power.last_busy, ktime_get_mono_fast_ns());
113	198	}
114	199
	200	+/**
	201	+ * pm_runtime_is_irq_safe - Check if runtime PM can work in interrupt context.
	202	+ * @dev: Target device.
	203	+ *
	204	+ * Return %true if @dev has been marked as an "IRQ-safe" device (with respect
	205	+ * to runtime PM), in which case its runtime PM callabcks can be expected to
	206	+ * work correctly when invoked from interrupt handlers.
	207	+ */
115	208	static inline bool pm_runtime_is_irq_safe(struct device *dev)
116	209	{
117	210	return dev->power.irq_safe;
@@ -191,97 +284,250 @@ static inline void pm_runtime_drop_link(struct device *dev) {} · 73 unmodified lines
191	284
192	285	#endif /* !CONFIG_PM */
193	286
	287	+/**
	288	+ * pm_runtime_idle - Conditionally set up autosuspend of a device or suspend it.
	289	+ * @dev: Target device.
	290	+ *
	291	+ * Invoke the "idle check" callback of @dev and, depending on its return value,
	292	+ * set up autosuspend of @dev or suspend it (depending on whether or not
	293	+ * autosuspend has been enabled for it).
	294	+ */
194	295	static inline int pm_runtime_idle(struct device *dev)
195	296	{
196	297	return __pm_runtime_idle(dev, 0);
197	298	}
198	299
	300	+/**
	301	+ * pm_runtime_suspend - Suspend a device synchronously.
	302	+ * @dev: Target device.
	303	+ */
199	304	static inline int pm_runtime_suspend(struct device *dev)
200	305	{
201	306	return __pm_runtime_suspend(dev, 0);
202	307	}
203	308
	309	+/**
	310	+ * pm_runtime_autosuspend - Set up autosuspend of a device or suspend it.
	311	+ * @dev: Target device.
	312	+ *
	313	+ * Set up autosuspend of @dev or suspend it (depending on whether or not
	314	+ * autosuspend is enabled for it) without engaging its "idle check" callback.
	315	+ */
204	316	static inline int pm_runtime_autosuspend(struct device *dev)
205	317	{
206	318	return __pm_runtime_suspend(dev, RPM_AUTO);
207	319	}
208	320
	321	+/**
	322	+ * pm_runtime_resume - Resume a device synchronously.
	323	+ * @dev: Target device.
	324	+ */
209	325	static inline int pm_runtime_resume(struct device *dev)
210	326	{
211	327	return __pm_runtime_resume(dev, 0);
212	328	}
213	329
	330	+/**
	331	+ * pm_request_idle - Queue up "idle check" execution for a device.
	332	+ * @dev: Target device.
	333	+ *
	334	+ * Queue up a work item to run an equivalent of pm_runtime_idle() for @dev
	335	+ * asynchronously.
	336	+ */
214	337	static inline int pm_request_idle(struct device *dev)
215	338	{
216	339	return __pm_runtime_idle(dev, RPM_ASYNC);
217	340	}
218	341
	342	+/**
	343	+ * pm_request_resume - Queue up runtime-resume of a device.
	344	+ * @dev: Target device.
	345	+ */
219	346	static inline int pm_request_resume(struct device *dev)
220	347	{
221	348	return __pm_runtime_resume(dev, RPM_ASYNC);
222	349	}
223	350
	351	+/**
	352	+ * pm_request_autosuspend - Queue up autosuspend of a device.
	353	+ * @dev: Target device.
	354	+ *
	355	+ * Queue up a work item to run an equivalent pm_runtime_autosuspend() for @dev
	356	+ * asynchronously.
	357	+ */
224	358	static inline int pm_request_autosuspend(struct device *dev)
225	359	{
226	360	return __pm_runtime_suspend(dev, RPM_ASYNC \| RPM_AUTO);
227	361	}
228	362
	363	+/**
	364	+ * pm_runtime_get - Bump up usage counter and queue up resume of a device.
	365	+ * @dev: Target device.
	366	+ *
	367	+ * Bump up the runtime PM usage counter of @dev and queue up a work item to
	368	+ * carry out runtime-resume of it.
	369	+ */
229	370	static inline int pm_runtime_get(struct device *dev)
230	371	{
231	372	return __pm_runtime_resume(dev, RPM_GET_PUT \| RPM_ASYNC);
232	373	}
233	374
	375	+/**
	376	+ * pm_runtime_get_sync - Bump up usage counter of a device and resume it.
	377	+ * @dev: Target device.
	378	+ *
	379	+ * Bump up the runtime PM usage counter of @dev and carry out runtime-resume of
	380	+ * it synchronously.
	381	+ *
	382	+ * The possible return values of this function are the same as for
	383	+ * pm_runtime_resume() and the runtime PM usage counter of @dev remains
	384	+ * incremented in all cases, even if it returns an error code.
	385	+ */
234	386	static inline int pm_runtime_get_sync(struct device *dev)
235	387	{
236	388	return __pm_runtime_resume(dev, RPM_GET_PUT);
237	389	}
238	390
	391	+/**
	392	+ * pm_runtime_put - Drop device usage counter and queue up "idle check" if 0.
	393	+ * @dev: Target device.
	394	+ *
	395	+ * Decrement the runtime PM usage counter of @dev and if it turns out to be
	396	+ * equal to 0, queue up a work item for @dev like in pm_request_idle().
	397	+ */
239	398	static inline int pm_runtime_put(struct device *dev)
240	399	{
241	400	return __pm_runtime_idle(dev, RPM_GET_PUT \| RPM_ASYNC);
242	401	}
243	402
	403	+/**
	404	+ * pm_runtime_put_autosuspend - Drop device usage counter and queue autosuspend if 0.
	405	+ * @dev: Target device.
	406	+ *
	407	+ * Decrement the runtime PM usage counter of @dev and if it turns out to be
	408	+ * equal to 0, queue up a work item for @dev like in pm_request_autosuspend().
	409	+ */
244	410	static inline int pm_runtime_put_autosuspend(struct device *dev)
245	411	{
246	412	return __pm_runtime_suspend(dev,
247	413	RPM_GET_PUT \| RPM_ASYNC \| RPM_AUTO);
248	414	}
249	415
	416	+/**
	417	+ * pm_runtime_put_sync - Drop device usage counter and run "idle check" if 0.
	418	+ * @dev: Target device.
	419	+ *
	420	+ * Decrement the runtime PM usage counter of @dev and if it turns out to be
	421	+ * equal to 0, invoke the "idle check" callback of @dev and, depending on its
	422	+ * return value, set up autosuspend of @dev or suspend it (depending on whether
	423	+ * or not autosuspend has been enabled for it).
	424	+ *
	425	+ * The possible return values of this function are the same as for
	426	+ * pm_runtime_idle() and the runtime PM usage counter of @dev remains
	427	+ * decremented in all cases, even if it returns an error code.
	428	+ */
250	429	static inline int pm_runtime_put_sync(struct device *dev)
251	430	{
252	431	return __pm_runtime_idle(dev, RPM_GET_PUT);
253	432	}
254	433
	434	+/**
	435	+ * pm_runtime_put_sync_suspend - Drop device usage counter and suspend if 0.
	436	+ * @dev: Target device.
	437	+ *
	438	+ * Decrement the runtime PM usage counter of @dev and if it turns out to be
	439	+ * equal to 0, carry out runtime-suspend of @dev synchronously.
	440	+ *
	441	+ * The possible return values of this function are the same as for
	442	+ * pm_runtime_suspend() and the runtime PM usage counter of @dev remains
	443	+ * decremented in all cases, even if it returns an error code.
	444	+ */
255	445	static inline int pm_runtime_put_sync_suspend(struct device *dev)
256	446	{
257	447	return __pm_runtime_suspend(dev, RPM_GET_PUT);
258	448	}
259	449
	450	+/**
	451	+ * pm_runtime_put_sync_autosuspend - Drop device usage counter and autosuspend if 0.
	452	+ * @dev: Target device.
	453	+ *
	454	+ * Decrement the runtime PM usage counter of @dev and if it turns out to be
	455	+ * equal to 0, set up autosuspend of @dev or suspend it synchronously (depending
	456	+ * on whether or not autosuspend has been enabled for it).
	457	+ *
	458	+ * The possible return values of this function are the same as for
	459	+ * pm_runtime_autosuspend() and the runtime PM usage counter of @dev remains
	460	+ * decremented in all cases, even if it returns an error code.
	461	+ */
260	462	static inline int pm_runtime_put_sync_autosuspend(struct device *dev)
261	463	{
262	464	return __pm_runtime_suspend(dev, RPM_GET_PUT \| RPM_AUTO);
263	465	}
264	466
	467	+/**
	468	+ * pm_runtime_set_active - Set runtime PM status to "active".
	469	+ * @dev: Target device.
	470	+ *
	471	+ * Set the runtime PM status of @dev to %RPM_ACTIVE and ensure that dependencies
	472	+ * of it will be taken into account.
	473	+ *
	474	+ * It is not valid to call this function for devices with runtime PM enabled.
	475	+ */
265	476	static inline int pm_runtime_set_active(struct device *dev)
266	477	{
267	478	return __pm_runtime_set_status(dev, RPM_ACTIVE);
268	479	}
269	480
	481	+/**
	482	+ * pm_runtime_set_suspended - Set runtime PM status to "active".
	483	+ * @dev: Target device.
	484	+ *
	485	+ * Set the runtime PM status of @dev to %RPM_SUSPENDED and ensure that
	486	+ * dependencies of it will be taken into account.
	487	+ *
	488	+ * It is not valid to call this function for devices with runtime PM enabled.
	489	+ */
270	490	static inline int pm_runtime_set_suspended(struct device *dev)
271	491	{
272	492	return __pm_runtime_set_status(dev, RPM_SUSPENDED);
273	493	}
274	494
	495	+/**
	496	+ * pm_runtime_disable - Disable runtime PM for a device.
	497	+ * @dev: Target device.
	498	+ *
	499	+ * Prevent the runtime PM framework from working with @dev (by incrementing its
	500	+ * "blocking" counter).
	501	+ *
	502	+ * For each invocation of this function for @dev there must be a matching
	503	+ * pm_runtime_enable() call in order for runtime PM to be enabled for it.
	504	+ */
275	505	static inline void pm_runtime_disable(struct device *dev)
276	506	{
277	507	__pm_runtime_disable(dev, true);
278	508	}
279	509
	510	+/**
	511	+ * pm_runtime_use_autosuspend - Allow autosuspend to be used for a device.
	512	+ * @dev: Target device.
	513	+ *
	514	+ * Allow the runtime PM autosuspend mechanism to be used for @dev whenever
	515	+ * requested (or "autosuspend" will be handled as direct runtime-suspend for
	516	+ * it).
	517	+ */
280	518	static inline void pm_runtime_use_autosuspend(struct device *dev)
281	519	{
282	520	__pm_runtime_use_autosuspend(dev, true);
283	521	}
284	522
	523	+/**
	524	+ * pm_runtime_dont_use_autosuspend - Prevent autosuspend from being used.
	525	+ * @dev: Target device.
	526	+ *
	527	+ * Prevent the runtime PM autosuspend mechanism from being used for @dev which
	528	+ * means that "autosuspend" will be handled as direct runtime-suspend for it
	529	+ * going forward.
	530	+ */
285	531	static inline void pm_runtime_dont_use_autosuspend(struct device *dev)
286	532	{
287	533	__pm_runtime_use_autosuspend(dev, false);