Merge tag 'pm-5.19-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm

Pull power management updates from Rafael Wysocki: "These add support for 'artificial' Energy Models in which power numbers for different entities may be in different scales, add support for some new hardware, fix bugs and clean up code in multiple places. Specifics: - Update the Energy Model support code to allow the Energy Model to be artificial, which means that the power values may not be on a uniform scale with other devices providing power information, and update the cpufreq_cooling and devfreq_cooling thermal drivers to support artificial Energy Models (Lukasz Luba). - Make DTPM check the Energy Model type (Lukasz Luba). - Fix policy counter decrementation in cpufreq if Energy Model is in use (Pierre Gondois). - Add CPU-based scaling support to passive devfreq governor (Saravana Kannan, Chanwoo Choi). - Update the rk3399_dmc devfreq driver (Brian Norris). - Export dev_pm_ops instead of suspend() and resume() in the IIO chemical scd30 driver (Jonathan Cameron). - Add namespace variants of EXPORT[_GPL]_SIMPLE_DEV_PM_OPS and PM-runtime counterparts (Jonathan Cameron). - Move symbol exports in the IIO chemical scd30 driver into the IIO_SCD30 namespace (Jonathan Cameron). - Avoid device PM-runtime usage count underflows (Rafael Wysocki). - Allow dynamic debug to control printing of PM messages (David Cohen). - Fix some kernel-doc comments in hibernation code (Yang Li, Haowen Bai). - Preserve ACPI-table override during hibernation (Amadeusz Sławiński). - Improve support for suspend-to-RAM for PSCI OSI mode (Ulf Hansson). - Make Intel RAPL power capping driver support the RaptorLake and AlderLake N processors (Zhang Rui, Sumeet Pawnikar). - Remove redundant store to value after multiply in the RAPL power capping driver (Colin Ian King). - Add AlderLake processor support to the intel_idle driver (Zhang Rui). - Fix regression leading to no genpd governor in the PSCI cpuidle driver and fix the riscv-sbi cpuidle driver to allow a genpd governor to be used (Ulf Hansson). - Fix cpufreq governor clean up code to avoid using kfree() directly to free kobject-based items (Kevin Hao). - Prepare cpufreq for powerpc's asm/prom.h cleanup (Christophe Leroy). - Make intel_pstate notify frequency invariance code when no_turbo is turned on and off (Chen Yu). - Add Sapphire Rapids OOB mode support to intel_pstate (Srinivas Pandruvada). - Make cpufreq avoid unnecessary frequency updates due to mismatch between hardware and the frequency table (Viresh Kumar). - Make remove_cpu_dev_symlink() clear the real_cpus mask to simplify code (Viresh Kumar). - Rearrange cpufreq_offline() and cpufreq_remove_dev() to make the calling convention for some driver callbacks consistent (Rafael Wysocki). - Avoid accessing half-initialized cpufreq policies from the show() and store() sysfs functions (Schspa Shi). - Rearrange cpufreq_offline() to make the calling convention for some driver callbacks consistent (Schspa Shi). - Update CPPC handling in cpufreq (Pierre Gondois). - Extend dev_pm_domain_detach() doc (Krzysztof Kozlowski). - Move genpd's time-accounting to ktime_get_mono_fast_ns() (Ulf Hansson). - Improve the way genpd deals with its governors (Ulf Hansson). - Update the turbostat utility to version 2022.04.16 (Len Brown, Dan Merillat, Sumeet Pawnikar, Zephaniah E. Loss-Cutler-Hull, Chen Yu)" * tag 'pm-5.19-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm: (94 commits) PM: domains: Trust domain-idle-states from DT to be correct by genpd PM: domains: Measure power-on/off latencies in genpd based on a governor PM: domains: Allocate governor data dynamically based on a genpd governor PM: domains: Clean up some code in pm_genpd_init() and genpd_remove() PM: domains: Fix initialization of genpd's next_wakeup PM: domains: Fixup QoS latency measurements for IRQ safe devices in genpd PM: domains: Measure suspend/resume latencies in genpd based on governor PM: domains: Move the next_wakeup variable into the struct gpd_timing_data PM: domains: Allocate gpd_timing_data dynamically based on governor PM: domains: Skip another warning in irq_safe_dev_in_sleep_domain() PM: domains: Rename irq_safe_dev_in_no_sleep_domain() in genpd PM: domains: Don't check PM_QOS_FLAG_NO_POWER_OFF in genpd PM: domains: Drop redundant code for genpd always-on governor PM: domains: Add GENPD_FLAG_RPM_ALWAYS_ON for the always-on governor powercap: intel_rapl: remove redundant store to value after multiply cpufreq: CPPC: Enable dvfs_possible_from_any_cpu cpufreq: CPPC: Enable fast_switch ACPI: CPPC: Assume no transition latency if no PCCT ACPI: bus: Set CPPC _OSC bits for all and when CPPC_LIB is supported ACPI: CPPC: Check _OSC for flexible address space ...

Linus Torvalds committed 4 years ago

Commit 09583df2 parents 1961b06 0d6448260 files · +2463 −1005Browse files

Unified · Split

60 files

Documentation/devicetree/bindings/devfreq/rk3399_dmc.txt +0 −212
Documentation/devicetree/bindings/memory-controllers/rockchip,rk3399-dmc.yaml +384 −0
Documentation/power/energy-model.rst +22 −2
arch/arm64/kernel/smp.c +1 −0
arch/x86/include/asm/msr-index.h +1 −0
arch/x86/kernel/acpi/boot.c +1 −1
drivers/acpi/bus.c +26 −8
drivers/acpi/cppc_acpi.c +43 −1
drivers/base/power/common.c +4 −4
drivers/base/power/domain.c +170 −108
drivers/base/power/domain_governor.c +35 −30
drivers/base/power/runtime.c +42 −11
drivers/cpufreq/cppc_cpufreq.c +211 −0
drivers/cpufreq/cpufreq.c +69 −43
drivers/cpufreq/cpufreq_governor.c +13 −7
drivers/cpufreq/cpufreq_governor.h +1 −0
drivers/cpufreq/intel_pstate.c +2 −0
drivers/cpufreq/mediatek-cpufreq-hw.c +2 −2
drivers/cpufreq/pasemi-cpufreq.c +0 −1
drivers/cpufreq/pmac32-cpufreq.c +1 −1
drivers/cpufreq/pmac64-cpufreq.c +1 −1
drivers/cpufreq/ppc_cbe_cpufreq.c +0 −1
drivers/cpufreq/ppc_cbe_cpufreq_pmi.c +1 −1
drivers/cpufreq/scmi-cpufreq.c +2 −2
drivers/cpuidle/cpuidle-psci-domain.c +2 −2
drivers/cpuidle/cpuidle-psci.c +46 −0
drivers/cpuidle/cpuidle-riscv-sbi.c +2 −2
drivers/devfreq/devfreq.c +12 −8
drivers/devfreq/governor.h +27 −0
drivers/devfreq/governor_passive.c +335 −68
drivers/devfreq/rk3399_dmc.c +147 −165
drivers/idle/intel_idle.c +133 −0
drivers/iio/chemical/scd30.h +1 −4
drivers/iio/chemical/scd30_core.c +5 −5
drivers/iio/chemical/scd30_i2c.c +2 −1
drivers/iio/chemical/scd30_serial.c +2 −1
drivers/opp/of.c +3 −3
drivers/powercap/dtpm_cpu.c +1 −1
drivers/powercap/intel_rapl_common.c +3 −1
drivers/powercap/intel_rapl_msr.c +1 −0
drivers/soc/rockchip/pm_domains.c +118 −0
drivers/thermal/cpufreq_cooling.c +1 −1
drivers/thermal/devfreq_cooling.c +5 −3
include/acpi/cppc_acpi.h +5 −0
include/linux/acpi.h +2 −0
include/linux/devfreq.h +15 −2
include/linux/energy_model.h +31 −4
include/linux/pm.h +9 −5
include/linux/pm_domain.h +14 −10
include/linux/pm_runtime.h +8 −2
include/linux/suspend.h +39 −5
include/soc/rockchip/pm_domains.h +25 −0
kernel/power/Makefile +5 −1
kernel/power/energy_model.c +36 −29
kernel/power/main.c +0 −29
kernel/power/process.c +0 −3
kernel/power/snapshot.c +8 −4
tools/power/x86/turbostat/Makefile +1 −1
tools/power/x86/turbostat/turbostat.8 +1 −1
tools/power/x86/turbostat/turbostat.c +386 −208

Documentation/devicetree/bindings/devfreq/rk3399_dmc.txt+0−212

@@ -1,212 +0,0 @@
1		−* Rockchip rk3399 DMC (Dynamic Memory Controller) device
2		−
3		−Required properties:
4		−- compatible: Must be "rockchip,rk3399-dmc".
5		−- devfreq-events: Node to get DDR loading, Refer to
6		− Documentation/devicetree/bindings/devfreq/event/
7		− rockchip-dfi.txt
8		−- clocks: Phandles for clock specified in "clock-names" property
9		−- clock-names : The name of clock used by the DFI, must be
10		− "pclk_ddr_mon";
11		−- operating-points-v2: Refer to Documentation/devicetree/bindings/opp/opp-v2.yaml
12		− for details.
13		−- center-supply: DMC supply node.
14		−- status: Marks the node enabled/disabled.
15		−- rockchip,pmu: Phandle to the syscon managing the "PMU general register
16		− files".
17		−
18		−Optional properties:
19		−- interrupts: The CPU interrupt number. The interrupt specifier
20		− format depends on the interrupt controller.
21		− It should be a DCF interrupt. When DDR DVFS finishes
22		− a DCF interrupt is triggered.
23		−- rockchip,pmu: Phandle to the syscon managing the "PMU general register
24		− files".
25		−
26		−Following properties relate to DDR timing:
27		−
28		−- rockchip,dram_speed_bin : Value reference include/dt-bindings/clock/rk3399-ddr.h,
29		− it selects the DDR3 cl-trp-trcd type. It must be
30		− set according to "Speed Bin" in DDR3 datasheet,
31		− DO NOT use a smaller "Speed Bin" than specified
32		− for the DDR3 being used.
33		−
34		−- rockchip,pd_idle : Configure the PD_IDLE value. Defines the
35		− power-down idle period in which memories are
36		− placed into power-down mode if bus is idle
37		− for PD_IDLE DFI clock cycles.
38		−
39		−- rockchip,sr_idle : Configure the SR_IDLE value. Defines the
40		− self-refresh idle period in which memories are
41		− placed into self-refresh mode if bus is idle
42		− for SR_IDLE * 1024 DFI clock cycles (DFI
43		− clocks freq is half of DRAM clock), default
44		− value is "0".
45		−
46		−- rockchip,sr_mc_gate_idle : Defines the memory self-refresh and controller
47		− clock gating idle period. Memories are placed
48		− into self-refresh mode and memory controller
49		− clock arg gating started if bus is idle for
50		− sr_mc_gate_idle*1024 DFI clock cycles.
51		−
52		−- rockchip,srpd_lite_idle : Defines the self-refresh power down idle
53		− period in which memories are placed into
54		− self-refresh power down mode if bus is idle
55		− for srpd_lite_idle * 1024 DFI clock cycles.
56		− This parameter is for LPDDR4 only.
57		−
58		−- rockchip,standby_idle : Defines the standby idle period in which
59		− memories are placed into self-refresh mode.
60		− The controller, pi, PHY and DRAM clock will
61		− be gated if bus is idle for standby_idle * DFI
62		− clock cycles.
63		−
64		−- rockchip,dram_dll_dis_freq : Defines the DDR3 DLL bypass frequency in MHz.
65		− When DDR frequency is less than DRAM_DLL_DISB_FREQ,
66		− DDR3 DLL will be bypassed. Note: if DLL was bypassed,
67		− the odt will also stop working.
68		−
69		−- rockchip,phy_dll_dis_freq : Defines the PHY dll bypass frequency in
70		− MHz (Mega Hz). When DDR frequency is less than
71		− DRAM_DLL_DISB_FREQ, PHY DLL will be bypassed.
72		− Note: PHY DLL and PHY ODT are independent.
73		−
74		−- rockchip,ddr3_odt_dis_freq : When the DRAM type is DDR3, this parameter defines
75		− the ODT disable frequency in MHz (Mega Hz).
76		− when the DDR frequency is less then ddr3_odt_dis_freq,
77		− the ODT on the DRAM side and controller side are
78		− both disabled.
79		−
80		−- rockchip,ddr3_drv : When the DRAM type is DDR3, this parameter defines
81		− the DRAM side driver strength in ohms. Default
82		− value is 40.
83		−
84		−- rockchip,ddr3_odt : When the DRAM type is DDR3, this parameter defines
85		− the DRAM side ODT strength in ohms. Default value
86		− is 120.
87		−
88		−- rockchip,phy_ddr3_ca_drv : When the DRAM type is DDR3, this parameter defines
89		− the phy side CA line (incluing command line,
90		− address line and clock line) driver strength.
91		− Default value is 40.
92		−
93		−- rockchip,phy_ddr3_dq_drv : When the DRAM type is DDR3, this parameter defines
94		− the PHY side DQ line (including DQS/DQ/DM line)
95		− driver strength. Default value is 40.
96		−
97		−- rockchip,phy_ddr3_odt : When the DRAM type is DDR3, this parameter defines
98		− the PHY side ODT strength. Default value is 240.
99		−
100		−- rockchip,lpddr3_odt_dis_freq : When the DRAM type is LPDDR3, this parameter defines
101		− then ODT disable frequency in MHz (Mega Hz).
102		− When DDR frequency is less then ddr3_odt_dis_freq,
103		− the ODT on the DRAM side and controller side are
104		− both disabled.
105		−
106		−- rockchip,lpddr3_drv : When the DRAM type is LPDDR3, this parameter defines
107		− the DRAM side driver strength in ohms. Default
108		− value is 34.
109		−
110		−- rockchip,lpddr3_odt : When the DRAM type is LPDDR3, this parameter defines
111		− the DRAM side ODT strength in ohms. Default value
112		− is 240.
113		−
114		−- rockchip,phy_lpddr3_ca_drv : When the DRAM type is LPDDR3, this parameter defines
115		− the PHY side CA line (including command line,
116		− address line and clock line) driver strength.
117		− Default value is 40.
118		−
119		−- rockchip,phy_lpddr3_dq_drv : When the DRAM type is LPDDR3, this parameter defines
120		− the PHY side DQ line (including DQS/DQ/DM line)
121		− driver strength. Default value is 40.
122		−
123		−- rockchip,phy_lpddr3_odt : When dram type is LPDDR3, this parameter define
124		− the phy side odt strength, default value is 240.
125		−
126		−- rockchip,lpddr4_odt_dis_freq : When the DRAM type is LPDDR4, this parameter
127		− defines the ODT disable frequency in
128		− MHz (Mega Hz). When the DDR frequency is less then
129		− ddr3_odt_dis_freq, the ODT on the DRAM side and
130		− controller side are both disabled.
131		−
132		−- rockchip,lpddr4_drv : When the DRAM type is LPDDR4, this parameter defines
133		− the DRAM side driver strength in ohms. Default
134		− value is 60.
135		−
136		−- rockchip,lpddr4_dq_odt : When the DRAM type is LPDDR4, this parameter defines
137		− the DRAM side ODT on DQS/DQ line strength in ohms.
138		− Default value is 40.
139		−
140		−- rockchip,lpddr4_ca_odt : When the DRAM type is LPDDR4, this parameter defines
141		− the DRAM side ODT on CA line strength in ohms.
142		− Default value is 40.
143		−
144		−- rockchip,phy_lpddr4_ca_drv : When the DRAM type is LPDDR4, this parameter defines
145		− the PHY side CA line (including command address
146		− line) driver strength. Default value is 40.
147		−
148		−- rockchip,phy_lpddr4_ck_cs_drv : When the DRAM type is LPDDR4, this parameter defines
149		− the PHY side clock line and CS line driver
150		− strength. Default value is 80.
151		−
152		−- rockchip,phy_lpddr4_dq_drv : When the DRAM type is LPDDR4, this parameter defines
153		− the PHY side DQ line (including DQS/DQ/DM line)
154		− driver strength. Default value is 80.
155		−
156		−- rockchip,phy_lpddr4_odt : When the DRAM type is LPDDR4, this parameter defines
157		− the PHY side ODT strength. Default value is 60.
158		−
159		−Example:
160		− dmc_opp_table: dmc_opp_table {
161		− compatible = "operating-points-v2";
162		−
163		− opp00 {
164		− opp-hz = /bits/ 64 <300000000>;
165		− opp-microvolt = <900000>;
166		− };
167		− opp01 {
168		− opp-hz = /bits/ 64 <666000000>;
169		− opp-microvolt = <900000>;
170		− };
171		− };
172		−
173		− dmc: dmc {
174		− compatible = "rockchip,rk3399-dmc";
175		− devfreq-events = <&dfi>;
176		− interrupts = <GIC_SPI 1 IRQ_TYPE_LEVEL_HIGH>;
177		− clocks = <&cru SCLK_DDRC>;
178		− clock-names = "dmc_clk";
179		− operating-points-v2 = <&dmc_opp_table>;
180		− center-supply = <&ppvar_centerlogic>;
181		− upthreshold = <15>;
182		− downdifferential = <10>;
183		− rockchip,ddr3_speed_bin = <21>;
184		− rockchip,pd_idle = <0x40>;
185		− rockchip,sr_idle = <0x2>;
186		− rockchip,sr_mc_gate_idle = <0x3>;
187		− rockchip,srpd_lite_idle = <0x4>;
188		− rockchip,standby_idle = <0x2000>;
189		− rockchip,dram_dll_dis_freq = <300>;
190		− rockchip,phy_dll_dis_freq = <125>;
191		− rockchip,auto_pd_dis_freq = <666>;
192		− rockchip,ddr3_odt_dis_freq = <333>;
193		− rockchip,ddr3_drv = <40>;
194		− rockchip,ddr3_odt = <120>;
195		− rockchip,phy_ddr3_ca_drv = <40>;
196		− rockchip,phy_ddr3_dq_drv = <40>;
197		− rockchip,phy_ddr3_odt = <240>;
198		− rockchip,lpddr3_odt_dis_freq = <333>;
199		− rockchip,lpddr3_drv = <34>;
200		− rockchip,lpddr3_odt = <240>;
201		− rockchip,phy_lpddr3_ca_drv = <40>;
202		− rockchip,phy_lpddr3_dq_drv = <40>;
203		− rockchip,phy_lpddr3_odt = <240>;
204		− rockchip,lpddr4_odt_dis_freq = <333>;
205		− rockchip,lpddr4_drv = <60>;
206		− rockchip,lpddr4_dq_odt = <40>;
207		− rockchip,lpddr4_ca_odt = <40>;
208		− rockchip,phy_lpddr4_ca_drv = <40>;
209		− rockchip,phy_lpddr4_ck_cs_drv = <80>;
210		− rockchip,phy_lpddr4_dq_drv = <80>;
211		− rockchip,phy_lpddr4_odt = <60>;
212		− };

Documentation/devicetree/bindings/memory-controllers/rockchip,rk3399-dmc.yaml+384−0

@@ -0,0 +1,384 @@
	1	+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
	2	+# %YAML 1.2
	3	+---
	4	+$id: http://devicetree.org/schemas/memory-controllers/rockchip,rk3399-dmc.yaml#
	5	+$schema: http://devicetree.org/meta-schemas/core.yaml#
	6	+
	7	+title: Rockchip rk3399 DMC (Dynamic Memory Controller) device
	8	+
	9	+maintainers:
	10	+ - Brian Norris <briannorris@chromium.org>
	11	+
	12	+properties:
	13	+ compatible:
	14	+ enum:
	15	+ - rockchip,rk3399-dmc
	16	+
	17	+ devfreq-events:
	18	+ $ref: /schemas/types.yaml#/definitions/phandle
	19	+ description:
	20	+ Node to get DDR loading. Refer to
	21	+ Documentation/devicetree/bindings/devfreq/event/rockchip-dfi.txt.
	22	+
	23	+ clocks:
	24	+ maxItems: 1
	25	+
	26	+ clock-names:
	27	+ items:
	28	+ - const: dmc_clk
	29	+
	30	+ operating-points-v2: true
	31	+
	32	+ center-supply:
	33	+ description:
	34	+ DMC regulator supply.
	35	+
	36	+ rockchip,pmu:
	37	+ $ref: /schemas/types.yaml#/definitions/phandle
	38	+ description:
	39	+ Phandle to the syscon managing the "PMU general register files".
	40	+
	41	+ interrupts:
	42	+ maxItems: 1
	43	+ description:
	44	+ The CPU interrupt number. It should be a DCF interrupt. When DDR DVFS
	45	+ finishes, a DCF interrupt is triggered.
	46	+
	47	+ rockchip,ddr3_speed_bin:
	48	+ deprecated: true
	49	+ $ref: /schemas/types.yaml#/definitions/uint32
	50	+ description:
	51	+ For values, reference include/dt-bindings/clock/rk3399-ddr.h. Selects the
	52	+ DDR3 cl-trp-trcd type. It must be set according to "Speed Bin" in DDR3
	53	+ datasheet; DO NOT use a smaller "Speed Bin" than specified for the DDR3
	54	+ being used.
	55	+
	56	+ rockchip,pd_idle:
	57	+ deprecated: true
	58	+ $ref: /schemas/types.yaml#/definitions/uint32
	59	+ description:
	60	+ Configure the PD_IDLE value. Defines the power-down idle period in which
	61	+ memories are placed into power-down mode if bus is idle for PD_IDLE DFI
	62	+ clock cycles.
	63	+ See also rockchip,pd-idle-ns.
	64	+
	65	+ rockchip,sr_idle:
	66	+ deprecated: true
	67	+ $ref: /schemas/types.yaml#/definitions/uint32
	68	+ description:
	69	+ Configure the SR_IDLE value. Defines the self-refresh idle period in
	70	+ which memories are placed into self-refresh mode if bus is idle for
	71	+ SR_IDLE * 1024 DFI clock cycles (DFI clocks freq is half of DRAM clock).
	72	+ See also rockchip,sr-idle-ns.
	73	+ default: 0
	74	+
	75	+ rockchip,sr_mc_gate_idle:
	76	+ deprecated: true
	77	+ $ref: /schemas/types.yaml#/definitions/uint32
	78	+ description:
	79	+ Defines the memory self-refresh and controller clock gating idle period.
	80	+ Memories are placed into self-refresh mode and memory controller clock
	81	+ arg gating started if bus is idle for sr_mc_gate_idle*1024 DFI clock
	82	+ cycles.
	83	+ See also rockchip,sr-mc-gate-idle-ns.
	84	+
	85	+ rockchip,srpd_lite_idle:
	86	+ deprecated: true
	87	+ $ref: /schemas/types.yaml#/definitions/uint32
	88	+ description:
	89	+ Defines the self-refresh power down idle period in which memories are
	90	+ placed into self-refresh power down mode if bus is idle for
	91	+ srpd_lite_idle * 1024 DFI clock cycles. This parameter is for LPDDR4
	92	+ only.
	93	+ See also rockchip,srpd-lite-idle-ns.
	94	+
	95	+ rockchip,standby_idle:
	96	+ deprecated: true
	97	+ $ref: /schemas/types.yaml#/definitions/uint32
	98	+ description:
	99	+ Defines the standby idle period in which memories are placed into
	100	+ self-refresh mode. The controller, pi, PHY and DRAM clock will be gated
	101	+ if bus is idle for standby_idle * DFI clock cycles.
	102	+ See also rockchip,standby-idle-ns.
	103	+
	104	+ rockchip,dram_dll_dis_freq:
	105	+ deprecated: true
	106	+ $ref: /schemas/types.yaml#/definitions/uint32
	107	+ description: \|
	108	+ Defines the DDR3 DLL bypass frequency in MHz. When DDR frequency is less
	109	+ than DRAM_DLL_DISB_FREQ, DDR3 DLL will be bypassed.
	110	+ Note: if DLL was bypassed, the odt will also stop working.
	111	+
	112	+ rockchip,phy_dll_dis_freq:
	113	+ deprecated: true
	114	+ $ref: /schemas/types.yaml#/definitions/uint32
	115	+ description: \|
	116	+ Defines the PHY dll bypass frequency in MHz (Mega Hz). When DDR frequency
	117	+ is less than DRAM_DLL_DISB_FREQ, PHY DLL will be bypassed.
	118	+ Note: PHY DLL and PHY ODT are independent.
	119	+
	120	+ rockchip,auto_pd_dis_freq:
	121	+ deprecated: true
	122	+ $ref: /schemas/types.yaml#/definitions/uint32
	123	+ description:
	124	+ Defines the auto PD disable frequency in MHz.
	125	+
	126	+ rockchip,ddr3_odt_dis_freq:
	127	+ $ref: /schemas/types.yaml#/definitions/uint32
	128	+ minimum: 1000000 # In case anyone thought this was MHz.
	129	+ description:
	130	+ When the DRAM type is DDR3, this parameter defines the ODT disable
	131	+ frequency in Hz. When the DDR frequency is less then ddr3_odt_dis_freq,
	132	+ the ODT on the DRAM side and controller side are both disabled.
	133	+
	134	+ rockchip,ddr3_drv:
	135	+ deprecated: true
	136	+ $ref: /schemas/types.yaml#/definitions/uint32
	137	+ description:
	138	+ When the DRAM type is DDR3, this parameter defines the DRAM side drive
	139	+ strength in ohms.
	140	+ default: 40
	141	+
	142	+ rockchip,ddr3_odt:
	143	+ deprecated: true
	144	+ $ref: /schemas/types.yaml#/definitions/uint32
	145	+ description:
	146	+ When the DRAM type is DDR3, this parameter defines the DRAM side ODT
	147	+ strength in ohms.
	148	+ default: 120
	149	+
	150	+ rockchip,phy_ddr3_ca_drv:
	151	+ deprecated: true
	152	+ $ref: /schemas/types.yaml#/definitions/uint32
	153	+ description:
	154	+ When the DRAM type is DDR3, this parameter defines the phy side CA line
	155	+ (incluing command line, address line and clock line) drive strength.
	156	+ default: 40
	157	+
	158	+ rockchip,phy_ddr3_dq_drv:
	159	+ deprecated: true
	160	+ $ref: /schemas/types.yaml#/definitions/uint32
	161	+ description:
	162	+ When the DRAM type is DDR3, this parameter defines the PHY side DQ line
	163	+ (including DQS/DQ/DM line) drive strength.
	164	+ default: 40
	165	+
	166	+ rockchip,phy_ddr3_odt:
	167	+ deprecated: true
	168	+ $ref: /schemas/types.yaml#/definitions/uint32
	169	+ description:
	170	+ When the DRAM type is DDR3, this parameter defines the PHY side ODT
	171	+ strength.
	172	+ default: 240
	173	+
	174	+ rockchip,lpddr3_odt_dis_freq:
	175	+ $ref: /schemas/types.yaml#/definitions/uint32
	176	+ minimum: 1000000 # In case anyone thought this was MHz.
	177	+ description:
	178	+ When the DRAM type is LPDDR3, this parameter defines then ODT disable
	179	+ frequency in Hz. When DDR frequency is less then ddr3_odt_dis_freq, the
	180	+ ODT on the DRAM side and controller side are both disabled.
	181	+
	182	+ rockchip,lpddr3_drv:
	183	+ deprecated: true
	184	+ $ref: /schemas/types.yaml#/definitions/uint32
	185	+ description:
	186	+ When the DRAM type is LPDDR3, this parameter defines the DRAM side drive
	187	+ strength in ohms.
	188	+ default: 34
	189	+
	190	+ rockchip,lpddr3_odt:
	191	+ deprecated: true
	192	+ $ref: /schemas/types.yaml#/definitions/uint32
	193	+ description:
	194	+ When the DRAM type is LPDDR3, this parameter defines the DRAM side ODT
	195	+ strength in ohms.
	196	+ default: 240
	197	+
	198	+ rockchip,phy_lpddr3_ca_drv:
	199	+ deprecated: true
	200	+ $ref: /schemas/types.yaml#/definitions/uint32
	201	+ description:
	202	+ When the DRAM type is LPDDR3, this parameter defines the PHY side CA line
	203	+ (including command line, address line and clock line) drive strength.
	204	+ default: 40
	205	+
	206	+ rockchip,phy_lpddr3_dq_drv:
	207	+ deprecated: true
	208	+ $ref: /schemas/types.yaml#/definitions/uint32
	209	+ description:
	210	+ When the DRAM type is LPDDR3, this parameter defines the PHY side DQ line
	211	+ (including DQS/DQ/DM line) drive strength.
	212	+ default: 40
	213	+
	214	+ rockchip,phy_lpddr3_odt:
	215	+ deprecated: true
	216	+ $ref: /schemas/types.yaml#/definitions/uint32
	217	+ description:
	218	+ When dram type is LPDDR3, this parameter define the phy side odt
	219	+ strength, default value is 240.
	220	+
	221	+ rockchip,lpddr4_odt_dis_freq:
	222	+ $ref: /schemas/types.yaml#/definitions/uint32
	223	+ minimum: 1000000 # In case anyone thought this was MHz.
	224	+ description:
	225	+ When the DRAM type is LPDDR4, this parameter defines the ODT disable
	226	+ frequency in Hz. When the DDR frequency is less then ddr3_odt_dis_freq,
	227	+ the ODT on the DRAM side and controller side are both disabled.
	228	+
	229	+ rockchip,lpddr4_drv:
	230	+ deprecated: true
	231	+ $ref: /schemas/types.yaml#/definitions/uint32
	232	+ description:
	233	+ When the DRAM type is LPDDR4, this parameter defines the DRAM side drive
	234	+ strength in ohms.
	235	+ default: 60
	236	+
	237	+ rockchip,lpddr4_dq_odt:
	238	+ deprecated: true
	239	+ $ref: /schemas/types.yaml#/definitions/uint32
	240	+ description:
	241	+ When the DRAM type is LPDDR4, this parameter defines the DRAM side ODT on
	242	+ DQS/DQ line strength in ohms.
	243	+ default: 40
	244	+
	245	+ rockchip,lpddr4_ca_odt:
	246	+ deprecated: true
	247	+ $ref: /schemas/types.yaml#/definitions/uint32
	248	+ description:
	249	+ When the DRAM type is LPDDR4, this parameter defines the DRAM side ODT on
	250	+ CA line strength in ohms.
	251	+ default: 40
	252	+
	253	+ rockchip,phy_lpddr4_ca_drv:
	254	+ deprecated: true
	255	+ $ref: /schemas/types.yaml#/definitions/uint32
	256	+ description:
	257	+ When the DRAM type is LPDDR4, this parameter defines the PHY side CA line
	258	+ (including command address line) drive strength.
	259	+ default: 40
	260	+
	261	+ rockchip,phy_lpddr4_ck_cs_drv:
	262	+ deprecated: true
	263	+ $ref: /schemas/types.yaml#/definitions/uint32
	264	+ description:
	265	+ When the DRAM type is LPDDR4, this parameter defines the PHY side clock
	266	+ line and CS line drive strength.
	267	+ default: 80
	268	+
	269	+ rockchip,phy_lpddr4_dq_drv:
	270	+ deprecated: true
	271	+ $ref: /schemas/types.yaml#/definitions/uint32
	272	+ description:
	273	+ When the DRAM type is LPDDR4, this parameter defines the PHY side DQ line
	274	+ (including DQS/DQ/DM line) drive strength.
	275	+ default: 80
	276	+
	277	+ rockchip,phy_lpddr4_odt:
	278	+ deprecated: true
	279	+ $ref: /schemas/types.yaml#/definitions/uint32
	280	+ description:
	281	+ When the DRAM type is LPDDR4, this parameter defines the PHY side ODT
	282	+ strength.
	283	+ default: 60
	284	+
	285	+ rockchip,pd-idle-ns:
	286	+ description:
	287	+ Configure the PD_IDLE value in nanoseconds. Defines the power-down idle
	288	+ period in which memories are placed into power-down mode if bus is idle
	289	+ for PD_IDLE nanoseconds.
	290	+
	291	+ rockchip,sr-idle-ns:
	292	+ description:
	293	+ Configure the SR_IDLE value in nanoseconds. Defines the self-refresh idle
	294	+ period in which memories are placed into self-refresh mode if bus is idle
	295	+ for SR_IDLE nanoseconds.
	296	+ default: 0
	297	+
	298	+ rockchip,sr-mc-gate-idle-ns:
	299	+ description:
	300	+ Defines the memory self-refresh and controller clock gating idle period in nanoseconds.
	301	+ Memories are placed into self-refresh mode and memory controller clock
	302	+ arg gating started if bus is idle for sr_mc_gate_idle nanoseconds.
	303	+
	304	+ rockchip,srpd-lite-idle-ns:
	305	+ description:
	306	+ Defines the self-refresh power down idle period in which memories are
	307	+ placed into self-refresh power down mode if bus is idle for
	308	+ srpd_lite_idle nanoseonds. This parameter is for LPDDR4 only.
	309	+
	310	+ rockchip,standby-idle-ns:
	311	+ description:
	312	+ Defines the standby idle period in which memories are placed into
	313	+ self-refresh mode. The controller, pi, PHY and DRAM clock will be gated
	314	+ if bus is idle for standby_idle nanoseconds.
	315	+
	316	+ rockchip,pd-idle-dis-freq-hz:
	317	+ description:
	318	+ Defines the power-down idle disable frequency in Hz. When the DDR
	319	+ frequency is greater than pd-idle-dis-freq, power-down idle is disabled.
	320	+ See also rockchip,pd-idle-ns.
	321	+
	322	+ rockchip,sr-idle-dis-freq-hz:
	323	+ description:
	324	+ Defines the self-refresh idle disable frequency in Hz. When the DDR
	325	+ frequency is greater than sr-idle-dis-freq, self-refresh idle is
	326	+ disabled. See also rockchip,sr-idle-ns.
	327	+
	328	+ rockchip,sr-mc-gate-idle-dis-freq-hz:
	329	+ description:
	330	+ Defines the self-refresh and memory-controller clock gating disable
	331	+ frequency in Hz. When the DDR frequency is greater than
	332	+ sr-mc-gate-idle-dis-freq, the clock will not be gated when idle. See also
	333	+ rockchip,sr-mc-gate-idle-ns.
	334	+
	335	+ rockchip,srpd-lite-idle-dis-freq-hz:
	336	+ description:
	337	+ Defines the self-refresh power down idle disable frequency in Hz. When
	338	+ the DDR frequency is greater than srpd-lite-idle-dis-freq, memory will
	339	+ not be placed into self-refresh power down mode when idle. See also
	340	+ rockchip,srpd-lite-idle-ns.
	341	+
	342	+ rockchip,standby-idle-dis-freq-hz:
	343	+ description:
	344	+ Defines the standby idle disable frequency in Hz. When the DDR frequency
	345	+ is greater than standby-idle-dis-freq, standby idle is disabled. See also
	346	+ rockchip,standby-idle-ns.
	347	+
	348	+required:
	349	+ - compatible
	350	+ - devfreq-events
	351	+ - clocks
	352	+ - clock-names
	353	+ - operating-points-v2
	354	+ - center-supply
	355	+
	356	+additionalProperties: false
	357	+
	358	+examples:
	359	+ - \|
	360	+ #include <dt-bindings/clock/rk3399-cru.h>
	361	+ #include <dt-bindings/interrupt-controller/arm-gic.h>
	362	+ memory-controller {
	363	+ compatible = "rockchip,rk3399-dmc";
	364	+ devfreq-events = <&dfi>;
	365	+ rockchip,pmu = <&pmu>;
	366	+ interrupts = <GIC_SPI 1 IRQ_TYPE_LEVEL_HIGH>;
	367	+ clocks = <&cru SCLK_DDRC>;
	368	+ clock-names = "dmc_clk";
	369	+ operating-points-v2 = <&dmc_opp_table>;
	370	+ center-supply = <&ppvar_centerlogic>;
	371	+ rockchip,pd-idle-ns = <160>;
	372	+ rockchip,sr-idle-ns = <10240>;
	373	+ rockchip,sr-mc-gate-idle-ns = <40960>;
	374	+ rockchip,srpd-lite-idle-ns = <61440>;
	375	+ rockchip,standby-idle-ns = <81920>;
	376	+ rockchip,ddr3_odt_dis_freq = <333000000>;
	377	+ rockchip,lpddr3_odt_dis_freq = <333000000>;
	378	+ rockchip,lpddr4_odt_dis_freq = <333000000>;
	379	+ rockchip,pd-idle-dis-freq-hz = <1000000000>;
	380	+ rockchip,sr-idle-dis-freq-hz = <1000000000>;
	381	+ rockchip,sr-mc-gate-idle-dis-freq-hz = <1000000000>;
	382	+ rockchip,srpd-lite-idle-dis-freq-hz = <0>;
	383	+ rockchip,standby-idle-dis-freq-hz = <928000000>;
	384	+ };

Documentation/power/energy-model.rst+22−2

@@ -123,6 +123,26 @@ allows a platform to register EM power values which are reflecting total power · 122 unmodified lines
123	123	(static + dynamic). These power values might be coming directly from
124	124	experiments and measurements.
125	125
	126	+Registration of 'artificial' EM
	127	+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
	128	+
	129	+There is an option to provide a custom callback for drivers missing detailed
	130	+knowledge about power value for each performance state. The callback
	131	+.get_cost() is optional and provides the 'cost' values used by the EAS.
	132	+This is useful for platforms that only provide information on relative
	133	+efficiency between CPU types, where one could use the information to
	134	+create an abstract power model. But even an abstract power model can
	135	+sometimes be hard to fit in, given the input power value size restrictions.
	136	+The .get_cost() allows to provide the 'cost' values which reflect the
	137	+efficiency of the CPUs. This would allow to provide EAS information which
	138	+has different relation than what would be forced by the EM internal
	139	+formulas calculating 'cost' values. To register an EM for such platform, the
	140	+driver must set the flag 'milliwatts' to 0, provide .get_power() callback
	141	+and provide .get_cost() callback. The EM framework would handle such platform
	142	+properly during registration. A flag EM_PERF_DOMAIN_ARTIFICIAL is set for such
	143	+platform. Special care should be taken by other frameworks which are using EM
	144	+to test and treat this flag properly.
	145	+
126	146	Registration of 'simple' EM
127	147	~~~~~~~~~~~~~~~~~~~~~~~~~~~
128	148
@@ -181,8 +201,8 @@ EM framework:: · 52 unmodified lines
181	201
182	202	-> drivers/cpufreq/foo_cpufreq.c
183	203
184		− 01 static int est_power(unsigned long mW, unsigned long KHz,
185		− 02 struct device *dev)
	204	+ 01 static int est_power(struct device dev, unsigned long mW,
	205	+ 02 unsigned long *KHz)
186	206	03 {
187	207	04 long freq, power;
188	208	05

arch/arm64/kernel/smp.c+1−0

@@ -512,6 +512,7 @@ struct acpi_madt_generic_interrupt *acpi_cpu_get_madt_gicc(int cpu) · 511 unmodified lines
512	512	{
513	513	return &cpu_madt_gicc[cpu];
514	514	}
	515	+EXPORT_SYMBOL_GPL(acpi_cpu_get_madt_gicc);
515	516
516	517	/*
517	518	* acpi_map_gic_cpu_interface - parse processor MADT entry

arch/x86/include/asm/msr-index.h+1−0

@@ -319,6 +319,7 @@ · 318 unmodified lines
319	319
320	320	/* Run Time Average Power Limiting (RAPL) Interface */
321	321
	322	+#define MSR_VR_CURRENT_CONFIG 0x00000601
322	323	#define MSR_RAPL_POWER_UNIT 0x00000606
323	324
324	325	#define MSR_PKG_POWER_LIMIT 0x00000610

arch/x86/kernel/acpi/boot.c+1−1

@@ -1862,7 +1862,7 @@ int __acpi_release_global_lock(unsigned int *lock) · 1861 unmodified lines
1862	1862
1863	1863	void __init arch_reserve_mem_area(acpi_physical_address addr, size_t size)
1864	1864	{
1865		− e820__range_add(addr, size, E820_TYPE_ACPI);
	1865	+ e820__range_add(addr, size, E820_TYPE_NVS);
1866	1866	e820__update_table_print();
1867	1867	}
1868	1868

drivers/acpi/bus.c+26−8

@@ -278,6 +278,20 @@ bool osc_sb_apei_support_acked; · 277 unmodified lines
278	278	bool osc_pc_lpi_support_confirmed;
279	279	EXPORT_SYMBOL_GPL(osc_pc_lpi_support_confirmed);
280	280
	281	+/*
	282	+ * ACPI 6.2 Section 6.2.11.2 'Platform-Wide OSPM Capabilities':
	283	+ * Starting with ACPI Specification 6.2, all _CPC registers can be in
	284	+ * PCC, System Memory, System IO, or Functional Fixed Hardware address
	285	+ * spaces. OSPM support for this more flexible register space scheme is
	286	+ * indicated by the “Flexible Address Space for CPPC Registers” _OSC bit.
	287	+ *
	288	+ * Otherwise (cf ACPI 6.1, s8.4.7.1.1.X), _CPC registers must be in:
	289	+ * - PCC or Functional Fixed Hardware address space if defined
	290	+ * - SystemMemory address space (NULL register) if not defined
	291	+ */
	292	+bool osc_cpc_flexible_adr_space_confirmed;
	293	+EXPORT_SYMBOL_GPL(osc_cpc_flexible_adr_space_confirmed);
	294	+
281	295	/*
282	296	* ACPI 6.4 Operating System Capabilities for USB.
283	297	*/
@@ -315,12 +329,15 @@ static void acpi_bus_osc_negotiate_platform_control(void) · 31 unmodified lines
315	329	#endif
316	330	#ifdef CONFIG_X86
317	331	capbuf[OSC_SUPPORT_DWORD] \|= OSC_SB_GENERIC_INITIATOR_SUPPORT;
318		− if (boot_cpu_has(X86_FEATURE_HWP)) {
319		− capbuf[OSC_SUPPORT_DWORD] \|= OSC_SB_CPC_SUPPORT;
320		− capbuf[OSC_SUPPORT_DWORD] \|= OSC_SB_CPCV2_SUPPORT;
321		− }
322	332	#endif
323	333
	334	+#ifdef CONFIG_ACPI_CPPC_LIB
	335	+ capbuf[OSC_SUPPORT_DWORD] \|= OSC_SB_CPC_SUPPORT;
	336	+ capbuf[OSC_SUPPORT_DWORD] \|= OSC_SB_CPCV2_SUPPORT;
	337	+#endif
	338	+
	339	+ capbuf[OSC_SUPPORT_DWORD] \|= OSC_SB_CPC_FLEXIBLE_ADR_SPACE;
	340	+
324	341	if (IS_ENABLED(CONFIG_SCHED_MC_PRIO))
325	342	capbuf[OSC_SUPPORT_DWORD] \|= OSC_SB_CPC_DIVERSE_HIGH_SUPPORT;
326	343
@@ -341,10 +358,9 @@ static void acpi_bus_osc_negotiate_platform_control(void) · 14 unmodified lines
341	358	return;
342	359	}
343	360
344		−#ifdef CONFIG_X86
345		− if (boot_cpu_has(X86_FEATURE_HWP))
346		− osc_sb_cppc_not_supported = !(capbuf_ret[OSC_SUPPORT_DWORD] &
347		− (OSC_SB_CPC_SUPPORT \| OSC_SB_CPCV2_SUPPORT));
	361	+#ifdef CONFIG_ACPI_CPPC_LIB
	362	+ osc_sb_cppc_not_supported = !(capbuf_ret[OSC_SUPPORT_DWORD] &
	363	+ (OSC_SB_CPC_SUPPORT \| OSC_SB_CPCV2_SUPPORT));
348	364	#endif
349	365
350	366	/*
@@ -366,6 +382,8 @@ static void acpi_bus_osc_negotiate_platform_control(void) · 15 unmodified lines
366	382	capbuf_ret[OSC_SUPPORT_DWORD] & OSC_SB_PCLPI_SUPPORT;
367	383	osc_sb_native_usb4_support_confirmed =
368	384	capbuf_ret[OSC_SUPPORT_DWORD] & OSC_SB_NATIVE_USB4_SUPPORT;
	385	+ osc_cpc_flexible_adr_space_confirmed =
	386	+ capbuf_ret[OSC_SUPPORT_DWORD] & OSC_SB_CPC_FLEXIBLE_ADR_SPACE;
369	387	}
370	388
371	389	kfree(context.ret.pointer);

drivers/acpi/cppc_acpi.c+43−1

@@ -100,6 +100,16 @@ static DEFINE_PER_CPU(struct cpc_desc *, cpc_desc_ptr); · 99 unmodified lines
100	100	(cpc)->cpc_entry.reg.space_id == \
101	101	ACPI_ADR_SPACE_PLATFORM_COMM)
102	102
	103	+/* Check if a CPC register is in SystemMemory */
	104	+#define CPC_IN_SYSTEM_MEMORY(cpc) ((cpc)->type == ACPI_TYPE_BUFFER && \
	105	+ (cpc)->cpc_entry.reg.space_id == \
	106	+ ACPI_ADR_SPACE_SYSTEM_MEMORY)
	107	+
	108	+/* Check if a CPC register is in SystemIo */
	109	+#define CPC_IN_SYSTEM_IO(cpc) ((cpc)->type == ACPI_TYPE_BUFFER && \
	110	+ (cpc)->cpc_entry.reg.space_id == \
	111	+ ACPI_ADR_SPACE_SYSTEM_IO)
	112	+
103	113	/* Evaluates to True if reg is a NULL register descriptor */
104	114	#define IS_NULL_REG(reg) ((reg)->space_id == ACPI_ADR_SPACE_SYSTEM_MEMORY && \
105	115	(reg)->address == 0 && \
@@ -424,6 +434,24 @@ bool acpi_cpc_valid(void) · 318 unmodified lines
424	434	}
425	435	EXPORT_SYMBOL_GPL(acpi_cpc_valid);
426	436
	437	+bool cppc_allow_fast_switch(void)
	438	+{
	439	+ struct cpc_register_resource *desired_reg;
	440	+ struct cpc_desc *cpc_ptr;
	441	+ int cpu;
	442	+
	443	+ for_each_possible_cpu(cpu) {
	444	+ cpc_ptr = per_cpu(cpc_desc_ptr, cpu);
	445	+ desired_reg = &cpc_ptr->cpc_regs[DESIRED_PERF];
	446	+ if (!CPC_IN_SYSTEM_MEMORY(desired_reg) &&
	447	+ !CPC_IN_SYSTEM_IO(desired_reg))
	448	+ return false;
	449	+ }
	450	+
	451	+ return true;
	452	+}
	453	+EXPORT_SYMBOL_GPL(cppc_allow_fast_switch);
	454	+
427	455	/**
428	456	* acpi_get_psd_map - Map the CPUs in the freq domain of a given cpu
429	457	* @cpu: Find all CPUs that share a domain with cpu.
@@ -736,6 +764,11 @@ int acpi_cppc_processor_probe(struct acpi_processor *pr) · 306 unmodified lines
736	764	if (gas_t->address) {
737	765	void __iomem *addr;
738	766
	767	+ if (!osc_cpc_flexible_adr_space_confirmed) {
	768	+ pr_debug("Flexible address space capability not supported\n");
	769	+ goto out_free;
	770	+ }
	771	+
739	772	addr = ioremap(gas_t->address, gas_t->bit_width/8);
740	773	if (!addr)
741	774	goto out_free;
@@ -758,6 +791,10 @@ int acpi_cppc_processor_probe(struct acpi_processor *pr) · 16 unmodified lines
758	791	gas_t->address);
759	792	goto out_free;
760	793	}
	794	+ if (!osc_cpc_flexible_adr_space_confirmed) {
	795	+ pr_debug("Flexible address space capability not supported\n");
	796	+ goto out_free;
	797	+ }
761	798	} else {
762	799	if (gas_t->space_id != ACPI_ADR_SPACE_FIXED_HARDWARE \|\| !cpc_ffh_supported()) {
763	800	/* Support only PCC, SystemMemory, SystemIO, and FFH type regs. */
@@ -1447,6 +1484,9 @@ EXPORT_SYMBOL_GPL(cppc_set_perf); · 683 unmodified lines
1447	1484	* transition latency for performance change requests. The closest we have
1448	1485	* is the timing information from the PCCT tables which provides the info
1449	1486	* on the number and frequency of PCC commands the platform can handle.
	1487	+ *
	1488	+ * If desired_reg is in the SystemMemory or SystemIo ACPI address space,
	1489	+ * then assume there is no latency.
1450	1490	*/
1451	1491	unsigned int cppc_get_transition_latency(int cpu_num)
1452	1492	{
@@ -1472,7 +1512,9 @@ unsigned int cppc_get_transition_latency(int cpu_num) · 19 unmodified lines
1472	1512	return CPUFREQ_ETERNAL;
1473	1513
1474	1514	desired_reg = &cpc_desc->cpc_regs[DESIRED_PERF];
1475		− if (!CPC_IN_PCC(desired_reg))
	1515	+ if (CPC_IN_SYSTEM_MEMORY(desired_reg) \|\| CPC_IN_SYSTEM_IO(desired_reg))
	1516	+ return 0;
	1517	+ else if (!CPC_IN_PCC(desired_reg))
1476	1518	return CPUFREQ_ETERNAL;
1477	1519
1478	1520	if (pcc_ss_id < 0)

drivers/base/power/common.c+4−4

@@ -172,10 +172,10 @@ EXPORT_SYMBOL_GPL(dev_pm_domain_attach_by_name); · 171 unmodified lines
172	172	* @dev: Device to detach.
173	173	* @power_off: Used to indicate whether we should power off the device.
174	174	*
175		− * This functions will reverse the actions from dev_pm_domain_attach() and
176		− * dev_pm_domain_attach_by_id(), thus it detaches @dev from its PM domain.
177		− * Typically it should be invoked during the remove phase, either from
178		− * subsystem level code or from drivers.
	175	+ * This functions will reverse the actions from dev_pm_domain_attach(),
	176	+ * dev_pm_domain_attach_by_id() and dev_pm_domain_attach_by_name(), thus it
	177	+ * detaches @dev from its PM domain. Typically it should be invoked during the
	178	+ * remove phase, either from subsystem level code or from drivers.
179	179	*
180	180	* Callers must ensure proper synchronization of this function with power
181	181	* management callbacks.

drivers/base/power/domain.c+170−108

@@ -131,7 +131,7 @@ static const struct genpd_lock_ops genpd_spin_ops = { · 130 unmodified lines
131	131	#define genpd_is_cpu_domain(genpd) (genpd->flags & GENPD_FLAG_CPU_DOMAIN)
132	132	#define genpd_is_rpm_always_on(genpd) (genpd->flags & GENPD_FLAG_RPM_ALWAYS_ON)
133	133
134		−static inline bool irq_safe_dev_in_no_sleep_domain(struct device *dev,
	134	+static inline bool irq_safe_dev_in_sleep_domain(struct device *dev,
135	135	const struct generic_pm_domain *genpd)
136	136	{
137	137	bool ret;
@@ -139,11 +139,14 @@ static inline bool irq_safe_dev_in_no_sleep_domain(struct device *dev, · 1 unmodified line
139	139	ret = pm_runtime_is_irq_safe(dev) && !genpd_is_irq_safe(genpd);
140	140
141	141	/*
142		− * Warn once if an IRQ safe device is attached to a no sleep domain, as
143		− * to indicate a suboptimal configuration for PM. For an always on
144		− * domain this isn't case, thus don't warn.
	142	+ * Warn once if an IRQ safe device is attached to a domain, which
	143	+ * callbacks are allowed to sleep. This indicates a suboptimal
	144	+ * configuration for PM, but it doesn't matter for an always on domain.
145	145	*/
146		− if (ret && !genpd_is_always_on(genpd))
	146	+ if (genpd_is_always_on(genpd) \|\| genpd_is_rpm_always_on(genpd))
	147	+ return ret;
	148	+
	149	+ if (ret)
147	150	dev_warn_once(dev, "PM domain %s will not be powered off\n",
148	151	genpd->name);
149	152
@@ -225,24 +228,23 @@ static void genpd_debug_remove(struct generic_pm_domain *genpd) · 75 unmodified lines
225	228
226	229	static void genpd_update_accounting(struct generic_pm_domain *genpd)
227	230	{
228		− ktime_t delta, now;
	231	+ u64 delta, now;
229	232
230		− now = ktime_get();
231		− delta = ktime_sub(now, genpd->accounting_time);
	233	+ now = ktime_get_mono_fast_ns();
	234	+ if (now <= genpd->accounting_time)
	235	+ return;
	236	+
	237	+ delta = now - genpd->accounting_time;
232	238
233	239	/*
234	240	* If genpd->status is active, it means we are just
235	241	* out of off and so update the idle time and vice
236	242	* versa.
237	243	*/
238		− if (genpd->status == GENPD_STATE_ON) {
239		− int state_idx = genpd->state_idx;
240		−
241		− genpd->states[state_idx].idle_time =
242		− ktime_add(genpd->states[state_idx].idle_time, delta);
243		− } else {
244		− genpd->on_time = ktime_add(genpd->on_time, delta);
245		− }
	244	+ if (genpd->status == GENPD_STATE_ON)
	245	+ genpd->states[genpd->state_idx].idle_time += delta;
	246	+ else
	247	+ genpd->on_time += delta;
246	248
247	249	genpd->accounting_time = now;
248	250	}
@@ -476,15 +478,16 @@ EXPORT_SYMBOL_GPL(dev_pm_genpd_set_performance_state); · 227 unmodified lines
476	478	*/
477	479	void dev_pm_genpd_set_next_wakeup(struct device *dev, ktime_t next)
478	480	{
479		− struct generic_pm_domain_data *gpd_data;
480	481	struct generic_pm_domain *genpd;
	482	+ struct gpd_timing_data *td;
481	483
482	484	genpd = dev_to_genpd_safe(dev);
483	485	if (!genpd)
484	486	return;
485	487
486		− gpd_data = to_gpd_data(dev->power.subsys_data->domain_data);
487		− gpd_data->next_wakeup = next;
	488	+ td = to_gpd_data(dev->power.subsys_data->domain_data)->td;
	489	+ if (td)
	490	+ td->next_wakeup = next;
488	491	}
489	492	EXPORT_SYMBOL_GPL(dev_pm_genpd_set_next_wakeup);
490	493
@@ -506,6 +509,7 @@ static int _genpd_power_on(struct generic_pm_domain *genpd, bool timed) · 15 unmodified lines
506	509	if (!genpd->power_on)
507	510	goto out;
508	511
	512	+ timed = timed && genpd->gd && !genpd->states[state_idx].fwnode;
509	513	if (!timed) {
510	514	ret = genpd->power_on(genpd);
511	515	if (ret)
@@ -524,7 +528,7 @@ static int _genpd_power_on(struct generic_pm_domain *genpd, bool timed) · 12 unmodified lines
524	528	goto out;
525	529
526	530	genpd->states[state_idx].power_on_latency_ns = elapsed_ns;
527		− genpd->max_off_time_changed = true;
	531	+ genpd->gd->max_off_time_changed = true;
528	532	pr_debug("%s: Power-%s latency exceeded, new value %lld ns\n",
529	533	genpd->name, "on", elapsed_ns);
530	534
@@ -555,6 +559,7 @@ static int _genpd_power_off(struct generic_pm_domain *genpd, bool timed) · 24 unmodified lines
555	559	if (!genpd->power_off)
556	560	goto out;
557	561
	562	+ timed = timed && genpd->gd && !genpd->states[state_idx].fwnode;
558	563	if (!timed) {
559	564	ret = genpd->power_off(genpd);
560	565	if (ret)
@@ -573,7 +578,7 @@ static int _genpd_power_off(struct generic_pm_domain *genpd, bool timed) · 12 unmodified lines
573	578	goto out;
574	579
575	580	genpd->states[state_idx].power_off_latency_ns = elapsed_ns;
576		− genpd->max_off_time_changed = true;
	581	+ genpd->gd->max_off_time_changed = true;
577	582	pr_debug("%s: Power-%s latency exceeded, new value %lld ns\n",
578	583	genpd->name, "off", elapsed_ns);
579	584
@@ -649,18 +654,12 @@ static int genpd_power_off(struct generic_pm_domain *genpd, bool one_dev_on, · 69 unmodified lines
649	654	}
650	655
651	656	list_for_each_entry(pdd, &genpd->dev_list, list_node) {
652		− enum pm_qos_flags_status stat;
653		−
654		− stat = dev_pm_qos_flags(pdd->dev, PM_QOS_FLAG_NO_POWER_OFF);
655		− if (stat > PM_QOS_FLAGS_NONE)
656		− return -EBUSY;
657		−
658	657	/*
659	658	* Do not allow PM domain to be powered off, when an IRQ safe
660	659	* device is part of a non-IRQ safe domain.
661	660	*/
662	661	if (!pm_runtime_suspended(pdd->dev) \|\|
663		− irq_safe_dev_in_no_sleep_domain(pdd->dev, genpd))
	662	+ irq_safe_dev_in_sleep_domain(pdd->dev, genpd))
664	663	not_suspended++;
665	664	}
666	665
@@ -775,25 +774,27 @@ static int genpd_dev_pm_qos_notifier(struct notifier_block *nb, · 108 unmodified lines
775	774	dev = gpd_data->base.dev;
776	775
777	776	for (;;) {
778		− struct generic_pm_domain *genpd;
	777	+ struct generic_pm_domain *genpd = ERR_PTR(-ENODATA);
779	778	struct pm_domain_data *pdd;
	779	+ struct gpd_timing_data *td;
780	780
781	781	spin_lock_irq(&dev->power.lock);
782	782
783	783	pdd = dev->power.subsys_data ?
784	784	dev->power.subsys_data->domain_data : NULL;
785	785	if (pdd) {
786		− to_gpd_data(pdd)->td.constraint_changed = true;
787		− genpd = dev_to_genpd(dev);
788		− } else {
789		− genpd = ERR_PTR(-ENODATA);
	786	+ td = to_gpd_data(pdd)->td;
	787	+ if (td) {
	788	+ td->constraint_changed = true;
	789	+ genpd = dev_to_genpd(dev);
	790	+ }
790	791	}
791	792
792	793	spin_unlock_irq(&dev->power.lock);
793	794
794	795	if (!IS_ERR(genpd)) {
795	796	genpd_lock(genpd);
796		− genpd->max_off_time_changed = true;
	797	+ genpd->gd->max_off_time_changed = true;
797	798	genpd_unlock(genpd);
798	799	}
799	800
@@ -879,9 +880,9 @@ static int genpd_runtime_suspend(struct device *dev) · 79 unmodified lines
879	880	struct generic_pm_domain *genpd;
880	881	bool (suspend_ok)(struct device __dev);
881	882	struct generic_pm_domain_data *gpd_data = dev_gpd_data(dev);
882		− struct gpd_timing_data *td = &gpd_data->td;
	883	+ struct gpd_timing_data *td = gpd_data->td;
883	884	bool runtime_pm = pm_runtime_enabled(dev);
884		− ktime_t time_start;
	885	+ ktime_t time_start = 0;
885	886	s64 elapsed_ns;
886	887	int ret;
887	888
@@ -902,8 +903,7 @@ static int genpd_runtime_suspend(struct device *dev) · 14 unmodified lines
902	903	return -EBUSY;
903	904
904	905	/* Measure suspend latency. */
905		− time_start = 0;
906		− if (runtime_pm)
	906	+ if (td && runtime_pm)
907	907	time_start = ktime_get();
908	908
909	909	ret = __genpd_runtime_suspend(dev);
@@ -917,13 +917,13 @@ static int genpd_runtime_suspend(struct device *dev) · 7 unmodified lines
917	917	}
918	918
919	919	/* Update suspend latency value if the measured time exceeds it. */
920		− if (runtime_pm) {
	920	+ if (td && runtime_pm) {
921	921	elapsed_ns = ktime_to_ns(ktime_sub(ktime_get(), time_start));
922	922	if (elapsed_ns > td->suspend_latency_ns) {
923	923	td->suspend_latency_ns = elapsed_ns;
924	924	dev_dbg(dev, "suspend latency exceeded, %lld ns\n",
925	925	elapsed_ns);
926		− genpd->max_off_time_changed = true;
	926	+ genpd->gd->max_off_time_changed = true;
927	927	td->constraint_changed = true;
928	928	}
929	929	}
@@ -932,7 +932,7 @@ static int genpd_runtime_suspend(struct device *dev) · 2 unmodified lines
932	932	* If power.irq_safe is set, this routine may be run with
933	933	* IRQs disabled, so suspend only if the PM domain also is irq_safe.
934	934	*/
935		− if (irq_safe_dev_in_no_sleep_domain(dev, genpd))
	935	+ if (irq_safe_dev_in_sleep_domain(dev, genpd))
936	936	return 0;
937	937
938	938	genpd_lock(genpd);
@@ -955,12 +955,11 @@ static int genpd_runtime_resume(struct device *dev) · 16 unmodified lines
955	955	{
956	956	struct generic_pm_domain *genpd;
957	957	struct generic_pm_domain_data *gpd_data = dev_gpd_data(dev);
958		− struct gpd_timing_data *td = &gpd_data->td;
959		− bool runtime_pm = pm_runtime_enabled(dev);
960		− ktime_t time_start;
	958	+ struct gpd_timing_data *td = gpd_data->td;
	959	+ bool timed = td && pm_runtime_enabled(dev);
	960	+ ktime_t time_start = 0;
961	961	s64 elapsed_ns;
962	962	int ret;
963		− bool timed = true;
964	963
965	964	dev_dbg(dev, "%s()\n", __func__);
966	965
@@ -972,10 +971,8 @@ static int genpd_runtime_resume(struct device *dev) · 5 unmodified lines
972	971	* As we don't power off a non IRQ safe domain, which holds
973	972	* an IRQ safe device, we don't need to restore power to it.
974	973	*/
975		− if (irq_safe_dev_in_no_sleep_domain(dev, genpd)) {
976		− timed = false;
	974	+ if (irq_safe_dev_in_sleep_domain(dev, genpd))
977	975	goto out;
978		− }
979	976
980	977	genpd_lock(genpd);
981	978	ret = genpd_power_on(genpd, 0);
@@ -988,8 +985,7 @@ static int genpd_runtime_resume(struct device *dev) · 6 unmodified lines
988	985
989	986	out:
990	987	/* Measure resume latency. */
991		− time_start = 0;
992		− if (timed && runtime_pm)
	988	+ if (timed)
993	989	time_start = ktime_get();
994	990
995	991	ret = genpd_start_dev(genpd, dev);
@@ -1001,13 +997,13 @@ static int genpd_runtime_resume(struct device *dev) · 5 unmodified lines
1001	997	goto err_stop;
1002	998
1003	999	/* Update resume latency value if the measured time exceeds it. */
1004		− if (timed && runtime_pm) {
	1000	+ if (timed) {
1005	1001	elapsed_ns = ktime_to_ns(ktime_sub(ktime_get(), time_start));
1006	1002	if (elapsed_ns > td->resume_latency_ns) {
1007	1003	td->resume_latency_ns = elapsed_ns;
1008	1004	dev_dbg(dev, "resume latency exceeded, %lld ns\n",
1009	1005	elapsed_ns);
1010		− genpd->max_off_time_changed = true;
	1006	+ genpd->gd->max_off_time_changed = true;
1011	1007	td->constraint_changed = true;
1012	1008	}
1013	1009	}
@@ -1500,9 +1496,11 @@ EXPORT_SYMBOL_GPL(dev_pm_genpd_resume); · 486 unmodified lines
1500	1496
1501	1497	#endif /* CONFIG_PM_SLEEP */
1502	1498
1503		−static struct generic_pm_domain_data genpd_alloc_dev_data(struct device dev)
	1499	+static struct generic_pm_domain_data genpd_alloc_dev_data(struct device dev,
	1500	+ bool has_governor)
1504	1501	{
1505	1502	struct generic_pm_domain_data *gpd_data;
	1503	+ struct gpd_timing_data *td;
1506	1504	int ret;
1507	1505
1508	1506	ret = dev_pm_get_subsys_data(dev);
@@ -1516,26 +1514,38 @@ static struct generic_pm_domain_data genpd_alloc_dev_data(struct device dev) · 7 unmodified lines
1516	1514	}
1517	1515
1518	1516	gpd_data->base.dev = dev;
1519		− gpd_data->td.constraint_changed = true;
1520		− gpd_data->td.effective_constraint_ns = PM_QOS_RESUME_LATENCY_NO_CONSTRAINT_NS;
1521	1517	gpd_data->nb.notifier_call = genpd_dev_pm_qos_notifier;
1522		− gpd_data->next_wakeup = KTIME_MAX;
1523	1518
1524		− spin_lock_irq(&dev->power.lock);
	1519	+ /* Allocate data used by a governor. */
	1520	+ if (has_governor) {
	1521	+ td = kzalloc(sizeof(*td), GFP_KERNEL);
	1522	+ if (!td) {
	1523	+ ret = -ENOMEM;
	1524	+ goto err_free;
	1525	+ }
1525	1526
1526		− if (dev->power.subsys_data->domain_data) {
1527		− ret = -EINVAL;
1528		− goto err_free;
	1527	+ td->constraint_changed = true;
	1528	+ td->effective_constraint_ns = PM_QOS_RESUME_LATENCY_NO_CONSTRAINT_NS;
	1529	+ td->next_wakeup = KTIME_MAX;
	1530	+ gpd_data->td = td;
1529	1531	}
1530	1532
1531		− dev->power.subsys_data->domain_data = &gpd_data->base;
	1533	+ spin_lock_irq(&dev->power.lock);
	1534	+
	1535	+ if (dev->power.subsys_data->domain_data)
	1536	+ ret = -EINVAL;
	1537	+ else
	1538	+ dev->power.subsys_data->domain_data = &gpd_data->base;
1532	1539
1533	1540	spin_unlock_irq(&dev->power.lock);
1534	1541
	1542	+ if (ret)
	1543	+ goto err_free;
	1544	+
1535	1545	return gpd_data;
1536	1546
1537	1547	err_free:
1538		− spin_unlock_irq(&dev->power.lock);
	1548	+ kfree(gpd_data->td);
1539	1549	kfree(gpd_data);
1540	1550	err_put:
1541	1551	dev_pm_put_subsys_data(dev);
@@ -1551,6 +1561,7 @@ static void genpd_free_dev_data(struct device *dev, · 9 unmodified lines
1551	1561
1552	1562	spin_unlock_irq(&dev->power.lock);
1553	1563
	1564	+ kfree(gpd_data->td);
1554	1565	kfree(gpd_data);
1555	1566	dev_pm_put_subsys_data(dev);
1556	1567	}
@@ -1607,6 +1618,7 @@ static int genpd_get_cpu(struct generic_pm_domain genpd, struct device dev) · 50 unmodified lines
1607	1618	static int genpd_add_device(struct generic_pm_domain genpd, struct device dev,
1608	1619	struct device *base_dev)
1609	1620	{
	1621	+ struct genpd_governor_data *gd = genpd->gd;
1610	1622	struct generic_pm_domain_data *gpd_data;
1611	1623	int ret;
1612	1624
@@ -1615,7 +1627,7 @@ static int genpd_add_device(struct generic_pm_domain genpd, struct device dev, · 2 unmodified lines
1615	1627	if (IS_ERR_OR_NULL(genpd) \|\| IS_ERR_OR_NULL(dev))
1616	1628	return -EINVAL;
1617	1629
1618		− gpd_data = genpd_alloc_dev_data(dev);
	1630	+ gpd_data = genpd_alloc_dev_data(dev, gd);
1619	1631	if (IS_ERR(gpd_data))
1620	1632	return PTR_ERR(gpd_data);
1621	1633
@@ -1631,7 +1643,8 @@ static int genpd_add_device(struct generic_pm_domain genpd, struct device dev, · 9 unmodified lines
1631	1643	dev_pm_domain_set(dev, &genpd->domain);
1632	1644
1633	1645	genpd->device_count++;
1634		− genpd->max_off_time_changed = true;
	1646	+ if (gd)
	1647	+ gd->max_off_time_changed = true;
1635	1648
1636	1649	list_add_tail(&gpd_data->base.list_node, &genpd->dev_list);
1637	1650
@@ -1685,7 +1698,8 @@ static int genpd_remove_device(struct generic_pm_domain *genpd, · 47 unmodified lines
1685	1698	}
1686	1699
1687	1700	genpd->device_count--;
1688		− genpd->max_off_time_changed = true;
	1701	+ if (genpd->gd)
	1702	+ genpd->gd->max_off_time_changed = true;
1689	1703
1690	1704	genpd_clear_cpumask(genpd, gpd_data->cpu);
1691	1705	dev_pm_domain_set(dev, NULL);
@@ -1958,6 +1972,53 @@ static int genpd_set_default_power_state(struct generic_pm_domain *genpd) · 266 unmodified lines
1958	1972	return 0;
1959	1973	}
1960	1974
	1975	+static int genpd_alloc_data(struct generic_pm_domain *genpd)
	1976	+{
	1977	+ struct genpd_governor_data *gd = NULL;
	1978	+ int ret;
	1979	+
	1980	+ if (genpd_is_cpu_domain(genpd) &&
	1981	+ !zalloc_cpumask_var(&genpd->cpus, GFP_KERNEL))
	1982	+ return -ENOMEM;
	1983	+
	1984	+ if (genpd->gov) {
	1985	+ gd = kzalloc(sizeof(*gd), GFP_KERNEL);
	1986	+ if (!gd) {
	1987	+ ret = -ENOMEM;
	1988	+ goto free;
	1989	+ }
	1990	+
	1991	+ gd->max_off_time_ns = -1;
	1992	+ gd->max_off_time_changed = true;
	1993	+ gd->next_wakeup = KTIME_MAX;
	1994	+ }
	1995	+
	1996	+ /* Use only one "off" state if there were no states declared */
	1997	+ if (genpd->state_count == 0) {
	1998	+ ret = genpd_set_default_power_state(genpd);
	1999	+ if (ret)
	2000	+ goto free;
	2001	+ }
	2002	+
	2003	+ genpd->gd = gd;
	2004	+ return 0;
	2005	+
	2006	+free:
	2007	+ if (genpd_is_cpu_domain(genpd))
	2008	+ free_cpumask_var(genpd->cpus);
	2009	+ kfree(gd);
	2010	+ return ret;
	2011	+}
	2012	+
	2013	+static void genpd_free_data(struct generic_pm_domain *genpd)
	2014	+{
	2015	+ if (genpd_is_cpu_domain(genpd))
	2016	+ free_cpumask_var(genpd->cpus);
	2017	+ if (genpd->free_states)
	2018	+ genpd->free_states(genpd->states, genpd->state_count);
	2019	+ kfree(genpd->gd);
	2020	+}
	2021	+
1961	2022	static void genpd_lock_init(struct generic_pm_domain *genpd)
1962	2023	{
1963	2024	if (genpd->flags & GENPD_FLAG_IRQ_SAFE) {
@@ -1995,11 +2056,9 @@ int pm_genpd_init(struct generic_pm_domain *genpd, · 31 unmodified lines
1995	2056	atomic_set(&genpd->sd_count, 0);
1996	2057	genpd->status = is_off ? GENPD_STATE_OFF : GENPD_STATE_ON;
1997	2058	genpd->device_count = 0;
1998		− genpd->max_off_time_ns = -1;
1999		− genpd->max_off_time_changed = true;
2000	2059	genpd->provider = NULL;
2001	2060	genpd->has_provider = false;
2002		− genpd->accounting_time = ktime_get();
	2061	+ genpd->accounting_time = ktime_get_mono_fast_ns();
2003	2062	genpd->domain.ops.runtime_suspend = genpd_runtime_suspend;
2004	2063	genpd->domain.ops.runtime_resume = genpd_runtime_resume;
2005	2064	genpd->domain.ops.prepare = genpd_prepare;
@@ -2017,26 +2076,22 @@ int pm_genpd_init(struct generic_pm_domain *genpd, · 11 unmodified lines
2017	2076	genpd->dev_ops.start = pm_clk_resume;
2018	2077	}
2019	2078
	2079	+ /* The always-on governor works better with the corresponding flag. */
	2080	+ if (gov == &pm_domain_always_on_gov)
	2081	+ genpd->flags \|= GENPD_FLAG_RPM_ALWAYS_ON;
	2082	+
2020	2083	/* Always-on domains must be powered on at initialization. */
2021	2084	if ((genpd_is_always_on(genpd) \|\| genpd_is_rpm_always_on(genpd)) &&
2022	2085	!genpd_status_on(genpd))
2023	2086	return -EINVAL;
2024	2087
2025		− if (genpd_is_cpu_domain(genpd) &&
2026		− !zalloc_cpumask_var(&genpd->cpus, GFP_KERNEL))
2027		− return -ENOMEM;
2028		−
2029		− /* Use only one "off" state if there were no states declared */
2030		− if (genpd->state_count == 0) {
2031		− ret = genpd_set_default_power_state(genpd);
2032		− if (ret) {
2033		− if (genpd_is_cpu_domain(genpd))
2034		− free_cpumask_var(genpd->cpus);
2035		− return ret;
2036		− }
2037		− } else if (!gov && genpd->state_count > 1) {
	2088	+ /* Multiple states but no governor doesn't make sense. */
	2089	+ if (!gov && genpd->state_count > 1)
2038	2090	pr_warn("%s: no governor for states\n", genpd->name);
2039		− }
	2091	+
	2092	+ ret = genpd_alloc_data(genpd);
	2093	+ if (ret)
	2094	+ return ret;
2040	2095
2041	2096	device_initialize(&genpd->dev);
2042	2097	dev_set_name(&genpd->dev, "%s", genpd->name);
@@ -2081,10 +2136,7 @@ static int genpd_remove(struct generic_pm_domain *genpd) · 38 unmodified lines
2081	2136	genpd_unlock(genpd);
2082	2137	genpd_debug_remove(genpd);
2083	2138	cancel_work_sync(&genpd->power_off_work);
2084		− if (genpd_is_cpu_domain(genpd))
2085		− free_cpumask_var(genpd->cpus);
2086		− if (genpd->free_states)
2087		− genpd->free_states(genpd->states, genpd->state_count);
	2139	+ genpd_free_data(genpd);
2088	2140
2089	2141	pr_debug("%s: removed %s\n", __func__, genpd->name);
2090	2142
@@ -3163,6 +3215,7 @@ static int sub_domains_show(struct seq_file s, void data) · 1072 unmodified lines
3163	3215	static int idle_states_show(struct seq_file s, void data)
3164	3216	{
3165	3217	struct generic_pm_domain *genpd = s->private;
	3218	+ u64 now, delta, idle_time = 0;
3166	3219	unsigned int i;
3167	3220	int ret = 0;
3168	3221
@@ -3173,17 +3226,19 @@ static int idle_states_show(struct seq_file s, void data) · 4 unmodified lines
3173	3226	seq_puts(s, "State Time Spent(ms) Usage Rejected\n");
3174	3227
3175	3228	for (i = 0; i < genpd->state_count; i++) {
3176		− ktime_t delta = 0;
3177		− s64 msecs;
	3229	+ idle_time += genpd->states[i].idle_time;
3178	3230
3179		− if ((genpd->status == GENPD_STATE_OFF) &&
3180		− (genpd->state_idx == i))
3181		− delta = ktime_sub(ktime_get(), genpd->accounting_time);
	3231	+ if (genpd->status == GENPD_STATE_OFF && genpd->state_idx == i) {
	3232	+ now = ktime_get_mono_fast_ns();
	3233	+ if (now > genpd->accounting_time) {
	3234	+ delta = now - genpd->accounting_time;
	3235	+ idle_time += delta;
	3236	+ }
	3237	+ }
3182	3238
3183		− msecs = ktime_to_ms(
3184		− ktime_add(genpd->states[i].idle_time, delta));
3185		− seq_printf(s, "S%-13i %-14lld %-14llu %llu\n", i, msecs,
3186		− genpd->states[i].usage, genpd->states[i].rejected);
	3239	+ do_div(idle_time, NSEC_PER_MSEC);
	3240	+ seq_printf(s, "S%-13i %-14llu %-14llu %llu\n", i, idle_time,
	3241	+ genpd->states[i].usage, genpd->states[i].rejected);
3187	3242	}
3188	3243
3189	3244	genpd_unlock(genpd);
@@ -3193,18 +3248,22 @@ static int idle_states_show(struct seq_file s, void data) · 3 unmodified lines
3193	3248	static int active_time_show(struct seq_file s, void data)
3194	3249	{
3195	3250	struct generic_pm_domain *genpd = s->private;
3196		− ktime_t delta = 0;
	3251	+ u64 now, on_time, delta = 0;
3197	3252	int ret = 0;
3198	3253
3199	3254	ret = genpd_lock_interruptible(genpd);
3200	3255	if (ret)
3201	3256	return -ERESTARTSYS;
3202	3257
3203		− if (genpd->status == GENPD_STATE_ON)
3204		− delta = ktime_sub(ktime_get(), genpd->accounting_time);
	3258	+ if (genpd->status == GENPD_STATE_ON) {
	3259	+ now = ktime_get_mono_fast_ns();
	3260	+ if (now > genpd->accounting_time)
	3261	+ delta = now - genpd->accounting_time;
	3262	+ }
3205	3263
3206		− seq_printf(s, "%lld ms\n", ktime_to_ms(
3207		− ktime_add(genpd->on_time, delta)));
	3264	+ on_time = genpd->on_time + delta;
	3265	+ do_div(on_time, NSEC_PER_MSEC);
	3266	+ seq_printf(s, "%llu ms\n", on_time);
3208	3267
3209	3268	genpd_unlock(genpd);
3210	3269	return ret;
@@ -3213,7 +3272,7 @@ static int active_time_show(struct seq_file s, void data) · 2 unmodified lines
3213	3272	static int total_idle_time_show(struct seq_file s, void data)
3214	3273	{
3215	3274	struct generic_pm_domain *genpd = s->private;
3216		− ktime_t delta = 0, total = 0;
	3275	+ u64 now, delta, total = 0;
3217	3276	unsigned int i;
3218	3277	int ret = 0;
3219	3278
@@ -3222,16 +3281,19 @@ static int total_idle_time_show(struct seq_file s, void data) · 2 unmodified lines
3222	3281	return -ERESTARTSYS;
3223	3282
3224	3283	for (i = 0; i < genpd->state_count; i++) {
	3284	+ total += genpd->states[i].idle_time;
3225	3285
3226		− if ((genpd->status == GENPD_STATE_OFF) &&
3227		− (genpd->state_idx == i))
3228		− delta = ktime_sub(ktime_get(), genpd->accounting_time);
3229		−
3230		− total = ktime_add(total, genpd->states[i].idle_time);
	3286	+ if (genpd->status == GENPD_STATE_OFF && genpd->state_idx == i) {
	3287	+ now = ktime_get_mono_fast_ns();
	3288	+ if (now > genpd->accounting_time) {
	3289	+ delta = now - genpd->accounting_time;
	3290	+ total += delta;
	3291	+ }
	3292	+ }
3231	3293	}
3232		− total = ktime_add(total, delta);
3233	3294
3234		− seq_printf(s, "%lld ms\n", ktime_to_ms(total));
	3295	+ do_div(total, NSEC_PER_MSEC);
	3296	+ seq_printf(s, "%llu ms\n", total);
3235	3297
3236	3298	genpd_unlock(genpd);
3237	3299	return ret;

drivers/base/power/domain_governor.c+35−30

@@ -18,6 +18,8 @@ static int dev_update_qos_constraint(struct device dev, void data) · 17 unmodified lines
18	18	s64 constraint_ns;
19	19
20	20	if (dev->power.subsys_data && dev->power.subsys_data->domain_data) {
	21	+ struct gpd_timing_data *td = dev_gpd_data(dev)->td;
	22	+
21	23	/*
22	24	* Only take suspend-time QoS constraints of devices into
23	25	* account, because constraints updated after the device has
@@ -25,7 +27,8 @@ static int dev_update_qos_constraint(struct device dev, void data) · 1 unmodified line
25	27	* anyway. In order for them to take effect, the device has to
26	28	* be resumed and suspended again.
27	29	*/
28		− constraint_ns = dev_gpd_data(dev)->td.effective_constraint_ns;
	30	+ constraint_ns = td ? td->effective_constraint_ns :
	31	+ PM_QOS_RESUME_LATENCY_NO_CONSTRAINT_NS;
29	32	} else {
30	33	/*
31	34	* The child is not in a domain and there's no info on its
@@ -49,7 +52,7 @@ static int dev_update_qos_constraint(struct device dev, void data) · 17 unmodified lines
49	52	*/
50	53	static bool default_suspend_ok(struct device *dev)
51	54	{
52		− struct gpd_timing_data *td = &dev_gpd_data(dev)->td;
	55	+ struct gpd_timing_data *td = dev_gpd_data(dev)->td;
53	56	unsigned long flags;
54	57	s64 constraint_ns;
55	58
@@ -136,26 +139,28 @@ static void update_domain_next_wakeup(struct generic_pm_domain *genpd, ktime_t n · 80 unmodified lines
136	139	* is able to enter its optimal idle state.
137	140	*/
138	141	list_for_each_entry(pdd, &genpd->dev_list, list_node) {
139		− next_wakeup = to_gpd_data(pdd)->next_wakeup;
	142	+ next_wakeup = to_gpd_data(pdd)->td->next_wakeup;
140	143	if (next_wakeup != KTIME_MAX && !ktime_before(next_wakeup, now))
141	144	if (ktime_before(next_wakeup, domain_wakeup))
142	145	domain_wakeup = next_wakeup;
143	146	}
144	147
145	148	list_for_each_entry(link, &genpd->parent_links, parent_node) {
146		− next_wakeup = link->child->next_wakeup;
	149	+ struct genpd_governor_data *cgd = link->child->gd;
	150	+
	151	+ next_wakeup = cgd ? cgd->next_wakeup : KTIME_MAX;
147	152	if (next_wakeup != KTIME_MAX && !ktime_before(next_wakeup, now))
148	153	if (ktime_before(next_wakeup, domain_wakeup))
149	154	domain_wakeup = next_wakeup;
150	155	}
151	156
152		− genpd->next_wakeup = domain_wakeup;
	157	+ genpd->gd->next_wakeup = domain_wakeup;
153	158	}
154	159
155	160	static bool next_wakeup_allows_state(struct generic_pm_domain *genpd,
156	161	unsigned int state, ktime_t now)
157	162	{
158		− ktime_t domain_wakeup = genpd->next_wakeup;
	163	+ ktime_t domain_wakeup = genpd->gd->next_wakeup;
159	164	s64 idle_time_ns, min_sleep_ns;
160	165
161	166	min_sleep_ns = genpd->states[state].power_off_latency_ns +
@@ -185,8 +190,9 @@ static bool __default_power_down_ok(struct dev_pm_domain *pd, · 23 unmodified lines
185	190	* All subdomains have been powered off already at this point.
186	191	*/
187	192	list_for_each_entry(link, &genpd->parent_links, parent_node) {
188		− struct generic_pm_domain *sd = link->child;
189		− s64 sd_max_off_ns = sd->max_off_time_ns;
	193	+ struct genpd_governor_data *cgd = link->child->gd;
	194	+
	195	+ s64 sd_max_off_ns = cgd ? cgd->max_off_time_ns : -1;
190	196
191	197	if (sd_max_off_ns < 0)
192	198	continue;
@@ -215,7 +221,7 @@ static bool __default_power_down_ok(struct dev_pm_domain *pd, · 22 unmodified lines
215	221	* domain to turn off and on (that's how much time it will
216	222	* have to wait worst case).
217	223	*/
218		− td = &to_gpd_data(pdd)->td;
	224	+ td = to_gpd_data(pdd)->td;
219	225	constraint_ns = td->effective_constraint_ns;
220	226	/*
221	227	* Zero means "no suspend at all" and this runs only when all
@@ -244,7 +250,7 @@ static bool __default_power_down_ok(struct dev_pm_domain *pd, · 22 unmodified lines
244	250	* time and the time needed to turn the domain on is the maximum
245	251	* theoretical time this domain can spend in the "off" state.
246	252	*/
247		− genpd->max_off_time_ns = min_off_time_ns -
	253	+ genpd->gd->max_off_time_ns = min_off_time_ns -
248	254	genpd->states[state].power_on_latency_ns;
249	255	return true;
250	256	}
@@ -259,6 +265,7 @@ static bool __default_power_down_ok(struct dev_pm_domain *pd, · 8 unmodified lines
259	265	static bool _default_power_down_ok(struct dev_pm_domain *pd, ktime_t now)
260	266	{
261	267	struct generic_pm_domain *genpd = pd_to_genpd(pd);
	268	+ struct genpd_governor_data *gd = genpd->gd;
262	269	int state_idx = genpd->state_count - 1;
263	270	struct gpd_link *link;
264	271
@@ -269,11 +276,11 @@ static bool _default_power_down_ok(struct dev_pm_domain *pd, ktime_t now) · 4 unmodified lines
269	276	* cannot be met.
270	277	*/
271	278	update_domain_next_wakeup(genpd, now);
272		− if ((genpd->flags & GENPD_FLAG_MIN_RESIDENCY) && (genpd->next_wakeup != KTIME_MAX)) {
	279	+ if ((genpd->flags & GENPD_FLAG_MIN_RESIDENCY) && (gd->next_wakeup != KTIME_MAX)) {
273	280	/* Let's find out the deepest domain idle state, the devices prefer */
274	281	while (state_idx >= 0) {
275	282	if (next_wakeup_allows_state(genpd, state_idx, now)) {
276		− genpd->max_off_time_changed = true;
	283	+ gd->max_off_time_changed = true;
277	284	break;
278	285	}
279	286	state_idx--;
@@ -281,14 +288,14 @@ static bool _default_power_down_ok(struct dev_pm_domain *pd, ktime_t now) · 1 unmodified line
281	288
282	289	if (state_idx < 0) {
283	290	state_idx = 0;
284		− genpd->cached_power_down_ok = false;
	291	+ gd->cached_power_down_ok = false;
285	292	goto done;
286	293	}
287	294	}
288	295
289		− if (!genpd->max_off_time_changed) {
290		− genpd->state_idx = genpd->cached_power_down_state_idx;
291		− return genpd->cached_power_down_ok;
	296	+ if (!gd->max_off_time_changed) {
	297	+ genpd->state_idx = gd->cached_power_down_state_idx;
	298	+ return gd->cached_power_down_ok;
292	299	}
293	300
294	301	/*
@@ -297,12 +304,16 @@ static bool _default_power_down_ok(struct dev_pm_domain *pd, ktime_t now) · 2 unmodified lines
297	304	* going to be called for any parent until this instance
298	305	* returns.
299	306	*/
300		− list_for_each_entry(link, &genpd->child_links, child_node)
301		− link->parent->max_off_time_changed = true;
	307	+ list_for_each_entry(link, &genpd->child_links, child_node) {
	308	+ struct genpd_governor_data *pgd = link->parent->gd;
	309	+
	310	+ if (pgd)
	311	+ pgd->max_off_time_changed = true;
	312	+ }
302	313
303		− genpd->max_off_time_ns = -1;
304		− genpd->max_off_time_changed = false;
305		− genpd->cached_power_down_ok = true;
	314	+ gd->max_off_time_ns = -1;
	315	+ gd->max_off_time_changed = false;
	316	+ gd->cached_power_down_ok = true;
306	317
307	318	/*
308	319	* Find a state to power down to, starting from the state
@@ -310,7 +321,7 @@ static bool _default_power_down_ok(struct dev_pm_domain *pd, ktime_t now) · 1 unmodified line
310	321	*/
311	322	while (!__default_power_down_ok(pd, state_idx)) {
312	323	if (state_idx == 0) {
313		− genpd->cached_power_down_ok = false;
	324	+ gd->cached_power_down_ok = false;
314	325	break;
315	326	}
316	327	state_idx--;
@@ -318,8 +329,8 @@ static bool _default_power_down_ok(struct dev_pm_domain *pd, ktime_t now) · 1 unmodified line
318	329
319	330	done:
320	331	genpd->state_idx = state_idx;
321		− genpd->cached_power_down_state_idx = genpd->state_idx;
322		− return genpd->cached_power_down_ok;
	332	+ gd->cached_power_down_state_idx = genpd->state_idx;
	333	+ return gd->cached_power_down_ok;
323	334	}
324	335
325	336	static bool default_power_down_ok(struct dev_pm_domain *pd)
@@ -327,11 +338,6 @@ static bool default_power_down_ok(struct dev_pm_domain *pd) · 1 unmodified line
327	338	return _default_power_down_ok(pd, ktime_get());
328	339	}
329	340
330		−static bool always_on_power_down_ok(struct dev_pm_domain *domain)
331		−{
332		− return false;
333		−}
334		−
335	341	#ifdef CONFIG_CPU_IDLE
336	342	static bool cpu_power_down_ok(struct dev_pm_domain *pd)
337	343	{
@@ -401,6 +407,5 @@ struct dev_power_governor simple_qos_governor = { · 63 unmodified lines
401	407	* pm_genpd_gov_always_on - A governor implementing an always-on policy
402	408	*/
403	409	struct dev_power_governor pm_domain_always_on_gov = {
404		− .power_down_ok = always_on_power_down_ok,
405	410	.suspend_ok = default_suspend_ok,
406	411	};

drivers/base/power/runtime.c+42−11

@@ -263,7 +263,7 @@ static int rpm_check_suspend_allowed(struct device *dev) · 262 unmodified lines
263	263	retval = -EINVAL;
264	264	else if (dev->power.disable_depth > 0)
265	265	retval = -EACCES;
266		− else if (atomic_read(&dev->power.usage_count) > 0)
	266	+ else if (atomic_read(&dev->power.usage_count))
267	267	retval = -EAGAIN;
268	268	else if (!dev->power.ignore_children &&
269	269	atomic_read(&dev->power.child_count))
@@ -1039,13 +1039,33 @@ int pm_schedule_suspend(struct device *dev, unsigned int delay) · 769 unmodified lines
1039	1039	}
1040	1040	EXPORT_SYMBOL_GPL(pm_schedule_suspend);
1041	1041
	1042	+static int rpm_drop_usage_count(struct device *dev)
	1043	+{
	1044	+ int ret;
	1045	+
	1046	+ ret = atomic_sub_return(1, &dev->power.usage_count);
	1047	+ if (ret >= 0)
	1048	+ return ret;
	1049	+
	1050	+ /*
	1051	+ * Because rpm_resume() does not check the usage counter, it will resume
	1052	+ * the device even if the usage counter is 0 or negative, so it is
	1053	+ * sufficient to increment the usage counter here to reverse the change
	1054	+ * made above.
	1055	+ */
	1056	+ atomic_inc(&dev->power.usage_count);
	1057	+ dev_warn(dev, "Runtime PM usage count underflow!\n");
	1058	+ return -EINVAL;
	1059	+}
	1060	+
1042	1061	/**
1043	1062	* __pm_runtime_idle - Entry point for runtime idle operations.
1044	1063	* @dev: Device to send idle notification for.
1045	1064	* @rpmflags: Flag bits.
1046	1065	*
1047	1066	* If the RPM_GET_PUT flag is set, decrement the device's usage count and
1048		− * return immediately if it is larger than zero. Then carry out an idle
	1067	+ * return immediately if it is larger than zero (if it becomes negative, log a
	1068	+ * warning, increment it, and return an error). Then carry out an idle
1049	1069	* notification, either synchronous or asynchronous.
1050	1070	*
1051	1071	* This routine may be called in atomic context if the RPM_ASYNC flag is set,
@@ -1057,7 +1077,10 @@ int __pm_runtime_idle(struct device *dev, int rpmflags) · 5 unmodified lines
1057	1077	int retval;
1058	1078
1059	1079	if (rpmflags & RPM_GET_PUT) {
1060		− if (!atomic_dec_and_test(&dev->power.usage_count)) {
	1080	+ retval = rpm_drop_usage_count(dev);
	1081	+ if (retval < 0) {
	1082	+ return retval;
	1083	+ } else if (retval > 0) {
1061	1084	trace_rpm_usage_rcuidle(dev, rpmflags);
1062	1085	return 0;
1063	1086	}
@@ -1079,7 +1102,8 @@ EXPORT_SYMBOL_GPL(__pm_runtime_idle); · 15 unmodified lines
1079	1102	* @rpmflags: Flag bits.
1080	1103	*
1081	1104	* If the RPM_GET_PUT flag is set, decrement the device's usage count and
1082		− * return immediately if it is larger than zero. Then carry out a suspend,
	1105	+ * return immediately if it is larger than zero (if it becomes negative, log a
	1106	+ * warning, increment it, and return an error). Then carry out a suspend,
1083	1107	* either synchronous or asynchronous.
1084	1108	*
1085	1109	* This routine may be called in atomic context if the RPM_ASYNC flag is set,
@@ -1091,7 +1115,10 @@ int __pm_runtime_suspend(struct device *dev, int rpmflags) · 5 unmodified lines
1091	1115	int retval;
1092	1116
1093	1117	if (rpmflags & RPM_GET_PUT) {
1094		− if (!atomic_dec_and_test(&dev->power.usage_count)) {
	1118	+ retval = rpm_drop_usage_count(dev);
	1119	+ if (retval < 0) {
	1120	+ return retval;
	1121	+ } else if (retval > 0) {
1095	1122	trace_rpm_usage_rcuidle(dev, rpmflags);
1096	1123	return 0;
1097	1124	}
@@ -1210,12 +1237,13 @@ int __pm_runtime_set_status(struct device *dev, unsigned int status) · 112 unmodified lines
1210	1237	{
1211	1238	struct device *parent = dev->parent;
1212	1239	bool notify_parent = false;
	1240	+ unsigned long flags;
1213	1241	int error = 0;
1214	1242
1215	1243	if (status != RPM_ACTIVE && status != RPM_SUSPENDED)
1216	1244	return -EINVAL;
1217	1245
1218		− spin_lock_irq(&dev->power.lock);
	1246	+ spin_lock_irqsave(&dev->power.lock, flags);
1219	1247
1220	1248	/*
1221	1249	* Prevent PM-runtime from being enabled for the device or return an
@@ -1226,7 +1254,7 @@ int __pm_runtime_set_status(struct device *dev, unsigned int status) · 4 unmodified lines
1226	1254	else
1227	1255	error = -EAGAIN;
1228	1256
1229		− spin_unlock_irq(&dev->power.lock);
	1257	+ spin_unlock_irqrestore(&dev->power.lock, flags);
1230	1258
1231	1259	if (error)
1232	1260	return error;
@@ -1247,7 +1275,7 @@ int __pm_runtime_set_status(struct device *dev, unsigned int status) · 14 unmodified lines
1247	1275	device_links_read_unlock(idx);
1248	1276	}
1249	1277
1250		− spin_lock_irq(&dev->power.lock);
	1278	+ spin_lock_irqsave(&dev->power.lock, flags);
1251	1279
1252	1280	if (dev->power.runtime_status == status \|\| !parent)
1253	1281	goto out_set;
@@ -1288,7 +1316,7 @@ int __pm_runtime_set_status(struct device *dev, unsigned int status) · 34 unmodified lines
1288	1316	dev->power.runtime_error = 0;
1289	1317
1290	1318	out:
1291		− spin_unlock_irq(&dev->power.lock);
	1319	+ spin_unlock_irqrestore(&dev->power.lock, flags);
1292	1320
1293	1321	if (notify_parent)
1294	1322	pm_request_idle(parent);
@@ -1527,14 +1555,17 @@ EXPORT_SYMBOL_GPL(pm_runtime_forbid); · 232 unmodified lines
1527	1555	*/
1528	1556	void pm_runtime_allow(struct device *dev)
1529	1557	{
	1558	+ int ret;
	1559	+
1530	1560	spin_lock_irq(&dev->power.lock);
1531	1561	if (dev->power.runtime_auto)
1532	1562	goto out;
1533	1563
1534	1564	dev->power.runtime_auto = true;
1535		− if (atomic_dec_and_test(&dev->power.usage_count))
	1565	+ ret = rpm_drop_usage_count(dev);
	1566	+ if (ret == 0)
1536	1567	rpm_idle(dev, RPM_AUTO \| RPM_ASYNC);
1537		− else
	1568	+ else if (ret > 0)
1538	1569	trace_rpm_usage_rcuidle(dev, RPM_AUTO \| RPM_ASYNC);
1539	1570
1540	1571	out:

drivers/cpufreq/cppc_cpufreq.c+211−0

@@ -389,6 +389,27 @@ static int cppc_cpufreq_set_target(struct cpufreq_policy *policy, · 388 unmodified lines
389	389	return ret;
390	390	}
391	391
	392	+static unsigned int cppc_cpufreq_fast_switch(struct cpufreq_policy *policy,
	393	+ unsigned int target_freq)
	394	+{
	395	+ struct cppc_cpudata *cpu_data = policy->driver_data;
	396	+ unsigned int cpu = policy->cpu;
	397	+ u32 desired_perf;
	398	+ int ret;
	399	+
	400	+ desired_perf = cppc_cpufreq_khz_to_perf(cpu_data, target_freq);
	401	+ cpu_data->perf_ctrls.desired_perf = desired_perf;
	402	+ ret = cppc_set_perf(cpu, &cpu_data->perf_ctrls);
	403	+
	404	+ if (ret) {
	405	+ pr_debug("Failed to set target on CPU:%d. ret:%d\n",
	406	+ cpu, ret);
	407	+ return 0;
	408	+ }
	409	+
	410	+ return target_freq;
	411	+}
	412	+
392	413	static int cppc_verify_policy(struct cpufreq_policy_data *policy)
393	414	{
394	415	cpufreq_verify_within_cpu_limits(policy);
@@ -420,12 +441,197 @@ static unsigned int cppc_cpufreq_get_transition_delay_us(unsigned int cpu) · 25 unmodified lines
420	441	return cppc_get_transition_latency(cpu) / NSEC_PER_USEC;
421	442	}
422	443
	444	+static DEFINE_PER_CPU(unsigned int, efficiency_class);
	445	+static void cppc_cpufreq_register_em(struct cpufreq_policy *policy);
	446	+
	447	+/* Create an artificial performance state every CPPC_EM_CAP_STEP capacity unit. */
	448	+#define CPPC_EM_CAP_STEP (20)
	449	+/* Increase the cost value by CPPC_EM_COST_STEP every performance state. */
	450	+#define CPPC_EM_COST_STEP (1)
	451	+/* Add a cost gap correspnding to the energy of 4 CPUs. */
	452	+#define CPPC_EM_COST_GAP (4 * SCHED_CAPACITY_SCALE * CPPC_EM_COST_STEP \
	453	+ / CPPC_EM_CAP_STEP)
	454	+
	455	+static unsigned int get_perf_level_count(struct cpufreq_policy *policy)
	456	+{
	457	+ struct cppc_perf_caps *perf_caps;
	458	+ unsigned int min_cap, max_cap;
	459	+ struct cppc_cpudata *cpu_data;
	460	+ int cpu = policy->cpu;
	461	+
	462	+ cpu_data = policy->driver_data;
	463	+ perf_caps = &cpu_data->perf_caps;
	464	+ max_cap = arch_scale_cpu_capacity(cpu);
	465	+ min_cap = div_u64(max_cap * perf_caps->lowest_perf, perf_caps->highest_perf);
	466	+ if ((min_cap == 0) \|\| (max_cap < min_cap))
	467	+ return 0;
	468	+ return 1 + max_cap / CPPC_EM_CAP_STEP - min_cap / CPPC_EM_CAP_STEP;
	469	+}
	470	+
	471	+/*
	472	+ * The cost is defined as:
	473	+ * cost = power * max_frequency / frequency
	474	+ */
	475	+static inline unsigned long compute_cost(int cpu, int step)
	476	+{
	477	+ return CPPC_EM_COST_GAP * per_cpu(efficiency_class, cpu) +
	478	+ step * CPPC_EM_COST_STEP;
	479	+}
	480	+
	481	+static int cppc_get_cpu_power(struct device *cpu_dev,
	482	+ unsigned long power, unsigned long KHz)
	483	+{
	484	+ unsigned long perf_step, perf_prev, perf, perf_check;
	485	+ unsigned int min_step, max_step, step, step_check;
	486	+ unsigned long prev_freq = *KHz;
	487	+ unsigned int min_cap, max_cap;
	488	+ struct cpufreq_policy *policy;
	489	+
	490	+ struct cppc_perf_caps *perf_caps;
	491	+ struct cppc_cpudata *cpu_data;
	492	+
	493	+ policy = cpufreq_cpu_get_raw(cpu_dev->id);
	494	+ cpu_data = policy->driver_data;
	495	+ perf_caps = &cpu_data->perf_caps;
	496	+ max_cap = arch_scale_cpu_capacity(cpu_dev->id);
	497	+ min_cap = div_u64(max_cap * perf_caps->lowest_perf,
	498	+ perf_caps->highest_perf);
	499	+
	500	+ perf_step = CPPC_EM_CAP_STEP * perf_caps->highest_perf / max_cap;
	501	+ min_step = min_cap / CPPC_EM_CAP_STEP;
	502	+ max_step = max_cap / CPPC_EM_CAP_STEP;
	503	+
	504	+ perf_prev = cppc_cpufreq_khz_to_perf(cpu_data, *KHz);
	505	+ step = perf_prev / perf_step;
	506	+
	507	+ if (step > max_step)
	508	+ return -EINVAL;
	509	+
	510	+ if (min_step == max_step) {
	511	+ step = max_step;
	512	+ perf = perf_caps->highest_perf;
	513	+ } else if (step < min_step) {
	514	+ step = min_step;
	515	+ perf = perf_caps->lowest_perf;
	516	+ } else {
	517	+ step++;
	518	+ if (step == max_step)
	519	+ perf = perf_caps->highest_perf;
	520	+ else
	521	+ perf = step * perf_step;
	522	+ }
	523	+
	524	+ *KHz = cppc_cpufreq_perf_to_khz(cpu_data, perf);
	525	+ perf_check = cppc_cpufreq_khz_to_perf(cpu_data, *KHz);
	526	+ step_check = perf_check / perf_step;
	527	+
	528	+ /*
	529	+ * To avoid bad integer approximation, check that new frequency value
	530	+ * increased and that the new frequency will be converted to the
	531	+ * desired step value.
	532	+ */
	533	+ while ((*KHz == prev_freq) \|\| (step_check != step)) {
	534	+ perf++;
	535	+ *KHz = cppc_cpufreq_perf_to_khz(cpu_data, perf);
	536	+ perf_check = cppc_cpufreq_khz_to_perf(cpu_data, *KHz);
	537	+ step_check = perf_check / perf_step;
	538	+ }
	539	+
	540	+ /*
	541	+ * With an artificial EM, only the cost value is used. Still the power
	542	+ * is populated such as 0 < power < EM_MAX_POWER. This allows to add
	543	+ * more sense to the artificial performance states.
	544	+ */
	545	+ *power = compute_cost(cpu_dev->id, step);
	546	+
	547	+ return 0;
	548	+}
	549	+
	550	+static int cppc_get_cpu_cost(struct device *cpu_dev, unsigned long KHz,
	551	+ unsigned long *cost)
	552	+{
	553	+ unsigned long perf_step, perf_prev;
	554	+ struct cppc_perf_caps *perf_caps;
	555	+ struct cpufreq_policy *policy;
	556	+ struct cppc_cpudata *cpu_data;
	557	+ unsigned int max_cap;
	558	+ int step;
	559	+
	560	+ policy = cpufreq_cpu_get_raw(cpu_dev->id);
	561	+ cpu_data = policy->driver_data;
	562	+ perf_caps = &cpu_data->perf_caps;
	563	+ max_cap = arch_scale_cpu_capacity(cpu_dev->id);
	564	+
	565	+ perf_prev = cppc_cpufreq_khz_to_perf(cpu_data, KHz);
	566	+ perf_step = CPPC_EM_CAP_STEP * perf_caps->highest_perf / max_cap;
	567	+ step = perf_prev / perf_step;
	568	+
	569	+ *cost = compute_cost(cpu_dev->id, step);
	570	+
	571	+ return 0;
	572	+}
	573	+
	574	+static int populate_efficiency_class(void)
	575	+{
	576	+ struct acpi_madt_generic_interrupt *gicc;
	577	+ DECLARE_BITMAP(used_classes, 256) = {};
	578	+ int class, cpu, index;
	579	+
	580	+ for_each_possible_cpu(cpu) {
	581	+ gicc = acpi_cpu_get_madt_gicc(cpu);
	582	+ class = gicc->efficiency_class;
	583	+ bitmap_set(used_classes, class, 1);
	584	+ }
	585	+
	586	+ if (bitmap_weight(used_classes, 256) <= 1) {
	587	+ pr_debug("Efficiency classes are all equal (=%d). "
	588	+ "No EM registered", class);
	589	+ return -EINVAL;
	590	+ }
	591	+
	592	+ /*
	593	+ * Squeeze efficiency class values on [0:#efficiency_class-1].
	594	+ * Values are per spec in [0:255].
	595	+ */
	596	+ index = 0;
	597	+ for_each_set_bit(class, used_classes, 256) {
	598	+ for_each_possible_cpu(cpu) {
	599	+ gicc = acpi_cpu_get_madt_gicc(cpu);
	600	+ if (gicc->efficiency_class == class)
	601	+ per_cpu(efficiency_class, cpu) = index;
	602	+ }
	603	+ index++;
	604	+ }
	605	+ cppc_cpufreq_driver.register_em = cppc_cpufreq_register_em;
	606	+
	607	+ return 0;
	608	+}
	609	+
	610	+static void cppc_cpufreq_register_em(struct cpufreq_policy *policy)
	611	+{
	612	+ struct cppc_cpudata *cpu_data;
	613	+ struct em_data_callback em_cb =
	614	+ EM_ADV_DATA_CB(cppc_get_cpu_power, cppc_get_cpu_cost);
	615	+
	616	+ cpu_data = policy->driver_data;
	617	+ em_dev_register_perf_domain(get_cpu_device(policy->cpu),
	618	+ get_perf_level_count(policy), &em_cb,
	619	+ cpu_data->shared_cpu_map, 0);
	620	+}
	621	+
423	622	#else
424	623
425	624	static unsigned int cppc_cpufreq_get_transition_delay_us(unsigned int cpu)
426	625	{
427	626	return cppc_get_transition_latency(cpu) / NSEC_PER_USEC;
428	627	}
	628	+static int populate_efficiency_class(void)
	629	+{
	630	+ return 0;
	631	+}
	632	+static void cppc_cpufreq_register_em(struct cpufreq_policy *policy)
	633	+{
	634	+}
429	635	#endif
430	636
431	637
@@ -536,6 +742,9 @@ static int cppc_cpufreq_cpu_init(struct cpufreq_policy *policy) · 104 unmodified lines
536	742	goto out;
537	743	}
538	744
	745	+ policy->fast_switch_possible = cppc_allow_fast_switch();
	746	+ policy->dvfs_possible_from_any_cpu = true;
	747	+
539	748	/*
540	749	* If 'highest_perf' is greater than 'nominal_perf', we assume CPU Boost
541	750	* is supported.
@@ -681,6 +890,7 @@ static struct cpufreq_driver cppc_cpufreq_driver = { · 139 unmodified lines
681	890	.verify = cppc_verify_policy,
682	891	.target = cppc_cpufreq_set_target,
683	892	.get = cppc_cpufreq_get_rate,
	893	+ .fast_switch = cppc_cpufreq_fast_switch,
684	894	.init = cppc_cpufreq_cpu_init,
685	895	.exit = cppc_cpufreq_cpu_exit,
686	896	.set_boost = cppc_cpufreq_set_boost,
@@ -742,6 +952,7 @@ static int __init cppc_cpufreq_init(void) · 55 unmodified lines
742	952
743	953	cppc_check_hisi_workaround();
744	954	cppc_freq_invariance_init();
	955	+ populate_efficiency_class();
745	956
746	957	ret = cpufreq_register_driver(&cppc_cpufreq_driver);
747	958	if (ret)

drivers/cpufreq/cpufreq.c+69−43

@@ -28,6 +28,7 @@ · 27 unmodified lines
28	28	#include <linux/suspend.h>
29	29	#include <linux/syscore_ops.h>
30	30	#include <linux/tick.h>
	31	+#include <linux/units.h>
31	32	#include <trace/events/power.h>
32	33
33	34	static LIST_HEAD(cpufreq_policy_list);
@@ -947,13 +948,14 @@ static ssize_t show(struct kobject kobj, struct attribute attr, char *buf) · 913 unmodified lines
947	948	{
948	949	struct cpufreq_policy *policy = to_policy(kobj);
949	950	struct freq_attr *fattr = to_attr(attr);
950		− ssize_t ret;
	951	+ ssize_t ret = -EBUSY;
951	952
952	953	if (!fattr->show)
953	954	return -EIO;
954	955
955	956	down_read(&policy->rwsem);
956		− ret = fattr->show(policy, buf);
	957	+ if (likely(!policy_is_inactive(policy)))
	958	+ ret = fattr->show(policy, buf);
957	959	up_read(&policy->rwsem);
958	960
959	961	return ret;
@@ -964,7 +966,7 @@ static ssize_t store(struct kobject kobj, struct attribute attr, · 4 unmodified lines
964	966	{
965	967	struct cpufreq_policy *policy = to_policy(kobj);
966	968	struct freq_attr *fattr = to_attr(attr);
967		− ssize_t ret = -EINVAL;
	969	+ ssize_t ret = -EBUSY;
968	970
969	971	if (!fattr->store)
970	972	return -EIO;
@@ -978,7 +980,8 @@ static ssize_t store(struct kobject kobj, struct attribute attr, · 7 unmodified lines
978	980
979	981	if (cpu_online(policy->cpu)) {
980	982	down_write(&policy->rwsem);
981		− ret = fattr->store(policy, buf, count);
	983	+ if (likely(!policy_is_inactive(policy)))
	984	+ ret = fattr->store(policy, buf, count);
982	985	up_write(&policy->rwsem);
983	986	}
984	987
@@ -1019,11 +1022,12 @@ static void add_cpu_dev_symlink(struct cpufreq_policy *policy, unsigned int cpu, · 34 unmodified lines
1019	1022	dev_err(dev, "cpufreq symlink creation failed\n");
1020	1023	}
1021	1024
1022		−static void remove_cpu_dev_symlink(struct cpufreq_policy *policy,
	1025	+static void remove_cpu_dev_symlink(struct cpufreq_policy *policy, int cpu,
1023	1026	struct device *dev)
1024	1027	{
1025	1028	dev_dbg(dev, "%s: Removing symlink\n", __func__);
1026	1029	sysfs_remove_link(&dev->kobj, "cpufreq");
	1030	+ cpumask_clear_cpu(cpu, policy->real_cpus);
1027	1031	}
1028	1032
1029	1033	static int cpufreq_add_dev_interface(struct cpufreq_policy *policy)
@@ -1337,12 +1341,12 @@ static int cpufreq_online(unsigned int cpu) · 307 unmodified lines
1337	1341	down_write(&policy->rwsem);
1338	1342	policy->cpu = cpu;
1339	1343	policy->governor = NULL;
1340		− up_write(&policy->rwsem);
1341	1344	} else {
1342	1345	new_policy = true;
1343	1346	policy = cpufreq_policy_alloc(cpu);
1344	1347	if (!policy)
1345	1348	return -ENOMEM;
	1349	+ down_write(&policy->rwsem);
1346	1350	}
1347	1351
1348	1352	if (!new_policy && cpufreq_driver->online) {
@@ -1382,7 +1386,6 @@ static int cpufreq_online(unsigned int cpu) · 33 unmodified lines
1382	1386	cpumask_copy(policy->related_cpus, policy->cpus);
1383	1387	}
1384	1388
1385		− down_write(&policy->rwsem);
1386	1389	/*
1387	1390	* affected cpus must always be the one, which are online. We aren't
1388	1391	* managing offline cpus here.
@@ -1531,9 +1534,9 @@ static int cpufreq_online(unsigned int cpu) · 142 unmodified lines
1531	1534
1532	1535	out_destroy_policy:
1533	1536	for_each_cpu(j, policy->real_cpus)
1534		− remove_cpu_dev_symlink(policy, get_cpu_device(j));
	1537	+ remove_cpu_dev_symlink(policy, j, get_cpu_device(j));
1535	1538
1536		− up_write(&policy->rwsem);
	1539	+ cpumask_clear(policy->cpus);
1537	1540
1538	1541	out_offline_policy:
1539	1542	if (cpufreq_driver->offline)
@@ -1544,6 +1547,8 @@ out_exit_policy: · 4 unmodified lines
1544	1547	cpufreq_driver->exit(policy);
1545	1548
1546	1549	out_free_policy:
	1550	+ up_write(&policy->rwsem);
	1551	+
1547	1552	cpufreq_policy_free(policy);
1548	1553	return ret;
1549	1554	}
@@ -1575,47 +1580,36 @@ static int cpufreq_add_dev(struct device dev, struct subsys_interface sif) · 25 unmodified lines
1575	1580	return 0;
1576	1581	}
1577	1582
1578		−static int cpufreq_offline(unsigned int cpu)
	1583	+static void __cpufreq_offline(unsigned int cpu, struct cpufreq_policy *policy)
1579	1584	{
1580		− struct cpufreq_policy *policy;
1581	1585	int ret;
1582	1586
1583		− pr_debug("%s: unregistering CPU %u\n", __func__, cpu);
1584		−
1585		− policy = cpufreq_cpu_get_raw(cpu);
1586		− if (!policy) {
1587		− pr_debug("%s: No cpu_data found\n", __func__);
1588		− return 0;
1589		− }
1590		−
1591		− down_write(&policy->rwsem);
1592	1587	if (has_target())
1593	1588	cpufreq_stop_governor(policy);
1594	1589
1595	1590	cpumask_clear_cpu(cpu, policy->cpus);
1596	1591
1597		− if (policy_is_inactive(policy)) {
1598		− if (has_target())
1599		− strncpy(policy->last_governor, policy->governor->name,
1600		− CPUFREQ_NAME_LEN);
1601		− else
1602		− policy->last_policy = policy->policy;
1603		− } else if (cpu == policy->cpu) {
1604		− /* Nominate new CPU */
1605		− policy->cpu = cpumask_any(policy->cpus);
1606		− }
1607		−
1608		− /* Start governor again for active policy */
1609	1592	if (!policy_is_inactive(policy)) {
	1593	+ /* Nominate a new CPU if necessary. */
	1594	+ if (cpu == policy->cpu)
	1595	+ policy->cpu = cpumask_any(policy->cpus);
	1596	+
	1597	+ /* Start the governor again for the active policy. */
1610	1598	if (has_target()) {
1611	1599	ret = cpufreq_start_governor(policy);
1612	1600	if (ret)
1613	1601	pr_err("%s: Failed to start governor\n", __func__);
1614	1602	}
1615	1603
1616		− goto unlock;
	1604	+ return;
1617	1605	}
1618	1606
	1607	+ if (has_target())
	1608	+ strncpy(policy->last_governor, policy->governor->name,
	1609	+ CPUFREQ_NAME_LEN);
	1610	+ else
	1611	+ policy->last_policy = policy->policy;
	1612	+
1619	1613	if (cpufreq_thermal_control_enabled(cpufreq_driver)) {
1620	1614	cpufreq_cooling_unregister(policy->cdev);
1621	1615	policy->cdev = NULL;
@@ -1634,8 +1628,24 @@ static int cpufreq_offline(unsigned int cpu) · 12 unmodified lines
1634	1628	cpufreq_driver->exit(policy);
1635	1629	policy->freq_table = NULL;
1636	1630	}
	1631	+}
	1632	+
	1633	+static int cpufreq_offline(unsigned int cpu)
	1634	+{
	1635	+ struct cpufreq_policy *policy;
	1636	+
	1637	+ pr_debug("%s: unregistering CPU %u\n", __func__, cpu);
	1638	+
	1639	+ policy = cpufreq_cpu_get_raw(cpu);
	1640	+ if (!policy) {
	1641	+ pr_debug("%s: No cpu_data found\n", __func__);
	1642	+ return 0;
	1643	+ }
	1644	+
	1645	+ down_write(&policy->rwsem);
	1646	+
	1647	+ __cpufreq_offline(cpu, policy);
1637	1648
1638		−unlock:
1639	1649	up_write(&policy->rwsem);
1640	1650	return 0;
1641	1651	}
@@ -1653,19 +1663,25 @@ static void cpufreq_remove_dev(struct device dev, struct subsys_interface sif) · 11 unmodified lines
1653	1663	if (!policy)
1654	1664	return;
1655	1665
1656		− if (cpu_online(cpu))
1657		− cpufreq_offline(cpu);
	1666	+ down_write(&policy->rwsem);
1658	1667
1659		− cpumask_clear_cpu(cpu, policy->real_cpus);
1660		− remove_cpu_dev_symlink(policy, dev);
	1668	+ if (cpu_online(cpu))
	1669	+ __cpufreq_offline(cpu, policy);
1661	1670
1662		− if (cpumask_empty(policy->real_cpus)) {
1663		− /* We did light-weight exit earlier, do full tear down now */
1664		− if (cpufreq_driver->offline)
1665		− cpufreq_driver->exit(policy);
	1671	+ remove_cpu_dev_symlink(policy, cpu, dev);
1666	1672
1667		− cpufreq_policy_free(policy);
	1673	+ if (!cpumask_empty(policy->real_cpus)) {
	1674	+ up_write(&policy->rwsem);
	1675	+ return;
1668	1676	}
	1677	+
	1678	+ /* We did light-weight exit earlier, do full tear down now */
	1679	+ if (cpufreq_driver->offline)
	1680	+ cpufreq_driver->exit(policy);
	1681	+
	1682	+ up_write(&policy->rwsem);
	1683	+
	1684	+ cpufreq_policy_free(policy);
1669	1685	}
1670	1686
1671	1687	/**
@@ -1707,6 +1723,16 @@ static unsigned int cpufreq_verify_current_freq(struct cpufreq_policy *policy, b · 35 unmodified lines
1707	1723	return new_freq;
1708	1724
1709	1725	if (policy->cur != new_freq) {
	1726	+ /*
	1727	+ * For some platforms, the frequency returned by hardware may be
	1728	+ * slightly different from what is provided in the frequency
	1729	+ * table, for example hardware may return 499 MHz instead of 500
	1730	+ * MHz. In such cases it is better to avoid getting into
	1731	+ * unnecessary frequency updates.
	1732	+ */
	1733	+ if (abs(policy->cur - new_freq) < HZ_PER_MHZ)
	1734	+ return policy->cur;
	1735	+
1710	1736	cpufreq_out_of_sync(policy, new_freq);
1711	1737	if (update)
1712	1738	schedule_work(&policy->update);

drivers/cpufreq/cpufreq_governor.c+13−7

@@ -388,6 +388,15 @@ static void free_policy_dbs_info(struct policy_dbs_info *policy_dbs, · 387 unmodified lines
388	388	gov->free(policy_dbs);
389	389	}
390	390
	391	+static void cpufreq_dbs_data_release(struct kobject *kobj)
	392	+{
	393	+ struct dbs_data *dbs_data = to_dbs_data(to_gov_attr_set(kobj));
	394	+ struct dbs_governor *gov = dbs_data->gov;
	395	+
	396	+ gov->exit(dbs_data);
	397	+ kfree(dbs_data);
	398	+}
	399	+
391	400	int cpufreq_dbs_governor_init(struct cpufreq_policy *policy)
392	401	{
393	402	struct dbs_governor *gov = dbs_governor_of(policy);
@@ -425,6 +434,7 @@ int cpufreq_dbs_governor_init(struct cpufreq_policy *policy) · 31 unmodified lines
425	434	goto free_policy_dbs_info;
426	435	}
427	436
	437	+ dbs_data->gov = gov;
428	438	gov_attr_set_init(&dbs_data->attr_set, &policy_dbs->list);
429	439
430	440	ret = gov->init(dbs_data);
@@ -447,6 +457,7 @@ int cpufreq_dbs_governor_init(struct cpufreq_policy *policy) · 16 unmodified lines
447	457	policy->governor_data = policy_dbs;
448	458
449	459	gov->kobj_type.sysfs_ops = &governor_sysfs_ops;
	460	+ gov->kobj_type.release = cpufreq_dbs_data_release;
450	461	ret = kobject_init_and_add(&dbs_data->attr_set.kobj, &gov->kobj_type,
451	462	get_governor_parent_kobj(policy),
452	463	"%s", gov->gov.name);
@@ -488,13 +499,8 @@ void cpufreq_dbs_governor_exit(struct cpufreq_policy *policy) · 35 unmodified lines
488	499
489	500	policy->governor_data = NULL;
490	501
491		− if (!count) {
492		− if (!have_governor_per_policy())
493		− gov->gdbs_data = NULL;
494		−
495		− gov->exit(dbs_data);
496		− kfree(dbs_data);
497		− }
	502	+ if (!count && !have_governor_per_policy())
	503	+ gov->gdbs_data = NULL;
498	504
499	505	free_policy_dbs_info(policy_dbs, gov);
500	506

drivers/cpufreq/cpufreq_governor.h+1−0

@@ -37,6 +37,7 @@ enum {OD_NORMAL_SAMPLE, OD_SUB_SAMPLE}; · 36 unmodified lines
37	37	/* Governor demand based switching data (per-policy or global). */
38	38	struct dbs_data {
39	39	struct gov_attr_set attr_set;
	40	+ struct dbs_governor *gov;
40	41	void *tuners;
41	42	unsigned int ignore_nice_load;
42	43	unsigned int sampling_rate;

drivers/cpufreq/intel_pstate.c+2−0

@@ -1322,6 +1322,7 @@ static ssize_t store_no_turbo(struct kobject a, struct kobj_attribute b, · 1321 unmodified lines
1322	1322	mutex_unlock(&intel_pstate_limits_lock);
1323	1323
1324	1324	intel_pstate_update_policies();
	1325	+ arch_set_max_freq_ratio(global.no_turbo);
1325	1326
1326	1327	mutex_unlock(&intel_pstate_driver_lock);
1327	1328
@@ -2424,6 +2425,7 @@ static const struct x86_cpu_id intel_pstate_cpu_oob_ids[] __initconst = { · 1096 unmodified lines
2424	2425	X86_MATCH(BROADWELL_X, core_funcs),
2425	2426	X86_MATCH(SKYLAKE_X, core_funcs),
2426	2427	X86_MATCH(ICELAKE_X, core_funcs),
	2428	+ X86_MATCH(SAPPHIRERAPIDS_X, core_funcs),
2427	2429	{}
2428	2430	};
2429	2431

drivers/cpufreq/mediatek-cpufreq-hw.c+2−2

@@ -51,8 +51,8 @@ static const u16 cpufreq_mtk_offsets[REG_ARRAY_SIZE] = { · 50 unmodified lines
51	51	};
52	52
53	53	static int __maybe_unused
54		−mtk_cpufreq_get_cpu_power(unsigned long *mW,
55		− unsigned long KHz, struct device cpu_dev)
	54	+mtk_cpufreq_get_cpu_power(struct device cpu_dev, unsigned long mW,
	55	+ unsigned long *KHz)
56	56	{
57	57	struct mtk_cpufreq_data *data;
58	58	struct cpufreq_policy *policy;

drivers/cpufreq/pasemi-cpufreq.c+0−1

@@ -18,7 +18,6 @@ · 17 unmodified lines
18	18
19	19	#include <asm/hw_irq.h>
20	20	#include <asm/io.h>
21		−#include <asm/prom.h>
22	21	#include <asm/time.h>
23	22	#include <asm/smp.h>
24	23

drivers/cpufreq/pmac32-cpufreq.c+1−1

@@ -24,7 +24,7 @@ · 23 unmodified lines
24	24	#include <linux/device.h>
25	25	#include <linux/hardirq.h>
26	26	#include <linux/of_device.h>
27		−#include <asm/prom.h>
	27	+
28	28	#include <asm/machdep.h>
29	29	#include <asm/irq.h>
30	30	#include <asm/pmac_feature.h>

drivers/cpufreq/pmac64-cpufreq.c+1−1

@@ -22,7 +22,7 @@ · 21 unmodified lines
22	22	#include <linux/completion.h>
23	23	#include <linux/mutex.h>
24	24	#include <linux/of_device.h>
25		−#include <asm/prom.h>
	25	+
26	26	#include <asm/machdep.h>
27	27	#include <asm/irq.h>
28	28	#include <asm/sections.h>

drivers/cpufreq/ppc_cbe_cpufreq.c+0−1

@@ -12,7 +12,6 @@ · 11 unmodified lines
12	12	#include <linux/of_platform.h>
13	13
14	14	#include <asm/machdep.h>
15		−#include <asm/prom.h>
16	15	#include <asm/cell-regs.h>
17	16
18	17	#include "ppc_cbe_cpufreq.h"

drivers/cpufreq/ppc_cbe_cpufreq_pmi.c+1−1

@@ -13,9 +13,9 @@ · 12 unmodified lines
13	13	#include <linux/init.h>
14	14	#include <linux/of_platform.h>
15	15	#include <linux/pm_qos.h>
	16	+#include <linux/slab.h>
16	17
17	18	#include <asm/processor.h>
18		−#include <asm/prom.h>
19	19	#include <asm/pmi.h>
20	20	#include <asm/cell-regs.h>
21	21

drivers/cpufreq/scmi-cpufreq.c+2−2

@@ -96,8 +96,8 @@ scmi_get_sharing_cpus(struct device cpu_dev, struct cpumask cpumask) · 95 unmodified lines
96	96	}
97	97
98	98	static int __maybe_unused
99		−scmi_get_cpu_power(unsigned long power, unsigned long KHz,
100		− struct device *cpu_dev)
	99	+scmi_get_cpu_power(struct device cpu_dev, unsigned long power,
	100	+ unsigned long *KHz)
101	101	{
102	102	unsigned long Hz;
103	103	int ret, domain;

drivers/cpuidle/cpuidle-psci-domain.c+2−2

@@ -52,7 +52,7 @@ static int psci_pd_init(struct device_node *np, bool use_osi) · 51 unmodified lines
52	52	struct generic_pm_domain *pd;
53	53	struct psci_pd_provider *pd_provider;
54	54	struct dev_power_governor *pd_gov;
55		− int ret = -ENOMEM, state_count = 0;
	55	+ int ret = -ENOMEM;
56	56
57	57	pd = dt_idle_pd_alloc(np, psci_dt_parse_state_node);
58	58	if (!pd)
@@ -71,7 +71,7 @@ static int psci_pd_init(struct device_node *np, bool use_osi) · 12 unmodified lines
71	71	pd->flags \|= GENPD_FLAG_ALWAYS_ON;
72	72
73	73	/* Use governor for CPU PM domains if it has some states to manage. */
74		− pd_gov = state_count > 0 ? &pm_domain_cpu_gov : NULL;
	74	+ pd_gov = pd->states ? &pm_domain_cpu_gov : NULL;
75	75
76	76	ret = pm_genpd_init(pd, pd_gov, false);
77	77	if (ret)

drivers/cpuidle/cpuidle-psci.c+46−0

@@ -23,6 +23,7 @@ · 22 unmodified lines
23	23	#include <linux/pm_runtime.h>
24	24	#include <linux/slab.h>
25	25	#include <linux/string.h>
	26	+#include <linux/syscore_ops.h>
26	27
27	28	#include <asm/cpuidle.h>
28	29
@@ -131,6 +132,49 @@ static int psci_idle_cpuhp_down(unsigned int cpu) · 102 unmodified lines
131	132	return 0;
132	133	}
133	134
	135	+static void psci_idle_syscore_switch(bool suspend)
	136	+{
	137	+ bool cleared = false;
	138	+ struct device *dev;
	139	+ int cpu;
	140	+
	141	+ for_each_possible_cpu(cpu) {
	142	+ dev = per_cpu_ptr(&psci_cpuidle_data, cpu)->dev;
	143	+
	144	+ if (dev && suspend) {
	145	+ dev_pm_genpd_suspend(dev);
	146	+ } else if (dev) {
	147	+ dev_pm_genpd_resume(dev);
	148	+
	149	+ /* Account for userspace having offlined a CPU. */
	150	+ if (pm_runtime_status_suspended(dev))
	151	+ pm_runtime_set_active(dev);
	152	+
	153	+ /* Clear domain state to re-start fresh. */
	154	+ if (!cleared) {
	155	+ psci_set_domain_state(0);
	156	+ cleared = true;
	157	+ }
	158	+ }
	159	+ }
	160	+}
	161	+
	162	+static int psci_idle_syscore_suspend(void)
	163	+{
	164	+ psci_idle_syscore_switch(true);
	165	+ return 0;
	166	+}
	167	+
	168	+static void psci_idle_syscore_resume(void)
	169	+{
	170	+ psci_idle_syscore_switch(false);
	171	+}
	172	+
	173	+static struct syscore_ops psci_idle_syscore_ops = {
	174	+ .suspend = psci_idle_syscore_suspend,
	175	+ .resume = psci_idle_syscore_resume,
	176	+};
	177	+
134	178	static void psci_idle_init_cpuhp(void)
135	179	{
136	180	int err;
@@ -138,6 +182,8 @@ static void psci_idle_init_cpuhp(void) · 1 unmodified line
138	182	if (!psci_cpuidle_use_cpuhp)
139	183	return;
140	184
	185	+ register_syscore_ops(&psci_idle_syscore_ops);
	186	+
141	187	err = cpuhp_setup_state_nocalls(CPUHP_AP_CPU_PM_STARTING,
142	188	"cpuidle/psci:online",
143	189	psci_idle_cpuhp_up,

drivers/cpuidle/cpuidle-riscv-sbi.c+2−2

@@ -414,7 +414,7 @@ static int sbi_pd_init(struct device_node *np) · 413 unmodified lines
414	414	struct generic_pm_domain *pd;
415	415	struct sbi_pd_provider *pd_provider;
416	416	struct dev_power_governor *pd_gov;
417		− int ret = -ENOMEM, state_count = 0;
	417	+ int ret = -ENOMEM;
418	418
419	419	pd = dt_idle_pd_alloc(np, sbi_dt_parse_state_node);
420	420	if (!pd)
@@ -433,7 +433,7 @@ static int sbi_pd_init(struct device_node *np) · 12 unmodified lines
433	433	pd->flags \|= GENPD_FLAG_ALWAYS_ON;
434	434
435	435	/* Use governor for CPU PM domains if it has some states to manage. */
436		− pd_gov = state_count > 0 ? &pm_domain_cpu_gov : NULL;
	436	+ pd_gov = pd->states ? &pm_domain_cpu_gov : NULL;
437	437
438	438	ret = pm_genpd_init(pd, pd_gov, false);
439	439	if (ret)

drivers/devfreq/devfreq.c+12−8

@@ -112,16 +112,16 @@ static unsigned long find_available_max_freq(struct devfreq *devfreq) · 111 unmodified lines
112	112	}
113	113
114	114	/**
115		− * get_freq_range() - Get the current freq range
	115	+ * devfreq_get_freq_range() - Get the current freq range
116	116	* @devfreq: the devfreq instance
117	117	* @min_freq: the min frequency
118	118	* @max_freq: the max frequency
119	119	*
120	120	* This takes into consideration all constraints.
121	121	*/
122		−static void get_freq_range(struct devfreq *devfreq,
123		− unsigned long *min_freq,
124		− unsigned long *max_freq)
	122	+void devfreq_get_freq_range(struct devfreq *devfreq,
	123	+ unsigned long *min_freq,
	124	+ unsigned long *max_freq)
125	125	{
126	126	unsigned long *freq_table = devfreq->profile->freq_table;
127	127	s32 qos_min_freq, qos_max_freq;
@@ -158,6 +158,7 @@ static void get_freq_range(struct devfreq *devfreq, · 30 unmodified lines
158	158	if (min_freq > max_freq)
159	159	min_freq = max_freq;
160	160	}
	161	+EXPORT_SYMBOL(devfreq_get_freq_range);
161	162
162	163	/**
163	164	* devfreq_get_freq_level() - Lookup freq_table for the frequency
@@ -418,7 +419,7 @@ int devfreq_update_target(struct devfreq *devfreq, unsigned long freq) · 254 unmodified lines
418	419	err = devfreq->governor->get_target_freq(devfreq, &freq);
419	420	if (err)
420	421	return err;
421		− get_freq_range(devfreq, &min_freq, &max_freq);
	422	+ devfreq_get_freq_range(devfreq, &min_freq, &max_freq);
422	423
423	424	if (freq < min_freq) {
424	425	freq = min_freq;
@@ -785,6 +786,7 @@ struct devfreq devfreq_add_device(struct device dev, · 360 unmodified lines
785	786	{
786	787	struct devfreq *devfreq;
787	788	struct devfreq_governor *governor;
	789	+ unsigned long min_freq, max_freq;
788	790	int err = 0;
789	791
790	792	if (!dev \|\| !profile \|\| !governor_name) {
@@ -849,6 +851,8 @@ struct devfreq devfreq_add_device(struct device dev, · 58 unmodified lines
849	851	goto err_dev;
850	852	}
851	853
	854	+ devfreq_get_freq_range(devfreq, &min_freq, &max_freq);
	855	+
852	856	devfreq->suspend_freq = dev_pm_opp_get_suspend_opp_freq(dev);
853	857	devfreq->opp_table = dev_pm_opp_get_opp_table(dev);
854	858	if (IS_ERR(devfreq->opp_table))
@@ -1587,7 +1591,7 @@ static ssize_t min_freq_show(struct device dev, struct device_attribute attr, · 732 unmodified lines
1587	1591	unsigned long min_freq, max_freq;
1588	1592
1589	1593	mutex_lock(&df->lock);
1590		− get_freq_range(df, &min_freq, &max_freq);
	1594	+ devfreq_get_freq_range(df, &min_freq, &max_freq);
1591	1595	mutex_unlock(&df->lock);
1592	1596
1593	1597	return sprintf(buf, "%lu\n", min_freq);
@@ -1641,7 +1645,7 @@ static ssize_t max_freq_show(struct device dev, struct device_attribute attr, · 47 unmodified lines
1641	1645	unsigned long min_freq, max_freq;
1642	1646
1643	1647	mutex_lock(&df->lock);
1644		− get_freq_range(df, &min_freq, &max_freq);
	1648	+ devfreq_get_freq_range(df, &min_freq, &max_freq);
1645	1649	mutex_unlock(&df->lock);
1646	1650
1647	1651	return sprintf(buf, "%lu\n", max_freq);
@@ -1955,7 +1959,7 @@ static int devfreq_summary_show(struct seq_file s, void data) · 307 unmodified lines
1955	1959
1956	1960	mutex_lock(&devfreq->lock);
1957	1961	cur_freq = devfreq->previous_freq;
1958		− get_freq_range(devfreq, &min_freq, &max_freq);
	1962	+ devfreq_get_freq_range(devfreq, &min_freq, &max_freq);
1959	1963	timer = devfreq->profile->timer;
1960	1964
1961	1965	if (IS_SUPPORTED_ATTR(devfreq->governor->attrs, POLLING_INTERVAL))

drivers/devfreq/governor.h+27−0

@@ -47,6 +47,31 @@ · 46 unmodified lines
47	47	#define DEVFREQ_GOV_ATTR_POLLING_INTERVAL BIT(0)
48	48	#define DEVFREQ_GOV_ATTR_TIMER BIT(1)
49	49
	50	+/**
	51	+ * struct devfreq_cpu_data - Hold the per-cpu data
	52	+ * @node: list node
	53	+ * @dev: reference to cpu device.
	54	+ * @first_cpu: the cpumask of the first cpu of a policy.
	55	+ * @opp_table: reference to cpu opp table.
	56	+ * @cur_freq: the current frequency of the cpu.
	57	+ * @min_freq: the min frequency of the cpu.
	58	+ * @max_freq: the max frequency of the cpu.
	59	+ *
	60	+ * This structure stores the required cpu_data of a cpu.
	61	+ * This is auto-populated by the governor.
	62	+ */
	63	+struct devfreq_cpu_data {
	64	+ struct list_head node;
	65	+
	66	+ struct device *dev;
	67	+ unsigned int first_cpu;
	68	+
	69	+ struct opp_table *opp_table;
	70	+ unsigned int cur_freq;
	71	+ unsigned int min_freq;
	72	+ unsigned int max_freq;
	73	+};
	74	+
50	75	/**
51	76	* struct devfreq_governor - Devfreq policy governor
52	77	* @node: list node - contains registered devfreq governors
@@ -89,6 +114,8 @@ int devm_devfreq_add_governor(struct device *dev, · 36 unmodified lines
89	114
90	115	int devfreq_update_status(struct devfreq *devfreq, unsigned long freq);
91	116	int devfreq_update_target(struct devfreq *devfreq, unsigned long freq);
	117	+void devfreq_get_freq_range(struct devfreq devfreq, unsigned long min_freq,
	118	+ unsigned long *max_freq);
92	119
93	120	static inline int devfreq_update_stats(struct devfreq *df)
94	121	{

drivers/devfreq/governor_passive.c+335−68

@@ -1,4 +1,4 @@
1		−// SPDX-License-Identifier: GPL-2.0-only
	1	+ // SPDX-License-Identifier: GPL-2.0-only
2	2	/*
3	3	* linux/drivers/devfreq/governor_passive.c
4	4	*
@@ -8,76 +8,129 @@ · 3 unmodified lines
8	8	*/
9	9
10	10	#include <linux/module.h>
	11	+#include <linux/cpu.h>
	12	+#include <linux/cpufreq.h>
	13	+#include <linux/cpumask.h>
	14	+#include <linux/slab.h>
11	15	#include <linux/device.h>
12	16	#include <linux/devfreq.h>
13	17	#include "governor.h"
14	18
15		−static int devfreq_passive_get_target_freq(struct devfreq *devfreq,
16		− unsigned long *freq)
	19	+#define HZ_PER_KHZ 1000
	20	+
	21	+static struct devfreq_cpu_data *
	22	+get_parent_cpu_data(struct devfreq_passive_data *p_data,
	23	+ struct cpufreq_policy *policy)
17	24	{
18		− struct devfreq_passive_data *p_data
19		− = (struct devfreq_passive_data *)devfreq->data;
20		− struct devfreq parent_devfreq = (struct devfreq )p_data->parent;
21		− unsigned long child_freq = ULONG_MAX;
22		− struct dev_pm_opp opp, p_opp;
23		− int i, count;
	25	+ struct devfreq_cpu_data *parent_cpu_data;
24	26
25		− /*
26		− * If the devfreq device with passive governor has the specific method
27		− * to determine the next frequency, should use the get_target_freq()
28		− * of struct devfreq_passive_data.
29		− */
30		− if (p_data->get_target_freq)
31		− return p_data->get_target_freq(devfreq, freq);
	27	+ if (!p_data \|\| !policy)
	28	+ return NULL;
32	29
33		− /*
34		− * If the parent and passive devfreq device uses the OPP table,
35		− * get the next frequency by using the OPP table.
36		− */
	30	+ list_for_each_entry(parent_cpu_data, &p_data->cpu_data_list, node)
	31	+ if (parent_cpu_data->first_cpu == cpumask_first(policy->related_cpus))
	32	+ return parent_cpu_data;
37	33
38		− /*
39		− * - parent devfreq device uses the governors except for passive.
40		− * - passive devfreq device uses the passive governor.
41		− *
42		− * Each devfreq has the OPP table. After deciding the new frequency
43		− * from the governor of parent devfreq device, the passive governor
44		− * need to get the index of new frequency on OPP table of parent
45		− * device. And then the index is used for getting the suitable
46		− * new frequency for passive devfreq device.
47		− */
48		− if (!devfreq->profile \|\| !devfreq->profile->freq_table
49		− \|\| devfreq->profile->max_state <= 0)
50		− return -EINVAL;
	34	+ return NULL;
	35	+}
51	36
52		− /*
53		− * The passive governor have to get the correct frequency from OPP
54		− * list of parent device. Because in this case, *freq is temporary
55		− * value which is decided by ondemand governor.
56		− */
57		− if (devfreq->opp_table && parent_devfreq->opp_table) {
58		− p_opp = devfreq_recommended_opp(parent_devfreq->dev.parent,
59		− freq, 0);
60		− if (IS_ERR(p_opp))
61		− return PTR_ERR(p_opp);
	37	+static unsigned long get_target_freq_by_required_opp(struct device *p_dev,
	38	+ struct opp_table *p_opp_table,
	39	+ struct opp_table *opp_table,
	40	+ unsigned long *freq)
	41	+{
	42	+ struct dev_pm_opp opp = NULL, p_opp = NULL;
	43	+ unsigned long target_freq;
62	44
63		− opp = dev_pm_opp_xlate_required_opp(parent_devfreq->opp_table,
64		− devfreq->opp_table, p_opp);
65		− dev_pm_opp_put(p_opp);
	45	+ if (!p_dev \|\| !p_opp_table \|\| !opp_table \|\| !freq)
	46	+ return 0;
66	47
67		− if (IS_ERR(opp))
68		− goto no_required_opp;
	48	+ p_opp = devfreq_recommended_opp(p_dev, freq, 0);
	49	+ if (IS_ERR(p_opp))
	50	+ return 0;
69	51
70		− *freq = dev_pm_opp_get_freq(opp);
71		− dev_pm_opp_put(opp);
	52	+ opp = dev_pm_opp_xlate_required_opp(p_opp_table, opp_table, p_opp);
	53	+ dev_pm_opp_put(p_opp);
72	54
	55	+ if (IS_ERR(opp))
73	56	return 0;
	57	+
	58	+ target_freq = dev_pm_opp_get_freq(opp);
	59	+ dev_pm_opp_put(opp);
	60	+
	61	+ return target_freq;
	62	+}
	63	+
	64	+static int get_target_freq_with_cpufreq(struct devfreq *devfreq,
	65	+ unsigned long *target_freq)
	66	+{
	67	+ struct devfreq_passive_data *p_data =
	68	+ (struct devfreq_passive_data *)devfreq->data;
	69	+ struct devfreq_cpu_data *parent_cpu_data;
	70	+ struct cpufreq_policy *policy;
	71	+ unsigned long cpu, cpu_cur, cpu_min, cpu_max, cpu_percent;
	72	+ unsigned long dev_min, dev_max;
	73	+ unsigned long freq = 0;
	74	+ int ret = 0;
	75	+
	76	+ for_each_online_cpu(cpu) {
	77	+ policy = cpufreq_cpu_get(cpu);
	78	+ if (!policy) {
	79	+ ret = -EINVAL;
	80	+ continue;
	81	+ }
	82	+
	83	+ parent_cpu_data = get_parent_cpu_data(p_data, policy);
	84	+ if (!parent_cpu_data) {
	85	+ cpufreq_cpu_put(policy);
	86	+ continue;
	87	+ }
	88	+
	89	+ /* Get target freq via required opps */
	90	+ cpu_cur = parent_cpu_data->cur_freq * HZ_PER_KHZ;
	91	+ freq = get_target_freq_by_required_opp(parent_cpu_data->dev,
	92	+ parent_cpu_data->opp_table,
	93	+ devfreq->opp_table, &cpu_cur);
	94	+ if (freq) {
	95	+ target_freq = max(freq, target_freq);
	96	+ cpufreq_cpu_put(policy);
	97	+ continue;
	98	+ }
	99	+
	100	+ /* Use interpolation if required opps is not available */
	101	+ devfreq_get_freq_range(devfreq, &dev_min, &dev_max);
	102	+
	103	+ cpu_min = parent_cpu_data->min_freq;
	104	+ cpu_max = parent_cpu_data->max_freq;
	105	+ cpu_cur = parent_cpu_data->cur_freq;
	106	+
	107	+ cpu_percent = ((cpu_cur - cpu_min) * 100) / (cpu_max - cpu_min);
	108	+ freq = dev_min + mult_frac(dev_max - dev_min, cpu_percent, 100);
	109	+
	110	+ target_freq = max(freq, target_freq);
	111	+ cpufreq_cpu_put(policy);
74	112	}
75	113
76		−no_required_opp:
77		− /*
78		− * Get the OPP table's index of decided frequency by governor
79		− * of parent device.
80		− */
	114	+ return ret;
	115	+}
	116	+
	117	+static int get_target_freq_with_devfreq(struct devfreq *devfreq,
	118	+ unsigned long *freq)
	119	+{
	120	+ struct devfreq_passive_data *p_data
	121	+ = (struct devfreq_passive_data *)devfreq->data;
	122	+ struct devfreq parent_devfreq = (struct devfreq )p_data->parent;
	123	+ unsigned long child_freq = ULONG_MAX;
	124	+ int i, count;
	125	+
	126	+ /* Get target freq via required opps */
	127	+ child_freq = get_target_freq_by_required_opp(parent_devfreq->dev.parent,
	128	+ parent_devfreq->opp_table,
	129	+ devfreq->opp_table, freq);
	130	+ if (child_freq)
	131	+ goto out;
	132	+
	133	+ /* Use interpolation if required opps is not available */
81	134	for (i = 0; i < parent_devfreq->profile->max_state; i++)
82	135	if (parent_devfreq->profile->freq_table[i] == *freq)
83	136	break;
@@ -85,7 +138,6 @@ no_required_opp: · 1 unmodified line
85	138	if (i == parent_devfreq->profile->max_state)
86	139	return -EINVAL;
87	140
88		− /* Get the suitable frequency by using index of parent device. */
89	141	if (i < devfreq->profile->max_state) {
90	142	child_freq = devfreq->profile->freq_table[i];
91	143	} else {
@@ -93,12 +145,202 @@ no_required_opp: · 1 unmodified line
93	145	child_freq = devfreq->profile->freq_table[count - 1];
94	146	}
95	147
96		− /* Return the suitable frequency for passive device. */
	148	+out:
97	149	*freq = child_freq;
98	150
99	151	return 0;
100	152	}
101	153
	154	+static int devfreq_passive_get_target_freq(struct devfreq *devfreq,
	155	+ unsigned long *freq)
	156	+{
	157	+ struct devfreq_passive_data *p_data =
	158	+ (struct devfreq_passive_data *)devfreq->data;
	159	+ int ret;
	160	+
	161	+ if (!p_data)
	162	+ return -EINVAL;
	163	+
	164	+ /*
	165	+ * If the devfreq device with passive governor has the specific method
	166	+ * to determine the next frequency, should use the get_target_freq()
	167	+ * of struct devfreq_passive_data.
	168	+ */
	169	+ if (p_data->get_target_freq)
	170	+ return p_data->get_target_freq(devfreq, freq);
	171	+
	172	+ switch (p_data->parent_type) {
	173	+ case DEVFREQ_PARENT_DEV:
	174	+ ret = get_target_freq_with_devfreq(devfreq, freq);
	175	+ break;
	176	+ case CPUFREQ_PARENT_DEV:
	177	+ ret = get_target_freq_with_cpufreq(devfreq, freq);
	178	+ break;
	179	+ default:
	180	+ ret = -EINVAL;
	181	+ dev_err(&devfreq->dev, "Invalid parent type\n");
	182	+ break;
	183	+ }
	184	+
	185	+ return ret;
	186	+}
	187	+
	188	+static int cpufreq_passive_notifier_call(struct notifier_block *nb,
	189	+ unsigned long event, void *ptr)
	190	+{
	191	+ struct devfreq_passive_data *p_data =
	192	+ container_of(nb, struct devfreq_passive_data, nb);
	193	+ struct devfreq devfreq = (struct devfreq )p_data->this;
	194	+ struct devfreq_cpu_data *parent_cpu_data;
	195	+ struct cpufreq_freqs *freqs = ptr;
	196	+ unsigned int cur_freq;
	197	+ int ret;
	198	+
	199	+ if (event != CPUFREQ_POSTCHANGE \|\| !freqs)
	200	+ return 0;
	201	+
	202	+ parent_cpu_data = get_parent_cpu_data(p_data, freqs->policy);
	203	+ if (!parent_cpu_data \|\| parent_cpu_data->cur_freq == freqs->new)
	204	+ return 0;
	205	+
	206	+ cur_freq = parent_cpu_data->cur_freq;
	207	+ parent_cpu_data->cur_freq = freqs->new;
	208	+
	209	+ mutex_lock(&devfreq->lock);
	210	+ ret = devfreq_update_target(devfreq, freqs->new);
	211	+ mutex_unlock(&devfreq->lock);
	212	+ if (ret) {
	213	+ parent_cpu_data->cur_freq = cur_freq;
	214	+ dev_err(&devfreq->dev, "failed to update the frequency.\n");
	215	+ return ret;
	216	+ }
	217	+
	218	+ return 0;
	219	+}
	220	+
	221	+static int cpufreq_passive_unregister_notifier(struct devfreq *devfreq)
	222	+{
	223	+ struct devfreq_passive_data *p_data
	224	+ = (struct devfreq_passive_data *)devfreq->data;
	225	+ struct devfreq_cpu_data *parent_cpu_data;
	226	+ int cpu, ret = 0;
	227	+
	228	+ if (p_data->nb.notifier_call) {
	229	+ ret = cpufreq_unregister_notifier(&p_data->nb,
	230	+ CPUFREQ_TRANSITION_NOTIFIER);
	231	+ if (ret < 0)
	232	+ return ret;
	233	+ }
	234	+
	235	+ for_each_possible_cpu(cpu) {
	236	+ struct cpufreq_policy *policy = cpufreq_cpu_get(cpu);
	237	+ if (!policy) {
	238	+ ret = -EINVAL;
	239	+ continue;
	240	+ }
	241	+
	242	+ parent_cpu_data = get_parent_cpu_data(p_data, policy);
	243	+ if (!parent_cpu_data) {
	244	+ cpufreq_cpu_put(policy);
	245	+ continue;
	246	+ }
	247	+
	248	+ list_del(&parent_cpu_data->node);
	249	+ if (parent_cpu_data->opp_table)
	250	+ dev_pm_opp_put_opp_table(parent_cpu_data->opp_table);
	251	+ kfree(parent_cpu_data);
	252	+ cpufreq_cpu_put(policy);
	253	+ }
	254	+
	255	+ return ret;
	256	+}
	257	+
	258	+static int cpufreq_passive_register_notifier(struct devfreq *devfreq)
	259	+{
	260	+ struct devfreq_passive_data *p_data
	261	+ = (struct devfreq_passive_data *)devfreq->data;
	262	+ struct device *dev = devfreq->dev.parent;
	263	+ struct opp_table *opp_table = NULL;
	264	+ struct devfreq_cpu_data *parent_cpu_data;
	265	+ struct cpufreq_policy *policy;
	266	+ struct device *cpu_dev;
	267	+ unsigned int cpu;
	268	+ int ret;
	269	+
	270	+ p_data->cpu_data_list
	271	+ = (struct list_head)LIST_HEAD_INIT(p_data->cpu_data_list);
	272	+
	273	+ p_data->nb.notifier_call = cpufreq_passive_notifier_call;
	274	+ ret = cpufreq_register_notifier(&p_data->nb, CPUFREQ_TRANSITION_NOTIFIER);
	275	+ if (ret) {
	276	+ dev_err(dev, "failed to register cpufreq notifier\n");
	277	+ p_data->nb.notifier_call = NULL;
	278	+ goto err;
	279	+ }
	280	+
	281	+ for_each_possible_cpu(cpu) {
	282	+ policy = cpufreq_cpu_get(cpu);
	283	+ if (!policy) {
	284	+ ret = -EPROBE_DEFER;
	285	+ goto err;
	286	+ }
	287	+
	288	+ parent_cpu_data = get_parent_cpu_data(p_data, policy);
	289	+ if (parent_cpu_data) {
	290	+ cpufreq_cpu_put(policy);
	291	+ continue;
	292	+ }
	293	+
	294	+ parent_cpu_data = kzalloc(sizeof(*parent_cpu_data),
	295	+ GFP_KERNEL);
	296	+ if (!parent_cpu_data) {
	297	+ ret = -ENOMEM;
	298	+ goto err_put_policy;
	299	+ }
	300	+
	301	+ cpu_dev = get_cpu_device(cpu);
	302	+ if (!cpu_dev) {
	303	+ dev_err(dev, "failed to get cpu device\n");
	304	+ ret = -ENODEV;
	305	+ goto err_free_cpu_data;
	306	+ }
	307	+
	308	+ opp_table = dev_pm_opp_get_opp_table(cpu_dev);
	309	+ if (IS_ERR(opp_table)) {
	310	+ dev_err(dev, "failed to get opp_table of cpu%d\n", cpu);
	311	+ ret = PTR_ERR(opp_table);
	312	+ goto err_free_cpu_data;
	313	+ }
	314	+
	315	+ parent_cpu_data->dev = cpu_dev;
	316	+ parent_cpu_data->opp_table = opp_table;
	317	+ parent_cpu_data->first_cpu = cpumask_first(policy->related_cpus);
	318	+ parent_cpu_data->cur_freq = policy->cur;
	319	+ parent_cpu_data->min_freq = policy->cpuinfo.min_freq;
	320	+ parent_cpu_data->max_freq = policy->cpuinfo.max_freq;
	321	+
	322	+ list_add_tail(&parent_cpu_data->node, &p_data->cpu_data_list);
	323	+ cpufreq_cpu_put(policy);
	324	+ }
	325	+
	326	+ mutex_lock(&devfreq->lock);
	327	+ ret = devfreq_update_target(devfreq, 0L);
	328	+ mutex_unlock(&devfreq->lock);
	329	+ if (ret)
	330	+ dev_err(dev, "failed to update the frequency\n");
	331	+
	332	+ return ret;
	333	+
	334	+err_free_cpu_data:
	335	+ kfree(parent_cpu_data);
	336	+err_put_policy:
	337	+ cpufreq_cpu_put(policy);
	338	+err:
	339	+ WARN_ON(cpufreq_passive_unregister_notifier(devfreq));
	340	+
	341	+ return ret;
	342	+}
	343	+
102	344	static int devfreq_passive_notifier_call(struct notifier_block *nb,
103	345	unsigned long event, void *ptr)
104	346	{
@@ -131,30 +373,55 @@ static int devfreq_passive_notifier_call(struct notifier_block *nb, · 26 unmodified lines
131	373	return NOTIFY_DONE;
132	374	}
133	375
134		−static int devfreq_passive_event_handler(struct devfreq *devfreq,
135		− unsigned int event, void *data)
	376	+static int devfreq_passive_unregister_notifier(struct devfreq *devfreq)
	377	+{
	378	+ struct devfreq_passive_data *p_data
	379	+ = (struct devfreq_passive_data *)devfreq->data;
	380	+ struct devfreq parent = (struct devfreq )p_data->parent;
	381	+ struct notifier_block *nb = &p_data->nb;
	382	+
	383	+ return devfreq_unregister_notifier(parent, nb, DEVFREQ_TRANSITION_NOTIFIER);
	384	+}
	385	+
	386	+static int devfreq_passive_register_notifier(struct devfreq *devfreq)
136	387	{
137	388	struct devfreq_passive_data *p_data
138	389	= (struct devfreq_passive_data *)devfreq->data;
139	390	struct devfreq parent = (struct devfreq )p_data->parent;
140	391	struct notifier_block *nb = &p_data->nb;
141		− int ret = 0;
142	392
143	393	if (!parent)
144	394	return -EPROBE_DEFER;
145	395
	396	+ nb->notifier_call = devfreq_passive_notifier_call;
	397	+ return devfreq_register_notifier(parent, nb, DEVFREQ_TRANSITION_NOTIFIER);
	398	+}
	399	+
	400	+static int devfreq_passive_event_handler(struct devfreq *devfreq,
	401	+ unsigned int event, void *data)
	402	+{
	403	+ struct devfreq_passive_data *p_data
	404	+ = (struct devfreq_passive_data *)devfreq->data;
	405	+ int ret = 0;
	406	+
	407	+ if (!p_data)
	408	+ return -EINVAL;
	409	+
	410	+ if (!p_data->this)
	411	+ p_data->this = devfreq;
	412	+
146	413	switch (event) {
147	414	case DEVFREQ_GOV_START:
148		− if (!p_data->this)
149		− p_data->this = devfreq;
150		−
151		− nb->notifier_call = devfreq_passive_notifier_call;
152		− ret = devfreq_register_notifier(parent, nb,
153		− DEVFREQ_TRANSITION_NOTIFIER);
	415	+ if (p_data->parent_type == DEVFREQ_PARENT_DEV)
	416	+ ret = devfreq_passive_register_notifier(devfreq);
	417	+ else if (p_data->parent_type == CPUFREQ_PARENT_DEV)
	418	+ ret = cpufreq_passive_register_notifier(devfreq);
154	419	break;
155	420	case DEVFREQ_GOV_STOP:
156		− WARN_ON(devfreq_unregister_notifier(parent, nb,
157		− DEVFREQ_TRANSITION_NOTIFIER));
	421	+ if (p_data->parent_type == DEVFREQ_PARENT_DEV)
	422	+ WARN_ON(devfreq_passive_unregister_notifier(devfreq));
	423	+ else if (p_data->parent_type == CPUFREQ_PARENT_DEV)
	424	+ WARN_ON(cpufreq_passive_unregister_notifier(devfreq));
158	425	break;
159	426	default:
160	427	break;

drivers/devfreq/rk3399_dmc.c+147−165

@@ -5,6 +5,7 @@ · 4 unmodified lines
5	5	*/
6	6
7	7	#include <linux/arm-smccc.h>
	8	+#include <linux/bitfield.h>
8	9	#include <linux/clk.h>
9	10	#include <linux/delay.h>
10	11	#include <linux/devfreq.h>
@@ -20,55 +21,49 @@ · 9 unmodified lines
20	21	#include <linux/rwsem.h>
21	22	#include <linux/suspend.h>
22	23
	24	+#include <soc/rockchip/pm_domains.h>
23	25	#include <soc/rockchip/rk3399_grf.h>
24	26	#include <soc/rockchip/rockchip_sip.h>
25	27
26		−struct dram_timing {
27		− unsigned int ddr3_speed_bin;
28		− unsigned int pd_idle;
29		− unsigned int sr_idle;
30		− unsigned int sr_mc_gate_idle;
31		− unsigned int srpd_lite_idle;
32		− unsigned int standby_idle;
33		− unsigned int auto_pd_dis_freq;
34		− unsigned int dram_dll_dis_freq;
35		− unsigned int phy_dll_dis_freq;
36		− unsigned int ddr3_odt_dis_freq;
37		− unsigned int ddr3_drv;
38		− unsigned int ddr3_odt;
39		− unsigned int phy_ddr3_ca_drv;
40		− unsigned int phy_ddr3_dq_drv;
41		− unsigned int phy_ddr3_odt;
42		− unsigned int lpddr3_odt_dis_freq;
43		− unsigned int lpddr3_drv;
44		− unsigned int lpddr3_odt;
45		− unsigned int phy_lpddr3_ca_drv;
46		− unsigned int phy_lpddr3_dq_drv;
47		− unsigned int phy_lpddr3_odt;
48		− unsigned int lpddr4_odt_dis_freq;
49		− unsigned int lpddr4_drv;
50		− unsigned int lpddr4_dq_odt;
51		− unsigned int lpddr4_ca_odt;
52		− unsigned int phy_lpddr4_ca_drv;
53		− unsigned int phy_lpddr4_ck_cs_drv;
54		− unsigned int phy_lpddr4_dq_drv;
55		− unsigned int phy_lpddr4_odt;
56		−};
	28	+#define NS_TO_CYCLE(NS, MHz) (((NS) * (MHz)) / NSEC_PER_USEC)
	29	+
	30	+#define RK3399_SET_ODT_PD_0_SR_IDLE GENMASK(7, 0)
	31	+#define RK3399_SET_ODT_PD_0_SR_MC_GATE_IDLE GENMASK(15, 8)
	32	+#define RK3399_SET_ODT_PD_0_STANDBY_IDLE GENMASK(31, 16)
	33	+
	34	+#define RK3399_SET_ODT_PD_1_PD_IDLE GENMASK(11, 0)
	35	+#define RK3399_SET_ODT_PD_1_SRPD_LITE_IDLE GENMASK(27, 16)
	36	+
	37	+#define RK3399_SET_ODT_PD_2_ODT_ENABLE BIT(0)
57	38
58	39	struct rk3399_dmcfreq {
59	40	struct device *dev;
60	41	struct devfreq *devfreq;
	42	+ struct devfreq_dev_profile profile;
61	43	struct devfreq_simple_ondemand_data ondemand_data;
62	44	struct clk *dmc_clk;
63	45	struct devfreq_event_dev *edev;
64	46	struct mutex lock;
65		− struct dram_timing timing;
66	47	struct regulator *vdd_center;
67	48	struct regmap *regmap_pmu;
68	49	unsigned long rate, target_rate;
69	50	unsigned long volt, target_volt;
70	51	unsigned int odt_dis_freq;
71		− int odt_pd_arg0, odt_pd_arg1;
	52	+
	53	+ unsigned int pd_idle_ns;
	54	+ unsigned int sr_idle_ns;
	55	+ unsigned int sr_mc_gate_idle_ns;
	56	+ unsigned int srpd_lite_idle_ns;
	57	+ unsigned int standby_idle_ns;
	58	+ unsigned int ddr3_odt_dis_freq;
	59	+ unsigned int lpddr3_odt_dis_freq;
	60	+ unsigned int lpddr4_odt_dis_freq;
	61	+
	62	+ unsigned int pd_idle_dis_freq;
	63	+ unsigned int sr_idle_dis_freq;
	64	+ unsigned int sr_mc_gate_idle_dis_freq;
	65	+ unsigned int srpd_lite_idle_dis_freq;
	66	+ unsigned int standby_idle_dis_freq;
72	67	};
73	68
74	69	static int rk3399_dmcfreq_target(struct device dev, unsigned long freq,
@@ -78,10 +73,14 @@ static int rk3399_dmcfreq_target(struct device dev, unsigned long freq, · 3 unmodified lines
78	73	struct dev_pm_opp *opp;
79	74	unsigned long old_clk_rate = dmcfreq->rate;
80	75	unsigned long target_volt, target_rate;
	76	+ unsigned int ddrcon_mhz;
81	77	struct arm_smccc_res res;
82		− bool odt_enable = false;
83	78	int err;
84	79
	80	+ u32 odt_pd_arg0 = 0;
	81	+ u32 odt_pd_arg1 = 0;
	82	+ u32 odt_pd_arg2 = 0;
	83	+
85	84	opp = devfreq_recommended_opp(dev, freq, flags);
86	85	if (IS_ERR(opp))
87	86	return PTR_ERR(opp);
@@ -95,19 +94,71 @@ static int rk3399_dmcfreq_target(struct device dev, unsigned long freq, · 7 unmodified lines
95	94
96	95	mutex_lock(&dmcfreq->lock);
97	96
	97	+ /*
	98	+ * Ensure power-domain transitions don't interfere with ARM Trusted
	99	+ * Firmware power-domain idling.
	100	+ */
	101	+ err = rockchip_pmu_block();
	102	+ if (err) {
	103	+ dev_err(dev, "Failed to block PMU: %d\n", err);
	104	+ goto out_unlock;
	105	+ }
	106	+
	107	+ /*
	108	+ * Some idle parameters may be based on the DDR controller clock, which
	109	+ * is half of the DDR frequency.
	110	+ * pd_idle and standby_idle are based on the controller clock cycle.
	111	+ * sr_idle_cycle, sr_mc_gate_idle_cycle, and srpd_lite_idle_cycle
	112	+ * are based on the 1024 controller clock cycle
	113	+ */
	114	+ ddrcon_mhz = target_rate / USEC_PER_SEC / 2;
	115	+
	116	+ u32p_replace_bits(&odt_pd_arg1,
	117	+ NS_TO_CYCLE(dmcfreq->pd_idle_ns, ddrcon_mhz),
	118	+ RK3399_SET_ODT_PD_1_PD_IDLE);
	119	+ u32p_replace_bits(&odt_pd_arg0,
	120	+ NS_TO_CYCLE(dmcfreq->standby_idle_ns, ddrcon_mhz),
	121	+ RK3399_SET_ODT_PD_0_STANDBY_IDLE);
	122	+ u32p_replace_bits(&odt_pd_arg0,
	123	+ DIV_ROUND_UP(NS_TO_CYCLE(dmcfreq->sr_idle_ns,
	124	+ ddrcon_mhz), 1024),
	125	+ RK3399_SET_ODT_PD_0_SR_IDLE);
	126	+ u32p_replace_bits(&odt_pd_arg0,
	127	+ DIV_ROUND_UP(NS_TO_CYCLE(dmcfreq->sr_mc_gate_idle_ns,
	128	+ ddrcon_mhz), 1024),
	129	+ RK3399_SET_ODT_PD_0_SR_MC_GATE_IDLE);
	130	+ u32p_replace_bits(&odt_pd_arg1,
	131	+ DIV_ROUND_UP(NS_TO_CYCLE(dmcfreq->srpd_lite_idle_ns,
	132	+ ddrcon_mhz), 1024),
	133	+ RK3399_SET_ODT_PD_1_SRPD_LITE_IDLE);
	134	+
98	135	if (dmcfreq->regmap_pmu) {
	136	+ if (target_rate >= dmcfreq->sr_idle_dis_freq)
	137	+ odt_pd_arg0 &= ~RK3399_SET_ODT_PD_0_SR_IDLE;
	138	+
	139	+ if (target_rate >= dmcfreq->sr_mc_gate_idle_dis_freq)
	140	+ odt_pd_arg0 &= ~RK3399_SET_ODT_PD_0_SR_MC_GATE_IDLE;
	141	+
	142	+ if (target_rate >= dmcfreq->standby_idle_dis_freq)
	143	+ odt_pd_arg0 &= ~RK3399_SET_ODT_PD_0_STANDBY_IDLE;
	144	+
	145	+ if (target_rate >= dmcfreq->pd_idle_dis_freq)
	146	+ odt_pd_arg1 &= ~RK3399_SET_ODT_PD_1_PD_IDLE;
	147	+
	148	+ if (target_rate >= dmcfreq->srpd_lite_idle_dis_freq)
	149	+ odt_pd_arg1 &= ~RK3399_SET_ODT_PD_1_SRPD_LITE_IDLE;
	150	+
99	151	if (target_rate >= dmcfreq->odt_dis_freq)
100		− odt_enable = true;
	152	+ odt_pd_arg2 \|= RK3399_SET_ODT_PD_2_ODT_ENABLE;
101	153
102	154	/*
103	155	* This makes a SMC call to the TF-A to set the DDR PD
104	156	* (power-down) timings and to enable or disable the
105	157	* ODT (on-die termination) resistors.
106	158	*/
107		− arm_smccc_smc(ROCKCHIP_SIP_DRAM_FREQ, dmcfreq->odt_pd_arg0,
108		− dmcfreq->odt_pd_arg1,
109		− ROCKCHIP_SIP_CONFIG_DRAM_SET_ODT_PD,
110		− odt_enable, 0, 0, 0, &res);
	159	+ arm_smccc_smc(ROCKCHIP_SIP_DRAM_FREQ, odt_pd_arg0, odt_pd_arg1,
	160	+ ROCKCHIP_SIP_CONFIG_DRAM_SET_ODT_PD, odt_pd_arg2,
	161	+ 0, 0, 0, &res);
111	162	}
112	163
113	164	/*
@@ -158,6 +209,8 @@ static int rk3399_dmcfreq_target(struct device dev, unsigned long freq, · 44 unmodified lines
158	209	dmcfreq->volt = target_volt;
159	210
160	211	out:
	212	+ rockchip_pmu_unblock();
	213	+out_unlock:
161	214	mutex_unlock(&dmcfreq->lock);
162	215	return err;
163	216	}
@@ -189,13 +242,6 @@ static int rk3399_dmcfreq_get_cur_freq(struct device dev, unsigned long freq) · 25 unmodified lines
189	242	return 0;
190	243	}
191	244
192		−static struct devfreq_dev_profile rk3399_devfreq_dmc_profile = {
193		− .polling_ms = 200,
194		− .target = rk3399_dmcfreq_target,
195		− .get_dev_status = rk3399_dmcfreq_get_dev_status,
196		− .get_cur_freq = rk3399_dmcfreq_get_cur_freq,
197		−};
198		−
199	245	static __maybe_unused int rk3399_dmcfreq_suspend(struct device *dev)
200	246	{
201	247	struct rk3399_dmcfreq *dmcfreq = dev_get_drvdata(dev);
@@ -238,69 +284,48 @@ static __maybe_unused int rk3399_dmcfreq_resume(struct device *dev) · 36 unmodified lines
238	284	static SIMPLE_DEV_PM_OPS(rk3399_dmcfreq_pm, rk3399_dmcfreq_suspend,
239	285	rk3399_dmcfreq_resume);
240	286
241		−static int of_get_ddr_timings(struct dram_timing *timing,
242		− struct device_node *np)
	287	+static int rk3399_dmcfreq_of_props(struct rk3399_dmcfreq *data,
	288	+ struct device_node *np)
243	289	{
244	290	int ret = 0;
245	291
246		− ret = of_property_read_u32(np, "rockchip,ddr3_speed_bin",
247		− &timing->ddr3_speed_bin);
248		− ret \|= of_property_read_u32(np, "rockchip,pd_idle",
249		− &timing->pd_idle);
250		− ret \|= of_property_read_u32(np, "rockchip,sr_idle",
251		− &timing->sr_idle);
252		− ret \|= of_property_read_u32(np, "rockchip,sr_mc_gate_idle",
253		− &timing->sr_mc_gate_idle);
254		− ret \|= of_property_read_u32(np, "rockchip,srpd_lite_idle",
255		− &timing->srpd_lite_idle);
256		− ret \|= of_property_read_u32(np, "rockchip,standby_idle",
257		− &timing->standby_idle);
258		− ret \|= of_property_read_u32(np, "rockchip,auto_pd_dis_freq",
259		− &timing->auto_pd_dis_freq);
260		− ret \|= of_property_read_u32(np, "rockchip,dram_dll_dis_freq",
261		− &timing->dram_dll_dis_freq);
262		− ret \|= of_property_read_u32(np, "rockchip,phy_dll_dis_freq",
263		− &timing->phy_dll_dis_freq);
	292	+ /*
	293	+ * These are all optional, and serve as minimum bounds. Give them large
	294	+ * (i.e., never "disabled") values if the DT doesn't specify one.
	295	+ */
	296	+ data->pd_idle_dis_freq =
	297	+ data->sr_idle_dis_freq =
	298	+ data->sr_mc_gate_idle_dis_freq =
	299	+ data->srpd_lite_idle_dis_freq =
	300	+ data->standby_idle_dis_freq = UINT_MAX;
	301	+
	302	+ ret \|= of_property_read_u32(np, "rockchip,pd-idle-ns",
	303	+ &data->pd_idle_ns);
	304	+ ret \|= of_property_read_u32(np, "rockchip,sr-idle-ns",
	305	+ &data->sr_idle_ns);
	306	+ ret \|= of_property_read_u32(np, "rockchip,sr-mc-gate-idle-ns",
	307	+ &data->sr_mc_gate_idle_ns);
	308	+ ret \|= of_property_read_u32(np, "rockchip,srpd-lite-idle-ns",
	309	+ &data->srpd_lite_idle_ns);
	310	+ ret \|= of_property_read_u32(np, "rockchip,standby-idle-ns",
	311	+ &data->standby_idle_ns);
264	312	ret \|= of_property_read_u32(np, "rockchip,ddr3_odt_dis_freq",
265		− &timing->ddr3_odt_dis_freq);
266		− ret \|= of_property_read_u32(np, "rockchip,ddr3_drv",
267		− &timing->ddr3_drv);
268		− ret \|= of_property_read_u32(np, "rockchip,ddr3_odt",
269		− &timing->ddr3_odt);
270		− ret \|= of_property_read_u32(np, "rockchip,phy_ddr3_ca_drv",
271		− &timing->phy_ddr3_ca_drv);
272		− ret \|= of_property_read_u32(np, "rockchip,phy_ddr3_dq_drv",
273		− &timing->phy_ddr3_dq_drv);
274		− ret \|= of_property_read_u32(np, "rockchip,phy_ddr3_odt",
275		− &timing->phy_ddr3_odt);
	313	+ &data->ddr3_odt_dis_freq);
276	314	ret \|= of_property_read_u32(np, "rockchip,lpddr3_odt_dis_freq",
277		− &timing->lpddr3_odt_dis_freq);
278		− ret \|= of_property_read_u32(np, "rockchip,lpddr3_drv",
279		− &timing->lpddr3_drv);
280		− ret \|= of_property_read_u32(np, "rockchip,lpddr3_odt",
281		− &timing->lpddr3_odt);
282		− ret \|= of_property_read_u32(np, "rockchip,phy_lpddr3_ca_drv",
283		− &timing->phy_lpddr3_ca_drv);
284		− ret \|= of_property_read_u32(np, "rockchip,phy_lpddr3_dq_drv",
285		− &timing->phy_lpddr3_dq_drv);
286		− ret \|= of_property_read_u32(np, "rockchip,phy_lpddr3_odt",
287		− &timing->phy_lpddr3_odt);
	315	+ &data->lpddr3_odt_dis_freq);
288	316	ret \|= of_property_read_u32(np, "rockchip,lpddr4_odt_dis_freq",
289		− &timing->lpddr4_odt_dis_freq);
290		− ret \|= of_property_read_u32(np, "rockchip,lpddr4_drv",
291		− &timing->lpddr4_drv);
292		− ret \|= of_property_read_u32(np, "rockchip,lpddr4_dq_odt",
293		− &timing->lpddr4_dq_odt);
294		− ret \|= of_property_read_u32(np, "rockchip,lpddr4_ca_odt",
295		− &timing->lpddr4_ca_odt);
296		− ret \|= of_property_read_u32(np, "rockchip,phy_lpddr4_ca_drv",
297		− &timing->phy_lpddr4_ca_drv);
298		− ret \|= of_property_read_u32(np, "rockchip,phy_lpddr4_ck_cs_drv",
299		− &timing->phy_lpddr4_ck_cs_drv);
300		− ret \|= of_property_read_u32(np, "rockchip,phy_lpddr4_dq_drv",
301		− &timing->phy_lpddr4_dq_drv);
302		− ret \|= of_property_read_u32(np, "rockchip,phy_lpddr4_odt",
303		− &timing->phy_lpddr4_odt);
	317	+ &data->lpddr4_odt_dis_freq);
	318	+
	319	+ ret \|= of_property_read_u32(np, "rockchip,pd-idle-dis-freq-hz",
	320	+ &data->pd_idle_dis_freq);
	321	+ ret \|= of_property_read_u32(np, "rockchip,sr-idle-dis-freq-hz",
	322	+ &data->sr_idle_dis_freq);
	323	+ ret \|= of_property_read_u32(np, "rockchip,sr-mc-gate-idle-dis-freq-hz",
	324	+ &data->sr_mc_gate_idle_dis_freq);
	325	+ ret \|= of_property_read_u32(np, "rockchip,srpd-lite-idle-dis-freq-hz",
	326	+ &data->srpd_lite_idle_dis_freq);
	327	+ ret \|= of_property_read_u32(np, "rockchip,standby-idle-dis-freq-hz",
	328	+ &data->standby_idle_dis_freq);
304	329
305	330	return ret;
306	331	}
@@ -311,8 +336,7 @@ static int rk3399_dmcfreq_probe(struct platform_device *pdev) · 4 unmodified lines
311	336	struct device *dev = &pdev->dev;
312	337	struct device_node np = pdev->dev.of_node, node;
313	338	struct rk3399_dmcfreq *data;
314		− int ret, index, size;
315		− uint32_t *timing;
	339	+ int ret;
316	340	struct dev_pm_opp *opp;
317	341	u32 ddr_type;
318	342	u32 val;
@@ -343,26 +367,7 @@ static int rk3399_dmcfreq_probe(struct platform_device *pdev) · 24 unmodified lines
343	367	return ret;
344	368	}
345	369
346		− /*
347		− * Get dram timing and pass it to arm trust firmware,
348		− * the dram driver in arm trust firmware will get these
349		− * timing and to do dram initial.
350		− */
351		− if (!of_get_ddr_timings(&data->timing, np)) {
352		− timing = &data->timing.ddr3_speed_bin;
353		− size = sizeof(struct dram_timing) / 4;
354		− for (index = 0; index < size; index++) {
355		− arm_smccc_smc(ROCKCHIP_SIP_DRAM_FREQ, *timing++, index,
356		− ROCKCHIP_SIP_CONFIG_DRAM_SET_PARAM,
357		− 0, 0, 0, 0, &res);
358		− if (res.a0) {
359		− dev_err(dev, "Failed to set dram param: %ld\n",
360		− res.a0);
361		− ret = -EINVAL;
362		− goto err_edev;
363		− }
364		− }
365		− }
	370	+ rk3399_dmcfreq_of_props(data, np);
366	371
367	372	node = of_parse_phandle(np, "rockchip,pmu", 0);
368	373	if (!node)
@@ -381,13 +386,13 @@ static int rk3399_dmcfreq_probe(struct platform_device *pdev) · 12 unmodified lines
381	386
382	387	switch (ddr_type) {
383	388	case RK3399_PMUGRF_DDRTYPE_DDR3:
384		− data->odt_dis_freq = data->timing.ddr3_odt_dis_freq;
	389	+ data->odt_dis_freq = data->ddr3_odt_dis_freq;
385	390	break;
386	391	case RK3399_PMUGRF_DDRTYPE_LPDDR3:
387		− data->odt_dis_freq = data->timing.lpddr3_odt_dis_freq;
	392	+ data->odt_dis_freq = data->lpddr3_odt_dis_freq;
388	393	break;
389	394	case RK3399_PMUGRF_DDRTYPE_LPDDR4:
390		− data->odt_dis_freq = data->timing.lpddr4_odt_dis_freq;
	395	+ data->odt_dis_freq = data->lpddr4_odt_dis_freq;
391	396	break;
392	397	default:
393	398	ret = -EINVAL;
@@ -399,63 +404,46 @@ no_pmu: · 5 unmodified lines
399	404	ROCKCHIP_SIP_CONFIG_DRAM_INIT,
400	405	0, 0, 0, 0, &res);
401	406
402		− /*
403		− * In TF-A there is a platform SIP call to set the PD (power-down)
404		− * timings and to enable or disable the ODT (on-die termination).
405		− * This call needs three arguments as follows:
406		− *
407		− * arg0:
408		− * bit[0-7] : sr_idle
409		− * bit[8-15] : sr_mc_gate_idle
410		− * bit[16-31] : standby idle
411		− * arg1:
412		− * bit[0-11] : pd_idle
413		− * bit[16-27] : srpd_lite_idle
414		− * arg2:
415		− * bit[0] : odt enable
416		− */
417		− data->odt_pd_arg0 = (data->timing.sr_idle & 0xff) \|
418		− ((data->timing.sr_mc_gate_idle & 0xff) << 8) \|
419		− ((data->timing.standby_idle & 0xffff) << 16);
420		− data->odt_pd_arg1 = (data->timing.pd_idle & 0xfff) \|
421		− ((data->timing.srpd_lite_idle & 0xfff) << 16);
422		−
423	407	/*
424	408	* We add a devfreq driver to our parent since it has a device tree node
425	409	* with operating points.
426	410	*/
427		− if (dev_pm_opp_of_add_table(dev)) {
	411	+ if (devm_pm_opp_of_add_table(dev)) {
428	412	dev_err(dev, "Invalid operating-points in device tree.\n");
429	413	ret = -EINVAL;
430	414	goto err_edev;
431	415	}
432	416
433		− of_property_read_u32(np, "upthreshold",
434		− &data->ondemand_data.upthreshold);
435		− of_property_read_u32(np, "downdifferential",
436		− &data->ondemand_data.downdifferential);
	417	+ data->ondemand_data.upthreshold = 25;
	418	+ data->ondemand_data.downdifferential = 15;
437	419
438	420	data->rate = clk_get_rate(data->dmc_clk);
439	421
440	422	opp = devfreq_recommended_opp(dev, &data->rate, 0);
441	423	if (IS_ERR(opp)) {
442	424	ret = PTR_ERR(opp);
443		− goto err_free_opp;
	425	+ goto err_edev;
444	426	}
445	427
446	428	data->rate = dev_pm_opp_get_freq(opp);
447	429	data->volt = dev_pm_opp_get_voltage(opp);
448	430	dev_pm_opp_put(opp);
449	431
450		− rk3399_devfreq_dmc_profile.initial_freq = data->rate;
	432	+ data->profile = (struct devfreq_dev_profile) {
	433	+ .polling_ms = 200,
	434	+ .target = rk3399_dmcfreq_target,
	435	+ .get_dev_status = rk3399_dmcfreq_get_dev_status,
	436	+ .get_cur_freq = rk3399_dmcfreq_get_cur_freq,
	437	+ .initial_freq = data->rate,
	438	+ };
451	439
452	440	data->devfreq = devm_devfreq_add_device(dev,
453		− &rk3399_devfreq_dmc_profile,
	441	+ &data->profile,
454	442	DEVFREQ_GOV_SIMPLE_ONDEMAND,
455	443	&data->ondemand_data);
456	444	if (IS_ERR(data->devfreq)) {
457	445	ret = PTR_ERR(data->devfreq);
458		− goto err_free_opp;
	446	+ goto err_edev;
459	447	}
460	448
461	449	devm_devfreq_register_opp_notifier(dev, data->devfreq);
@@ -465,8 +453,6 @@ no_pmu: · 3 unmodified lines
465	453
466	454	return 0;
467	455
468		−err_free_opp:
469		− dev_pm_opp_of_remove_table(&pdev->dev);
470	456	err_edev:
471	457	devfreq_event_disable_edev(data->edev);
472	458
@@ -477,11 +463,7 @@ static int rk3399_dmcfreq_remove(struct platform_device *pdev) · 4 unmodified lines
477	463	{
478	464	struct rk3399_dmcfreq *dmcfreq = dev_get_drvdata(&pdev->dev);
479	465
480		− /*
481		− * Before remove the opp table we need to unregister the opp notifier.
482		− */
483		− devm_devfreq_unregister_opp_notifier(dmcfreq->dev, dmcfreq->devfreq);
484		− dev_pm_opp_of_remove_table(dmcfreq->dev);
	466	+ devfreq_event_disable_edev(dmcfreq->edev);
485	467
486	468	return 0;
487	469	}

drivers/idle/intel_idle.c+133−0

@@ -764,6 +764,106 @@ static struct cpuidle_state icx_cstates[] __initdata = { · 763 unmodified lines
764	764	.enter = NULL }
765	765	};
766	766
	767	+/*
	768	+ * On AlderLake C1 has to be disabled if C1E is enabled, and vice versa.
	769	+ * C1E is enabled only if "C1E promotion" bit is set in MSR_IA32_POWER_CTL.
	770	+ * But in this case there is effectively no C1, because C1 requests are
	771	+ * promoted to C1E. If the "C1E promotion" bit is cleared, then both C1
	772	+ * and C1E requests end up with C1, so there is effectively no C1E.
	773	+ *
	774	+ * By default we enable C1E and disable C1 by marking it with
	775	+ * 'CPUIDLE_FLAG_UNUSABLE'.
	776	+ */
	777	+static struct cpuidle_state adl_cstates[] __initdata = {
	778	+ {
	779	+ .name = "C1",
	780	+ .desc = "MWAIT 0x00",
	781	+ .flags = MWAIT2flg(0x00) \| CPUIDLE_FLAG_UNUSABLE,
	782	+ .exit_latency = 1,
	783	+ .target_residency = 1,
	784	+ .enter = &intel_idle,
	785	+ .enter_s2idle = intel_idle_s2idle, },
	786	+ {
	787	+ .name = "C1E",
	788	+ .desc = "MWAIT 0x01",
	789	+ .flags = MWAIT2flg(0x01) \| CPUIDLE_FLAG_ALWAYS_ENABLE,
	790	+ .exit_latency = 2,
	791	+ .target_residency = 4,
	792	+ .enter = &intel_idle,
	793	+ .enter_s2idle = intel_idle_s2idle, },
	794	+ {
	795	+ .name = "C6",
	796	+ .desc = "MWAIT 0x20",
	797	+ .flags = MWAIT2flg(0x20) \| CPUIDLE_FLAG_TLB_FLUSHED,
	798	+ .exit_latency = 220,
	799	+ .target_residency = 600,
	800	+ .enter = &intel_idle,
	801	+ .enter_s2idle = intel_idle_s2idle, },
	802	+ {
	803	+ .name = "C8",
	804	+ .desc = "MWAIT 0x40",
	805	+ .flags = MWAIT2flg(0x40) \| CPUIDLE_FLAG_TLB_FLUSHED,
	806	+ .exit_latency = 280,
	807	+ .target_residency = 800,
	808	+ .enter = &intel_idle,
	809	+ .enter_s2idle = intel_idle_s2idle, },
	810	+ {
	811	+ .name = "C10",
	812	+ .desc = "MWAIT 0x60",
	813	+ .flags = MWAIT2flg(0x60) \| CPUIDLE_FLAG_TLB_FLUSHED,
	814	+ .exit_latency = 680,
	815	+ .target_residency = 2000,
	816	+ .enter = &intel_idle,
	817	+ .enter_s2idle = intel_idle_s2idle, },
	818	+ {
	819	+ .enter = NULL }
	820	+};
	821	+
	822	+static struct cpuidle_state adl_l_cstates[] __initdata = {
	823	+ {
	824	+ .name = "C1",
	825	+ .desc = "MWAIT 0x00",
	826	+ .flags = MWAIT2flg(0x00) \| CPUIDLE_FLAG_UNUSABLE,
	827	+ .exit_latency = 1,
	828	+ .target_residency = 1,
	829	+ .enter = &intel_idle,
	830	+ .enter_s2idle = intel_idle_s2idle, },
	831	+ {
	832	+ .name = "C1E",
	833	+ .desc = "MWAIT 0x01",
	834	+ .flags = MWAIT2flg(0x01) \| CPUIDLE_FLAG_ALWAYS_ENABLE,
	835	+ .exit_latency = 2,
	836	+ .target_residency = 4,
	837	+ .enter = &intel_idle,
	838	+ .enter_s2idle = intel_idle_s2idle, },
	839	+ {
	840	+ .name = "C6",
	841	+ .desc = "MWAIT 0x20",
	842	+ .flags = MWAIT2flg(0x20) \| CPUIDLE_FLAG_TLB_FLUSHED,
	843	+ .exit_latency = 170,
	844	+ .target_residency = 500,
	845	+ .enter = &intel_idle,
	846	+ .enter_s2idle = intel_idle_s2idle, },
	847	+ {
	848	+ .name = "C8",
	849	+ .desc = "MWAIT 0x40",
	850	+ .flags = MWAIT2flg(0x40) \| CPUIDLE_FLAG_TLB_FLUSHED,
	851	+ .exit_latency = 200,
	852	+ .target_residency = 600,
	853	+ .enter = &intel_idle,
	854	+ .enter_s2idle = intel_idle_s2idle, },
	855	+ {
	856	+ .name = "C10",
	857	+ .desc = "MWAIT 0x60",
	858	+ .flags = MWAIT2flg(0x60) \| CPUIDLE_FLAG_TLB_FLUSHED,
	859	+ .exit_latency = 230,
	860	+ .target_residency = 700,
	861	+ .enter = &intel_idle,
	862	+ .enter_s2idle = intel_idle_s2idle, },
	863	+ {
	864	+ .enter = NULL }
	865	+};
	866	+
767	867	/*
768	868	* On Sapphire Rapids Xeon C1 has to be disabled if C1E is enabled, and vice
769	869	* versa. On SPR C1E is enabled only if "C1E promotion" bit is set in
@@ -1147,6 +1247,14 @@ static const struct idle_cpu idle_cpu_icx __initconst = { · 377 unmodified lines
1147	1247	.use_acpi = true,
1148	1248	};
1149	1249
	1250	+static const struct idle_cpu idle_cpu_adl __initconst = {
	1251	+ .state_table = adl_cstates,
	1252	+};
	1253	+
	1254	+static const struct idle_cpu idle_cpu_adl_l __initconst = {
	1255	+ .state_table = adl_l_cstates,
	1256	+};
	1257	+
1150	1258	static const struct idle_cpu idle_cpu_spr __initconst = {
1151	1259	.state_table = spr_cstates,
1152	1260	.disable_promotion_to_c1e = true,
@@ -1215,6 +1323,8 @@ static const struct x86_cpu_id intel_idle_ids[] __initconst = { · 62 unmodified lines
1215	1323	X86_MATCH_INTEL_FAM6_MODEL(SKYLAKE_X, &idle_cpu_skx),
1216	1324	X86_MATCH_INTEL_FAM6_MODEL(ICELAKE_X, &idle_cpu_icx),
1217	1325	X86_MATCH_INTEL_FAM6_MODEL(ICELAKE_D, &idle_cpu_icx),
	1326	+ X86_MATCH_INTEL_FAM6_MODEL(ALDERLAKE, &idle_cpu_adl),
	1327	+ X86_MATCH_INTEL_FAM6_MODEL(ALDERLAKE_L, &idle_cpu_adl_l),
1218	1328	X86_MATCH_INTEL_FAM6_MODEL(SAPPHIRERAPIDS_X, &idle_cpu_spr),
1219	1329	X86_MATCH_INTEL_FAM6_MODEL(XEON_PHI_KNL, &idle_cpu_knl),
1220	1330	X86_MATCH_INTEL_FAM6_MODEL(XEON_PHI_KNM, &idle_cpu_knl),
@@ -1573,6 +1683,25 @@ static void __init skx_idle_state_table_update(void) · 352 unmodified lines
1573	1683	}
1574	1684	}
1575	1685
	1686	+/**
	1687	+ * adl_idle_state_table_update - Adjust AlderLake idle states table.
	1688	+ */
	1689	+static void __init adl_idle_state_table_update(void)
	1690	+{
	1691	+ /* Check if user prefers C1 over C1E. */
	1692	+ if (preferred_states_mask & BIT(1) && !(preferred_states_mask & BIT(2))) {
	1693	+ cpuidle_state_table[0].flags &= ~CPUIDLE_FLAG_UNUSABLE;
	1694	+ cpuidle_state_table[1].flags \|= CPUIDLE_FLAG_UNUSABLE;
	1695	+
	1696	+ /* Disable C1E by clearing the "C1E promotion" bit. */
	1697	+ c1e_promotion = C1E_PROMOTION_DISABLE;
	1698	+ return;
	1699	+ }
	1700	+
	1701	+ /* Make sure C1E is enabled by default */
	1702	+ c1e_promotion = C1E_PROMOTION_ENABLE;
	1703	+}
	1704	+
1576	1705	/**
1577	1706	* spr_idle_state_table_update - Adjust Sapphire Rapids idle states table.
1578	1707	*/
@@ -1642,6 +1771,10 @@ static void __init intel_idle_init_cstates_icpu(struct cpuidle_driver *drv) · 63 unmodified lines
1642	1771	case INTEL_FAM6_SAPPHIRERAPIDS_X:
1643	1772	spr_idle_state_table_update();
1644	1773	break;
	1774	+ case INTEL_FAM6_ALDERLAKE:
	1775	+ case INTEL_FAM6_ALDERLAKE_L:
	1776	+ adl_idle_state_table_update();
	1777	+ break;
1645	1778	}
1646	1779
1647	1780	for (cstate = 0; cstate < CPUIDLE_STATE_MAX; ++cstate) {

drivers/iio/chemical/scd30.h+1−4

@@ -68,10 +68,7 @@ struct scd30_state { · 67 unmodified lines
68	68	scd30_command_t command;
69	69	};
70	70
71		−int scd30_suspend(struct device *dev);
72		−int scd30_resume(struct device *dev);
73		−
74		−static __maybe_unused SIMPLE_DEV_PM_OPS(scd30_pm_ops, scd30_suspend, scd30_resume);
	71	+extern const struct dev_pm_ops scd30_pm_ops;
75	72
76	73	int scd30_probe(struct device dev, int irq, const char name, void *priv, scd30_command_t command);
77	74

drivers/iio/chemical/scd30_core.c+5−5

@@ -517,7 +517,7 @@ static const struct iio_chan_spec scd30_channels[] = { · 516 unmodified lines
517	517	IIO_CHAN_SOFT_TIMESTAMP(3),
518	518	};
519	519
520		−int __maybe_unused scd30_suspend(struct device *dev)
	520	+static int scd30_suspend(struct device *dev)
521	521	{
522	522	struct iio_dev *indio_dev = dev_get_drvdata(dev);
523	523	struct scd30_state *state = iio_priv(indio_dev);
@@ -529,9 +529,8 @@ int __maybe_unused scd30_suspend(struct device *dev) · 5 unmodified lines
529	529
530	530	return regulator_disable(state->vdd);
531	531	}
532		−EXPORT_SYMBOL(scd30_suspend);
533	532
534		−int __maybe_unused scd30_resume(struct device *dev)
	533	+static int scd30_resume(struct device *dev)
535	534	{
536	535	struct iio_dev *indio_dev = dev_get_drvdata(dev);
537	536	struct scd30_state *state = iio_priv(indio_dev);
@@ -543,7 +542,8 @@ int __maybe_unused scd30_resume(struct device *dev) · 5 unmodified lines
543	542
544	543	return scd30_command_write(state, CMD_START_MEAS, state->pressure_comp);
545	544	}
546		−EXPORT_SYMBOL(scd30_resume);
	545	+
	546	+EXPORT_NS_SIMPLE_DEV_PM_OPS(scd30_pm_ops, scd30_suspend, scd30_resume, IIO_SCD30);
547	547
548	548	static void scd30_stop_meas(void *data)
549	549	{
@@ -759,7 +759,7 @@ int scd30_probe(struct device dev, int irq, const char name, void *priv, · 209 unmodified lines
759	759
760	760	return devm_iio_device_register(dev, indio_dev);
761	761	}
762		−EXPORT_SYMBOL(scd30_probe);
	762	+EXPORT_SYMBOL_NS(scd30_probe, IIO_SCD30);
763	763
764	764	MODULE_AUTHOR("Tomasz Duszynski <tomasz.duszynski@octakon.com>");
765	765	MODULE_DESCRIPTION("Sensirion SCD30 carbon dioxide sensor core driver");

drivers/iio/chemical/scd30_i2c.c+2−1

@@ -128,7 +128,7 @@ static struct i2c_driver scd30_i2c_driver = { · 127 unmodified lines
128	128	.driver = {
129	129	.name = KBUILD_MODNAME,
130	130	.of_match_table = scd30_i2c_of_match,
131		− .pm = &scd30_pm_ops,
	131	+ .pm = pm_sleep_ptr(&scd30_pm_ops),
132	132	},
133	133	.probe_new = scd30_i2c_probe,
134	134	};
@@ -137,3 +137,4 @@ module_i2c_driver(scd30_i2c_driver); · 2 unmodified lines
137	137	MODULE_AUTHOR("Tomasz Duszynski <tomasz.duszynski@octakon.com>");
138	138	MODULE_DESCRIPTION("Sensirion SCD30 carbon dioxide sensor i2c driver");
139	139	MODULE_LICENSE("GPL v2");
	140	+MODULE_IMPORT_NS(IIO_SCD30);

drivers/iio/chemical/scd30_serial.c+2−1

@@ -252,7 +252,7 @@ static struct serdev_device_driver scd30_serdev_driver = { · 251 unmodified lines
252	252	.driver = {
253	253	.name = KBUILD_MODNAME,
254	254	.of_match_table = scd30_serdev_of_match,
255		− .pm = &scd30_pm_ops,
	255	+ .pm = pm_sleep_ptr(&scd30_pm_ops),
256	256	},
257	257	.probe = scd30_serdev_probe,
258	258	};
@@ -261,3 +261,4 @@ module_serdev_device_driver(scd30_serdev_driver); · 2 unmodified lines
261	261	MODULE_AUTHOR("Tomasz Duszynski <tomasz.duszynski@octakon.com>");
262	262	MODULE_DESCRIPTION("Sensirion SCD30 carbon dioxide sensor serial driver");
263	263	MODULE_LICENSE("GPL v2");
	264	+MODULE_IMPORT_NS(IIO_SCD30);

drivers/opp/of.c+3−3

@@ -1448,7 +1448,7 @@ EXPORT_SYMBOL_GPL(dev_pm_opp_get_of_node); · 1447 unmodified lines
1448	1448	* Returns 0 on success or a proper -EINVAL value in case of error.
1449	1449	*/
1450	1450	static int __maybe_unused
1451		−_get_dt_power(unsigned long mW, unsigned long kHz, struct device *dev)
	1451	+_get_dt_power(struct device dev, unsigned long mW, unsigned long *kHz)
1452	1452	{
1453	1453	struct dev_pm_opp *opp;
1454	1454	unsigned long opp_freq, opp_power;
@@ -1482,8 +1482,8 @@ _get_dt_power(unsigned long mW, unsigned long kHz, struct device *dev) · 27 unmodified lines
1482	1482	* Returns -EINVAL if the power calculation failed because of missing
1483	1483	* parameters, 0 otherwise.
1484	1484	*/
1485		−static int __maybe_unused _get_power(unsigned long mW, unsigned long kHz,
1486		− struct device *dev)
	1485	+static int __maybe_unused _get_power(struct device dev, unsigned long mW,
	1486	+ unsigned long *kHz)
1487	1487	{
1488	1488	struct dev_pm_opp *opp;
1489	1489	struct device_node *np;

drivers/powercap/dtpm_cpu.c+1−1

@@ -211,7 +211,7 @@ static int __dtpm_cpu_setup(int cpu, struct dtpm *parent) · 210 unmodified lines
211	211	return 0;
212	212
213	213	pd = em_cpu_get(cpu);
214		− if (!pd)
	214	+ if (!pd \|\| em_is_artificial(pd))
215	215	return -EINVAL;
216	216
217	217	dtpm_cpu = kzalloc(sizeof(*dtpm_cpu), GFP_KERNEL);

drivers/powercap/intel_rapl_common.c+3−1

@@ -1010,7 +1010,7 @@ static u64 rapl_compute_time_window_atom(struct rapl_package *rp, u64 value, · 1009 unmodified lines
1010	1010	* where time_unit is default to 1 sec. Never 0.
1011	1011	*/
1012	1012	if (!to_raw)
1013		− return (value) ? value *= rp->time_unit : rp->time_unit;
	1013	+ return (value) ? value * rp->time_unit : rp->time_unit;
1014	1014
1015	1015	value = div64_u64(value, rp->time_unit);
1016	1016
@@ -1107,6 +1107,8 @@ static const struct x86_cpu_id rapl_ids[] __initconst = { · 90 unmodified lines
1107	1107	X86_MATCH_INTEL_FAM6_MODEL(ROCKETLAKE, &rapl_defaults_core),
1108	1108	X86_MATCH_INTEL_FAM6_MODEL(ALDERLAKE, &rapl_defaults_core),
1109	1109	X86_MATCH_INTEL_FAM6_MODEL(ALDERLAKE_L, &rapl_defaults_core),
	1110	+ X86_MATCH_INTEL_FAM6_MODEL(ALDERLAKE_N, &rapl_defaults_core),
	1111	+ X86_MATCH_INTEL_FAM6_MODEL(RAPTORLAKE, &rapl_defaults_core),
1110	1112	X86_MATCH_INTEL_FAM6_MODEL(SAPPHIRERAPIDS_X, &rapl_defaults_spr_server),
1111	1113	X86_MATCH_INTEL_FAM6_MODEL(LAKEFIELD, &rapl_defaults_core),
1112	1114

drivers/powercap/intel_rapl_msr.c+1−0

@@ -140,6 +140,7 @@ static const struct x86_cpu_id pl4_support_ids[] = { · 139 unmodified lines
140	140	{ X86_VENDOR_INTEL, 6, INTEL_FAM6_TIGERLAKE_L, X86_FEATURE_ANY },
141	141	{ X86_VENDOR_INTEL, 6, INTEL_FAM6_ALDERLAKE, X86_FEATURE_ANY },
142	142	{ X86_VENDOR_INTEL, 6, INTEL_FAM6_ALDERLAKE_L, X86_FEATURE_ANY },
	143	+ { X86_VENDOR_INTEL, 6, INTEL_FAM6_RAPTORLAKE, X86_FEATURE_ANY },
143	144	{}
144	145	};
145	146

drivers/soc/rockchip/pm_domains.c+118−0

@@ -8,6 +8,7 @@ · 7 unmodified lines
8	8	#include <linux/io.h>
9	9	#include <linux/iopoll.h>
10	10	#include <linux/err.h>
	11	+#include <linux/mutex.h>
11	12	#include <linux/pm_clock.h>
12	13	#include <linux/pm_domain.h>
13	14	#include <linux/of_address.h>
@@ -16,6 +17,7 @@ · 2 unmodified lines
16	17	#include <linux/clk.h>
17	18	#include <linux/regmap.h>
18	19	#include <linux/mfd/syscon.h>
	20	+#include <soc/rockchip/pm_domains.h>
19	21	#include <dt-bindings/power/px30-power.h>
20	22	#include <dt-bindings/power/rk3036-power.h>
21	23	#include <dt-bindings/power/rk3066-power.h>
@@ -139,6 +141,109 @@ struct rockchip_pmu { · 117 unmodified lines
139	141	#define DOMAIN_RK3568(name, pwr, req, wakeup) \
140	142	DOMAIN_M(name, pwr, pwr, req, req, req, wakeup)
141	143
	144	+/*
	145	+ * Dynamic Memory Controller may need to coordinate with us -- see
	146	+ * rockchip_pmu_block().
	147	+ *
	148	+ * dmc_pmu_mutex protects registration-time races, so DMC driver doesn't try to
	149	+ * block() while we're initializing the PMU.
	150	+ */
	151	+static DEFINE_MUTEX(dmc_pmu_mutex);
	152	+static struct rockchip_pmu *dmc_pmu;
	153	+
	154	+/*
	155	+ * Block PMU transitions and make sure they don't interfere with ARM Trusted
	156	+ * Firmware operations. There are two conflicts, noted in the comments below.
	157	+ *
	158	+ * Caller must unblock PMU transitions via rockchip_pmu_unblock().
	159	+ */
	160	+int rockchip_pmu_block(void)
	161	+{
	162	+ struct rockchip_pmu *pmu;
	163	+ struct generic_pm_domain *genpd;
	164	+ struct rockchip_pm_domain *pd;
	165	+ int i, ret;
	166	+
	167	+ mutex_lock(&dmc_pmu_mutex);
	168	+
	169	+ /* No PMU (yet)? Then we just block rockchip_pmu_probe(). */
	170	+ if (!dmc_pmu)
	171	+ return 0;
	172	+ pmu = dmc_pmu;
	173	+
	174	+ /*
	175	+ * mutex blocks all idle transitions: we can't touch the
	176	+ * PMU_BUS_IDLE_REQ (our ".idle_offset") register while ARM Trusted
	177	+ * Firmware might be using it.
	178	+ */
	179	+ mutex_lock(&pmu->mutex);
	180	+
	181	+ /*
	182	+ * Power domain clocks: Per Rockchip, we must keep certain clocks
	183	+ * enabled for the duration of power-domain transitions. Most
	184	+ * transitions are handled by this driver, but some cases (in
	185	+ * particular, DRAM DVFS / memory-controller idle) must be handled by
	186	+ * firmware. Firmware can handle most clock management via a special
	187	+ * "ungate" register (PMU_CRU_GATEDIS_CON0), but unfortunately, this
	188	+ * doesn't handle PLLs. We can assist this transition by doing the
	189	+ * clock management on behalf of firmware.
	190	+ */
	191	+ for (i = 0; i < pmu->genpd_data.num_domains; i++) {
	192	+ genpd = pmu->genpd_data.domains[i];
	193	+ if (genpd) {
	194	+ pd = to_rockchip_pd(genpd);
	195	+ ret = clk_bulk_enable(pd->num_clks, pd->clks);
	196	+ if (ret < 0) {
	197	+ dev_err(pmu->dev,
	198	+ "failed to enable clks for domain '%s': %d\n",
	199	+ genpd->name, ret);
	200	+ goto err;
	201	+ }
	202	+ }
	203	+ }
	204	+
	205	+ return 0;
	206	+
	207	+err:
	208	+ for (i = i - 1; i >= 0; i--) {
	209	+ genpd = pmu->genpd_data.domains[i];
	210	+ if (genpd) {
	211	+ pd = to_rockchip_pd(genpd);
	212	+ clk_bulk_disable(pd->num_clks, pd->clks);
	213	+ }
	214	+ }
	215	+ mutex_unlock(&pmu->mutex);
	216	+ mutex_unlock(&dmc_pmu_mutex);
	217	+
	218	+ return ret;
	219	+}
	220	+EXPORT_SYMBOL_GPL(rockchip_pmu_block);
	221	+
	222	+/* Unblock PMU transitions. */
	223	+void rockchip_pmu_unblock(void)
	224	+{
	225	+ struct rockchip_pmu *pmu;
	226	+ struct generic_pm_domain *genpd;
	227	+ struct rockchip_pm_domain *pd;
	228	+ int i;
	229	+
	230	+ if (dmc_pmu) {
	231	+ pmu = dmc_pmu;
	232	+ for (i = 0; i < pmu->genpd_data.num_domains; i++) {
	233	+ genpd = pmu->genpd_data.domains[i];
	234	+ if (genpd) {
	235	+ pd = to_rockchip_pd(genpd);
	236	+ clk_bulk_disable(pd->num_clks, pd->clks);
	237	+ }
	238	+ }
	239	+
	240	+ mutex_unlock(&pmu->mutex);
	241	+ }
	242	+
	243	+ mutex_unlock(&dmc_pmu_mutex);
	244	+}
	245	+EXPORT_SYMBOL_GPL(rockchip_pmu_unblock);
	246	+
142	247	static bool rockchip_pmu_domain_is_idle(struct rockchip_pm_domain *pd)
143	248	{
144	249	struct rockchip_pmu *pmu = pd->pmu;
@@ -690,6 +795,12 @@ static int rockchip_pm_domain_probe(struct platform_device *pdev) · 545 unmodified lines
690	795
691	796	error = -ENODEV;
692	797
	798	+ /*
	799	+ * Prevent any rockchip_pmu_block() from racing with the remainder of
	800	+ * setup (clocks, register initialization).
	801	+ */
	802	+ mutex_lock(&dmc_pmu_mutex);
	803	+
693	804	for_each_available_child_of_node(np, node) {
694	805	error = rockchip_pm_add_one_domain(pmu, node);
695	806	if (error) {
@@ -719,10 +830,17 @@ static int rockchip_pm_domain_probe(struct platform_device *pdev) · 23 unmodified lines
719	830	goto err_out;
720	831	}
721	832
	833	+ /* We only expect one PMU. */
	834	+ if (!WARN_ON_ONCE(dmc_pmu))
	835	+ dmc_pmu = pmu;
	836	+
	837	+ mutex_unlock(&dmc_pmu_mutex);
	838	+
722	839	return 0;
723	840
724	841	err_out:
725	842	rockchip_pm_domain_cleanup(pmu);
	843	+ mutex_unlock(&dmc_pmu_mutex);
726	844	return error;
727	845	}
728	846

drivers/thermal/cpufreq_cooling.c+1−1

@@ -328,7 +328,7 @@ static inline bool em_is_sane(struct cpufreq_cooling_device *cpufreq_cdev, · 327 unmodified lines
328	328	struct cpufreq_policy *policy;
329	329	unsigned int nr_levels;
330	330
331		− if (!em)
	331	+ if (!em \|\| em_is_artificial(em))
332	332	return false;
333	333
334	334	policy = cpufreq_cdev->policy;

drivers/thermal/devfreq_cooling.c+5−3

@@ -358,6 +358,7 @@ of_devfreq_cooling_register_power(struct device_node np, struct devfreq df, · 357 unmodified lines
358	358	struct thermal_cooling_device *cdev;
359	359	struct device *dev = df->dev.parent;
360	360	struct devfreq_cooling_device *dfc;
	361	+ struct em_perf_domain *em;
361	362	char *name;
362	363	int err, num_opps;
363	364
@@ -367,8 +368,9 @@ of_devfreq_cooling_register_power(struct device_node np, struct devfreq df, · 3 unmodified lines
367	368
368	369	dfc->devfreq = df;
369	370
370		− dfc->em_pd = em_pd_get(dev);
371		− if (dfc->em_pd) {
	371	+ em = em_pd_get(dev);
	372	+ if (em && !em_is_artificial(em)) {
	373	+ dfc->em_pd = em;
372	374	devfreq_cooling_ops.get_requested_power =
373	375	devfreq_cooling_get_requested_power;
374	376	devfreq_cooling_ops.state2power = devfreq_cooling_state2power;
@@ -379,7 +381,7 @@ of_devfreq_cooling_register_power(struct device_node np, struct devfreq df, · 4 unmodified lines
379	381	num_opps = em_pd_nr_perf_states(dfc->em_pd);
380	382	} else {
381	383	/* Backward compatibility for drivers which do not use IPA */
382		− dev_dbg(dev, "missing EM for cooling device\n");
	384	+ dev_dbg(dev, "missing proper EM for cooling device\n");
383	385
384	386	num_opps = dev_pm_opp_get_opp_count(dev);
385	387

include/acpi/cppc_acpi.h+5−0

@@ -141,6 +141,7 @@ extern int cppc_set_perf(int cpu, struct cppc_perf_ctrls *perf_ctrls); · 140 unmodified lines
141	141	extern int cppc_set_enable(int cpu, bool enable);
142	142	extern int cppc_get_perf_caps(int cpu, struct cppc_perf_caps *caps);
143	143	extern bool acpi_cpc_valid(void);
	144	+extern bool cppc_allow_fast_switch(void);
144	145	extern int acpi_get_psd_map(unsigned int cpu, struct cppc_cpudata *cpu_data);
145	146	extern unsigned int cppc_get_transition_latency(int cpu);
146	147	extern bool cpc_ffh_supported(void);
@@ -175,6 +176,10 @@ static inline bool acpi_cpc_valid(void) · 28 unmodified lines
175	176	{
176	177	return false;
177	178	}
	179	+static inline bool cppc_allow_fast_switch(void)
	180	+{
	181	+ return false;
	182	+}
178	183	static inline unsigned int cppc_get_transition_latency(int cpu)
179	184	{
180	185	return CPUFREQ_ETERNAL;

include/linux/acpi.h+2−0

@@ -574,6 +574,7 @@ acpi_status acpi_run_osc(acpi_handle handle, struct acpi_osc_context *context); · 573 unmodified lines
574	574	#define OSC_SB_OSLPI_SUPPORT 0x00000100
575	575	#define OSC_SB_CPC_DIVERSE_HIGH_SUPPORT 0x00001000
576	576	#define OSC_SB_GENERIC_INITIATOR_SUPPORT 0x00002000
	577	+#define OSC_SB_CPC_FLEXIBLE_ADR_SPACE 0x00004000
577	578	#define OSC_SB_NATIVE_USB4_SUPPORT 0x00040000
578	579	#define OSC_SB_PRM_SUPPORT 0x00200000
579	580
@@ -581,6 +582,7 @@ extern bool osc_sb_apei_support_acked; · 1 unmodified line
581	582	extern bool osc_pc_lpi_support_confirmed;
582	583	extern bool osc_sb_native_usb4_support_confirmed;
583	584	extern bool osc_sb_cppc_not_supported;
	585	+extern bool osc_cpc_flexible_adr_space_confirmed;
584	586
585	587	/* USB4 Capabilities */
586	588	#define OSC_USB_USB3_TUNNELING 0x00000001

include/linux/devfreq.h+15−2

@@ -38,6 +38,7 @@ enum devfreq_timer { · 37 unmodified lines
38	38
39	39	struct devfreq;
40	40	struct devfreq_governor;
	41	+struct devfreq_cpu_data;
41	42	struct thermal_cooling_device;
42	43
43	44	/**
@@ -288,6 +289,11 @@ struct devfreq_simple_ondemand_data { · 244 unmodified lines
288	289	#endif
289	290
290	291	#if IS_ENABLED(CONFIG_DEVFREQ_GOV_PASSIVE)
	292	+enum devfreq_parent_dev_type {
	293	+ DEVFREQ_PARENT_DEV,
	294	+ CPUFREQ_PARENT_DEV,
	295	+};
	296	+
291	297	/**
292	298	* struct devfreq_passive_data - ``void *data`` fed to struct devfreq
293	299	* and devfreq_add_device
@@ -299,8 +305,11 @@ struct devfreq_simple_ondemand_data { · 5 unmodified lines
299	305	* using governors except for passive governor.
300	306	* If the devfreq device has the specific method to decide
301	307	* the next frequency, should use this callback.
302		− * @this: the devfreq instance of own device.
303		− * @nb: the notifier block for DEVFREQ_TRANSITION_NOTIFIER list
	308	+ * @parent_type: the parent type of the device.
	309	+ * @this: the devfreq instance of own device.
	310	+ * @nb: the notifier block for DEVFREQ_TRANSITION_NOTIFIER or
	311	+ * CPUFREQ_TRANSITION_NOTIFIER list.
	312	+ * @cpu_data_list: the list of cpu frequency data for all cpufreq_policy.
304	313	*
305	314	* The devfreq_passive_data have to set the devfreq instance of parent
306	315	* device with governors except for the passive governor. But, don't need to
@@ -314,9 +323,13 @@ struct devfreq_passive_data { · 7 unmodified lines
314	323	/* Optional callback to decide the next frequency of passvice device */
315	324	int (get_target_freq)(struct devfreq this, unsigned long *freq);
316	325
	326	+ /* Should set the type of parent device */
	327	+ enum devfreq_parent_dev_type parent_type;
	328	+
317	329	/* For passive governor's internal use. Don't need to set them */
318	330	struct devfreq *this;
319	331	struct notifier_block nb;
	332	+ struct list_head cpu_data_list;
320	333	};
321	334	#endif
322	335

include/linux/energy_model.h+31−4

@@ -67,11 +67,16 @@ struct em_perf_domain { · 66 unmodified lines
67	67	*
68	68	* EM_PERF_DOMAIN_SKIP_INEFFICIENCIES: Skip inefficient states when estimating
69	69	* energy consumption.
	70	+ *
	71	+ * EM_PERF_DOMAIN_ARTIFICIAL: The power values are artificial and might be
	72	+ * created by platform missing real power information
70	73	*/
71	74	#define EM_PERF_DOMAIN_MILLIWATTS BIT(0)
72	75	#define EM_PERF_DOMAIN_SKIP_INEFFICIENCIES BIT(1)
	76	+#define EM_PERF_DOMAIN_ARTIFICIAL BIT(2)
73	77
74	78	#define em_span_cpus(em) (to_cpumask((em)->cpus))
	79	+#define em_is_artificial(em) ((em)->flags & EM_PERF_DOMAIN_ARTIFICIAL)
75	80
76	81	#ifdef CONFIG_ENERGY_MODEL
77	82	#define EM_MAX_POWER 0xFFFF
@@ -96,11 +101,11 @@ struct em_data_callback { · 18 unmodified lines
96	101	/**
97	102	* active_power() - Provide power at the next performance state of
98	103	* a device
	104	+ * @dev : Device for which we do this operation (can be a CPU)
99	105	* @power : Active power at the performance state
100	106	* (modified)
101	107	* @freq : Frequency at the performance state in kHz
102	108	* (modified)
103		− * @dev : Device for which we do this operation (can be a CPU)
104	109	*
105	110	* active_power() must find the lowest performance state of 'dev' above
106	111	* 'freq' and update 'power' and 'freq' to the matching active power
@@ -112,11 +117,32 @@ struct em_data_callback { · 5 unmodified lines
112	117	*
113	118	* Return 0 on success.
114	119	*/
115		− int (active_power)(unsigned long power, unsigned long *freq,
116		− struct device *dev);
	120	+ int (active_power)(struct device dev, unsigned long *power,
	121	+ unsigned long *freq);
	122	+
	123	+ /**
	124	+ * get_cost() - Provide the cost at the given performance state of
	125	+ * a device
	126	+ * @dev : Device for which we do this operation (can be a CPU)
	127	+ * @freq : Frequency at the performance state in kHz
	128	+ * @cost : The cost value for the performance state
	129	+ * (modified)
	130	+ *
	131	+ * In case of CPUs, the cost is the one of a single CPU in the domain.
	132	+ * It is expected to fit in the [0, EM_MAX_POWER] range due to internal
	133	+ * usage in EAS calculation.
	134	+ *
	135	+ * Return 0 on success, or appropriate error value in case of failure.
	136	+ */
	137	+ int (get_cost)(struct device dev, unsigned long freq,
	138	+ unsigned long *cost);
117	139	};
118		−#define EM_DATA_CB(_active_power_cb) { .active_power = &_active_power_cb }
119	140	#define EM_SET_ACTIVE_POWER_CB(em_cb, cb) ((em_cb).active_power = cb)
	141	+#define EM_ADV_DATA_CB(_active_power_cb, _cost_cb) \
	142	+ { .active_power = _active_power_cb, \
	143	+ .get_cost = _cost_cb }
	144	+#define EM_DATA_CB(_active_power_cb) \
	145	+ EM_ADV_DATA_CB(_active_power_cb, NULL)
120	146
121	147	struct em_perf_domain *em_cpu_get(int cpu);
122	148	struct em_perf_domain em_pd_get(struct device dev);
@@ -264,6 +290,7 @@ static inline int em_pd_nr_perf_states(struct em_perf_domain *pd) · 141 unmodified lines
264	290
265	291	#else
266	292	struct em_data_callback {};
	293	+#define EM_ADV_DATA_CB(_active_power_cb, _cost_cb) { }
267	294	#define EM_DATA_CB(_active_power_cb) { }
268	295	#define EM_SET_ACTIVE_POWER_CB(em_cb, cb) do { } while (0)
269	296

include/linux/pm.h+9−5

@@ -368,13 +368,13 @@ const struct dev_pm_ops name = { \ · 367 unmodified lines
368	368
369	369	#ifdef CONFIG_PM
370	370	#define _EXPORT_DEV_PM_OPS(name, suspend_fn, resume_fn, runtime_suspend_fn, \
371		− runtime_resume_fn, idle_fn, sec) \
	371	+ runtime_resume_fn, idle_fn, sec, ns) \
372	372	_DEFINE_DEV_PM_OPS(name, suspend_fn, resume_fn, runtime_suspend_fn, \
373	373	runtime_resume_fn, idle_fn); \
374		− _EXPORT_SYMBOL(name, sec)
	374	+ __EXPORT_SYMBOL(name, sec, ns)
375	375	#else
376	376	#define _EXPORT_DEV_PM_OPS(name, suspend_fn, resume_fn, runtime_suspend_fn, \
377		− runtime_resume_fn, idle_fn, sec) \
	377	+ runtime_resume_fn, idle_fn, sec, ns) \
378	378	static __maybe_unused _DEFINE_DEV_PM_OPS(__static_##name, suspend_fn, \
379	379	resume_fn, runtime_suspend_fn, \
380	380	runtime_resume_fn, idle_fn)
@@ -391,9 +391,13 @@ static __maybe_unused _DEFINE_DEV_PM_OPS(__static_##name, suspend_fn, \ · 10 unmodified lines
391	391	_DEFINE_DEV_PM_OPS(name, suspend_fn, resume_fn, NULL, NULL, NULL)
392	392
393	393	#define EXPORT_SIMPLE_DEV_PM_OPS(name, suspend_fn, resume_fn) \
394		− _EXPORT_DEV_PM_OPS(name, suspend_fn, resume_fn, NULL, NULL, NULL, "")
	394	+ _EXPORT_DEV_PM_OPS(name, suspend_fn, resume_fn, NULL, NULL, NULL, "", "")
395	395	#define EXPORT_GPL_SIMPLE_DEV_PM_OPS(name, suspend_fn, resume_fn) \
396		− _EXPORT_DEV_PM_OPS(name, suspend_fn, resume_fn, NULL, NULL, NULL, "_gpl")
	396	+ _EXPORT_DEV_PM_OPS(name, suspend_fn, resume_fn, NULL, NULL, NULL, "_gpl", "")
	397	+#define EXPORT_NS_SIMPLE_DEV_PM_OPS(name, suspend_fn, resume_fn, ns) \
	398	+ _EXPORT_DEV_PM_OPS(name, suspend_fn, resume_fn, NULL, NULL, NULL, "", #ns)
	399	+#define EXPORT_NS_GPL_SIMPLE_DEV_PM_OPS(name, suspend_fn, resume_fn, ns) \
	400	+ _EXPORT_DEV_PM_OPS(name, suspend_fn, resume_fn, NULL, NULL, NULL, "_gpl", #ns)
397	401
398	402	/* Deprecated. Use DEFINE_SIMPLE_DEV_PM_OPS() instead. */
399	403	#define SIMPLE_DEV_PM_OPS(name, suspend_fn, resume_fn) \

include/linux/pm_domain.h+14−10

@@ -91,6 +91,14 @@ struct gpd_dev_ops { · 90 unmodified lines
91	91	int (stop)(struct device dev);
92	92	};
93	93
	94	+struct genpd_governor_data {
	95	+ s64 max_off_time_ns;
	96	+ bool max_off_time_changed;
	97	+ ktime_t next_wakeup;
	98	+ bool cached_power_down_ok;
	99	+ bool cached_power_down_state_idx;
	100	+};
	101	+
94	102	struct genpd_power_state {
95	103	s64 power_off_latency_ns;
96	104	s64 power_on_latency_ns;
@@ -98,7 +106,7 @@ struct genpd_power_state { · 1 unmodified line
98	106	u64 usage;
99	107	u64 rejected;
100	108	struct fwnode_handle *fwnode;
101		− ktime_t idle_time;
	109	+ u64 idle_time;
102	110	void *data;
103	111	};
104	112
@@ -114,6 +122,7 @@ struct generic_pm_domain { · 9 unmodified lines
114	122	struct list_head child_links; /* Links with PM domain as a child */
115	123	struct list_head dev_list; /* List of devices */
116	124	struct dev_power_governor *gov;
	125	+ struct genpd_governor_data gd; / Data used by a genpd governor. */
117	126	struct work_struct power_off_work;
118	127	struct fwnode_handle provider; / Identity of the domain provider */
119	128	bool has_provider;
@@ -134,11 +143,6 @@ struct generic_pm_domain { · 14 unmodified lines
134	143	int (set_performance_state)(struct generic_pm_domain genpd,
135	144	unsigned int state);
136	145	struct gpd_dev_ops dev_ops;
137		− s64 max_off_time_ns; /* Maximum allowed "suspended" time. */
138		− ktime_t next_wakeup; /* Maintained by the domain governor */
139		− bool max_off_time_changed;
140		− bool cached_power_down_ok;
141		− bool cached_power_down_state_idx;
142	146	int (attach_dev)(struct generic_pm_domain domain,
143	147	struct device *dev);
144	148	void (detach_dev)(struct generic_pm_domain domain,
@@ -149,8 +153,8 @@ struct generic_pm_domain { · 4 unmodified lines
149	153	unsigned int state_count);
150	154	unsigned int state_count; /* number of states */
151	155	unsigned int state_idx; /* state that genpd will go to when off */
152		− ktime_t on_time;
153		− ktime_t accounting_time;
	156	+ u64 on_time;
	157	+ u64 accounting_time;
154	158	const struct genpd_lock_ops *lock_ops;
155	159	union {
156	160	struct mutex mlock;
@@ -182,6 +186,7 @@ struct gpd_timing_data { · 25 unmodified lines
182	186	s64 suspend_latency_ns;
183	187	s64 resume_latency_ns;
184	188	s64 effective_constraint_ns;
	189	+ ktime_t next_wakeup;
185	190	bool constraint_changed;
186	191	bool cached_suspend_ok;
187	192	};
@@ -193,14 +198,13 @@ struct pm_domain_data { · 5 unmodified lines
193	198
194	199	struct generic_pm_domain_data {
195	200	struct pm_domain_data base;
196		− struct gpd_timing_data td;
	201	+ struct gpd_timing_data *td;
197	202	struct notifier_block nb;
198	203	struct notifier_block *power_nb;
199	204	int cpu;
200	205	unsigned int performance_state;
201	206	unsigned int default_pstate;
202	207	unsigned int rpm_pstate;
203		− ktime_t next_wakeup;
204	208	void *data;
205	209	};
206	210

include/linux/pm_runtime.h+8−2

@@ -41,10 +41,16 @@ · 40 unmodified lines
41	41
42	42	#define EXPORT_RUNTIME_DEV_PM_OPS(name, suspend_fn, resume_fn, idle_fn) \
43	43	_EXPORT_DEV_PM_OPS(name, pm_runtime_force_suspend, pm_runtime_force_resume, \
44		− suspend_fn, resume_fn, idle_fn, "")
	44	+ suspend_fn, resume_fn, idle_fn, "", "")
45	45	#define EXPORT_GPL_RUNTIME_DEV_PM_OPS(name, suspend_fn, resume_fn, idle_fn) \
46	46	_EXPORT_DEV_PM_OPS(name, pm_runtime_force_suspend, pm_runtime_force_resume, \
47		− suspend_fn, resume_fn, idle_fn, "_gpl")
	47	+ suspend_fn, resume_fn, idle_fn, "_gpl", "")
	48	+#define EXPORT_NS_RUNTIME_DEV_PM_OPS(name, suspend_fn, resume_fn, idle_fn, ns) \
	49	+ _EXPORT_DEV_PM_OPS(name, pm_runtime_force_suspend, pm_runtime_force_resume, \
	50	+ suspend_fn, resume_fn, idle_fn, "", #ns)
	51	+#define EXPORT_NS_GPL_RUNTIME_DEV_PM_OPS(name, suspend_fn, resume_fn, idle_fn, ns) \
	52	+ _EXPORT_DEV_PM_OPS(name, pm_runtime_force_suspend, pm_runtime_force_resume, \
	53	+ suspend_fn, resume_fn, idle_fn, "_gpl", #ns)
48	54
49	55	#ifdef CONFIG_PM
50	56	extern struct workqueue_struct *pm_wq;

include/linux/suspend.h+39−5

@@ -542,22 +542,56 @@ static inline void unlock_system_sleep(void) {} · 541 unmodified lines
542	542	#ifdef CONFIG_PM_SLEEP_DEBUG
543	543	extern bool pm_print_times_enabled;
544	544	extern bool pm_debug_messages_on;
545		−extern __printf(2, 3) void __pm_pr_dbg(bool defer, const char *fmt, ...);
	545	+static inline int pm_dyn_debug_messages_on(void)
	546	+{
	547	+#ifdef CONFIG_DYNAMIC_DEBUG
	548	+ return 1;
	549	+#else
	550	+ return 0;
	551	+#endif
	552	+}
	553	+#ifndef pr_fmt
	554	+#define pr_fmt(fmt) "PM: " fmt
	555	+#endif
	556	+#define __pm_pr_dbg(fmt, ...) \
	557	+ do { \
	558	+ if (pm_debug_messages_on) \
	559	+ printk(KERN_DEBUG pr_fmt(fmt), ##__VA_ARGS__); \
	560	+ else if (pm_dyn_debug_messages_on()) \
	561	+ pr_debug(fmt, ##__VA_ARGS__); \
	562	+ } while (0)
	563	+#define __pm_deferred_pr_dbg(fmt, ...) \
	564	+ do { \
	565	+ if (pm_debug_messages_on) \
	566	+ printk_deferred(KERN_DEBUG pr_fmt(fmt), ##__VA_ARGS__); \
	567	+ } while (0)
546	568	#else
547	569	#define pm_print_times_enabled (false)
548	570	#define pm_debug_messages_on (false)
549	571
550	572	#include <linux/printk.h>
551	573
552		−#define __pm_pr_dbg(defer, fmt, ...) \
553		− no_printk(KERN_DEBUG fmt, ##__VA_ARGS__)
	574	+#define __pm_pr_dbg(fmt, ...) \
	575	+ no_printk(KERN_DEBUG pr_fmt(fmt), ##__VA_ARGS__)
	576	+#define __pm_deferred_pr_dbg(fmt, ...) \
	577	+ no_printk(KERN_DEBUG pr_fmt(fmt), ##__VA_ARGS__)
554	578	#endif
555	579
	580	+/**
	581	+ * pm_pr_dbg - print pm sleep debug messages
	582	+ *
	583	+ * If pm_debug_messages_on is enabled, print message.
	584	+ * If pm_debug_messages_on is disabled and CONFIG_DYNAMIC_DEBUG is enabled,
	585	+ * print message only from instances explicitly enabled on dynamic debug's
	586	+ * control.
	587	+ * If pm_debug_messages_on is disabled and CONFIG_DYNAMIC_DEBUG is disabled,
	588	+ * don't print message.
	589	+ */
556	590	#define pm_pr_dbg(fmt, ...) \
557		− __pm_pr_dbg(false, fmt, ##__VA_ARGS__)
	591	+ __pm_pr_dbg(fmt, ##__VA_ARGS__)
558	592
559	593	#define pm_deferred_pr_dbg(fmt, ...) \
560		− __pm_pr_dbg(true, fmt, ##__VA_ARGS__)
	594	+ __pm_deferred_pr_dbg(fmt, ##__VA_ARGS__)
561	595
562	596	#ifdef CONFIG_PM_AUTOSLEEP
563	597

include/soc/rockchip/pm_domains.h+25−0

@@ -0,0 +1,25 @@
	1	+/* SPDX-License-Identifier: GPL-2.0 */
	2	+/*
	3	+ * Copyright 2022, The Chromium OS Authors. All rights reserved.
	4	+ */
	5	+
	6	+#ifndef __SOC_ROCKCHIP_PM_DOMAINS_H__
	7	+#define __SOC_ROCKCHIP_PM_DOMAINS_H__
	8	+
	9	+#ifdef CONFIG_ROCKCHIP_PM_DOMAINS
	10	+
	11	+int rockchip_pmu_block(void);
	12	+void rockchip_pmu_unblock(void);
	13	+
	14	+#else /* CONFIG_ROCKCHIP_PM_DOMAINS */
	15	+
	16	+static inline int rockchip_pmu_block(void)
	17	+{
	18	+ return 0;
	19	+}
	20	+
	21	+static inline void rockchip_pmu_unblock(void) { }
	22	+
	23	+#endif /* CONFIG_ROCKCHIP_PM_DOMAINS */
	24	+
	25	+#endif /* __SOC_ROCKCHIP_PM_DOMAINS_H__ */

kernel/power/Makefile+5−1

@@ -1,6 +1,10 @@
1	1	# SPDX-License-Identifier: GPL-2.0
2	2
3		−ccflags-$(CONFIG_PM_DEBUG) := -DDEBUG
	3	+ifeq ($(CONFIG_DYNAMIC_DEBUG), y)
	4	+CFLAGS_swap.o := -DDEBUG
	5	+CFLAGS_snapshot.o := -DDEBUG
	6	+CFLAGS_energy_model.o := -DDEBUG
	7	+endif
4	8
5	9	KASAN_SANITIZE_snapshot.o := n
6	10

kernel/power/energy_model.c+36−29

@@ -54,28 +54,15 @@ static int em_debug_cpus_show(struct seq_file s, void unused) · 53 unmodified lines
54	54	}
55	55	DEFINE_SHOW_ATTRIBUTE(em_debug_cpus);
56	56
57		−static int em_debug_units_show(struct seq_file s, void unused)
	57	+static int em_debug_flags_show(struct seq_file s, void unused)
58	58	{
59	59	struct em_perf_domain *pd = s->private;
60		− char *units = (pd->flags & EM_PERF_DOMAIN_MILLIWATTS) ?
61		− "milliWatts" : "bogoWatts";
62	60
63		− seq_printf(s, "%s\n", units);
	61	+ seq_printf(s, "%#lx\n", pd->flags);
64	62
65	63	return 0;
66	64	}
67		−DEFINE_SHOW_ATTRIBUTE(em_debug_units);
68		−
69		−static int em_debug_skip_inefficiencies_show(struct seq_file s, void unused)
70		−{
71		− struct em_perf_domain *pd = s->private;
72		− int enabled = (pd->flags & EM_PERF_DOMAIN_SKIP_INEFFICIENCIES) ? 1 : 0;
73		−
74		− seq_printf(s, "%d\n", enabled);
75		−
76		− return 0;
77		−}
78		−DEFINE_SHOW_ATTRIBUTE(em_debug_skip_inefficiencies);
	65	+DEFINE_SHOW_ATTRIBUTE(em_debug_flags);
79	66
80	67	static void em_debug_create_pd(struct device *dev)
81	68	{
@@ -89,9 +76,8 @@ static void em_debug_create_pd(struct device *dev) · 7 unmodified lines
89	76	debugfs_create_file("cpus", 0444, d, dev->em_pd->cpus,
90	77	&em_debug_cpus_fops);
91	78
92		− debugfs_create_file("units", 0444, d, dev->em_pd, &em_debug_units_fops);
93		− debugfs_create_file("skip-inefficiencies", 0444, d, dev->em_pd,
94		− &em_debug_skip_inefficiencies_fops);
	79	+ debugfs_create_file("flags", 0444, d, dev->em_pd,
	80	+ &em_debug_flags_fops);
95	81
96	82	/* Create a sub-directory for each performance state */
97	83	for (i = 0; i < dev->em_pd->nr_perf_states; i++)
@@ -121,7 +107,8 @@ static void em_debug_remove_pd(struct device *dev) {} · 23 unmodified lines
121	107	#endif
122	108
123	109	static int em_create_perf_table(struct device dev, struct em_perf_domain pd,
124		− int nr_states, struct em_data_callback *cb)
	110	+ int nr_states, struct em_data_callback *cb,
	111	+ unsigned long flags)
125	112	{
126	113	unsigned long power, freq, prev_freq = 0, prev_cost = ULONG_MAX;
127	114	struct em_perf_state *table;
@@ -139,7 +126,7 @@ static int em_create_perf_table(struct device dev, struct em_perf_domain pd, · 11 unmodified lines
139	126	* lowest performance state of 'dev' above 'freq' and updates
140	127	* 'power' and 'freq' accordingly.
141	128	*/
142		− ret = cb->active_power(&power, &freq, dev);
	129	+ ret = cb->active_power(dev, &power, &freq);
143	130	if (ret) {
144	131	dev_err(dev, "EM: invalid perf. state: %d\n",
145	132	ret);
@@ -173,10 +160,22 @@ static int em_create_perf_table(struct device dev, struct em_perf_domain pd, · 27 unmodified lines
173	160	/* Compute the cost of each performance state. */
174	161	fmax = (u64) table[nr_states - 1].frequency;
175	162	for (i = nr_states - 1; i >= 0; i--) {
176		− unsigned long power_res = em_scale_power(table[i].power);
	163	+ unsigned long power_res, cost;
	164	+
	165	+ if (flags & EM_PERF_DOMAIN_ARTIFICIAL) {
	166	+ ret = cb->get_cost(dev, table[i].frequency, &cost);
	167	+ if (ret \|\| !cost \|\| cost > EM_MAX_POWER) {
	168	+ dev_err(dev, "EM: invalid cost %lu %d\n",
	169	+ cost, ret);
	170	+ goto free_ps_table;
	171	+ }
	172	+ } else {
	173	+ power_res = em_scale_power(table[i].power);
	174	+ cost = div64_u64(fmax * power_res, table[i].frequency);
	175	+ }
	176	+
	177	+ table[i].cost = cost;
177	178
178		− table[i].cost = div64_u64(fmax * power_res,
179		− table[i].frequency);
180	179	if (table[i].cost >= prev_cost) {
181	180	table[i].flags = EM_PERF_STATE_INEFFICIENT;
182	181	dev_dbg(dev, "EM: OPP:%lu is inefficient\n",
@@ -197,7 +196,8 @@ free_ps_table: · 14 unmodified lines
197	196	}
198	197
199	198	static int em_create_pd(struct device *dev, int nr_states,
200		− struct em_data_callback cb, cpumask_t cpus)
	199	+ struct em_data_callback cb, cpumask_t cpus,
	200	+ unsigned long flags)
201	201	{
202	202	struct em_perf_domain *pd;
203	203	struct device *cpu_dev;
@@ -215,7 +215,7 @@ static int em_create_pd(struct device *dev, int nr_states, · 11 unmodified lines
215	215	return -ENOMEM;
216	216	}
217	217
218		− ret = em_create_perf_table(dev, pd, nr_states, cb);
	218	+ ret = em_create_perf_table(dev, pd, nr_states, cb, flags);
219	219	if (ret) {
220	220	kfree(pd);
221	221	return ret;
@@ -259,6 +259,8 @@ static void em_cpufreq_update_efficiencies(struct device *dev) · 37 unmodified lines
259	259	found++;
260	260	}
261	261
	262	+ cpufreq_cpu_put(policy);
	263	+
262	264	if (!found)
263	265	return;
264	266
@@ -332,6 +334,7 @@ int em_dev_register_perf_domain(struct device *dev, unsigned int nr_states, · 67 unmodified lines
332	334	bool milliwatts)
333	335	{
334	336	unsigned long cap, prev_cap = 0;
	337	+ unsigned long flags = 0;
335	338	int cpu, ret;
336	339
337	340	if (!dev \|\| !nr_states \|\| !cb)
@@ -378,12 +381,16 @@ int em_dev_register_perf_domain(struct device *dev, unsigned int nr_states, · 40 unmodified lines
378	381	}
379	382	}
380	383
381		− ret = em_create_pd(dev, nr_states, cb, cpus);
	384	+ if (milliwatts)
	385	+ flags \|= EM_PERF_DOMAIN_MILLIWATTS;
	386	+ else if (cb->get_cost)
	387	+ flags \|= EM_PERF_DOMAIN_ARTIFICIAL;
	388	+
	389	+ ret = em_create_pd(dev, nr_states, cb, cpus, flags);
382	390	if (ret)
383	391	goto unlock;
384	392
385		− if (milliwatts)
386		− dev->em_pd->flags \|= EM_PERF_DOMAIN_MILLIWATTS;
	393	+ dev->em_pd->flags \|= flags;
387	394
388	395	em_cpufreq_update_efficiencies(dev);
389	396

kernel/power/main.c+0−29

@@ -545,35 +545,6 @@ static int __init pm_debug_messages_setup(char *str) · 544 unmodified lines
545	545	}
546	546	__setup("pm_debug_messages", pm_debug_messages_setup);
547	547
548		−/**
549		− * __pm_pr_dbg - Print a suspend debug message to the kernel log.
550		− * @defer: Whether or not to use printk_deferred() to print the message.
551		− * @fmt: Message format.
552		− *
553		− * The message will be emitted if enabled through the pm_debug_messages
554		− * sysfs attribute.
555		− */
556		−void __pm_pr_dbg(bool defer, const char *fmt, ...)
557		−{
558		− struct va_format vaf;
559		− va_list args;
560		−
561		− if (!pm_debug_messages_on)
562		− return;
563		−
564		− va_start(args, fmt);
565		−
566		− vaf.fmt = fmt;
567		− vaf.va = &args;
568		−
569		− if (defer)
570		− printk_deferred(KERN_DEBUG "PM: %pV", &vaf);
571		− else
572		− printk(KERN_DEBUG "PM: %pV", &vaf);
573		−
574		− va_end(args);
575		−}
576		−
577	548	#else /* !CONFIG_PM_SLEEP_DEBUG */
578	549	static inline void pm_print_times_init(void) {}
579	550	#endif /* CONFIG_PM_SLEEP_DEBUG */

kernel/power/process.c+0−3

@@ -6,9 +6,6 @@ · 5 unmodified lines
6	6	* Originally from swsusp.
7	7	*/
8	8
9		−
10		−#undef DEBUG
11		−
12	9	#include <linux/interrupt.h>
13	10	#include <linux/oom.h>
14	11	#include <linux/suspend.h>

kernel/power/snapshot.c+8−4

@@ -326,7 +326,7 @@ static void chain_alloc(struct chain_allocator ca, unsigned int size) · 325 unmodified lines
326	326	return ret;
327	327	}
328	328
329		−/**
	329	+/*
330	330	* Data types related to memory bitmaps.
331	331	*
332	332	* Memory bitmap is a structure consisting of many linked lists of
@@ -427,6 +427,10 @@ struct memory_bitmap { · 94 unmodified lines
427	427
428	428	/**
429	429	* alloc_rtree_node - Allocate a new node and add it to the radix tree.
	430	+ * @gfp_mask: GFP mask for the allocation.
	431	+ * @safe_needed: Get pages not used before hibernation (restore only)
	432	+ * @ca: Pointer to a linked list of pages ("a chain") to allocate from
	433	+ * @list: Radix Tree node to add.
430	434	*
431	435	* This function is used to allocate inner nodes as well as the
432	436	* leave nodes of the radix tree. It also adds the node to the
@@ -902,7 +906,7 @@ static bool rtree_next_node(struct memory_bitmap *bm) · 469 unmodified lines
902	906	}
903	907
904	908	/**
905		− * memory_bm_rtree_next_pfn - Find the next set bit in a memory bitmap.
	909	+ * memory_bm_next_pfn - Find the next set bit in a memory bitmap.
906	910	* @bm: Memory bitmap.
907	911	*
908	912	* Starting from the last returned position this function searches for the next
@@ -1937,7 +1941,7 @@ static inline int get_highmem_buffer(int safe_needed) · 1028 unmodified lines
1937	1941	}
1938	1942
1939	1943	/**
1940		− * alloc_highmem_image_pages - Allocate some highmem pages for the image.
	1944	+ * alloc_highmem_pages - Allocate some highmem pages for the image.
1941	1945	*
1942	1946	* Try to allocate as many pages as needed, but if the number of free highmem
1943	1947	* pages is less than that, allocate them all.
@@ -2224,7 +2228,7 @@ static int check_header(struct swsusp_info *info) · 280 unmodified lines
2224	2228	}
2225	2229
2226	2230	/**
2227		− * load header - Check the image header and copy the data from it.
	2231	+ * load_header - Check the image header and copy the data from it.
2228	2232	*/
2229	2233	static int load_header(struct swsusp_info *info)
2230	2234	{

tools/power/x86/turbostat/Makefile+1−1

@@ -9,7 +9,7 @@ ifeq ("$(origin O)", "command line") · 8 unmodified lines
9	9	endif
10	10
11	11	turbostat : turbostat.c
12		−override CFLAGS += -O2 -Wall -I../../../include
	12	+override CFLAGS += -O2 -Wall -Wextra -I../../../include
13	13	override CFLAGS += -DMSRHEADER='"../../../../arch/x86/include/asm/msr-index.h"'
14	14	override CFLAGS += -DINTEL_FAMILY_HEADER='"../../../../arch/x86/include/asm/intel-family.h"'
15	15	override CFLAGS += -D_FILE_OFFSET_BITS=64

tools/power/x86/turbostat/turbostat.8+1−1

@@ -292,7 +292,7 @@ starts a new interval. · 291 unmodified lines
292	292	must be run as root.
293	293	Alternatively, non-root users can be enabled to run turbostat this way:
294	294
295		−# setcap cap_sys_rawio=ep ./turbostat
	295	+# setcap cap_sys_admin,cap_sys_rawio,cap_sys_nice=+ep ./turbostat
296	296
297	297	# chmod +r /dev/cpu/*/msr
298	298

tools/power/x86/turbostat/turbostat.c+386−208

@@ -3,7 +3,7 @@ · 2 unmodified lines
3	3	* turbostat -- show CPU frequency and C-state residency
4	4	* on modern Intel and AMD processors.
5	5	*
6		− * Copyright (c) 2021 Intel Corporation.
	6	+ * Copyright (c) 2022 Intel Corporation.
7	7	* Len Brown <len.brown@intel.com>
8	8	*/
9	9
@@ -37,6 +37,171 @@ · 27 unmodified lines
37	37	#include <asm/unistd.h>
38	38	#include <stdbool.h>
39	39
	40	+#define UNUSED(x) (void)(x)
	41	+
	42	+/*
	43	+ * This list matches the column headers, except
	44	+ * 1. built-in only, the sysfs counters are not here -- we learn of those at run-time
	45	+ * 2. Core and CPU are moved to the end, we can't have strings that contain them
	46	+ * matching on them for --show and --hide.
	47	+ */
	48	+
	49	+/*
	50	+ * buffer size used by sscanf() for added column names
	51	+ * Usually truncated to 7 characters, but also handles 18 columns for raw 64-bit counters
	52	+ */
	53	+#define NAME_BYTES 20
	54	+#define PATH_BYTES 128
	55	+
	56	+enum counter_scope { SCOPE_CPU, SCOPE_CORE, SCOPE_PACKAGE };
	57	+enum counter_type { COUNTER_ITEMS, COUNTER_CYCLES, COUNTER_SECONDS, COUNTER_USEC };
	58	+enum counter_format { FORMAT_RAW, FORMAT_DELTA, FORMAT_PERCENT };
	59	+
	60	+struct msr_counter {
	61	+ unsigned int msr_num;
	62	+ char name[NAME_BYTES];
	63	+ char path[PATH_BYTES];
	64	+ unsigned int width;
	65	+ enum counter_type type;
	66	+ enum counter_format format;
	67	+ struct msr_counter *next;
	68	+ unsigned int flags;
	69	+#define FLAGS_HIDE (1 << 0)
	70	+#define FLAGS_SHOW (1 << 1)
	71	+#define SYSFS_PERCPU (1 << 1)
	72	+};
	73	+
	74	+struct msr_counter bic[] = {
	75	+ { 0x0, "usec", "", 0, 0, 0, NULL, 0 },
	76	+ { 0x0, "Time_Of_Day_Seconds", "", 0, 0, 0, NULL, 0 },
	77	+ { 0x0, "Package", "", 0, 0, 0, NULL, 0 },
	78	+ { 0x0, "Node", "", 0, 0, 0, NULL, 0 },
	79	+ { 0x0, "Avg_MHz", "", 0, 0, 0, NULL, 0 },
	80	+ { 0x0, "Busy%", "", 0, 0, 0, NULL, 0 },
	81	+ { 0x0, "Bzy_MHz", "", 0, 0, 0, NULL, 0 },
	82	+ { 0x0, "TSC_MHz", "", 0, 0, 0, NULL, 0 },
	83	+ { 0x0, "IRQ", "", 0, 0, 0, NULL, 0 },
	84	+ { 0x0, "SMI", "", 32, 0, FORMAT_DELTA, NULL, 0 },
	85	+ { 0x0, "sysfs", "", 0, 0, 0, NULL, 0 },
	86	+ { 0x0, "CPU%c1", "", 0, 0, 0, NULL, 0 },
	87	+ { 0x0, "CPU%c3", "", 0, 0, 0, NULL, 0 },
	88	+ { 0x0, "CPU%c6", "", 0, 0, 0, NULL, 0 },
	89	+ { 0x0, "CPU%c7", "", 0, 0, 0, NULL, 0 },
	90	+ { 0x0, "ThreadC", "", 0, 0, 0, NULL, 0 },
	91	+ { 0x0, "CoreTmp", "", 0, 0, 0, NULL, 0 },
	92	+ { 0x0, "CoreCnt", "", 0, 0, 0, NULL, 0 },
	93	+ { 0x0, "PkgTmp", "", 0, 0, 0, NULL, 0 },
	94	+ { 0x0, "GFX%rc6", "", 0, 0, 0, NULL, 0 },
	95	+ { 0x0, "GFXMHz", "", 0, 0, 0, NULL, 0 },
	96	+ { 0x0, "Pkg%pc2", "", 0, 0, 0, NULL, 0 },
	97	+ { 0x0, "Pkg%pc3", "", 0, 0, 0, NULL, 0 },
	98	+ { 0x0, "Pkg%pc6", "", 0, 0, 0, NULL, 0 },
	99	+ { 0x0, "Pkg%pc7", "", 0, 0, 0, NULL, 0 },
	100	+ { 0x0, "Pkg%pc8", "", 0, 0, 0, NULL, 0 },
	101	+ { 0x0, "Pkg%pc9", "", 0, 0, 0, NULL, 0 },
	102	+ { 0x0, "Pk%pc10", "", 0, 0, 0, NULL, 0 },
	103	+ { 0x0, "CPU%LPI", "", 0, 0, 0, NULL, 0 },
	104	+ { 0x0, "SYS%LPI", "", 0, 0, 0, NULL, 0 },
	105	+ { 0x0, "PkgWatt", "", 0, 0, 0, NULL, 0 },
	106	+ { 0x0, "CorWatt", "", 0, 0, 0, NULL, 0 },
	107	+ { 0x0, "GFXWatt", "", 0, 0, 0, NULL, 0 },
	108	+ { 0x0, "PkgCnt", "", 0, 0, 0, NULL, 0 },
	109	+ { 0x0, "RAMWatt", "", 0, 0, 0, NULL, 0 },
	110	+ { 0x0, "PKG_%", "", 0, 0, 0, NULL, 0 },
	111	+ { 0x0, "RAM_%", "", 0, 0, 0, NULL, 0 },
	112	+ { 0x0, "Pkg_J", "", 0, 0, 0, NULL, 0 },
	113	+ { 0x0, "Cor_J", "", 0, 0, 0, NULL, 0 },
	114	+ { 0x0, "GFX_J", "", 0, 0, 0, NULL, 0 },
	115	+ { 0x0, "RAM_J", "", 0, 0, 0, NULL, 0 },
	116	+ { 0x0, "Mod%c6", "", 0, 0, 0, NULL, 0 },
	117	+ { 0x0, "Totl%C0", "", 0, 0, 0, NULL, 0 },
	118	+ { 0x0, "Any%C0", "", 0, 0, 0, NULL, 0 },
	119	+ { 0x0, "GFX%C0", "", 0, 0, 0, NULL, 0 },
	120	+ { 0x0, "CPUGFX%", "", 0, 0, 0, NULL, 0 },
	121	+ { 0x0, "Core", "", 0, 0, 0, NULL, 0 },
	122	+ { 0x0, "CPU", "", 0, 0, 0, NULL, 0 },
	123	+ { 0x0, "APIC", "", 0, 0, 0, NULL, 0 },
	124	+ { 0x0, "X2APIC", "", 0, 0, 0, NULL, 0 },
	125	+ { 0x0, "Die", "", 0, 0, 0, NULL, 0 },
	126	+ { 0x0, "GFXAMHz", "", 0, 0, 0, NULL, 0 },
	127	+ { 0x0, "IPC", "", 0, 0, 0, NULL, 0 },
	128	+ { 0x0, "CoreThr", "", 0, 0, 0, NULL, 0 },
	129	+};
	130	+
	131	+#define MAX_BIC (sizeof(bic) / sizeof(struct msr_counter))
	132	+#define BIC_USEC (1ULL << 0)
	133	+#define BIC_TOD (1ULL << 1)
	134	+#define BIC_Package (1ULL << 2)
	135	+#define BIC_Node (1ULL << 3)
	136	+#define BIC_Avg_MHz (1ULL << 4)
	137	+#define BIC_Busy (1ULL << 5)
	138	+#define BIC_Bzy_MHz (1ULL << 6)
	139	+#define BIC_TSC_MHz (1ULL << 7)
	140	+#define BIC_IRQ (1ULL << 8)
	141	+#define BIC_SMI (1ULL << 9)
	142	+#define BIC_sysfs (1ULL << 10)
	143	+#define BIC_CPU_c1 (1ULL << 11)
	144	+#define BIC_CPU_c3 (1ULL << 12)
	145	+#define BIC_CPU_c6 (1ULL << 13)
	146	+#define BIC_CPU_c7 (1ULL << 14)
	147	+#define BIC_ThreadC (1ULL << 15)
	148	+#define BIC_CoreTmp (1ULL << 16)
	149	+#define BIC_CoreCnt (1ULL << 17)
	150	+#define BIC_PkgTmp (1ULL << 18)
	151	+#define BIC_GFX_rc6 (1ULL << 19)
	152	+#define BIC_GFXMHz (1ULL << 20)
	153	+#define BIC_Pkgpc2 (1ULL << 21)
	154	+#define BIC_Pkgpc3 (1ULL << 22)
	155	+#define BIC_Pkgpc6 (1ULL << 23)
	156	+#define BIC_Pkgpc7 (1ULL << 24)
	157	+#define BIC_Pkgpc8 (1ULL << 25)
	158	+#define BIC_Pkgpc9 (1ULL << 26)
	159	+#define BIC_Pkgpc10 (1ULL << 27)
	160	+#define BIC_CPU_LPI (1ULL << 28)
	161	+#define BIC_SYS_LPI (1ULL << 29)
	162	+#define BIC_PkgWatt (1ULL << 30)
	163	+#define BIC_CorWatt (1ULL << 31)
	164	+#define BIC_GFXWatt (1ULL << 32)
	165	+#define BIC_PkgCnt (1ULL << 33)
	166	+#define BIC_RAMWatt (1ULL << 34)
	167	+#define BIC_PKG__ (1ULL << 35)
	168	+#define BIC_RAM__ (1ULL << 36)
	169	+#define BIC_Pkg_J (1ULL << 37)
	170	+#define BIC_Cor_J (1ULL << 38)
	171	+#define BIC_GFX_J (1ULL << 39)
	172	+#define BIC_RAM_J (1ULL << 40)
	173	+#define BIC_Mod_c6 (1ULL << 41)
	174	+#define BIC_Totl_c0 (1ULL << 42)
	175	+#define BIC_Any_c0 (1ULL << 43)
	176	+#define BIC_GFX_c0 (1ULL << 44)
	177	+#define BIC_CPUGFX (1ULL << 45)
	178	+#define BIC_Core (1ULL << 46)
	179	+#define BIC_CPU (1ULL << 47)
	180	+#define BIC_APIC (1ULL << 48)
	181	+#define BIC_X2APIC (1ULL << 49)
	182	+#define BIC_Die (1ULL << 50)
	183	+#define BIC_GFXACTMHz (1ULL << 51)
	184	+#define BIC_IPC (1ULL << 52)
	185	+#define BIC_CORE_THROT_CNT (1ULL << 53)
	186	+
	187	+#define BIC_TOPOLOGY (BIC_Package \| BIC_Node \| BIC_CoreCnt \| BIC_PkgCnt \| BIC_Core \| BIC_CPU \| BIC_Die )
	188	+#define BIC_THERMAL_PWR ( BIC_CoreTmp \| BIC_PkgTmp \| BIC_PkgWatt \| BIC_CorWatt \| BIC_GFXWatt \| BIC_RAMWatt \| BIC_PKG__ \| BIC_RAM__)
	189	+#define BIC_FREQUENCY ( BIC_Avg_MHz \| BIC_Busy \| BIC_Bzy_MHz \| BIC_TSC_MHz \| BIC_GFXMHz \| BIC_GFXACTMHz )
	190	+#define BIC_IDLE ( BIC_sysfs \| BIC_CPU_c1 \| BIC_CPU_c3 \| BIC_CPU_c6 \| BIC_CPU_c7 \| BIC_GFX_rc6 \| BIC_Pkgpc2 \| BIC_Pkgpc3 \| BIC_Pkgpc6 \| BIC_Pkgpc7 \| BIC_Pkgpc8 \| BIC_Pkgpc9 \| BIC_Pkgpc10 \| BIC_CPU_LPI \| BIC_SYS_LPI \| BIC_Mod_c6 \| BIC_Totl_c0 \| BIC_Any_c0 \| BIC_GFX_c0 \| BIC_CPUGFX)
	191	+#define BIC_OTHER ( BIC_IRQ \| BIC_SMI \| BIC_ThreadC \| BIC_CoreTmp \| BIC_IPC)
	192	+
	193	+#define BIC_DISABLED_BY_DEFAULT (BIC_USEC \| BIC_TOD \| BIC_APIC \| BIC_X2APIC)
	194	+
	195	+unsigned long long bic_enabled = (0xFFFFFFFFFFFFFFFFULL & ~BIC_DISABLED_BY_DEFAULT);
	196	+unsigned long long bic_present = BIC_USEC \| BIC_TOD \| BIC_sysfs \| BIC_APIC \| BIC_X2APIC;
	197	+
	198	+#define DO_BIC(COUNTER_NAME) (bic_enabled & bic_present & COUNTER_NAME)
	199	+#define DO_BIC_READ(COUNTER_NAME) (bic_present & COUNTER_NAME)
	200	+#define ENABLE_BIC(COUNTER_NAME) (bic_enabled \|= COUNTER_NAME)
	201	+#define BIC_PRESENT(COUNTER_BIT) (bic_present \|= COUNTER_BIT)
	202	+#define BIC_NOT_PRESENT(COUNTER_BIT) (bic_present &= ~COUNTER_BIT)
	203	+#define BIC_IS_ENABLED(COUNTER_BIT) (bic_enabled & COUNTER_BIT)
	204	+
40	205	char *proc_stat = "/proc/stat";
41	206	FILE *outf;
42	207	int *fd_percpu;
@@ -48,6 +213,7 @@ struct timespec interval_ts = { 5, 0 }; · 5 unmodified lines
48	213	unsigned int model_orig;
49	214
50	215	unsigned int num_iterations;
	216	+unsigned int header_iterations;
51	217	unsigned int debug;
52	218	unsigned int quiet;
53	219	unsigned int shown;
@@ -159,13 +325,6 @@ int ignore_stdin; · 105 unmodified lines
159	325
160	326	#define MAX(a, b) ((a) > (b) ? (a) : (b))
161	327
162		−/*
163		− * buffer size used by sscanf() for added column names
164		− * Usually truncated to 7 characters, but also handles 18 columns for raw 64-bit counters
165		− */
166		−#define NAME_BYTES 20
167		−#define PATH_BYTES 128
168		−
169	328	int backwards_count;
170	329	char *progname;
171	330
@@ -205,6 +364,7 @@ struct core_data { · 33 unmodified lines
205	364	unsigned int core_temp_c;
206	365	unsigned int core_energy; /* MSR_CORE_ENERGY_STAT */
207	366	unsigned int core_id;
	367	+ unsigned long long core_throt_cnt;
208	368	unsigned long long counter[MAX_ADDED_COUNTERS];
209	369	} core_even, core_odd;
210	370
@@ -255,24 +415,6 @@ struct pkg_data { · 44 unmodified lines
255	415
256	416	#define GET_PKG(pkg_base, pkg_no) (pkg_base + pkg_no)
257	417
258		−enum counter_scope { SCOPE_CPU, SCOPE_CORE, SCOPE_PACKAGE };
259		−enum counter_type { COUNTER_ITEMS, COUNTER_CYCLES, COUNTER_SECONDS, COUNTER_USEC };
260		−enum counter_format { FORMAT_RAW, FORMAT_DELTA, FORMAT_PERCENT };
261		−
262		−struct msr_counter {
263		− unsigned int msr_num;
264		− char name[NAME_BYTES];
265		− char path[PATH_BYTES];
266		− unsigned int width;
267		− enum counter_type type;
268		− enum counter_format format;
269		− struct msr_counter *next;
270		− unsigned int flags;
271		−#define FLAGS_HIDE (1 << 0)
272		−#define FLAGS_SHOW (1 << 1)
273		−#define SYSFS_PERCPU (1 << 1)
274		−};
275		−
276	418	/*
277	419	* The accumulated sum of MSR is defined as a monotonic
278	420	* increasing MSR, it will be accumulated periodically,
@@ -522,8 +664,10 @@ static int perf_instr_count_open(int cpu_num) · 243 unmodified lines
522	664
523	665	/* counter for cpu_num, including user + kernel and all processes */
524	666	fd = perf_event_open(&pea, -1, cpu_num, -1, 0);
525		− if (fd == -1)
526		− err(-1, "cpu%d: perf instruction counter\n", cpu_num);
	667	+ if (fd == -1) {
	668	+ warn("cpu%d: perf instruction counter", cpu_num);
	669	+ BIC_NOT_PRESENT(BIC_IPC);
	670	+ }
527	671
528	672	return fd;
529	673	}
@@ -550,143 +694,10 @@ int get_msr(int cpu, off_t offset, unsigned long long *msr) · 20 unmodified lines
550	694	return 0;
551	695	}
552	696
553		−/*
554		− * This list matches the column headers, except
555		− * 1. built-in only, the sysfs counters are not here -- we learn of those at run-time
556		− * 2. Core and CPU are moved to the end, we can't have strings that contain them
557		− * matching on them for --show and --hide.
558		− */
559		−struct msr_counter bic[] = {
560		− { 0x0, "usec" },
561		− { 0x0, "Time_Of_Day_Seconds" },
562		− { 0x0, "Package" },
563		− { 0x0, "Node" },
564		− { 0x0, "Avg_MHz" },
565		− { 0x0, "Busy%" },
566		− { 0x0, "Bzy_MHz" },
567		− { 0x0, "TSC_MHz" },
568		− { 0x0, "IRQ" },
569		− { 0x0, "SMI", "", 32, 0, FORMAT_DELTA, NULL },
570		− { 0x0, "sysfs" },
571		− { 0x0, "CPU%c1" },
572		− { 0x0, "CPU%c3" },
573		− { 0x0, "CPU%c6" },
574		− { 0x0, "CPU%c7" },
575		− { 0x0, "ThreadC" },
576		− { 0x0, "CoreTmp" },
577		− { 0x0, "CoreCnt" },
578		− { 0x0, "PkgTmp" },
579		− { 0x0, "GFX%rc6" },
580		− { 0x0, "GFXMHz" },
581		− { 0x0, "Pkg%pc2" },
582		− { 0x0, "Pkg%pc3" },
583		− { 0x0, "Pkg%pc6" },
584		− { 0x0, "Pkg%pc7" },
585		− { 0x0, "Pkg%pc8" },
586		− { 0x0, "Pkg%pc9" },
587		− { 0x0, "Pk%pc10" },
588		− { 0x0, "CPU%LPI" },
589		− { 0x0, "SYS%LPI" },
590		− { 0x0, "PkgWatt" },
591		− { 0x0, "CorWatt" },
592		− { 0x0, "GFXWatt" },
593		− { 0x0, "PkgCnt" },
594		− { 0x0, "RAMWatt" },
595		− { 0x0, "PKG_%" },
596		− { 0x0, "RAM_%" },
597		− { 0x0, "Pkg_J" },
598		− { 0x0, "Cor_J" },
599		− { 0x0, "GFX_J" },
600		− { 0x0, "RAM_J" },
601		− { 0x0, "Mod%c6" },
602		− { 0x0, "Totl%C0" },
603		− { 0x0, "Any%C0" },
604		− { 0x0, "GFX%C0" },
605		− { 0x0, "CPUGFX%" },
606		− { 0x0, "Core" },
607		− { 0x0, "CPU" },
608		− { 0x0, "APIC" },
609		− { 0x0, "X2APIC" },
610		− { 0x0, "Die" },
611		− { 0x0, "GFXAMHz" },
612		− { 0x0, "IPC" },
613		−};
614		−
615		−#define MAX_BIC (sizeof(bic) / sizeof(struct msr_counter))
616		−#define BIC_USEC (1ULL << 0)
617		−#define BIC_TOD (1ULL << 1)
618		−#define BIC_Package (1ULL << 2)
619		−#define BIC_Node (1ULL << 3)
620		−#define BIC_Avg_MHz (1ULL << 4)
621		−#define BIC_Busy (1ULL << 5)
622		−#define BIC_Bzy_MHz (1ULL << 6)
623		−#define BIC_TSC_MHz (1ULL << 7)
624		−#define BIC_IRQ (1ULL << 8)
625		−#define BIC_SMI (1ULL << 9)
626		−#define BIC_sysfs (1ULL << 10)
627		−#define BIC_CPU_c1 (1ULL << 11)
628		−#define BIC_CPU_c3 (1ULL << 12)
629		−#define BIC_CPU_c6 (1ULL << 13)
630		−#define BIC_CPU_c7 (1ULL << 14)
631		−#define BIC_ThreadC (1ULL << 15)
632		−#define BIC_CoreTmp (1ULL << 16)
633		−#define BIC_CoreCnt (1ULL << 17)
634		−#define BIC_PkgTmp (1ULL << 18)
635		−#define BIC_GFX_rc6 (1ULL << 19)
636		−#define BIC_GFXMHz (1ULL << 20)
637		−#define BIC_Pkgpc2 (1ULL << 21)
638		−#define BIC_Pkgpc3 (1ULL << 22)
639		−#define BIC_Pkgpc6 (1ULL << 23)
640		−#define BIC_Pkgpc7 (1ULL << 24)
641		−#define BIC_Pkgpc8 (1ULL << 25)
642		−#define BIC_Pkgpc9 (1ULL << 26)
643		−#define BIC_Pkgpc10 (1ULL << 27)
644		−#define BIC_CPU_LPI (1ULL << 28)
645		−#define BIC_SYS_LPI (1ULL << 29)
646		−#define BIC_PkgWatt (1ULL << 30)
647		−#define BIC_CorWatt (1ULL << 31)
648		−#define BIC_GFXWatt (1ULL << 32)
649		−#define BIC_PkgCnt (1ULL << 33)
650		−#define BIC_RAMWatt (1ULL << 34)
651		−#define BIC_PKG__ (1ULL << 35)
652		−#define BIC_RAM__ (1ULL << 36)
653		−#define BIC_Pkg_J (1ULL << 37)
654		−#define BIC_Cor_J (1ULL << 38)
655		−#define BIC_GFX_J (1ULL << 39)
656		−#define BIC_RAM_J (1ULL << 40)
657		−#define BIC_Mod_c6 (1ULL << 41)
658		−#define BIC_Totl_c0 (1ULL << 42)
659		−#define BIC_Any_c0 (1ULL << 43)
660		−#define BIC_GFX_c0 (1ULL << 44)
661		−#define BIC_CPUGFX (1ULL << 45)
662		−#define BIC_Core (1ULL << 46)
663		−#define BIC_CPU (1ULL << 47)
664		−#define BIC_APIC (1ULL << 48)
665		−#define BIC_X2APIC (1ULL << 49)
666		−#define BIC_Die (1ULL << 50)
667		−#define BIC_GFXACTMHz (1ULL << 51)
668		−#define BIC_IPC (1ULL << 52)
669		−
670		−#define BIC_TOPOLOGY (BIC_Package \| BIC_Node \| BIC_CoreCnt \| BIC_PkgCnt \| BIC_Core \| BIC_CPU \| BIC_Die )
671		−#define BIC_THERMAL_PWR ( BIC_CoreTmp \| BIC_PkgTmp \| BIC_PkgWatt \| BIC_CorWatt \| BIC_GFXWatt \| BIC_RAMWatt \| BIC_PKG__ \| BIC_RAM__)
672		−#define BIC_FREQUENCY ( BIC_Avg_MHz \| BIC_Busy \| BIC_Bzy_MHz \| BIC_TSC_MHz \| BIC_GFXMHz \| BIC_GFXACTMHz )
673		−#define BIC_IDLE ( BIC_sysfs \| BIC_CPU_c1 \| BIC_CPU_c3 \| BIC_CPU_c6 \| BIC_CPU_c7 \| BIC_GFX_rc6 \| BIC_Pkgpc2 \| BIC_Pkgpc3 \| BIC_Pkgpc6 \| BIC_Pkgpc7 \| BIC_Pkgpc8 \| BIC_Pkgpc9 \| BIC_Pkgpc10 \| BIC_CPU_LPI \| BIC_SYS_LPI \| BIC_Mod_c6 \| BIC_Totl_c0 \| BIC_Any_c0 \| BIC_GFX_c0 \| BIC_CPUGFX)
674		−#define BIC_OTHER ( BIC_IRQ \| BIC_SMI \| BIC_ThreadC \| BIC_CoreTmp \| BIC_IPC)
675		−
676		−#define BIC_DISABLED_BY_DEFAULT (BIC_USEC \| BIC_TOD \| BIC_APIC \| BIC_X2APIC)
677		−
678		−unsigned long long bic_enabled = (0xFFFFFFFFFFFFFFFFULL & ~BIC_DISABLED_BY_DEFAULT);
679		−unsigned long long bic_present = BIC_USEC \| BIC_TOD \| BIC_sysfs \| BIC_APIC \| BIC_X2APIC;
680		−
681		−#define DO_BIC(COUNTER_NAME) (bic_enabled & bic_present & COUNTER_NAME)
682		−#define DO_BIC_READ(COUNTER_NAME) (bic_present & COUNTER_NAME)
683		−#define ENABLE_BIC(COUNTER_NAME) (bic_enabled \|= COUNTER_NAME)
684		−#define BIC_PRESENT(COUNTER_BIT) (bic_present \|= COUNTER_BIT)
685		−#define BIC_NOT_PRESENT(COUNTER_BIT) (bic_present &= ~COUNTER_BIT)
686		−#define BIC_IS_ENABLED(COUNTER_BIT) (bic_enabled & COUNTER_BIT)
687		−
688	697	#define MAX_DEFERRED 16
	698	+char *deferred_add_names[MAX_DEFERRED];
689	699	char *deferred_skip_names[MAX_DEFERRED];
	700	+int deferred_add_index;
690	701	int deferred_skip_index;
691	702
692	703	/*
@@ -720,6 +731,8 @@ void help(void) · 27 unmodified lines
720	731	" -l, --list list column headers only\n"
721	732	" -n, --num_iterations num\n"
722	733	" number of the measurement iterations\n"
	734	+ " -N, --header_iterations num\n"
	735	+ " print header every num iterations\n"
723	736	" -o, --out file\n"
724	737	" create or truncate \"file\" for all output\n"
725	738	" -q, --quiet skip decoding system configuration header\n"
@@ -741,7 +754,7 @@ void help(void) · 15 unmodified lines
741	754	*/
742	755	unsigned long long bic_lookup(char *name_list, enum show_hide_mode mode)
743	756	{
744		− int i;
	757	+ unsigned int i;
745	758	unsigned long long retval = 0;
746	759
747	760	while (name_list) {
@@ -752,40 +765,51 @@ unsigned long long bic_lookup(char *name_list, enum show_hide_mode mode) · 4 unmodified lines
752	765	if (comma)
753	766	*comma = '\0';
754	767
755		− if (!strcmp(name_list, "all"))
756		− return ~0;
757		− if (!strcmp(name_list, "topology"))
758		− return BIC_TOPOLOGY;
759		− if (!strcmp(name_list, "power"))
760		− return BIC_THERMAL_PWR;
761		− if (!strcmp(name_list, "idle"))
762		− return BIC_IDLE;
763		− if (!strcmp(name_list, "frequency"))
764		− return BIC_FREQUENCY;
765		− if (!strcmp(name_list, "other"))
766		− return BIC_OTHER;
767		− if (!strcmp(name_list, "all"))
768		− return 0;
769		−
770	768	for (i = 0; i < MAX_BIC; ++i) {
771	769	if (!strcmp(name_list, bic[i].name)) {
772	770	retval \|= (1ULL << i);
773	771	break;
774	772	}
	773	+ if (!strcmp(name_list, "all")) {
	774	+ retval \|= ~0;
	775	+ break;
	776	+ } else if (!strcmp(name_list, "topology")) {
	777	+ retval \|= BIC_TOPOLOGY;
	778	+ break;
	779	+ } else if (!strcmp(name_list, "power")) {
	780	+ retval \|= BIC_THERMAL_PWR;
	781	+ break;
	782	+ } else if (!strcmp(name_list, "idle")) {
	783	+ retval \|= BIC_IDLE;
	784	+ break;
	785	+ } else if (!strcmp(name_list, "frequency")) {
	786	+ retval \|= BIC_FREQUENCY;
	787	+ break;
	788	+ } else if (!strcmp(name_list, "other")) {
	789	+ retval \|= BIC_OTHER;
	790	+ break;
	791	+ }
	792	+
775	793	}
776	794	if (i == MAX_BIC) {
777	795	if (mode == SHOW_LIST) {
778		− fprintf(stderr, "Invalid counter name: %s\n", name_list);
779		− exit(-1);
780		− }
781		− deferred_skip_names[deferred_skip_index++] = name_list;
782		− if (debug)
783		− fprintf(stderr, "deferred \"%s\"\n", name_list);
784		− if (deferred_skip_index >= MAX_DEFERRED) {
785		− fprintf(stderr, "More than max %d un-recognized --skip options '%s'\n",
786		− MAX_DEFERRED, name_list);
787		− help();
788		− exit(1);
	796	+ deferred_add_names[deferred_add_index++] = name_list;
	797	+ if (deferred_add_index >= MAX_DEFERRED) {
	798	+ fprintf(stderr, "More than max %d un-recognized --add options '%s'\n",
	799	+ MAX_DEFERRED, name_list);
	800	+ help();
	801	+ exit(1);
	802	+ }
	803	+ } else {
	804	+ deferred_skip_names[deferred_skip_index++] = name_list;
	805	+ if (debug)
	806	+ fprintf(stderr, "deferred \"%s\"\n", name_list);
	807	+ if (deferred_skip_index >= MAX_DEFERRED) {
	808	+ fprintf(stderr, "More than max %d un-recognized --skip options '%s'\n",
	809	+ MAX_DEFERRED, name_list);
	810	+ help();
	811	+ exit(1);
	812	+ }
789	813	}
790	814	}
791	815
@@ -872,6 +896,9 @@ void print_header(char *delim) · 80 unmodified lines
872	896	if (DO_BIC(BIC_CoreTmp))
873	897	outp += sprintf(outp, "%sCoreTmp", (printed++ ? delim : ""));
874	898
	899	+ if (DO_BIC(BIC_CORE_THROT_CNT))
	900	+ outp += sprintf(outp, "%sCoreThr", (printed++ ? delim : ""));
	901	+
875	902	if (do_rapl && !rapl_joules) {
876	903	if (DO_BIC(BIC_CorWatt) && (do_rapl & RAPL_PER_CORE_ENERGY))
877	904	outp += sprintf(outp, "%sCorWatt", (printed++ ? delim : ""));
@@ -1011,6 +1038,7 @@ int dump_counters(struct thread_data t, struct core_data c, struct pkg_data *p · 133 unmodified lines
1011	1038	outp += sprintf(outp, "c6: %016llX\n", c->c6);
1012	1039	outp += sprintf(outp, "c7: %016llX\n", c->c7);
1013	1040	outp += sprintf(outp, "DTS: %dC\n", c->core_temp_c);
	1041	+ outp += sprintf(outp, "cpu_throt_count: %016llX\n", c->core_throt_cnt);
1014	1042	outp += sprintf(outp, "Joules: %0X\n", c->core_energy);
1015	1043
1016	1044	for (i = 0, mp = sys.cp; mp; i++, mp = mp->next) {
@@ -1225,6 +1253,10 @@ int format_counters(struct thread_data t, struct core_data c, struct pkg_data · 208 unmodified lines
1225	1253	if (DO_BIC(BIC_CoreTmp))
1226	1254	outp += sprintf(outp, "%s%d", (printed++ ? delim : ""), c->core_temp_c);
1227	1255
	1256	+ /* Core throttle count */
	1257	+ if (DO_BIC(BIC_CORE_THROT_CNT))
	1258	+ outp += sprintf(outp, "%s%lld", (printed++ ? delim : ""), c->core_throt_cnt);
	1259	+
1228	1260	for (i = 0, mp = sys.cp; mp; i++, mp = mp->next) {
1229	1261	if (mp->format == FORMAT_RAW) {
1230	1262	if (mp->width == 32)
@@ -1311,6 +1343,7 @@ int format_counters(struct thread_data t, struct core_data c, struct pkg_data · 80 unmodified lines
1311	1343	if (DO_BIC(BIC_PkgWatt))
1312	1344	outp +=
1313	1345	sprintf(outp, fmt8, (printed++ ? delim : ""), p->energy_pkg * rapl_energy_units / interval_float);
	1346	+
1314	1347	if (DO_BIC(BIC_CorWatt) && !(do_rapl & RAPL_PER_CORE_ENERGY))
1315	1348	outp +=
1316	1349	sprintf(outp, fmt8, (printed++ ? delim : ""), p->energy_cores * rapl_energy_units / interval_float);
@@ -1386,14 +1419,14 @@ void flush_output_stderr(void) · 69 unmodified lines
1386	1419
1387	1420	void format_all_counters(struct thread_data t, struct core_data c, struct pkg_data *p)
1388	1421	{
1389		− static int printed;
	1422	+ static int count;
1390	1423
1391		− if (!printed \|\| !summary_only)
	1424	+ if ((!count \|\| (header_iterations && !(count % header_iterations))) \|\| !summary_only)
1392	1425	print_header("\t");
1393	1426
1394	1427	format_counters(&average.threads, &average.cores, &average.packages);
1395	1428
1396		− printed = 1;
	1429	+ count++;
1397	1430
1398	1431	if (summary_only)
1399	1432	return;
@@ -1467,6 +1500,7 @@ void delta_core(struct core_data new, struct core_data old) · 67 unmodified lines
1467	1500	old->c6 = new->c6 - old->c6;
1468	1501	old->c7 = new->c7 - old->c7;
1469	1502	old->core_temp_c = new->core_temp_c;
	1503	+ old->core_throt_cnt = new->core_throt_cnt;
1470	1504	old->mc6_us = new->mc6_us - old->mc6_us;
1471	1505
1472	1506	DELTA_WRAP32(new->core_energy, old->core_energy);
@@ -1626,6 +1660,7 @@ void clear_counters(struct thread_data t, struct core_data c, struct pkg_data · 153 unmodified lines
1626	1660	c->mc6_us = 0;
1627	1661	c->core_temp_c = 0;
1628	1662	c->core_energy = 0;
	1663	+ c->core_throt_cnt = 0;
1629	1664
1630	1665	p->pkg_wtd_core_c0 = 0;
1631	1666	p->pkg_any_core_c0 = 0;
@@ -1710,6 +1745,7 @@ int sum_counters(struct thread_data t, struct core_data c, struct pkg_data *p) · 78 unmodified lines
1710	1745	average.cores.mc6_us += c->mc6_us;
1711	1746
1712	1747	average.cores.core_temp_c = MAX(average.cores.core_temp_c, c->core_temp_c);
	1748	+ average.cores.core_throt_cnt = MAX(average.cores.core_throt_cnt, c->core_throt_cnt);
1713	1749
1714	1750	average.cores.core_energy += c->core_energy;
1715	1751
@@ -1987,6 +2023,26 @@ void get_apic_id(struct thread_data *t) · 271 unmodified lines
1987	2023	fprintf(outf, "cpu%d: BIOS BUG: apic 0x%x x2apic 0x%x\n", t->cpu_id, t->apic_id, t->x2apic_id);
1988	2024	}
1989	2025
	2026	+int get_core_throt_cnt(int cpu, unsigned long long *cnt)
	2027	+{
	2028	+ char path[128 + PATH_BYTES];
	2029	+ unsigned long long tmp;
	2030	+ FILE *fp;
	2031	+ int ret;
	2032	+
	2033	+ sprintf(path, "/sys/devices/system/cpu/cpu%d/thermal_throttle/core_throttle_count", cpu);
	2034	+ fp = fopen(path, "r");
	2035	+ if (!fp)
	2036	+ return -1;
	2037	+ ret = fscanf(fp, "%lld", &tmp);
	2038	+ if (ret != 1)
	2039	+ return -1;
	2040	+ fclose(fp);
	2041	+ *cnt = tmp;
	2042	+
	2043	+ return 0;
	2044	+}
	2045	+
1990	2046	/*
1991	2047	* get_counters(...)
1992	2048	* migrate to cpu
@@ -2129,6 +2185,9 @@ retry: · 136 unmodified lines
2129	2185	c->core_temp_c = tj_max - ((msr >> 16) & 0x7F);
2130	2186	}
2131	2187
	2188	+ if (DO_BIC(BIC_CORE_THROT_CNT))
	2189	+ get_core_throt_cnt(cpu, &c->core_throt_cnt);
	2190	+
2132	2191	if (do_rapl & RAPL_AMD_F17H) {
2133	2192	if (get_msr(cpu, MSR_CORE_ENERGY_STAT, &msr))
2134	2193	return -14;
@@ -2428,6 +2487,9 @@ int has_turbo_ratio_group_limits(int family, int model) · 293 unmodified lines
2428	2487	if (!genuine_intel)
2429	2488	return 0;
2430	2489
	2490	+ if (family != 6)
	2491	+ return 0;
	2492	+
2431	2493	switch (model) {
2432	2494	case INTEL_FAM6_ATOM_GOLDMONT:
2433	2495	case INTEL_FAM6_SKYLAKE_X:
@@ -2435,8 +2497,9 @@ int has_turbo_ratio_group_limits(int family, int model) · 1 unmodified line
2435	2497	case INTEL_FAM6_ATOM_GOLDMONT_D:
2436	2498	case INTEL_FAM6_ATOM_TREMONT_D:
2437	2499	return 1;
	2500	+ default:
	2501	+ return 0;
2438	2502	}
2439		− return 0;
2440	2503	}
2441	2504
2442	2505	static void dump_turbo_ratio_limits(int family, int model)
@@ -3027,6 +3090,8 @@ void set_max_cpu_num(void) · 584 unmodified lines
3027	3090	*/
3028	3091	int count_cpus(int cpu)
3029	3092	{
	3093	+ UNUSED(cpu);
	3094	+
3030	3095	topo.num_cpus++;
3031	3096	return 0;
3032	3097	}
@@ -3361,6 +3426,9 @@ static int update_msr_sum(struct thread_data t, struct core_data c, struct pkg · 328 unmodified lines
3361	3426	int i, ret;
3362	3427	int cpu = t->cpu_id;
3363	3428
	3429	+ UNUSED(c);
	3430	+ UNUSED(p);
	3431	+
3364	3432	for (i = IDX_PKG_ENERGY; i < IDX_COUNT; i++) {
3365	3433	unsigned long long msr_cur, msr_last;
3366	3434	off_t offset;
@@ -3387,6 +3455,8 @@ static int update_msr_sum(struct thread_data t, struct core_data c, struct pkg · 20 unmodified lines
3387	3455
3388	3456	static void msr_record_handler(union sigval v)
3389	3457	{
	3458	+ UNUSED(v);
	3459	+
3390	3460	for_all_cpus(update_msr_sum, EVEN_COUNTERS);
3391	3461	}
3392	3462
@@ -3439,6 +3509,9 @@ release_msr: · 46 unmodified lines
3439	3509	/*
3440	3510	* set_my_sched_priority(pri)
3441	3511	* return previous
	3512	+ *
	3513	+ * if non-root, do this:
	3514	+ * # /sbin/setcap cap_sys_rawio,cap_sys_nice=+ep /usr/bin/turbostat
3442	3515	*/
3443	3516	int set_my_sched_priority(int priority)
3444	3517	{
@@ -3457,7 +3530,7 @@ int set_my_sched_priority(int priority) · 12 unmodified lines
3457	3530	errno = 0;
3458	3531	retval = getpriority(PRIO_PROCESS, 0);
3459	3532	if (retval != priority)
3460		− err(-1, "getpriority(%d) != setpriority(%d)", retval, priority);
	3533	+ err(retval, "getpriority(%d) != setpriority(%d)", retval, priority);
3461	3534
3462	3535	return original_priority;
3463	3536	}
@@ -3466,7 +3539,7 @@ void turbostat_loop() · 2 unmodified lines
3466	3539	{
3467	3540	int retval;
3468	3541	int restarted = 0;
3469		− int done_iters = 0;
	3542	+ unsigned int done_iters = 0;
3470	3543
3471	3544	setup_signal_handler();
3472	3545
@@ -3678,6 +3751,7 @@ int probe_nhm_msrs(unsigned int family, unsigned int model) · 205 unmodified lines
3678	3751	break;
3679	3752	case INTEL_FAM6_ATOM_SILVERMONT: /* BYT */
3680	3753	no_MSR_MISC_PWR_MGMT = 1;
	3754	+ /* FALLTHRU */
3681	3755	case INTEL_FAM6_ATOM_SILVERMONT_D: /* AVN */
3682	3756	pkg_cstate_limits = slv_pkg_cstate_limits;
3683	3757	break;
@@ -3721,6 +3795,9 @@ int has_slv_msrs(unsigned int family, unsigned int model) · 37 unmodified lines
3721	3795	if (!genuine_intel)
3722	3796	return 0;
3723	3797
	3798	+ if (family != 6)
	3799	+ return 0;
	3800	+
3724	3801	switch (model) {
3725	3802	case INTEL_FAM6_ATOM_SILVERMONT:
3726	3803	case INTEL_FAM6_ATOM_SILVERMONT_MID:
@@ -3736,6 +3813,9 @@ int is_dnv(unsigned int family, unsigned int model) · 9 unmodified lines
3736	3813	if (!genuine_intel)
3737	3814	return 0;
3738	3815
	3816	+ if (family != 6)
	3817	+ return 0;
	3818	+
3739	3819	switch (model) {
3740	3820	case INTEL_FAM6_ATOM_GOLDMONT_D:
3741	3821	return 1;
@@ -3749,6 +3829,9 @@ int is_bdx(unsigned int family, unsigned int model) · 7 unmodified lines
3749	3829	if (!genuine_intel)
3750	3830	return 0;
3751	3831
	3832	+ if (family != 6)
	3833	+ return 0;
	3834	+
3752	3835	switch (model) {
3753	3836	case INTEL_FAM6_BROADWELL_X:
3754	3837	return 1;
@@ -3762,6 +3845,9 @@ int is_skx(unsigned int family, unsigned int model) · 7 unmodified lines
3762	3845	if (!genuine_intel)
3763	3846	return 0;
3764	3847
	3848	+ if (family != 6)
	3849	+ return 0;
	3850	+
3765	3851	switch (model) {
3766	3852	case INTEL_FAM6_SKYLAKE_X:
3767	3853	return 1;
@@ -3775,6 +3861,9 @@ int is_icx(unsigned int family, unsigned int model) · 7 unmodified lines
3775	3861	if (!genuine_intel)
3776	3862	return 0;
3777	3863
	3864	+ if (family != 6)
	3865	+ return 0;
	3866	+
3778	3867	switch (model) {
3779	3868	case INTEL_FAM6_ICELAKE_X:
3780	3869	return 1;
@@ -3787,6 +3876,9 @@ int is_ehl(unsigned int family, unsigned int model) · 6 unmodified lines
3787	3876	if (!genuine_intel)
3788	3877	return 0;
3789	3878
	3879	+ if (family != 6)
	3880	+ return 0;
	3881	+
3790	3882	switch (model) {
3791	3883	case INTEL_FAM6_ATOM_TREMONT:
3792	3884	return 1;
@@ -3799,6 +3891,9 @@ int is_jvl(unsigned int family, unsigned int model) · 6 unmodified lines
3799	3891	if (!genuine_intel)
3800	3892	return 0;
3801	3893
	3894	+ if (family != 6)
	3895	+ return 0;
	3896	+
3802	3897	switch (model) {
3803	3898	case INTEL_FAM6_ATOM_TREMONT_D:
3804	3899	return 1;
@@ -3811,6 +3906,9 @@ int has_turbo_ratio_limit(unsigned int family, unsigned int model) · 6 unmodified lines
3811	3906	if (has_slv_msrs(family, model))
3812	3907	return 0;
3813	3908
	3909	+ if (family != 6)
	3910	+ return 0;
	3911	+
3814	3912	switch (model) {
3815	3913	/* Nehalem compatible, but do not include turbo-ratio limit support */
3816	3914	case INTEL_FAM6_NEHALEM_EX: /* Nehalem-EX Xeon - Beckton */
@@ -4125,6 +4223,9 @@ int print_epb(struct thread_data t, struct core_data c, struct pkg_data *p) · 308 unmodified lines
4125	4223	char *epb_string;
4126	4224	int cpu, epb;
4127	4225
	4226	+ UNUSED(c);
	4227	+ UNUSED(p);
	4228	+
4128	4229	if (!has_epb)
4129	4230	return 0;
4130	4231
@@ -4171,6 +4272,9 @@ int print_hwp(struct thread_data t, struct core_data c, struct pkg_data *p) · 40 unmodified lines
4171	4272	unsigned long long msr;
4172	4273	int cpu;
4173	4274
	4275	+ UNUSED(c);
	4276	+ UNUSED(p);
	4277	+
4174	4278	if (!has_hwp)
4175	4279	return 0;
4176	4280
@@ -4254,6 +4358,9 @@ int print_perf_limit(struct thread_data t, struct core_data c, struct pkg_data · 77 unmodified lines
4254	4358	unsigned long long msr;
4255	4359	int cpu;
4256	4360
	4361	+ UNUSED(c);
	4362	+ UNUSED(p);
	4363	+
4257	4364	cpu = t->cpu_id;
4258	4365
4259	4366	/* per-package */
@@ -4359,6 +4466,8 @@ double get_tdp_intel(unsigned int model) · 99 unmodified lines
4359	4466
4360	4467	double get_tdp_amd(unsigned int family)
4361	4468	{
	4469	+ UNUSED(family);
	4470	+
4362	4471	/* This is the max stock TDP of HEDT/Server Fam17h+ chips */
4363	4472	return 280.0;
4364	4473	}
@@ -4376,6 +4485,7 @@ static double rapl_dram_energy_units_probe(int model, double rapl_energy_units) · 11 unmodified lines
4376	4485	case INTEL_FAM6_BROADWELL_X: /* BDX */
4377	4486	case INTEL_FAM6_SKYLAKE_X: /* SKX */
4378	4487	case INTEL_FAM6_XEON_PHI_KNL: /* KNL */
	4488	+ case INTEL_FAM6_ICELAKE_X: /* ICX */
4379	4489	return (rapl_dram_energy_units = 15.3 / 1000000);
4380	4490	default:
4381	4491	return (rapl_energy_units);
@@ -4559,6 +4669,8 @@ void rapl_probe_amd(unsigned int family, unsigned int model) · 177 unmodified lines
4559	4669	unsigned int has_rapl = 0;
4560	4670	double tdp;
4561	4671
	4672	+ UNUSED(model);
	4673	+
4562	4674	if (max_extended_level >= 0x80000007) {
4563	4675	__cpuid(0x80000007, eax, ebx, ecx, edx);
4564	4676	/* RAPL (Fam 17h+) */
@@ -4617,6 +4729,7 @@ void perf_limit_reasons_probe(unsigned int family, unsigned int model) · 52 unmodified lines
4617	4729	case INTEL_FAM6_HASWELL_L: /* HSW */
4618	4730	case INTEL_FAM6_HASWELL_G: /* HSW */
4619	4731	do_gfx_perf_limit_reasons = 1;
	4732	+ /* FALLTHRU */
4620	4733	case INTEL_FAM6_HASWELL_X: /* HSX */
4621	4734	do_core_perf_limit_reasons = 1;
4622	4735	do_ring_perf_limit_reasons = 1;
@@ -4643,6 +4756,9 @@ int print_thermal(struct thread_data t, struct core_data c, struct pkg_data *p · 20 unmodified lines
4643	4756	unsigned int dts, dts2;
4644	4757	int cpu;
4645	4758
	4759	+ UNUSED(c);
	4760	+ UNUSED(p);
	4761	+
4646	4762	if (!(do_dts \|\| do_ptm))
4647	4763	return 0;
4648	4764
@@ -4698,7 +4814,7 @@ int print_thermal(struct thread_data t, struct core_data c, struct pkg_data *p · 49 unmodified lines
4698	4814
4699	4815	void print_power_limit_msr(int cpu, unsigned long long msr, char *label)
4700	4816	{
4701		− fprintf(outf, "cpu%d: %s: %sabled (%f Watts, %f sec, clamp %sabled)\n",
	4817	+ fprintf(outf, "cpu%d: %s: %sabled (%0.3f Watts, %f sec, clamp %sabled)\n",
4702	4818	cpu, label,
4703	4819	((msr >> 15) & 1) ? "EN" : "DIS",
4704	4820	((msr >> 0) & 0x7FFF) * rapl_power_units,
@@ -4714,6 +4830,9 @@ int print_rapl(struct thread_data t, struct core_data c, struct pkg_data *p) · 9 unmodified lines
4714	4830	const char *msr_name;
4715	4831	int cpu;
4716	4832
	4833	+ UNUSED(c);
	4834	+ UNUSED(p);
	4835	+
4717	4836	if (!do_rapl)
4718	4837	return 0;
4719	4838
@@ -4762,12 +4881,19 @@ int print_rapl(struct thread_data t, struct core_data c, struct pkg_data *p) · 42 unmodified lines
4762	4881	cpu, msr, (msr >> 63) & 1 ? "" : "UN");
4763	4882
4764	4883	print_power_limit_msr(cpu, msr, "PKG Limit #1");
4765		− fprintf(outf, "cpu%d: PKG Limit #2: %sabled (%f Watts, %f* sec, clamp %sabled)\n",
	4884	+ fprintf(outf, "cpu%d: PKG Limit #2: %sabled (%0.3f Watts, %f* sec, clamp %sabled)\n",
4766	4885	cpu,
4767	4886	((msr >> 47) & 1) ? "EN" : "DIS",
4768	4887	((msr >> 32) & 0x7FFF) * rapl_power_units,
4769	4888	(1.0 + (((msr >> 54) & 0x3) / 4.0)) * (1 << ((msr >> 49) & 0x1F)) * rapl_time_units,
4770	4889	((msr >> 48) & 1) ? "EN" : "DIS");
	4890	+
	4891	+ if (get_msr(cpu, MSR_VR_CURRENT_CONFIG, &msr))
	4892	+ return -9;
	4893	+
	4894	+ fprintf(outf, "cpu%d: MSR_VR_CURRENT_CONFIG: 0x%08llx\n", cpu, msr);
	4895	+ fprintf(outf, "cpu%d: PKG Limit #4: %f Watts (%slocked)\n",
	4896	+ cpu, ((msr >> 0) & 0x1FFF) * rapl_power_units, (msr >> 31) & 1 ? "" : "UN");
4771	4897	}
4772	4898
4773	4899	if (do_rapl & RAPL_DRAM_POWER_INFO) {
@@ -4830,6 +4956,9 @@ int has_snb_msrs(unsigned int family, unsigned int model) · 56 unmodified lines
4830	4956	if (!genuine_intel)
4831	4957	return 0;
4832	4958
	4959	+ if (family != 6)
	4960	+ return 0;
	4961	+
4833	4962	switch (model) {
4834	4963	case INTEL_FAM6_SANDYBRIDGE:
4835	4964	case INTEL_FAM6_SANDYBRIDGE_X:
@@ -4873,6 +5002,9 @@ int has_c8910_msrs(unsigned int family, unsigned int model) · 37 unmodified lines
4873	5002	if (!genuine_intel)
4874	5003	return 0;
4875	5004
	5005	+ if (family != 6)
	5006	+ return 0;
	5007	+
4876	5008	switch (model) {
4877	5009	case INTEL_FAM6_HASWELL_L: /* HSW */
4878	5010	case INTEL_FAM6_BROADWELL: /* BDW */
@@ -4899,6 +5031,9 @@ int has_skl_msrs(unsigned int family, unsigned int model) · 20 unmodified lines
4899	5031	if (!genuine_intel)
4900	5032	return 0;
4901	5033
	5034	+ if (family != 6)
	5035	+ return 0;
	5036	+
4902	5037	switch (model) {
4903	5038	case INTEL_FAM6_SKYLAKE_L: /* SKL */
4904	5039	case INTEL_FAM6_CANNONLAKE_L: /* CNL */
@@ -4911,6 +5046,10 @@ int is_slm(unsigned int family, unsigned int model) · 6 unmodified lines
4911	5046	{
4912	5047	if (!genuine_intel)
4913	5048	return 0;
	5049	+
	5050	+ if (family != 6)
	5051	+ return 0;
	5052	+
4914	5053	switch (model) {
4915	5054	case INTEL_FAM6_ATOM_SILVERMONT: /* BYT */
4916	5055	case INTEL_FAM6_ATOM_SILVERMONT_D: /* AVN */
@@ -4923,6 +5062,10 @@ int is_knl(unsigned int family, unsigned int model) · 6 unmodified lines
4923	5062	{
4924	5063	if (!genuine_intel)
4925	5064	return 0;
	5065	+
	5066	+ if (family != 6)
	5067	+ return 0;
	5068	+
4926	5069	switch (model) {
4927	5070	case INTEL_FAM6_XEON_PHI_KNL: /* KNL */
4928	5071	return 1;
@@ -4935,6 +5078,9 @@ int is_cnl(unsigned int family, unsigned int model) · 6 unmodified lines
4935	5078	if (!genuine_intel)
4936	5079	return 0;
4937	5080
	5081	+ if (family != 6)
	5082	+ return 0;
	5083	+
4938	5084	switch (model) {
4939	5085	case INTEL_FAM6_CANNONLAKE_L: /* CNL */
4940	5086	return 1;
@@ -4989,6 +5135,9 @@ int get_cpu_type(struct thread_data t, struct core_data c, struct pkg_data *p) · 48 unmodified lines
4989	5135	{
4990	5136	unsigned int eax, ebx, ecx, edx;
4991	5137
	5138	+ UNUSED(c);
	5139	+ UNUSED(p);
	5140	+
4992	5141	if (!genuine_intel)
4993	5142	return 0;
4994	5143
@@ -5025,6 +5174,9 @@ int set_temperature_target(struct thread_data t, struct core_data c, struct pk · 30 unmodified lines
5025	5174	unsigned int tcc_default, tcc_offset;
5026	5175	int cpu;
5027	5176
	5177	+ UNUSED(c);
	5178	+ UNUSED(p);
	5179	+
5028	5180	/* tj_max is used only for dts or ptm */
5029	5181	if (!(do_dts \|\| do_ptm))
5030	5182	return 0;
@@ -5572,6 +5724,11 @@ void process_cpuid() · 541 unmodified lines
5572	5724	else
5573	5725	BIC_NOT_PRESENT(BIC_CPU_LPI);
5574	5726
	5727	+ if (!access("/sys/devices/system/cpu/cpu0/thermal_throttle/core_throttle_count", R_OK))
	5728	+ BIC_PRESENT(BIC_CORE_THROT_CNT);
	5729	+ else
	5730	+ BIC_NOT_PRESENT(BIC_CORE_THROT_CNT);
	5731	+
5575	5732	if (!access(sys_lpi_file_sysfs, R_OK)) {
5576	5733	sys_lpi_file = sys_lpi_file_sysfs;
5577	5734	BIC_PRESENT(BIC_SYS_LPI);
@@ -5601,11 +5758,6 @@ int dir_filter(const struct dirent *dirp) · 23 unmodified lines
5601	5758	return 0;
5602	5759	}
5603	5760
5604		−int open_dev_cpu_msr(int dummy1)
5605		−{
5606		− return 0;
5607		−}
5608		−
5609	5761	void topology_probe()
5610	5762	{
5611	5763	int i;
@@ -5896,6 +6048,9 @@ void turbostat_init() · 284 unmodified lines
5896	6048
5897	6049	if (!quiet && do_irtl_snb)
5898	6050	print_irtl();
	6051	+
	6052	+ if (DO_BIC(BIC_IPC))
	6053	+ (void)get_instr_count_fd(base_cpu);
5899	6054	}
5900	6055
5901	6056	int fork_it(char **argv)
@@ -5973,7 +6128,7 @@ int get_and_dump_counters(void) · 71 unmodified lines
5973	6128
5974	6129	void print_version()
5975	6130	{
5976		− fprintf(outf, "turbostat version 21.05.04" " - Len Brown <lenb@kernel.org>\n");
	6131	+ fprintf(outf, "turbostat version 2022.04.16 - Len Brown <lenb@kernel.org>\n");
5977	6132	}
5978	6133
5979	6134	int add_counter(unsigned int msr_num, char path, char name,
@@ -6138,6 +6293,16 @@ next: · 158 unmodified lines
6138	6293	}
6139	6294	}
6140	6295
	6296	+int is_deferred_add(char *name)
	6297	+{
	6298	+ int i;
	6299	+
	6300	+ for (i = 0; i < deferred_add_index; ++i)
	6301	+ if (!strcmp(name, deferred_add_names[i]))
	6302	+ return 1;
	6303	+ return 0;
	6304	+}
	6305	+
6141	6306	int is_deferred_skip(char *name)
6142	6307	{
6143	6308	int i;
@@ -6156,9 +6321,6 @@ void probe_sysfs(void) · 12 unmodified lines
6156	6321	int state;
6157	6322	char *sp;
6158	6323
6159		− if (!DO_BIC(BIC_sysfs))
6160		− return;
6161		−
6162	6324	for (state = 10; state >= 0; --state) {
6163	6325
6164	6326	sprintf(path, "/sys/devices/system/cpu/cpu%d/cpuidle/state%d/name", base_cpu, state);
@@ -6181,6 +6343,9 @@ void probe_sysfs(void) · 16 unmodified lines
6181	6343
6182	6344	sprintf(path, "cpuidle/state%d/time", state);
6183	6345
	6346	+ if (!DO_BIC(BIC_sysfs) && !is_deferred_add(name_buf))
	6347	+ continue;
	6348	+
6184	6349	if (is_deferred_skip(name_buf))
6185	6350	continue;
6186	6351
@@ -6206,6 +6371,9 @@ void probe_sysfs(void) · 19 unmodified lines
6206	6371
6207	6372	sprintf(path, "cpuidle/state%d/usage", state);
6208	6373
	6374	+ if (!DO_BIC(BIC_sysfs) && !is_deferred_add(name_buf))
	6375	+ continue;
	6376	+
6209	6377	if (is_deferred_skip(name_buf))
6210	6378	continue;
6211	6379
@@ -6313,6 +6481,7 @@ void cmdline(int argc, char **argv) · 101 unmodified lines
6313	6481	{ "interval", required_argument, 0, 'i' },
6314	6482	{ "IPC", no_argument, 0, 'I' },
6315	6483	{ "num_iterations", required_argument, 0, 'n' },
	6484	+ { "header_iterations", required_argument, 0, 'N' },
6316	6485	{ "help", no_argument, 0, 'h' },
6317	6486	{ "hide", required_argument, 0, 'H' }, // meh, -h taken by --help
6318	6487	{ "Joules", no_argument, 0, 'J' },
@@ -6394,6 +6563,14 @@ void cmdline(int argc, char **argv) · 75 unmodified lines
6394	6563	exit(2);
6395	6564	}
6396	6565	break;
	6566	+ case 'N':
	6567	+ header_iterations = strtod(optarg, NULL);
	6568	+
	6569	+ if (header_iterations <= 0) {
	6570	+ fprintf(outf, "iterations %d should be positive number\n", header_iterations);
	6571	+ exit(2);
	6572	+ }
	6573	+ break;
6397	6574	case 's':
6398	6575	/*
6399	6576	* --show: show only those specified
@@ -6432,6 +6609,8 @@ int main(int argc, char **argv) · 32 unmodified lines
6432	6609
6433	6610	turbostat_init();
6434	6611
	6612	+ msr_sum_record();
	6613	+
6435	6614	/* dump counters and exit */
6436	6615	if (dump_only)
6437	6616	return get_and_dump_counters();
@@ -6443,7 +6622,6 @@ int main(int argc, char **argv) · 5 unmodified lines
6443	6622	return 0;
6444	6623	}
6445	6624
6446		− msr_sum_record();
6447	6625	/*
6448	6626	* if any params left, it must be a command to fork
6449	6627	*/