History

Merge tag 'pm-5.17-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm

Pull power management updates from Rafael Wysocki: "The most signigicant change here is the addition of a new cpufreq 'P-state' driver for AMD processors as a better replacement for the venerable acpi-cpufreq driver. There are also other cpufreq updates (in the core, intel_pstate, ARM drivers), PM core updates (mostly related to adding new macros for declaring PM operations which should make the lives of driver developers somewhat easier), and a bunch of assorted fixes and cleanups. Summary: - Add new P-state driver for AMD processors (Huang Rui). - Fix initialization of min and max frequency QoS requests in the cpufreq core (Rafael Wysocki). - Fix EPP handling on Alder Lake in intel_pstate (Srinivas Pandruvada). - Make intel_pstate update cpuinfo.max_freq when notified of HWP capabilities changes and drop a redundant function call from that driver (Rafael Wysocki). - Improve IRQ support in the Qcom cpufreq driver (Ard Biesheuvel, Stephen Boyd, Vladimir Zapolskiy). - Fix double devm_remap() in the Mediatek cpufreq driver (Hector Yuan). - Introduce thermal pressure helpers for cpufreq CPU cooling (Lukasz Luba). - Make cpufreq use default_groups in kobj_type (Greg Kroah-Hartman). - Make cpuidle use default_groups in kobj_type (Greg Kroah-Hartman). - Fix two comments in cpuidle code (Jason Wang, Yang Li). - Allow model-specific normal EPB value to be used in the intel_epb sysfs attribute handling code (Srinivas Pandruvada). - Simplify locking in pm_runtime_put_suppliers() (Rafael Wysocki). - Add safety net to supplier device release in the runtime PM core code (Rafael Wysocki). - Capture device status before disabling runtime PM for it (Rafael Wysocki). - Add new macros for declaring PM operations to allow drivers to avoid guarding them with CONFIG_PM #ifdefs or __maybe_unused and update some drivers to use these macros (Paul Cercueil). - Allow ACPI hardware signature to be honoured during restore from hibernation (David Woodhouse). - Update outdated operating performance points (OPP) documentation (Tang Yizhou). - Reduce log severity for informative message regarding frequency transition failures in devfreq (Tzung-Bi Shih). - Add DRAM frequency controller devfreq driver for Allwinner sunXi SoCs (Samuel Holland). - Add missing COMMON_CLK dependency to sun8i devfreq driver (Arnd Bergmann). - Add support for new layout of Psys PowerLimit Register on SPR to the Intel RAPL power capping driver (Zhang Rui). - Fix typo in a comment in idle_inject.c (Jason Wang). - Remove unused function definition from the DTPM (Dynamit Thermal Power Management) power capping framework (Daniel Lezcano). - Reduce DTPM trace verbosity (Daniel Lezcano)" * tag 'pm-5.17-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm: (53 commits) x86, sched: Fix undefined reference to init_freq_invariance_cppc() build error cpufreq: amd-pstate: Fix Kconfig dependencies for AMD P-State cpufreq: amd-pstate: Fix struct amd_cpudata kernel-doc comment cpuidle: use default_groups in kobj_type x86: intel_epb: Allow model specific normal EPB value MAINTAINERS: Add AMD P-State driver maintainer entry Documentation: amd-pstate: Add AMD P-State driver introduction cpufreq: amd-pstate: Add AMD P-State performance attributes cpufreq: amd-pstate: Add AMD P-State frequencies attributes cpufreq: amd-pstate: Add boost mode support for AMD P-State cpufreq: amd-pstate: Add trace for AMD P-State module cpufreq: amd-pstate: Introduce the support for the processors with shared memory solution cpufreq: amd-pstate: Add fast switch function for AMD P-State cpufreq: amd-pstate: Introduce a new AMD P-State driver to support future processors ACPI: CPPC: Add CPPC enable register function ACPI: CPPC: Check present CPUs for determining _CPC is valid ACPI: CPPC: Implement support for SystemIO registers x86/msr: Add AMD CPPC MSR definitions x86/cpufeatures: Add AMD Collaborative Processor Performance Control feature flag cpufreq: use default_groups in kobj_type ...

Linus Torvalds committed 4 years ago

Commit b35b6d42 parents bca2175 78e6e4d55 files · +2274 −218Browse files

Unified · Split

55 files
  1. Documentation/admin-guide/acpi/cppc_sysfs.rst +2 −0
  2. Documentation/admin-guide/kernel-parameters.txt +12 −3
  3. Documentation/admin-guide/pm/amd-pstate.rst +382 −0
  4. Documentation/admin-guide/pm/working-state.rst +1 −0
  5. Documentation/power/opp.rst +7 −7
  6. Documentation/power/runtime_pm.rst +10 −4
  7. MAINTAINERS +7 −0
  8. arch/arm/include/asm/topology.h +1 −1
  9. arch/arm64/include/asm/topology.h +1 −1
  10. arch/x86/include/asm/cpufeatures.h +1 −0
  11. arch/x86/include/asm/msr-index.h +17 −0
  12. arch/x86/include/asm/topology.h +1 −1
  13. arch/x86/kernel/acpi/sleep.c +3 −1
  14. arch/x86/kernel/cpu/intel_epb.c +32 −13
  15. drivers/acpi/cppc_acpi.c +95 −4
  16. drivers/acpi/sleep.c +21 −5
  17. drivers/base/arch_topology.c +38 −4
  18. drivers/base/core.c +1 −2
  19. drivers/base/power/runtime.c +63 −35
  20. drivers/cpufreq/Kconfig.x86 +17 −0
  21. drivers/cpufreq/Makefile +5 −0
  22. drivers/cpufreq/amd-pstate-trace.c +2 −0
  23. drivers/cpufreq/amd-pstate-trace.h +77 −0
  24. drivers/cpufreq/amd-pstate.c +645 −0
  25. drivers/cpufreq/cpufreq.c +5 −4
  26. drivers/cpufreq/cpufreq_conservative.c +3 −2
  27. drivers/cpufreq/cpufreq_ondemand.c +3 −2
  28. drivers/cpufreq/intel_pstate.c +81 −40
  29. drivers/cpufreq/mediatek-cpufreq-hw.c +30 −3
  30. drivers/cpufreq/qcom-cpufreq-hw.c +19 −20
  31. drivers/cpuidle/governors/menu.c +1 −1
  32. drivers/cpuidle/sysfs.c +5 −3
  33. drivers/devfreq/Kconfig +9 −0
  34. drivers/devfreq/Makefile +1 −0
  35. drivers/devfreq/devfreq.c +2 −2
  36. drivers/devfreq/sun8i-a33-mbus.c +511 −0
  37. drivers/mmc/host/jz4740_mmc.c +4 −4
  38. drivers/mmc/host/mxcmmc.c +2 −4
  39. drivers/net/ethernet/realtek/r8169_main.c +3 −1
  40. drivers/powercap/dtpm.c +3 −3
  41. drivers/powercap/idle_inject.c +1 −1
  42. drivers/powercap/intel_rapl_common.c +59 −2
  43. drivers/thermal/cpufreq_cooling.c +1 −5
  44. include/acpi/cppc_acpi.h +5 −0
  45. include/linux/acpi.h +1 −1
  46. include/linux/arch_topology.h +2 −2
  47. include/linux/dtpm.h +0 −2
  48. include/linux/intel_rapl.h +6 −0
  49. include/linux/pm.h +53 −29
  50. include/linux/pm_runtime.h +3 −0
  51. include/linux/sched/topology.h +3 −3
  52. include/linux/suspend.h +1 −0
  53. init/Kconfig +1 −1
  54. kernel/power/power.h +1 −0
  55. kernel/power/swap.c +14 −2
Documentation/admin-guide/acpi/cppc_sysfs.rst+2−0
@@ -4,6 +4,8 @@ · 3 unmodified lines
44 Collaborative Processor Performance Control (CPPC)
55 ==================================================
66
7+.. _cppc_sysfs:
8+
79 CPPC
810 ====
911
Documentation/admin-guide/kernel-parameters.txt+12−3
@@ -225,14 +225,23 @@ · 224 unmodified lines
225225 For broken nForce2 BIOS resulting in XT-PIC timer.
226226
227227 acpi_sleep= [HW,ACPI] Sleep options
228− Format: { s3_bios, s3_mode, s3_beep, s4_nohwsig,
229− old_ordering, nonvs, sci_force_enable, nobl }
228+ Format: { s3_bios, s3_mode, s3_beep, s4_hwsig,
229+ s4_nohwsig, old_ordering, nonvs,
230+ sci_force_enable, nobl }
230231 See Documentation/power/video.rst for information on
231232 s3_bios and s3_mode.
232233 s3_beep is for debugging; it makes the PC's speaker beep
233234 as soon as the kernel's real-mode entry point is called.
235+ s4_hwsig causes the kernel to check the ACPI hardware
236+ signature during resume from hibernation, and gracefully
237+ refuse to resume if it has changed. This complies with
238+ the ACPI specification but not with reality, since
239+ Windows does not do this and many laptops do change it
240+ on docking. So the default behaviour is to allow resume
241+ and simply warn when the signature changes, unless the
242+ s4_hwsig option is enabled.
234243 s4_nohwsig prevents ACPI hardware signature from being
235− used during resume from hibernation.
244+ used (or even warned about) during resume.
236245 old_ordering causes the ACPI 1.0 ordering of the _PTS
237246 control method, with respect to putting devices into
238247 low power states, to be enforced (the ACPI 2.0 ordering
Documentation/admin-guide/pm/amd-pstate.rst+382−0
@@ -0,0 +1,382 @@
1+.. SPDX-License-Identifier: GPL-2.0
2+.. include:: <isonum.txt>
3+
4+===============================================
5+``amd-pstate`` CPU Performance Scaling Driver
6+===============================================
7+
8+:Copyright: |copy| 2021 Advanced Micro Devices, Inc.
9+
10+:Author: Huang Rui <ray.huang@amd.com>
11+
12+
13+Introduction
14+===================
15+
16+``amd-pstate`` is the AMD CPU performance scaling driver that introduces a
17+new CPU frequency control mechanism on modern AMD APU and CPU series in
18+Linux kernel. The new mechanism is based on Collaborative Processor
19+Performance Control (CPPC) which provides finer grain frequency management
20+than legacy ACPI hardware P-States. Current AMD CPU/APU platforms are using
21+the ACPI P-states driver to manage CPU frequency and clocks with switching
22+only in 3 P-states. CPPC replaces the ACPI P-states controls, allows a
23+flexible, low-latency interface for the Linux kernel to directly
24+communicate the performance hints to hardware.
25+
26+``amd-pstate`` leverages the Linux kernel governors such as ``schedutil``,
27+``ondemand``, etc. to manage the performance hints which are provided by
28+CPPC hardware functionality that internally follows the hardware
29+specification (for details refer to AMD64 Architecture Programmer's Manual
30+Volume 2: System Programming [1]_). Currently ``amd-pstate`` supports basic
31+frequency control function according to kernel governors on some of the
32+Zen2 and Zen3 processors, and we will implement more AMD specific functions
33+in future after we verify them on the hardware and SBIOS.
34+
35+
36+AMD CPPC Overview
37+=======================
38+
39+Collaborative Processor Performance Control (CPPC) interface enumerates a
40+continuous, abstract, and unit-less performance value in a scale that is
41+not tied to a specific performance state / frequency. This is an ACPI
42+standard [2]_ which software can specify application performance goals and
43+hints as a relative target to the infrastructure limits. AMD processors
44+provides the low latency register model (MSR) instead of AML code
45+interpreter for performance adjustments. ``amd-pstate`` will initialize a
46+``struct cpufreq_driver`` instance ``amd_pstate_driver`` with the callbacks
47+to manage each performance update behavior. ::
48+
49+ Highest Perf ------>+-----------------------+ +-----------------------+
50+ | | | |
51+ | | | |
52+ | | Max Perf ---->| |
53+ | | | |
54+ | | | |
55+ Nominal Perf ------>+-----------------------+ +-----------------------+
56+ | | | |
57+ | | | |
58+ | | | |
59+ | | | |
60+ | | | |
61+ | | | |
62+ | | Desired Perf ---->| |
63+ | | | |
64+ | | | |
65+ | | | |
66+ | | | |
67+ | | | |
68+ | | | |
69+ | | | |
70+ | | | |
71+ | | | |
72+ Lowest non- | | | |
73+ linear perf ------>+-----------------------+ +-----------------------+
74+ | | | |
75+ | | Lowest perf ---->| |
76+ | | | |
77+ Lowest perf ------>+-----------------------+ +-----------------------+
78+ | | | |
79+ | | | |
80+ | | | |
81+ 0 ------>+-----------------------+ +-----------------------+
82+
83+ AMD P-States Performance Scale
84+
85+
86+.. _perf_cap:
87+
88+AMD CPPC Performance Capability
89+--------------------------------
90+
91+Highest Performance (RO)
92+.........................
93+
94+It is the absolute maximum performance an individual processor may reach,
95+assuming ideal conditions. This performance level may not be sustainable
96+for long durations and may only be achievable if other platform components
97+are in a specific state; for example, it may require other processors be in
98+an idle state. This would be equivalent to the highest frequencies
99+supported by the processor.
100+
101+Nominal (Guaranteed) Performance (RO)
102+......................................
103+
104+It is the maximum sustained performance level of the processor, assuming
105+ideal operating conditions. In absence of an external constraint (power,
106+thermal, etc.) this is the performance level the processor is expected to
107+be able to maintain continuously. All cores/processors are expected to be
108+able to sustain their nominal performance state simultaneously.
109+
110+Lowest non-linear Performance (RO)
111+...................................
112+
113+It is the lowest performance level at which nonlinear power savings are
114+achieved, for example, due to the combined effects of voltage and frequency
115+scaling. Above this threshold, lower performance levels should be generally
116+more energy efficient than higher performance levels. This register
117+effectively conveys the most efficient performance level to ``amd-pstate``.
118+
119+Lowest Performance (RO)
120+........................
121+
122+It is the absolute lowest performance level of the processor. Selecting a
123+performance level lower than the lowest nonlinear performance level may
124+cause an efficiency penalty but should reduce the instantaneous power
125+consumption of the processor.
126+
127+AMD CPPC Performance Control
128+------------------------------
129+
130+``amd-pstate`` passes performance goals through these registers. The
131+register drives the behavior of the desired performance target.
132+
133+Minimum requested performance (RW)
134+...................................
135+
136+``amd-pstate`` specifies the minimum allowed performance level.
137+
138+Maximum requested performance (RW)
139+...................................
140+
141+``amd-pstate`` specifies a limit the maximum performance that is expected
142+to be supplied by the hardware.
143+
144+Desired performance target (RW)
145+...................................
146+
147+``amd-pstate`` specifies a desired target in the CPPC performance scale as
148+a relative number. This can be expressed as percentage of nominal
149+performance (infrastructure max). Below the nominal sustained performance
150+level, desired performance expresses the average performance level of the
151+processor subject to hardware. Above the nominal performance level,
152+processor must provide at least nominal performance requested and go higher
153+if current operating conditions allow.
154+
155+Energy Performance Preference (EPP) (RW)
156+.........................................
157+
158+Provides a hint to the hardware if software wants to bias toward performance
159+(0x0) or energy efficiency (0xff).
160+
161+
162+Key Governors Support
163+=======================
164+
165+``amd-pstate`` can be used with all the (generic) scaling governors listed
166+by the ``scaling_available_governors`` policy attribute in ``sysfs``. Then,
167+it is responsible for the configuration of policy objects corresponding to
168+CPUs and provides the ``CPUFreq`` core (and the scaling governors attached
169+to the policy objects) with accurate information on the maximum and minimum
170+operating frequencies supported by the hardware. Users can check the
171+``scaling_cur_freq`` information comes from the ``CPUFreq`` core.
172+
173+``amd-pstate`` mainly supports ``schedutil`` and ``ondemand`` for dynamic
174+frequency control. It is to fine tune the processor configuration on
175+``amd-pstate`` to the ``schedutil`` with CPU CFS scheduler. ``amd-pstate``
176+registers adjust_perf callback to implement the CPPC similar performance
177+update behavior. It is initialized by ``sugov_start`` and then populate the
178+CPU's update_util_data pointer to assign ``sugov_update_single_perf`` as
179+the utilization update callback function in CPU scheduler. CPU scheduler
180+will call ``cpufreq_update_util`` and assign the target performance
181+according to the ``struct sugov_cpu`` that utilization update belongs to.
182+Then ``amd-pstate`` updates the desired performance according to the CPU
183+scheduler assigned.
184+
185+
186+Processor Support
187+=======================
188+
189+The ``amd-pstate`` initialization will fail if the _CPC in ACPI SBIOS is
190+not existed at the detected processor, and it uses ``acpi_cpc_valid`` to
191+check the _CPC existence. All Zen based processors support legacy ACPI
192+hardware P-States function, so while the ``amd-pstate`` fails to be
193+initialized, the kernel will fall back to initialize ``acpi-cpufreq``
194+driver.
195+
196+There are two types of hardware implementations for ``amd-pstate``: one is
197+`Full MSR Support <perf_cap_>`_ and another is `Shared Memory Support
198+<perf_cap_>`_. It can use :c:macro:`X86_FEATURE_CPPC` feature flag (for
199+details refer to Processor Programming Reference (PPR) for AMD Family
200+19h Model 51h, Revision A1 Processors [3]_) to indicate the different
201+types. ``amd-pstate`` is to register different ``static_call`` instances
202+for different hardware implementations.
203+
204+Currently, some of Zen2 and Zen3 processors support ``amd-pstate``. In the
205+future, it will be supported on more and more AMD processors.
206+
207+Full MSR Support
208+-----------------
209+
210+Some new Zen3 processors such as Cezanne provide the MSR registers directly
211+while the :c:macro:`X86_FEATURE_CPPC` CPU feature flag is set.
212+``amd-pstate`` can handle the MSR register to implement the fast switch
213+function in ``CPUFreq`` that can shrink latency of frequency control on the
214+interrupt context. The functions with ``pstate_xxx`` prefix represent the
215+operations of MSR registers.
216+
217+Shared Memory Support
218+----------------------
219+
220+If :c:macro:`X86_FEATURE_CPPC` CPU feature flag is not set, that means the
221+processor supports shared memory solution. In this case, ``amd-pstate``
222+uses the ``cppc_acpi`` helper methods to implement the callback functions
223+that defined on ``static_call``. The functions with ``cppc_xxx`` prefix
224+represent the operations of acpi cppc helpers for shared memory solution.
225+
226+
227+AMD P-States and ACPI hardware P-States always can be supported in one
228+processor. But AMD P-States has the higher priority and if it is enabled
229+with :c:macro:`MSR_AMD_CPPC_ENABLE` or ``cppc_set_enable``, it will respond
230+to the request from AMD P-States.
231+
232+
233+User Space Interface in ``sysfs``
234+==================================
235+
236+``amd-pstate`` exposes several global attributes (files) in ``sysfs`` to
237+control its functionality at the system level. They located in the
238+``/sys/devices/system/cpu/cpufreq/policyX/`` directory and affect all CPUs. ::
239+
240+ root@hr-test1:/home/ray# ls /sys/devices/system/cpu/cpufreq/policy0/*amd*
241+ /sys/devices/system/cpu/cpufreq/policy0/amd_pstate_highest_perf
242+ /sys/devices/system/cpu/cpufreq/policy0/amd_pstate_lowest_nonlinear_freq
243+ /sys/devices/system/cpu/cpufreq/policy0/amd_pstate_max_freq
244+
245+
246+``amd_pstate_highest_perf / amd_pstate_max_freq``
247+
248+Maximum CPPC performance and CPU frequency that the driver is allowed to
249+set in percent of the maximum supported CPPC performance level (the highest
250+performance supported in `AMD CPPC Performance Capability <perf_cap_>`_).
251+In some of ASICs, the highest CPPC performance is not the one in the _CPC
252+table, so we need to expose it to sysfs. If boost is not active but
253+supported, this maximum frequency will be larger than the one in
254+``cpuinfo``.
255+This attribute is read-only.
256+
257+``amd_pstate_lowest_nonlinear_freq``
258+
259+The lowest non-linear CPPC CPU frequency that the driver is allowed to set
260+in percent of the maximum supported CPPC performance level (Please see the
261+lowest non-linear performance in `AMD CPPC Performance Capability
262+<perf_cap_>`_).
263+This attribute is read-only.
264+
265+For other performance and frequency values, we can read them back from
266+``/sys/devices/system/cpu/cpuX/acpi_cppc/``, see :ref:`cppc_sysfs`.
267+
268+
269+``amd-pstate`` vs ``acpi-cpufreq``
270+======================================
271+
272+On majority of AMD platforms supported by ``acpi-cpufreq``, the ACPI tables
273+provided by the platform firmware used for CPU performance scaling, but
274+only provides 3 P-states on AMD processors.
275+However, on modern AMD APU and CPU series, it provides the collaborative
276+processor performance control according to ACPI protocol and customize this
277+for AMD platforms. That is fine-grain and continuous frequency range
278+instead of the legacy hardware P-states. ``amd-pstate`` is the kernel
279+module which supports the new AMD P-States mechanism on most of future AMD
280+platforms. The AMD P-States mechanism will be the more performance and energy
281+efficiency frequency management method on AMD processors.
282+
283+Kernel Module Options for ``amd-pstate``
284+=========================================
285+
286+``shared_mem``
287+Use a module param (shared_mem) to enable related processors manually with
288+**amd_pstate.shared_mem=1**.
289+Due to the performance issue on the processors with `Shared Memory Support
290+<perf_cap_>`_, so we disable it for the moment and will enable this by default
291+once we address performance issue on this solution.
292+
293+The way to check whether current processor is `Full MSR Support <perf_cap_>`_
294+or `Shared Memory Support <perf_cap_>`_ : ::
295+
296+ ray@hr-test1:~$ lscpu | grep cppc
297+ Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate ssbd mba ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr rdpru wbnoinvd cppc arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif v_spec_ctrl umip pku ospke vaes vpclmulqdq rdpid overflow_recov succor smca fsrm
298+
299+If CPU Flags have cppc, then this processor supports `Full MSR Support
300+<perf_cap_>`_. Otherwise it supports `Shared Memory Support <perf_cap_>`_.
301+
302+
303+``cpupower`` tool support for ``amd-pstate``
304+===============================================
305+
306+``amd-pstate`` is supported on ``cpupower`` tool that can be used to dump the frequency
307+information. And it is in progress to support more and more operations for new
308+``amd-pstate`` module with this tool. ::
309+
310+ root@hr-test1:/home/ray# cpupower frequency-info
311+ analyzing CPU 0:
312+ driver: amd-pstate
313+ CPUs which run at the same hardware frequency: 0
314+ CPUs which need to have their frequency coordinated by software: 0
315+ maximum transition latency: 131 us
316+ hardware limits: 400 MHz - 4.68 GHz
317+ available cpufreq governors: ondemand conservative powersave userspace performance schedutil
318+ current policy: frequency should be within 400 MHz and 4.68 GHz.
319+ The governor "schedutil" may decide which speed to use
320+ within this range.
321+ current CPU frequency: Unable to call hardware
322+ current CPU frequency: 4.02 GHz (asserted by call to kernel)
323+ boost state support:
324+ Supported: yes
325+ Active: yes
326+ AMD PSTATE Highest Performance: 166. Maximum Frequency: 4.68 GHz.
327+ AMD PSTATE Nominal Performance: 117. Nominal Frequency: 3.30 GHz.
328+ AMD PSTATE Lowest Non-linear Performance: 39. Lowest Non-linear Frequency: 1.10 GHz.
329+ AMD PSTATE Lowest Performance: 15. Lowest Frequency: 400 MHz.
330+
331+
332+Diagnostics and Tuning
333+=======================
334+
335+Trace Events
336+--------------
337+
338+There are two static trace events that can be used for ``amd-pstate``
339+diagnostics. One of them is the cpu_frequency trace event generally used
340+by ``CPUFreq``, and the other one is the ``amd_pstate_perf`` trace event
341+specific to ``amd-pstate``. The following sequence of shell commands can
342+be used to enable them and see their output (if the kernel is generally
343+configured to support event tracing). ::
344+
345+ root@hr-test1:/home/ray# cd /sys/kernel/tracing/
346+ root@hr-test1:/sys/kernel/tracing# echo 1 > events/amd_cpu/enable
347+ root@hr-test1:/sys/kernel/tracing# cat trace
348+ # tracer: nop
349+ #
350+ # entries-in-buffer/entries-written: 47827/42233061 #P:2
351+ #
352+ # _-----=> irqs-off
353+ # / _----=> need-resched
354+ # | / _---=> hardirq/softirq
355+ # || / _--=> preempt-depth
356+ # ||| / delay
357+ # TASK-PID CPU# |||| TIMESTAMP FUNCTION
358+ # | | | |||| | |
359+ <idle>-0 [015] dN... 4995.979886: amd_pstate_perf: amd_min_perf=85 amd_des_perf=85 amd_max_perf=166 cpu_id=15 changed=false fast_switch=true
360+ <idle>-0 [007] d.h.. 4995.979893: amd_pstate_perf: amd_min_perf=85 amd_des_perf=85 amd_max_perf=166 cpu_id=7 changed=false fast_switch=true
361+ cat-2161 [000] d.... 4995.980841: amd_pstate_perf: amd_min_perf=85 amd_des_perf=85 amd_max_perf=166 cpu_id=0 changed=false fast_switch=true
362+ sshd-2125 [004] d.s.. 4995.980968: amd_pstate_perf: amd_min_perf=85 amd_des_perf=85 amd_max_perf=166 cpu_id=4 changed=false fast_switch=true
363+ <idle>-0 [007] d.s.. 4995.980968: amd_pstate_perf: amd_min_perf=85 amd_des_perf=85 amd_max_perf=166 cpu_id=7 changed=false fast_switch=true
364+ <idle>-0 [003] d.s.. 4995.980971: amd_pstate_perf: amd_min_perf=85 amd_des_perf=85 amd_max_perf=166 cpu_id=3 changed=false fast_switch=true
365+ <idle>-0 [011] d.s.. 4995.980996: amd_pstate_perf: amd_min_perf=85 amd_des_perf=85 amd_max_perf=166 cpu_id=11 changed=false fast_switch=true
366+
367+The cpu_frequency trace event will be triggered either by the ``schedutil`` scaling
368+governor (for the policies it is attached to), or by the ``CPUFreq`` core (for the
369+policies with other scaling governors).
370+
371+
372+Reference
373+===========
374+
375+.. [1] AMD64 Architecture Programmer's Manual Volume 2: System Programming,
376+ https://www.amd.com/system/files/TechDocs/24593.pdf
377+
378+.. [2] Advanced Configuration and Power Interface Specification,
379+ https://uefi.org/sites/default/files/resources/ACPI_Spec_6_4_Jan22.pdf
380+
381+.. [3] Processor Programming Reference (PPR) for AMD Family 19h Model 51h, Revision A1 Processors
382+ https://www.amd.com/system/files/TechDocs/56569-A1-PUB.zip
Documentation/admin-guide/pm/working-state.rst+1−0
@@ -11,6 +11,7 @@ Working-State Power Management · 10 unmodified lines
1111 intel_idle
1212 cpufreq
1313 intel_pstate
14+ amd-pstate
1415 cpufreq_drivers
1516 intel_epb
1617 intel-speed-select
Documentation/power/opp.rst+7−7
@@ -48,9 +48,9 @@ We can represent these as three OPPs as the following {Hz, uV} tuples: · 47 unmodified lines
4848 OPP library provides a set of helper functions to organize and query the OPP
4949 information. The library is located in drivers/opp/ directory and the header
5050 is located in include/linux/pm_opp.h. OPP library can be enabled by enabling
51−CONFIG_PM_OPP from power management menuconfig menu. OPP library depends on
52−CONFIG_PM as certain SoCs such as Texas Instrument's OMAP framework allows to
53−optionally boot at a certain OPP without needing cpufreq.
51+CONFIG_PM_OPP from power management menuconfig menu. Certain SoCs such as Texas
52+Instrument's OMAP framework allows to optionally boot at a certain OPP without
53+needing cpufreq.
5454
5555 Typical usage of the OPP library is as follows::
5656
@@ -75,8 +75,8 @@ operations until that OPP could be re-enabled if possible. · 18 unmodified lines
7575
7676 OPP library facilitates this concept in its implementation. The following
7777 operational functions operate only on available opps:
78−opp_find_freq_{ceil, floor}, dev_pm_opp_get_voltage, dev_pm_opp_get_freq,
79−dev_pm_opp_get_opp_count
78+dev_pm_opp_find_freq_{ceil, floor}, dev_pm_opp_get_voltage, dev_pm_opp_get_freq,
79+dev_pm_opp_get_opp_count.
8080
8181 dev_pm_opp_find_freq_exact is meant to be used to find the opp pointer
8282 which can then be used for dev_pm_opp_enable/disable functions to make an
@@ -103,7 +103,7 @@ dev_pm_opp_add · 20 unmodified lines
103103 The OPP is defined using the frequency and voltage. Once added, the OPP
104104 is assumed to be available and control of its availability can be done
105105 with the dev_pm_opp_enable/disable functions. OPP library
106− internally stores and manages this information in the opp struct.
106+ internally stores and manages this information in the dev_pm_opp struct.
107107 This function may be used by SoC framework to define a optimal list
108108 as per the demands of SoC usage environment.
109109
@@ -247,7 +247,7 @@ dev_pm_opp_disable · 137 unmodified lines
247247 5. OPP Data Retrieval Functions
248248 ===============================
249249 Since OPP library abstracts away the OPP information, a set of functions to pull
250−information from the OPP structure is necessary. Once an OPP pointer is
250+information from the dev_pm_opp structure is necessary. Once an OPP pointer is
251251 retrieved using the search functions, the following functions can be used by SoC
252252 framework to retrieve the information represented inside the OPP layer.
253253
Documentation/power/runtime_pm.rst+10−4
@@ -265,6 +265,10 @@ defined in include/linux/pm.h: · 264 unmodified lines
265265 RPM_SUSPENDED, which means that each device is initially regarded by the
266266 PM core as 'suspended', regardless of its real hardware status
267267
268+ `enum rpm_status last_status;`
269+ - the last runtime PM status of the device captured before disabling runtime
270+ PM for it (invalid initially and when disable_depth is 0)
271+
268272 `unsigned int runtime_auto;`
269273 - if set, indicates that the user space has allowed the device driver to
270274 power manage the device at run time via the /sys/devices/.../power/control
@@ -333,10 +337,12 @@ drivers/base/power/runtime.c and include/linux/pm_runtime.h: · 62 unmodified lines
333337
334338 `int pm_runtime_resume(struct device *dev);`
335339 - execute the subsystem-level resume callback for the device; returns 0 on
336− success, 1 if the device's runtime PM status was already 'active' or
337− error code on failure, where -EAGAIN means it may be safe to attempt to
338− resume the device again in future, but 'power.runtime_error' should be
339− checked additionally, and -EACCES means that 'power.disable_depth' is
340+ success, 1 if the device's runtime PM status is already 'active' (also if
341+ 'power.disable_depth' is nonzero, but the status was 'active' when it was
342+ changing from 0 to 1) or error code on failure, where -EAGAIN means it may
343+ be safe to attempt to resume the device again in future, but
344+ 'power.runtime_error' should be checked additionally, and -EACCES means
345+ that the callback could not be run, because 'power.disable_depth' was
340346 different from 0
341347
342348 `int pm_runtime_resume_and_get(struct device *dev);`
MAINTAINERS+7−0
@@ -994,6 +994,13 @@ S: Supported · 993 unmodified lines
994994 T: git https://gitlab.freedesktop.org/agd5f/linux.git
995995 F: drivers/gpu/drm/amd/pm/
996996
997+AMD PSTATE DRIVER
998+M: Huang Rui <ray.huang@amd.com>
999+L: linux-pm@vger.kernel.org
1000+S: Supported
1001+F: Documentation/admin-guide/pm/amd-pstate.rst
1002+F: drivers/cpufreq/amd-pstate*
1003+
9971004 AMD PTDMA DRIVER
9981005 M: Sanjay R Mehta <sanju.mehta@amd.com>
9991006 L: dmaengine@vger.kernel.org
arch/arm/include/asm/topology.h+1−1
@@ -23,7 +23,7 @@ · 22 unmodified lines
2323
2424 /* Replace task scheduler's default thermal pressure API */
2525 #define arch_scale_thermal_pressure topology_get_thermal_pressure
26−#define arch_set_thermal_pressure topology_set_thermal_pressure
26+#define arch_update_thermal_pressure topology_update_thermal_pressure
2727
2828 #else
2929
arch/arm64/include/asm/topology.h+1−1
@@ -32,7 +32,7 @@ void update_freq_counters_refs(void); · 31 unmodified lines
3232
3333 /* Replace task scheduler's default thermal pressure API */
3434 #define arch_scale_thermal_pressure topology_get_thermal_pressure
35−#define arch_set_thermal_pressure topology_set_thermal_pressure
35+#define arch_update_thermal_pressure topology_update_thermal_pressure
3636
3737 #include <asm-generic/topology.h>
3838
arch/x86/include/asm/cpufeatures.h+1−0
@@ -315,6 +315,7 @@ · 314 unmodified lines
315315 #define X86_FEATURE_AMD_SSBD (13*32+24) /* "" Speculative Store Bypass Disable */
316316 #define X86_FEATURE_VIRT_SSBD (13*32+25) /* Virtualized Speculative Store Bypass Disable */
317317 #define X86_FEATURE_AMD_SSB_NO (13*32+26) /* "" Speculative Store Bypass is fixed in hardware. */
318+#define X86_FEATURE_CPPC (13*32+27) /* Collaborative Processor Performance Control */
318319
319320 /* Thermal and Power Management Leaf, CPUID level 0x00000006 (EAX), word 14 */
320321 #define X86_FEATURE_DTHERM (14*32+ 0) /* Digital Thermal Sensor */
arch/x86/include/asm/msr-index.h+17−0
@@ -486,6 +486,23 @@ · 485 unmodified lines
486486
487487 #define MSR_AMD64_VIRT_SPEC_CTRL 0xc001011f
488488
489+/* AMD Collaborative Processor Performance Control MSRs */
490+#define MSR_AMD_CPPC_CAP1 0xc00102b0
491+#define MSR_AMD_CPPC_ENABLE 0xc00102b1
492+#define MSR_AMD_CPPC_CAP2 0xc00102b2
493+#define MSR_AMD_CPPC_REQ 0xc00102b3
494+#define MSR_AMD_CPPC_STATUS 0xc00102b4
495+
496+#define AMD_CPPC_LOWEST_PERF(x) (((x) >> 0) & 0xff)
497+#define AMD_CPPC_LOWNONLIN_PERF(x) (((x) >> 8) & 0xff)
498+#define AMD_CPPC_NOMINAL_PERF(x) (((x) >> 16) & 0xff)
499+#define AMD_CPPC_HIGHEST_PERF(x) (((x) >> 24) & 0xff)
500+
501+#define AMD_CPPC_MAX_PERF(x) (((x) & 0xff) << 0)
502+#define AMD_CPPC_MIN_PERF(x) (((x) & 0xff) << 8)
503+#define AMD_CPPC_DES_PERF(x) (((x) & 0xff) << 16)
504+#define AMD_CPPC_ENERGY_PERF_PREF(x) (((x) & 0xff) << 24)
505+
489506 /* Fam 17h MSRs */
490507 #define MSR_F17H_IRPERF 0xc00000e9
491508
arch/x86/include/asm/topology.h+1−1
@@ -221,7 +221,7 @@ static inline void arch_set_max_freq_ratio(bool turbo_disabled) · 220 unmodified lines
221221 }
222222 #endif
223223
224−#ifdef CONFIG_ACPI_CPPC_LIB
224+#if defined(CONFIG_ACPI_CPPC_LIB) && defined(CONFIG_SMP)
225225 void init_freq_invariance_cppc(void);
226226 #define init_freq_invariance_cppc init_freq_invariance_cppc
227227 #endif
arch/x86/kernel/acpi/sleep.c+3−1
@@ -139,8 +139,10 @@ static int __init acpi_sleep_setup(char *str) · 138 unmodified lines
139139 if (strncmp(str, "s3_beep", 7) == 0)
140140 acpi_realmode_flags |= 4;
141141 #ifdef CONFIG_HIBERNATION
142+ if (strncmp(str, "s4_hwsig", 8) == 0)
143+ acpi_check_s4_hw_signature(1);
142144 if (strncmp(str, "s4_nohwsig", 10) == 0)
143− acpi_no_s4_hw_signature();
145+ acpi_check_s4_hw_signature(0);
144146 #endif
145147 if (strncmp(str, "nonvs", 5) == 0)
146148 acpi_nvs_nosave();
arch/x86/kernel/cpu/intel_epb.c+32−13
@@ -16,6 +16,7 @@ · 15 unmodified lines
1616 #include <linux/syscore_ops.h>
1717 #include <linux/pm.h>
1818
19+#include <asm/cpu_device_id.h>
1920 #include <asm/cpufeature.h>
2021 #include <asm/msr.h>
2122
@@ -58,6 +59,22 @@ static DEFINE_PER_CPU(u8, saved_epb); · 36 unmodified lines
5859 #define EPB_SAVED 0x10ULL
5960 #define MAX_EPB EPB_MASK
6061
62+enum energy_perf_value_index {
63+ EPB_INDEX_PERFORMANCE,
64+ EPB_INDEX_BALANCE_PERFORMANCE,
65+ EPB_INDEX_NORMAL,
66+ EPB_INDEX_BALANCE_POWERSAVE,
67+ EPB_INDEX_POWERSAVE,
68+};
69+
70+static u8 energ_perf_values[] = {
71+ [EPB_INDEX_PERFORMANCE] = ENERGY_PERF_BIAS_PERFORMANCE,
72+ [EPB_INDEX_BALANCE_PERFORMANCE] = ENERGY_PERF_BIAS_BALANCE_PERFORMANCE,
73+ [EPB_INDEX_NORMAL] = ENERGY_PERF_BIAS_NORMAL,
74+ [EPB_INDEX_BALANCE_POWERSAVE] = ENERGY_PERF_BIAS_BALANCE_POWERSAVE,
75+ [EPB_INDEX_POWERSAVE] = ENERGY_PERF_BIAS_POWERSAVE,
76+};
77+
6178 static int intel_epb_save(void)
6279 {
6380 u64 epb;
@@ -90,7 +107,7 @@ static void intel_epb_restore(void) · 26 unmodified lines
90107 */
91108 val = epb & EPB_MASK;
92109 if (val == ENERGY_PERF_BIAS_PERFORMANCE) {
93− val = ENERGY_PERF_BIAS_NORMAL;
110+ val = energ_perf_values[EPB_INDEX_NORMAL];
94111 pr_warn_once("ENERGY_PERF_BIAS: Set to 'normal', was 'performance'\n");
95112 }
96113 }
@@ -103,18 +120,11 @@ static struct syscore_ops intel_epb_syscore_ops = { · 6 unmodified lines
103120 };
104121
105122 static const char * const energy_perf_strings[] = {
106− "performance",
107− "balance-performance",
108− "normal",
109− "balance-power",
110− "power"
111−};
112−static const u8 energ_perf_values[] = {
113− ENERGY_PERF_BIAS_PERFORMANCE,
114− ENERGY_PERF_BIAS_BALANCE_PERFORMANCE,
115− ENERGY_PERF_BIAS_NORMAL,
116− ENERGY_PERF_BIAS_BALANCE_POWERSAVE,
117− ENERGY_PERF_BIAS_POWERSAVE
123+ [EPB_INDEX_PERFORMANCE] = "performance",
124+ [EPB_INDEX_BALANCE_PERFORMANCE] = "balance-performance",
125+ [EPB_INDEX_NORMAL] = "normal",
126+ [EPB_INDEX_BALANCE_POWERSAVE] = "balance-power",
127+ [EPB_INDEX_POWERSAVE] = "power",
118128 };
119129
120130 static ssize_t energy_perf_bias_show(struct device *dev,
@@ -193,13 +203,22 @@ static int intel_epb_offline(unsigned int cpu) · 72 unmodified lines
193203 return 0;
194204 }
195205
206+static const struct x86_cpu_id intel_epb_normal[] = {
207+ X86_MATCH_INTEL_FAM6_MODEL(ALDERLAKE_L, 7),
208+ {}
209+};
210+
196211 static __init int intel_epb_init(void)
197212 {
213+ const struct x86_cpu_id *id = x86_match_cpu(intel_epb_normal);
198214 int ret;
199215
200216 if (!boot_cpu_has(X86_FEATURE_EPB))
201217 return -ENODEV;
202218
219+ if (id)
220+ energ_perf_values[EPB_INDEX_NORMAL] = id->driver_data;
221+
203222 ret = cpuhp_setup_state(CPUHP_AP_X86_INTEL_EPB_ONLINE,
204223 "x86/intel/epb:online", intel_epb_online,
205224 intel_epb_offline);
drivers/acpi/cppc_acpi.c+95−4
@@ -118,6 +118,8 @@ static DEFINE_PER_CPU(struct cpc_desc *, cpc_desc_ptr); · 117 unmodified lines
118118 */
119119 #define NUM_RETRIES 500ULL
120120
121+#define OVER_16BTS_MASK ~0xFFFFULL
122+
121123 #define define_one_cppc_ro(_name) \
122124 static struct kobj_attribute _name = \
123125 __ATTR(_name, 0444, show_##_name, NULL)
@@ -412,7 +414,7 @@ bool acpi_cpc_valid(void) · 288 unmodified lines
412414 struct cpc_desc *cpc_ptr;
413415 int cpu;
414416
415− for_each_possible_cpu(cpu) {
417+ for_each_present_cpu(cpu) {
416418 cpc_ptr = per_cpu(cpc_desc_ptr, cpu);
417419 if (!cpc_ptr)
418420 return false;
@@ -730,9 +732,26 @@ int acpi_cppc_processor_probe(struct acpi_processor *pr) · 311 unmodified lines
730732 goto out_free;
731733 cpc_ptr->cpc_regs[i-2].sys_mem_vaddr = addr;
732734 }
735+ } else if (gas_t->space_id == ACPI_ADR_SPACE_SYSTEM_IO) {
736+ if (gas_t->access_width < 1 || gas_t->access_width > 3) {
737+ /*
738+ * 1 = 8-bit, 2 = 16-bit, and 3 = 32-bit.
739+ * SystemIO doesn't implement 64-bit
740+ * registers.
741+ */
742+ pr_debug("Invalid access width %d for SystemIO register\n",
743+ gas_t->access_width);
744+ goto out_free;
745+ }
746+ if (gas_t->address & OVER_16BTS_MASK) {
747+ /* SystemIO registers use 16-bit integer addresses */
748+ pr_debug("Invalid IO port %llu for SystemIO register\n",
749+ gas_t->address);
750+ goto out_free;
751+ }
733752 } else {
734753 if (gas_t->space_id != ACPI_ADR_SPACE_FIXED_HARDWARE || !cpc_ffh_supported()) {
735− /* Support only PCC ,SYS MEM and FFH type regs */
754+ /* Support only PCC, SystemMemory, SystemIO, and FFH type regs. */
736755 pr_debug("Unsupported register type: %d\n", gas_t->space_id);
737756 goto out_free;
738757 }
@@ -907,7 +926,21 @@ static int cpc_read(int cpu, struct cpc_register_resource *reg_res, u64 *val) · 168 unmodified lines
907926 }
908927
909928 *val = 0;
910− if (reg->space_id == ACPI_ADR_SPACE_PLATFORM_COMM && pcc_ss_id >= 0)
929+
930+ if (reg->space_id == ACPI_ADR_SPACE_SYSTEM_IO) {
931+ u32 width = 8 << (reg->access_width - 1);
932+ acpi_status status;
933+
934+ status = acpi_os_read_port((acpi_io_address)reg->address,
935+ (u32 *)val, width);
936+ if (ACPI_FAILURE(status)) {
937+ pr_debug("Error: Failed to read SystemIO port %llx\n",
938+ reg->address);
939+ return -EFAULT;
940+ }
941+
942+ return 0;
943+ } else if (reg->space_id == ACPI_ADR_SPACE_PLATFORM_COMM && pcc_ss_id >= 0)
911944 vaddr = GET_PCC_VADDR(reg->address, pcc_ss_id);
912945 else if (reg->space_id == ACPI_ADR_SPACE_SYSTEM_MEMORY)
913946 vaddr = reg_res->sys_mem_vaddr;
@@ -946,7 +979,20 @@ static int cpc_write(int cpu, struct cpc_register_resource *reg_res, u64 val) · 32 unmodified lines
946979 int pcc_ss_id = per_cpu(cpu_pcc_subspace_idx, cpu);
947980 struct cpc_reg *reg = &reg_res->cpc_entry.reg;
948981
949− if (reg->space_id == ACPI_ADR_SPACE_PLATFORM_COMM && pcc_ss_id >= 0)
982+ if (reg->space_id == ACPI_ADR_SPACE_SYSTEM_IO) {
983+ u32 width = 8 << (reg->access_width - 1);
984+ acpi_status status;
985+
986+ status = acpi_os_write_port((acpi_io_address)reg->address,
987+ (u32)val, width);
988+ if (ACPI_FAILURE(status)) {
989+ pr_debug("Error: Failed to write SystemIO port %llx\n",
990+ reg->address);
991+ return -EFAULT;
992+ }
993+
994+ return 0;
995+ } else if (reg->space_id == ACPI_ADR_SPACE_PLATFORM_COMM && pcc_ss_id >= 0)
950996 vaddr = GET_PCC_VADDR(reg->address, pcc_ss_id);
951997 else if (reg->space_id == ACPI_ADR_SPACE_SYSTEM_MEMORY)
952998 vaddr = reg_res->sys_mem_vaddr;
@@ -1213,6 +1259,51 @@ out_err: · 260 unmodified lines
12131259 }
12141260 EXPORT_SYMBOL_GPL(cppc_get_perf_ctrs);
12151261
1262+/**
1263+ * cppc_set_enable - Set to enable CPPC on the processor by writing the
1264+ * Continuous Performance Control package EnableRegister field.
1265+ * @cpu: CPU for which to enable CPPC register.
1266+ * @enable: 0 - disable, 1 - enable CPPC feature on the processor.
1267+ *
1268+ * Return: 0 for success, -ERRNO or -EIO otherwise.
1269+ */
1270+int cppc_set_enable(int cpu, bool enable)
1271+{
1272+ int pcc_ss_id = per_cpu(cpu_pcc_subspace_idx, cpu);
1273+ struct cpc_register_resource *enable_reg;
1274+ struct cpc_desc *cpc_desc = per_cpu(cpc_desc_ptr, cpu);
1275+ struct cppc_pcc_data *pcc_ss_data = NULL;
1276+ int ret = -EINVAL;
1277+
1278+ if (!cpc_desc) {
1279+ pr_debug("No CPC descriptor for CPU:%d\n", cpu);
1280+ return -EINVAL;
1281+ }
1282+
1283+ enable_reg = &cpc_desc->cpc_regs[ENABLE];
1284+
1285+ if (CPC_IN_PCC(enable_reg)) {
1286+
1287+ if (pcc_ss_id < 0)
1288+ return -EIO;
1289+
1290+ ret = cpc_write(cpu, enable_reg, enable);
1291+ if (ret)
1292+ return ret;
1293+
1294+ pcc_ss_data = pcc_data[pcc_ss_id];
1295+
1296+ down_write(&pcc_ss_data->pcc_lock);
1297+ /* after writing CPC, transfer the ownership of PCC to platfrom */
1298+ ret = send_pcc_cmd(pcc_ss_id, CMD_WRITE);
1299+ up_write(&pcc_ss_data->pcc_lock);
1300+ return ret;
1301+ }
1302+
1303+ return cpc_write(cpu, enable_reg, enable);
1304+}
1305+EXPORT_SYMBOL_GPL(cppc_set_enable);
1306+
12161307 /**
12171308 * cppc_set_perf - Set a CPU's performance controls.
12181309 * @cpu: CPU for which to set performance controls.
drivers/acpi/sleep.c+21−5
@@ -874,11 +874,11 @@ static inline void acpi_sleep_syscore_init(void) {} · 873 unmodified lines
874874 #ifdef CONFIG_HIBERNATION
875875 static unsigned long s4_hardware_signature;
876876 static struct acpi_table_facs *facs;
877−static bool nosigcheck;
877+static int sigcheck = -1; /* Default behaviour is just to warn */
878878
879−void __init acpi_no_s4_hw_signature(void)
879+void __init acpi_check_s4_hw_signature(int check)
880880 {
881− nosigcheck = true;
881+ sigcheck = check;
882882 }
883883
884884 static int acpi_hibernation_begin(pm_message_t stage)
@@ -1004,12 +1004,28 @@ static void acpi_sleep_hibernate_setup(void) · 119 unmodified lines
10041004 hibernation_set_ops(old_suspend_ordering ?
10051005 &acpi_hibernation_ops_old : &acpi_hibernation_ops);
10061006 sleep_states[ACPI_STATE_S4] = 1;
1007− if (nosigcheck)
1007+ if (!sigcheck)
10081008 return;
10091009
10101010 acpi_get_table(ACPI_SIG_FACS, 1, (struct acpi_table_header **)&facs);
1011− if (facs)
1011+ if (facs) {
1012+ /*
1013+ * s4_hardware_signature is the local variable which is just
1014+ * used to warn about mismatch after we're attempting to
1015+ * resume (in violation of the ACPI specification.)
1016+ */
10121017 s4_hardware_signature = facs->hardware_signature;
1018+
1019+ if (sigcheck > 0) {
1020+ /*
1021+ * If we're actually obeying the ACPI specification
1022+ * then the signature is written out as part of the
1023+ * swsusp header, in order to allow the boot kernel
1024+ * to gracefully decline to resume.
1025+ */
1026+ swsusp_hardware_signature = facs->hardware_signature;
1027+ }
1028+ }
10131029 }
10141030 #else /* !CONFIG_HIBERNATION */
10151031 static inline void acpi_sleep_hibernate_setup(void) {}
drivers/base/arch_topology.c+38−4
@@ -22,6 +22,7 @@ · 21 unmodified lines
2222 static DEFINE_PER_CPU(struct scale_freq_data __rcu *, sft_data);
2323 static struct cpumask scale_freq_counters_mask;
2424 static bool scale_freq_invariant;
25+static DEFINE_PER_CPU(u32, freq_factor) = 1;
2526
2627 static bool supports_scale_freq_counters(const struct cpumask *cpus)
2728 {
@@ -155,15 +156,49 @@ void topology_set_cpu_scale(unsigned int cpu, unsigned long capacity) · 127 unmodified lines
155156
156157 DEFINE_PER_CPU(unsigned long, thermal_pressure);
157158
158−void topology_set_thermal_pressure(const struct cpumask *cpus,
159− unsigned long th_pressure)
159+/**
160+ * topology_update_thermal_pressure() - Update thermal pressure for CPUs
161+ * @cpus : The related CPUs for which capacity has been reduced
162+ * @capped_freq : The maximum allowed frequency that CPUs can run at
163+ *
164+ * Update the value of thermal pressure for all @cpus in the mask. The
165+ * cpumask should include all (online+offline) affected CPUs, to avoid
166+ * operating on stale data when hot-plug is used for some CPUs. The
167+ * @capped_freq reflects the currently allowed max CPUs frequency due to
168+ * thermal capping. It might be also a boost frequency value, which is bigger
169+ * than the internal 'freq_factor' max frequency. In such case the pressure
170+ * value should simply be removed, since this is an indication that there is
171+ * no thermal throttling. The @capped_freq must be provided in kHz.
172+ */
173+void topology_update_thermal_pressure(const struct cpumask *cpus,
174+ unsigned long capped_freq)
160175 {
176+ unsigned long max_capacity, capacity, th_pressure;
177+ u32 max_freq;
161178 int cpu;
162179
180+ cpu = cpumask_first(cpus);
181+ max_capacity = arch_scale_cpu_capacity(cpu);
182+ max_freq = per_cpu(freq_factor, cpu);
183+
184+ /* Convert to MHz scale which is used in 'freq_factor' */
185+ capped_freq /= 1000;
186+
187+ /*
188+ * Handle properly the boost frequencies, which should simply clean
189+ * the thermal pressure value.
190+ */
191+ if (max_freq <= capped_freq)
192+ capacity = max_capacity;
193+ else
194+ capacity = mult_frac(max_capacity, capped_freq, max_freq);
195+
196+ th_pressure = max_capacity - capacity;
197+
163198 for_each_cpu(cpu, cpus)
164199 WRITE_ONCE(per_cpu(thermal_pressure, cpu), th_pressure);
165200 }
166−EXPORT_SYMBOL_GPL(topology_set_thermal_pressure);
201+EXPORT_SYMBOL_GPL(topology_update_thermal_pressure);
167202
168203 static ssize_t cpu_capacity_show(struct device *dev,
169204 struct device_attribute *attr,
@@ -217,7 +252,6 @@ static void update_topology_flags_workfn(struct work_struct *work) · 47 unmodified lines
217252 update_topology = 0;
218253 }
219254
220−static DEFINE_PER_CPU(u32, freq_factor) = 1;
221255 static u32 *raw_capacity;
222256
223257 static int free_raw_capacity(void)
drivers/base/core.c+1−2
@@ -485,8 +485,7 @@ static void device_link_release_fn(struct work_struct *work) · 484 unmodified lines
485485 /* Ensure that all references to the link object have been dropped. */
486486 device_link_synchronize_removal();
487487
488− while (refcount_dec_not_one(&link->rpm_active))
489− pm_runtime_put(link->supplier);
488+ pm_runtime_release_supplier(link, true);
490489
491490 put_device(link->consumer);
492491 put_device(link->supplier);
drivers/base/power/runtime.c+63−35
@@ -305,19 +305,40 @@ static int rpm_get_suppliers(struct device *dev) · 304 unmodified lines
305305 return 0;
306306 }
307307
308+/**
309+ * pm_runtime_release_supplier - Drop references to device link's supplier.
310+ * @link: Target device link.
311+ * @check_idle: Whether or not to check if the supplier device is idle.
312+ *
313+ * Drop all runtime PM references associated with @link to its supplier device
314+ * and if @check_idle is set, check if that device is idle (and so it can be
315+ * suspended).
316+ */
317+void pm_runtime_release_supplier(struct device_link *link, bool check_idle)
318+{
319+ struct device *supplier = link->supplier;
320+
321+ /*
322+ * The additional power.usage_count check is a safety net in case
323+ * the rpm_active refcount becomes saturated, in which case
324+ * refcount_dec_not_one() would return true forever, but it is not
325+ * strictly necessary.
326+ */
327+ while (refcount_dec_not_one(&link->rpm_active) &&
328+ atomic_read(&supplier->power.usage_count) > 0)
329+ pm_runtime_put_noidle(supplier);
330+
331+ if (check_idle)
332+ pm_request_idle(supplier);
333+}
334+
308335 static void __rpm_put_suppliers(struct device *dev, bool try_to_suspend)
309336 {
310337 struct device_link *link;
311338
312339 list_for_each_entry_rcu(link, &dev->links.suppliers, c_node,
313− device_links_read_lock_held()) {
314−
315− while (refcount_dec_not_one(&link->rpm_active))
316− pm_runtime_put_noidle(link->supplier);
317−
318− if (try_to_suspend)
319− pm_request_idle(link->supplier);
320− }
340+ device_links_read_lock_held())
341+ pm_runtime_release_supplier(link, try_to_suspend);
321342 }
322343
323344 static void rpm_put_suppliers(struct device *dev)
@@ -742,13 +763,15 @@ static int rpm_resume(struct device *dev, int rpmflags) · 418 unmodified lines
742763 trace_rpm_resume_rcuidle(dev, rpmflags);
743764
744765 repeat:
745− if (dev->power.runtime_error)
766+ if (dev->power.runtime_error) {
746767 retval = -EINVAL;
747− else if (dev->power.disable_depth == 1 && dev->power.is_suspended
748− && dev->power.runtime_status == RPM_ACTIVE)
749− retval = 1;
750− else if (dev->power.disable_depth > 0)
751− retval = -EACCES;
768+ } else if (dev->power.disable_depth > 0) {
769+ if (dev->power.runtime_status == RPM_ACTIVE &&
770+ dev->power.last_status == RPM_ACTIVE)
771+ retval = 1;
772+ else
773+ retval = -EACCES;
774+ }
752775 if (retval)
753776 goto out;
754777
@@ -1410,8 +1433,10 @@ void __pm_runtime_disable(struct device *dev, bool check_resume) · 655 unmodified lines
14101433 /* Update time accounting before disabling PM-runtime. */
14111434 update_pm_runtime_accounting(dev);
14121435
1413− if (!dev->power.disable_depth++)
1436+ if (!dev->power.disable_depth++) {
14141437 __pm_runtime_barrier(dev);
1438+ dev->power.last_status = dev->power.runtime_status;
1439+ }
14151440
14161441 out:
14171442 spin_unlock_irq(&dev->power.lock);
@@ -1428,23 +1453,23 @@ void pm_runtime_enable(struct device *dev) · 10 unmodified lines
14281453
14291454 spin_lock_irqsave(&dev->power.lock, flags);
14301455
1431− if (dev->power.disable_depth > 0) {
1432− dev->power.disable_depth--;
1433−
1434− /* About to enable runtime pm, set accounting_timestamp to now */
1435− if (!dev->power.disable_depth)
1436− dev->power.accounting_timestamp = ktime_get_mono_fast_ns();
1437− } else {
1456+ if (!dev->power.disable_depth) {
14381457 dev_warn(dev, "Unbalanced %s!\n", __func__);
1458+ goto out;
14391459 }
14401460
1441− WARN(!dev->power.disable_depth &&
1442− dev->power.runtime_status == RPM_SUSPENDED &&
1443− !dev->power.ignore_children &&
1444− atomic_read(&dev->power.child_count) > 0,
1445− "Enabling runtime PM for inactive device (%s) with active children\n",
1446− dev_name(dev));
1461+ if (--dev->power.disable_depth > 0)
1462+ goto out;
1463+
1464+ dev->power.last_status = RPM_INVALID;
1465+ dev->power.accounting_timestamp = ktime_get_mono_fast_ns();
1466+
1467+ if (dev->power.runtime_status == RPM_SUSPENDED &&
1468+ !dev->power.ignore_children &&
1469+ atomic_read(&dev->power.child_count) > 0)
1470+ dev_warn(dev, "Enabling runtime PM for inactive device with active children\n");
14471471
1472+out:
14481473 spin_unlock_irqrestore(&dev->power.lock, flags);
14491474 }
14501475 EXPORT_SYMBOL_GPL(pm_runtime_enable);
@@ -1640,6 +1665,7 @@ EXPORT_SYMBOL_GPL(__pm_runtime_use_autosuspend); · 189 unmodified lines
16401665 void pm_runtime_init(struct device *dev)
16411666 {
16421667 dev->power.runtime_status = RPM_SUSPENDED;
1668+ dev->power.last_status = RPM_INVALID;
16431669 dev->power.idle_notification = false;
16441670
16451671 dev->power.disable_depth = 1;
@@ -1722,8 +1748,6 @@ void pm_runtime_get_suppliers(struct device *dev) · 76 unmodified lines
17221748 void pm_runtime_put_suppliers(struct device *dev)
17231749 {
17241750 struct device_link *link;
1725− unsigned long flags;
1726− bool put;
17271751 int idx;
17281752
17291753 idx = device_links_read_lock();
@@ -1731,11 +1755,17 @@ void pm_runtime_put_suppliers(struct device *dev) · 1 unmodified line
17311755 list_for_each_entry_rcu(link, &dev->links.suppliers, c_node,
17321756 device_links_read_lock_held())
17331757 if (link->supplier_preactivated) {
1758+ bool put;
1759+
17341760 link->supplier_preactivated = false;
1735− spin_lock_irqsave(&dev->power.lock, flags);
1761+
1762+ spin_lock_irq(&dev->power.lock);
1763+
17361764 put = pm_runtime_status_suspended(dev) &&
17371765 refcount_dec_not_one(&link->rpm_active);
1738− spin_unlock_irqrestore(&dev->power.lock, flags);
1766+
1767+ spin_unlock_irq(&dev->power.lock);
1768+
17391769 if (put)
17401770 pm_runtime_put(link->supplier);
17411771 }
@@ -1772,9 +1802,7 @@ void pm_runtime_drop_link(struct device_link *link) · 30 unmodified lines
17721802 return;
17731803
17741804 pm_runtime_drop_link_count(link->consumer);
1775−
1776− while (refcount_dec_not_one(&link->rpm_active))
1777− pm_runtime_put(link->supplier);
1805+ pm_runtime_release_supplier(link, true);
17781806 }
17791807
17801808 static bool pm_runtime_need_not_resume(struct device *dev)
drivers/cpufreq/Kconfig.x86+17−0
@@ -34,6 +34,23 @@ config X86_PCC_CPUFREQ · 33 unmodified lines
3434
3535 If in doubt, say N.
3636
37+config X86_AMD_PSTATE
38+ tristate "AMD Processor P-State driver"
39+ depends on X86 && ACPI
40+ select ACPI_PROCESSOR
41+ select ACPI_CPPC_LIB if X86_64
42+ select CPU_FREQ_GOV_SCHEDUTIL if SMP
43+ help
44+ This driver adds a CPUFreq driver which utilizes a fine grain
45+ processor performance frequency control range instead of legacy
46+ performance levels. _CPC needs to be present in the ACPI tables
47+ of the system.
48+
49+ For details, take a look at:
50+ <file:Documentation/admin-guide/pm/amd-pstate.rst>.
51+
52+ If in doubt, say N.
53+
3754 config X86_ACPI_CPUFREQ
3855 tristate "ACPI Processor P-States driver"
3956 depends on ACPI_PROCESSOR
drivers/cpufreq/Makefile+5−0
@@ -17,6 +17,10 @@ obj-$(CONFIG_CPU_FREQ_GOV_ATTR_SET) += cpufreq_governor_attr_set.o · 16 unmodified lines
1717 obj-$(CONFIG_CPUFREQ_DT) += cpufreq-dt.o
1818 obj-$(CONFIG_CPUFREQ_DT_PLATDEV) += cpufreq-dt-platdev.o
1919
20+# Traces
21+CFLAGS_amd-pstate-trace.o := -I$(src)
22+amd_pstate-y := amd-pstate.o amd-pstate-trace.o
23+
2024 ##################################################################################
2125 # x86 drivers.
2226 # Link order matters. K8 is preferred to ACPI because of firmware bugs in early
@@ -25,6 +29,7 @@ obj-$(CONFIG_CPUFREQ_DT_PLATDEV) += cpufreq-dt-platdev.o · 2 unmodified lines
2529 # speedstep-* is preferred over p4-clockmod.
2630
2731 obj-$(CONFIG_X86_ACPI_CPUFREQ) += acpi-cpufreq.o
32+obj-$(CONFIG_X86_AMD_PSTATE) += amd_pstate.o
2833 obj-$(CONFIG_X86_POWERNOW_K8) += powernow-k8.o
2934 obj-$(CONFIG_X86_PCC_CPUFREQ) += pcc-cpufreq.o
3035 obj-$(CONFIG_X86_POWERNOW_K6) += powernow-k6.o
drivers/cpufreq/amd-pstate-trace.c+2−0
@@ -0,0 +1,2 @@
1+#define CREATE_TRACE_POINTS
2+#include "amd-pstate-trace.h"
drivers/cpufreq/amd-pstate-trace.h+77−0
@@ -0,0 +1,77 @@
1+/* SPDX-License-Identifier: GPL-2.0 */
2+/*
3+ * amd-pstate-trace.h - AMD Processor P-state Frequency Driver Tracer
4+ *
5+ * Copyright (C) 2021 Advanced Micro Devices, Inc. All Rights Reserved.
6+ *
7+ * Author: Huang Rui <ray.huang@amd.com>
8+ */
9+
10+#if !defined(_AMD_PSTATE_TRACE_H) || defined(TRACE_HEADER_MULTI_READ)
11+#define _AMD_PSTATE_TRACE_H
12+
13+#include <linux/cpufreq.h>
14+#include <linux/tracepoint.h>
15+#include <linux/trace_events.h>
16+
17+#undef TRACE_SYSTEM
18+#define TRACE_SYSTEM amd_cpu
19+
20+#undef TRACE_INCLUDE_FILE
21+#define TRACE_INCLUDE_FILE amd-pstate-trace
22+
23+#define TPS(x) tracepoint_string(x)
24+
25+TRACE_EVENT(amd_pstate_perf,
26+
27+ TP_PROTO(unsigned long min_perf,
28+ unsigned long target_perf,
29+ unsigned long capacity,
30+ unsigned int cpu_id,
31+ bool changed,
32+ bool fast_switch
33+ ),
34+
35+ TP_ARGS(min_perf,
36+ target_perf,
37+ capacity,
38+ cpu_id,
39+ changed,
40+ fast_switch
41+ ),
42+
43+ TP_STRUCT__entry(
44+ __field(unsigned long, min_perf)
45+ __field(unsigned long, target_perf)
46+ __field(unsigned long, capacity)
47+ __field(unsigned int, cpu_id)
48+ __field(bool, changed)
49+ __field(bool, fast_switch)
50+ ),
51+
52+ TP_fast_assign(
53+ __entry->min_perf = min_perf;
54+ __entry->target_perf = target_perf;
55+ __entry->capacity = capacity;
56+ __entry->cpu_id = cpu_id;
57+ __entry->changed = changed;
58+ __entry->fast_switch = fast_switch;
59+ ),
60+
61+ TP_printk("amd_min_perf=%lu amd_des_perf=%lu amd_max_perf=%lu cpu_id=%u changed=%s fast_switch=%s",
62+ (unsigned long)__entry->min_perf,
63+ (unsigned long)__entry->target_perf,
64+ (unsigned long)__entry->capacity,
65+ (unsigned int)__entry->cpu_id,
66+ (__entry->changed) ? "true" : "false",
67+ (__entry->fast_switch) ? "true" : "false"
68+ )
69+);
70+
71+#endif /* _AMD_PSTATE_TRACE_H */
72+
73+/* This part must be outside protection */
74+#undef TRACE_INCLUDE_PATH
75+#define TRACE_INCLUDE_PATH .
76+
77+#include <trace/define_trace.h>
drivers/cpufreq/amd-pstate.c+645−0
@@ -0,0 +1,645 @@
1+// SPDX-License-Identifier: GPL-2.0-or-later
2+/*
3+ * amd-pstate.c - AMD Processor P-state Frequency Driver
4+ *
5+ * Copyright (C) 2021 Advanced Micro Devices, Inc. All Rights Reserved.
6+ *
7+ * Author: Huang Rui <ray.huang@amd.com>
8+ *
9+ * AMD P-State introduces a new CPU performance scaling design for AMD
10+ * processors using the ACPI Collaborative Performance and Power Control (CPPC)
11+ * feature which works with the AMD SMU firmware providing a finer grained
12+ * frequency control range. It is to replace the legacy ACPI P-States control,
13+ * allows a flexible, low-latency interface for the Linux kernel to directly
14+ * communicate the performance hints to hardware.
15+ *
16+ * AMD P-State is supported on recent AMD Zen base CPU series include some of
17+ * Zen2 and Zen3 processors. _CPC needs to be present in the ACPI tables of AMD
18+ * P-State supported system. And there are two types of hardware implementations
19+ * for AMD P-State: 1) Full MSR Solution and 2) Shared Memory Solution.
20+ * X86_FEATURE_CPPC CPU feature flag is used to distinguish the different types.
21+ */
22+
23+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
24+
25+#include <linux/kernel.h>
26+#include <linux/module.h>
27+#include <linux/init.h>
28+#include <linux/smp.h>
29+#include <linux/sched.h>
30+#include <linux/cpufreq.h>
31+#include <linux/compiler.h>
32+#include <linux/dmi.h>
33+#include <linux/slab.h>
34+#include <linux/acpi.h>
35+#include <linux/io.h>
36+#include <linux/delay.h>
37+#include <linux/uaccess.h>
38+#include <linux/static_call.h>
39+
40+#include <acpi/processor.h>
41+#include <acpi/cppc_acpi.h>
42+
43+#include <asm/msr.h>
44+#include <asm/processor.h>
45+#include <asm/cpufeature.h>
46+#include <asm/cpu_device_id.h>
47+#include "amd-pstate-trace.h"
48+
49+#define AMD_PSTATE_TRANSITION_LATENCY 0x20000
50+#define AMD_PSTATE_TRANSITION_DELAY 500
51+
52+/*
53+ * TODO: We need more time to fine tune processors with shared memory solution
54+ * with community together.
55+ *
56+ * There are some performance drops on the CPU benchmarks which reports from
57+ * Suse. We are co-working with them to fine tune the shared memory solution. So
58+ * we disable it by default to go acpi-cpufreq on these processors and add a
59+ * module parameter to be able to enable it manually for debugging.
60+ */
61+static bool shared_mem = false;
62+module_param(shared_mem, bool, 0444);
63+MODULE_PARM_DESC(shared_mem,
64+ "enable amd-pstate on processors with shared memory solution (false = disabled (default), true = enabled)");
65+
66+static struct cpufreq_driver amd_pstate_driver;
67+
68+/**
69+ * struct amd_cpudata - private CPU data for AMD P-State
70+ * @cpu: CPU number
71+ * @req: constraint request to apply
72+ * @cppc_req_cached: cached performance request hints
73+ * @highest_perf: the maximum performance an individual processor may reach,
74+ * assuming ideal conditions
75+ * @nominal_perf: the maximum sustained performance level of the processor,
76+ * assuming ideal operating conditions
77+ * @lowest_nonlinear_perf: the lowest performance level at which nonlinear power
78+ * savings are achieved
79+ * @lowest_perf: the absolute lowest performance level of the processor
80+ * @max_freq: the frequency that mapped to highest_perf
81+ * @min_freq: the frequency that mapped to lowest_perf
82+ * @nominal_freq: the frequency that mapped to nominal_perf
83+ * @lowest_nonlinear_freq: the frequency that mapped to lowest_nonlinear_perf
84+ * @boost_supported: check whether the Processor or SBIOS supports boost mode
85+ *
86+ * The amd_cpudata is key private data for each CPU thread in AMD P-State, and
87+ * represents all the attributes and goals that AMD P-State requests at runtime.
88+ */
89+struct amd_cpudata {
90+ int cpu;
91+
92+ struct freq_qos_request req[2];
93+ u64 cppc_req_cached;
94+
95+ u32 highest_perf;
96+ u32 nominal_perf;
97+ u32 lowest_nonlinear_perf;
98+ u32 lowest_perf;
99+
100+ u32 max_freq;
101+ u32 min_freq;
102+ u32 nominal_freq;
103+ u32 lowest_nonlinear_freq;
104+
105+ bool boost_supported;
106+};
107+
108+static inline int pstate_enable(bool enable)
109+{
110+ return wrmsrl_safe(MSR_AMD_CPPC_ENABLE, enable);
111+}
112+
113+static int cppc_enable(bool enable)
114+{
115+ int cpu, ret = 0;
116+
117+ for_each_present_cpu(cpu) {
118+ ret = cppc_set_enable(cpu, enable);
119+ if (ret)
120+ return ret;
121+ }
122+
123+ return ret;
124+}
125+
126+DEFINE_STATIC_CALL(amd_pstate_enable, pstate_enable);
127+
128+static inline int amd_pstate_enable(bool enable)
129+{
130+ return static_call(amd_pstate_enable)(enable);
131+}
132+
133+static int pstate_init_perf(struct amd_cpudata *cpudata)
134+{
135+ u64 cap1;
136+
137+ int ret = rdmsrl_safe_on_cpu(cpudata->cpu, MSR_AMD_CPPC_CAP1,
138+ &cap1);
139+ if (ret)
140+ return ret;
141+
142+ /*
143+ * TODO: Introduce AMD specific power feature.
144+ *
145+ * CPPC entry doesn't indicate the highest performance in some ASICs.
146+ */
147+ WRITE_ONCE(cpudata->highest_perf, amd_get_highest_perf());
148+
149+ WRITE_ONCE(cpudata->nominal_perf, AMD_CPPC_NOMINAL_PERF(cap1));
150+ WRITE_ONCE(cpudata->lowest_nonlinear_perf, AMD_CPPC_LOWNONLIN_PERF(cap1));
151+ WRITE_ONCE(cpudata->lowest_perf, AMD_CPPC_LOWEST_PERF(cap1));
152+
153+ return 0;
154+}
155+
156+static int cppc_init_perf(struct amd_cpudata *cpudata)
157+{
158+ struct cppc_perf_caps cppc_perf;
159+
160+ int ret = cppc_get_perf_caps(cpudata->cpu, &cppc_perf);
161+ if (ret)
162+ return ret;
163+
164+ WRITE_ONCE(cpudata->highest_perf, amd_get_highest_perf());
165+
166+ WRITE_ONCE(cpudata->nominal_perf, cppc_perf.nominal_perf);
167+ WRITE_ONCE(cpudata->lowest_nonlinear_perf,
168+ cppc_perf.lowest_nonlinear_perf);
169+ WRITE_ONCE(cpudata->lowest_perf, cppc_perf.lowest_perf);
170+
171+ return 0;
172+}
173+
174+DEFINE_STATIC_CALL(amd_pstate_init_perf, pstate_init_perf);
175+
176+static inline int amd_pstate_init_perf(struct amd_cpudata *cpudata)
177+{
178+ return static_call(amd_pstate_init_perf)(cpudata);
179+}
180+
181+static void pstate_update_perf(struct amd_cpudata *cpudata, u32 min_perf,
182+ u32 des_perf, u32 max_perf, bool fast_switch)
183+{
184+ if (fast_switch)
185+ wrmsrl(MSR_AMD_CPPC_REQ, READ_ONCE(cpudata->cppc_req_cached));
186+ else
187+ wrmsrl_on_cpu(cpudata->cpu, MSR_AMD_CPPC_REQ,
188+ READ_ONCE(cpudata->cppc_req_cached));
189+}
190+
191+static void cppc_update_perf(struct amd_cpudata *cpudata,
192+ u32 min_perf, u32 des_perf,
193+ u32 max_perf, bool fast_switch)
194+{
195+ struct cppc_perf_ctrls perf_ctrls;
196+
197+ perf_ctrls.max_perf = max_perf;
198+ perf_ctrls.min_perf = min_perf;
199+ perf_ctrls.desired_perf = des_perf;
200+
201+ cppc_set_perf(cpudata->cpu, &perf_ctrls);
202+}
203+
204+DEFINE_STATIC_CALL(amd_pstate_update_perf, pstate_update_perf);
205+
206+static inline void amd_pstate_update_perf(struct amd_cpudata *cpudata,
207+ u32 min_perf, u32 des_perf,
208+ u32 max_perf, bool fast_switch)
209+{
210+ static_call(amd_pstate_update_perf)(cpudata, min_perf, des_perf,
211+ max_perf, fast_switch);
212+}
213+
214+static void amd_pstate_update(struct amd_cpudata *cpudata, u32 min_perf,
215+ u32 des_perf, u32 max_perf, bool fast_switch)
216+{
217+ u64 prev = READ_ONCE(cpudata->cppc_req_cached);
218+ u64 value = prev;
219+
220+ value &= ~AMD_CPPC_MIN_PERF(~0L);
221+ value |= AMD_CPPC_MIN_PERF(min_perf);
222+
223+ value &= ~AMD_CPPC_DES_PERF(~0L);
224+ value |= AMD_CPPC_DES_PERF(des_perf);
225+
226+ value &= ~AMD_CPPC_MAX_PERF(~0L);
227+ value |= AMD_CPPC_MAX_PERF(max_perf);
228+
229+ trace_amd_pstate_perf(min_perf, des_perf, max_perf,
230+ cpudata->cpu, (value != prev), fast_switch);
231+
232+ if (value == prev)
233+ return;
234+
235+ WRITE_ONCE(cpudata->cppc_req_cached, value);
236+
237+ amd_pstate_update_perf(cpudata, min_perf, des_perf,
238+ max_perf, fast_switch);
239+}
240+
241+static int amd_pstate_verify(struct cpufreq_policy_data *policy)
242+{
243+ cpufreq_verify_within_cpu_limits(policy);
244+
245+ return 0;
246+}
247+
248+static int amd_pstate_target(struct cpufreq_policy *policy,
249+ unsigned int target_freq,
250+ unsigned int relation)
251+{
252+ struct cpufreq_freqs freqs;
253+ struct amd_cpudata *cpudata = policy->driver_data;
254+ unsigned long max_perf, min_perf, des_perf, cap_perf;
255+
256+ if (!cpudata->max_freq)
257+ return -ENODEV;
258+
259+ cap_perf = READ_ONCE(cpudata->highest_perf);
260+ min_perf = READ_ONCE(cpudata->lowest_nonlinear_perf);
261+ max_perf = cap_perf;
262+
263+ freqs.old = policy->cur;
264+ freqs.new = target_freq;
265+
266+ des_perf = DIV_ROUND_CLOSEST(target_freq * cap_perf,
267+ cpudata->max_freq);
268+
269+ cpufreq_freq_transition_begin(policy, &freqs);
270+ amd_pstate_update(cpudata, min_perf, des_perf,
271+ max_perf, false);
272+ cpufreq_freq_transition_end(policy, &freqs, false);
273+
274+ return 0;
275+}
276+
277+static void amd_pstate_adjust_perf(unsigned int cpu,
278+ unsigned long _min_perf,
279+ unsigned long target_perf,
280+ unsigned long capacity)
281+{
282+ unsigned long max_perf, min_perf, des_perf,
283+ cap_perf, lowest_nonlinear_perf;
284+ struct cpufreq_policy *policy = cpufreq_cpu_get(cpu);
285+ struct amd_cpudata *cpudata = policy->driver_data;
286+
287+ cap_perf = READ_ONCE(cpudata->highest_perf);
288+ lowest_nonlinear_perf = READ_ONCE(cpudata->lowest_nonlinear_perf);
289+
290+ des_perf = cap_perf;
291+ if (target_perf < capacity)
292+ des_perf = DIV_ROUND_UP(cap_perf * target_perf, capacity);
293+
294+ min_perf = READ_ONCE(cpudata->highest_perf);
295+ if (_min_perf < capacity)
296+ min_perf = DIV_ROUND_UP(cap_perf * _min_perf, capacity);
297+
298+ if (min_perf < lowest_nonlinear_perf)
299+ min_perf = lowest_nonlinear_perf;
300+
301+ max_perf = cap_perf;
302+ if (max_perf < min_perf)
303+ max_perf = min_perf;
304+
305+ des_perf = clamp_t(unsigned long, des_perf, min_perf, max_perf);
306+
307+ amd_pstate_update(cpudata, min_perf, des_perf, max_perf, true);
308+}
309+
310+static int amd_get_min_freq(struct amd_cpudata *cpudata)
311+{
312+ struct cppc_perf_caps cppc_perf;
313+
314+ int ret = cppc_get_perf_caps(cpudata->cpu, &cppc_perf);
315+ if (ret)
316+ return ret;
317+
318+ /* Switch to khz */
319+ return cppc_perf.lowest_freq * 1000;
320+}
321+
322+static int amd_get_max_freq(struct amd_cpudata *cpudata)
323+{
324+ struct cppc_perf_caps cppc_perf;
325+ u32 max_perf, max_freq, nominal_freq, nominal_perf;
326+ u64 boost_ratio;
327+
328+ int ret = cppc_get_perf_caps(cpudata->cpu, &cppc_perf);
329+ if (ret)
330+ return ret;
331+
332+ nominal_freq = cppc_perf.nominal_freq;
333+ nominal_perf = READ_ONCE(cpudata->nominal_perf);
334+ max_perf = READ_ONCE(cpudata->highest_perf);
335+
336+ boost_ratio = div_u64(max_perf << SCHED_CAPACITY_SHIFT,
337+ nominal_perf);
338+
339+ max_freq = nominal_freq * boost_ratio >> SCHED_CAPACITY_SHIFT;
340+
341+ /* Switch to khz */
342+ return max_freq * 1000;
343+}
344+
345+static int amd_get_nominal_freq(struct amd_cpudata *cpudata)
346+{
347+ struct cppc_perf_caps cppc_perf;
348+
349+ int ret = cppc_get_perf_caps(cpudata->cpu, &cppc_perf);
350+ if (ret)
351+ return ret;
352+
353+ /* Switch to khz */
354+ return cppc_perf.nominal_freq * 1000;
355+}
356+
357+static int amd_get_lowest_nonlinear_freq(struct amd_cpudata *cpudata)
358+{
359+ struct cppc_perf_caps cppc_perf;
360+ u32 lowest_nonlinear_freq, lowest_nonlinear_perf,
361+ nominal_freq, nominal_perf;
362+ u64 lowest_nonlinear_ratio;
363+
364+ int ret = cppc_get_perf_caps(cpudata->cpu, &cppc_perf);
365+ if (ret)
366+ return ret;
367+
368+ nominal_freq = cppc_perf.nominal_freq;
369+ nominal_perf = READ_ONCE(cpudata->nominal_perf);
370+
371+ lowest_nonlinear_perf = cppc_perf.lowest_nonlinear_perf;
372+
373+ lowest_nonlinear_ratio = div_u64(lowest_nonlinear_perf << SCHED_CAPACITY_SHIFT,
374+ nominal_perf);
375+
376+ lowest_nonlinear_freq = nominal_freq * lowest_nonlinear_ratio >> SCHED_CAPACITY_SHIFT;
377+
378+ /* Switch to khz */
379+ return lowest_nonlinear_freq * 1000;
380+}
381+
382+static int amd_pstate_set_boost(struct cpufreq_policy *policy, int state)
383+{
384+ struct amd_cpudata *cpudata = policy->driver_data;
385+ int ret;
386+
387+ if (!cpudata->boost_supported) {
388+ pr_err("Boost mode is not supported by this processor or SBIOS\n");
389+ return -EINVAL;
390+ }
391+
392+ if (state)
393+ policy->cpuinfo.max_freq = cpudata->max_freq;
394+ else
395+ policy->cpuinfo.max_freq = cpudata->nominal_freq;
396+
397+ policy->max = policy->cpuinfo.max_freq;
398+
399+ ret = freq_qos_update_request(&cpudata->req[1],
400+ policy->cpuinfo.max_freq);
401+ if (ret < 0)
402+ return ret;
403+
404+ return 0;
405+}
406+
407+static void amd_pstate_boost_init(struct amd_cpudata *cpudata)
408+{
409+ u32 highest_perf, nominal_perf;
410+
411+ highest_perf = READ_ONCE(cpudata->highest_perf);
412+ nominal_perf = READ_ONCE(cpudata->nominal_perf);
413+
414+ if (highest_perf <= nominal_perf)
415+ return;
416+
417+ cpudata->boost_supported = true;
418+ amd_pstate_driver.boost_enabled = true;
419+}
420+
421+static int amd_pstate_cpu_init(struct cpufreq_policy *policy)
422+{
423+ int min_freq, max_freq, nominal_freq, lowest_nonlinear_freq, ret;
424+ struct device *dev;
425+ struct amd_cpudata *cpudata;
426+
427+ dev = get_cpu_device(policy->cpu);
428+ if (!dev)
429+ return -ENODEV;
430+
431+ cpudata = kzalloc(sizeof(*cpudata), GFP_KERNEL);
432+ if (!cpudata)
433+ return -ENOMEM;
434+
435+ cpudata->cpu = policy->cpu;
436+
437+ ret = amd_pstate_init_perf(cpudata);
438+ if (ret)
439+ goto free_cpudata1;
440+
441+ min_freq = amd_get_min_freq(cpudata);
442+ max_freq = amd_get_max_freq(cpudata);
443+ nominal_freq = amd_get_nominal_freq(cpudata);
444+ lowest_nonlinear_freq = amd_get_lowest_nonlinear_freq(cpudata);
445+
446+ if (min_freq < 0 || max_freq < 0 || min_freq > max_freq) {
447+ dev_err(dev, "min_freq(%d) or max_freq(%d) value is incorrect\n",
448+ min_freq, max_freq);
449+ ret = -EINVAL;
450+ goto free_cpudata1;
451+ }
452+
453+ policy->cpuinfo.transition_latency = AMD_PSTATE_TRANSITION_LATENCY;
454+ policy->transition_delay_us = AMD_PSTATE_TRANSITION_DELAY;
455+
456+ policy->min = min_freq;
457+ policy->max = max_freq;
458+
459+ policy->cpuinfo.min_freq = min_freq;
460+ policy->cpuinfo.max_freq = max_freq;
461+
462+ /* It will be updated by governor */
463+ policy->cur = policy->cpuinfo.min_freq;
464+
465+ if (boot_cpu_has(X86_FEATURE_CPPC))
466+ policy->fast_switch_possible = true;
467+
468+ ret = freq_qos_add_request(&policy->constraints, &cpudata->req[0],
469+ FREQ_QOS_MIN, policy->cpuinfo.min_freq);
470+ if (ret < 0) {
471+ dev_err(dev, "Failed to add min-freq constraint (%d)\n", ret);
472+ goto free_cpudata1;
473+ }
474+
475+ ret = freq_qos_add_request(&policy->constraints, &cpudata->req[1],
476+ FREQ_QOS_MAX, policy->cpuinfo.max_freq);
477+ if (ret < 0) {
478+ dev_err(dev, "Failed to add max-freq constraint (%d)\n", ret);
479+ goto free_cpudata2;
480+ }
481+
482+ /* Initial processor data capability frequencies */
483+ cpudata->max_freq = max_freq;
484+ cpudata->min_freq = min_freq;
485+ cpudata->nominal_freq = nominal_freq;
486+ cpudata->lowest_nonlinear_freq = lowest_nonlinear_freq;
487+
488+ policy->driver_data = cpudata;
489+
490+ amd_pstate_boost_init(cpudata);
491+
492+ return 0;
493+
494+free_cpudata2:
495+ freq_qos_remove_request(&cpudata->req[0]);
496+free_cpudata1:
497+ kfree(cpudata);
498+ return ret;
499+}
500+
501+static int amd_pstate_cpu_exit(struct cpufreq_policy *policy)
502+{
503+ struct amd_cpudata *cpudata;
504+
505+ cpudata = policy->driver_data;
506+
507+ freq_qos_remove_request(&cpudata->req[1]);
508+ freq_qos_remove_request(&cpudata->req[0]);
509+ kfree(cpudata);
510+
511+ return 0;
512+}
513+
514+/* Sysfs attributes */
515+
516+/*
517+ * This frequency is to indicate the maximum hardware frequency.
518+ * If boost is not active but supported, the frequency will be larger than the
519+ * one in cpuinfo.
520+ */
521+static ssize_t show_amd_pstate_max_freq(struct cpufreq_policy *policy,
522+ char *buf)
523+{
524+ int max_freq;
525+ struct amd_cpudata *cpudata;
526+
527+ cpudata = policy->driver_data;
528+
529+ max_freq = amd_get_max_freq(cpudata);
530+ if (max_freq < 0)
531+ return max_freq;
532+
533+ return sprintf(&buf[0], "%u\n", max_freq);
534+}
535+
536+static ssize_t show_amd_pstate_lowest_nonlinear_freq(struct cpufreq_policy *policy,
537+ char *buf)
538+{
539+ int freq;
540+ struct amd_cpudata *cpudata;
541+
542+ cpudata = policy->driver_data;
543+
544+ freq = amd_get_lowest_nonlinear_freq(cpudata);
545+ if (freq < 0)
546+ return freq;
547+
548+ return sprintf(&buf[0], "%u\n", freq);
549+}
550+
551+/*
552+ * In some of ASICs, the highest_perf is not the one in the _CPC table, so we
553+ * need to expose it to sysfs.
554+ */
555+static ssize_t show_amd_pstate_highest_perf(struct cpufreq_policy *policy,
556+ char *buf)
557+{
558+ u32 perf;
559+ struct amd_cpudata *cpudata = policy->driver_data;
560+
561+ perf = READ_ONCE(cpudata->highest_perf);
562+
563+ return sprintf(&buf[0], "%u\n", perf);
564+}
565+
566+cpufreq_freq_attr_ro(amd_pstate_max_freq);
567+cpufreq_freq_attr_ro(amd_pstate_lowest_nonlinear_freq);
568+
569+cpufreq_freq_attr_ro(amd_pstate_highest_perf);
570+
571+static struct freq_attr *amd_pstate_attr[] = {
572+ &amd_pstate_max_freq,
573+ &amd_pstate_lowest_nonlinear_freq,
574+ &amd_pstate_highest_perf,
575+ NULL,
576+};
577+
578+static struct cpufreq_driver amd_pstate_driver = {
579+ .flags = CPUFREQ_CONST_LOOPS | CPUFREQ_NEED_UPDATE_LIMITS,
580+ .verify = amd_pstate_verify,
581+ .target = amd_pstate_target,
582+ .init = amd_pstate_cpu_init,
583+ .exit = amd_pstate_cpu_exit,
584+ .set_boost = amd_pstate_set_boost,
585+ .name = "amd-pstate",
586+ .attr = amd_pstate_attr,
587+};
588+
589+static int __init amd_pstate_init(void)
590+{
591+ int ret;
592+
593+ if (boot_cpu_data.x86_vendor != X86_VENDOR_AMD)
594+ return -ENODEV;
595+
596+ if (!acpi_cpc_valid()) {
597+ pr_debug("the _CPC object is not present in SBIOS\n");
598+ return -ENODEV;
599+ }
600+
601+ /* don't keep reloading if cpufreq_driver exists */
602+ if (cpufreq_get_current_driver())
603+ return -EEXIST;
604+
605+ /* capability check */
606+ if (boot_cpu_has(X86_FEATURE_CPPC)) {
607+ pr_debug("AMD CPPC MSR based functionality is supported\n");
608+ amd_pstate_driver.adjust_perf = amd_pstate_adjust_perf;
609+ } else if (shared_mem) {
610+ static_call_update(amd_pstate_enable, cppc_enable);
611+ static_call_update(amd_pstate_init_perf, cppc_init_perf);
612+ static_call_update(amd_pstate_update_perf, cppc_update_perf);
613+ } else {
614+ pr_info("This processor supports shared memory solution, you can enable it with amd_pstate.shared_mem=1\n");
615+ return -ENODEV;
616+ }
617+
618+ /* enable amd pstate feature */
619+ ret = amd_pstate_enable(true);
620+ if (ret) {
621+ pr_err("failed to enable amd-pstate with return %d\n", ret);
622+ return ret;
623+ }
624+
625+ ret = cpufreq_register_driver(&amd_pstate_driver);
626+ if (ret)
627+ pr_err("failed to register amd_pstate_driver with return %d\n",
628+ ret);
629+
630+ return ret;
631+}
632+
633+static void __exit amd_pstate_exit(void)
634+{
635+ cpufreq_unregister_driver(&amd_pstate_driver);
636+
637+ amd_pstate_enable(false);
638+}
639+
640+module_init(amd_pstate_init);
641+module_exit(amd_pstate_exit);
642+
643+MODULE_AUTHOR("Huang Rui <ray.huang@amd.com>");
644+MODULE_DESCRIPTION("AMD Processor P-state Frequency Driver");
645+MODULE_LICENSE("GPL");
drivers/cpufreq/cpufreq.c+5−4
@@ -924,7 +924,7 @@ cpufreq_freq_attr_rw(scaling_max_freq); · 923 unmodified lines
924924 cpufreq_freq_attr_rw(scaling_governor);
925925 cpufreq_freq_attr_rw(scaling_setspeed);
926926
927−static struct attribute *default_attrs[] = {
927+static struct attribute *cpufreq_attrs[] = {
928928 &cpuinfo_min_freq.attr,
929929 &cpuinfo_max_freq.attr,
930930 &cpuinfo_transition_latency.attr,
@@ -938,6 +938,7 @@ static struct attribute *default_attrs[] = { · 7 unmodified lines
938938 &scaling_setspeed.attr,
939939 NULL
940940 };
941+ATTRIBUTE_GROUPS(cpufreq);
941942
942943 #define to_policy(k) container_of(k, struct cpufreq_policy, kobj)
943944 #define to_attr(a) container_of(a, struct freq_attr, attr)
@@ -1000,7 +1001,7 @@ static const struct sysfs_ops sysfs_ops = { · 56 unmodified lines
10001001
10011002 static struct kobj_type ktype_cpufreq = {
10021003 .sysfs_ops = &sysfs_ops,
1003− .default_attrs = default_attrs,
1004+ .default_groups = cpufreq_groups,
10041005 .release = cpufreq_sysfs_release,
10051006 };
10061007
@@ -1403,7 +1404,7 @@ static int cpufreq_online(unsigned int cpu) · 396 unmodified lines
14031404
14041405 ret = freq_qos_add_request(&policy->constraints,
14051406 policy->min_freq_req, FREQ_QOS_MIN,
1406− policy->min);
1407+ FREQ_QOS_MIN_DEFAULT_VALUE);
14071408 if (ret < 0) {
14081409 /*
14091410 * So we don't call freq_qos_remove_request() for an
@@ -1423,7 +1424,7 @@ static int cpufreq_online(unsigned int cpu) · 13 unmodified lines
14231424
14241425 ret = freq_qos_add_request(&policy->constraints,
14251426 policy->max_freq_req, FREQ_QOS_MAX,
1426− policy->max);
1427+ FREQ_QOS_MAX_DEFAULT_VALUE);
14271428 if (ret < 0) {
14281429 policy->max_freq_req = NULL;
14291430 goto out_destroy_policy;
drivers/cpufreq/cpufreq_conservative.c+3−2
@@ -257,7 +257,7 @@ gov_attr_rw(ignore_nice_load); · 256 unmodified lines
257257 gov_attr_rw(down_threshold);
258258 gov_attr_rw(freq_step);
259259
260−static struct attribute *cs_attributes[] = {
260+static struct attribute *cs_attrs[] = {
261261 &sampling_rate.attr,
262262 &sampling_down_factor.attr,
263263 &up_threshold.attr,
@@ -266,6 +266,7 @@ static struct attribute *cs_attributes[] = { · 2 unmodified lines
266266 &freq_step.attr,
267267 NULL
268268 };
269+ATTRIBUTE_GROUPS(cs);
269270
270271 /************************** sysfs end ************************/
271272
@@ -315,7 +316,7 @@ static void cs_start(struct cpufreq_policy *policy) · 43 unmodified lines
315316
316317 static struct dbs_governor cs_governor = {
317318 .gov = CPUFREQ_DBS_GOVERNOR_INITIALIZER("conservative"),
318− .kobj_type = { .default_attrs = cs_attributes },
319+ .kobj_type = { .default_groups = cs_groups },
319320 .gov_dbs_update = cs_dbs_update,
320321 .alloc = cs_alloc,
321322 .free = cs_free,
drivers/cpufreq/cpufreq_ondemand.c+3−2
@@ -328,7 +328,7 @@ gov_attr_rw(sampling_down_factor); · 327 unmodified lines
328328 gov_attr_rw(ignore_nice_load);
329329 gov_attr_rw(powersave_bias);
330330
331−static struct attribute *od_attributes[] = {
331+static struct attribute *od_attrs[] = {
332332 &sampling_rate.attr,
333333 &up_threshold.attr,
334334 &sampling_down_factor.attr,
@@ -337,6 +337,7 @@ static struct attribute *od_attributes[] = { · 2 unmodified lines
337337 &io_is_busy.attr,
338338 NULL
339339 };
340+ATTRIBUTE_GROUPS(od);
340341
341342 /************************** sysfs end ************************/
342343
@@ -401,7 +402,7 @@ static struct od_ops od_ops = { · 58 unmodified lines
401402
402403 static struct dbs_governor od_dbs_gov = {
403404 .gov = CPUFREQ_DBS_GOVERNOR_INITIALIZER("ondemand"),
404− .kobj_type = { .default_attrs = od_attributes },
405+ .kobj_type = { .default_groups = od_groups },
405406 .gov_dbs_update = od_dbs_update,
406407 .alloc = od_alloc,
407408 .free = od_free,
drivers/cpufreq/intel_pstate.c+81−40
@@ -664,19 +664,29 @@ static int intel_pstate_set_epb(int cpu, s16 pref) · 663 unmodified lines
664664 * 3 balance_power
665665 * 4 power
666666 */
667+
668+enum energy_perf_value_index {
669+ EPP_INDEX_DEFAULT = 0,
670+ EPP_INDEX_PERFORMANCE,
671+ EPP_INDEX_BALANCE_PERFORMANCE,
672+ EPP_INDEX_BALANCE_POWERSAVE,
673+ EPP_INDEX_POWERSAVE,
674+};
675+
667676 static const char * const energy_perf_strings[] = {
668− "default",
669− "performance",
670− "balance_performance",
671− "balance_power",
672− "power",
677+ [EPP_INDEX_DEFAULT] = "default",
678+ [EPP_INDEX_PERFORMANCE] = "performance",
679+ [EPP_INDEX_BALANCE_PERFORMANCE] = "balance_performance",
680+ [EPP_INDEX_BALANCE_POWERSAVE] = "balance_power",
681+ [EPP_INDEX_POWERSAVE] = "power",
673682 NULL
674683 };
675−static const unsigned int epp_values[] = {
676− HWP_EPP_PERFORMANCE,
677− HWP_EPP_BALANCE_PERFORMANCE,
678− HWP_EPP_BALANCE_POWERSAVE,
679− HWP_EPP_POWERSAVE
684+static unsigned int epp_values[] = {
685+ [EPP_INDEX_DEFAULT] = 0, /* Unused index */
686+ [EPP_INDEX_PERFORMANCE] = HWP_EPP_PERFORMANCE,
687+ [EPP_INDEX_BALANCE_PERFORMANCE] = HWP_EPP_BALANCE_PERFORMANCE,
688+ [EPP_INDEX_BALANCE_POWERSAVE] = HWP_EPP_BALANCE_POWERSAVE,
689+ [EPP_INDEX_POWERSAVE] = HWP_EPP_POWERSAVE,
680690 };
681691
682692 static int intel_pstate_get_energy_pref_index(struct cpudata *cpu_data, int *raw_epp)
@@ -690,14 +700,14 @@ static int intel_pstate_get_energy_pref_index(struct cpudata *cpu_data, int *raw · 7 unmodified lines
690700 return epp;
691701
692702 if (boot_cpu_has(X86_FEATURE_HWP_EPP)) {
693− if (epp == HWP_EPP_PERFORMANCE)
694− return 1;
695− if (epp == HWP_EPP_BALANCE_PERFORMANCE)
696− return 2;
697− if (epp == HWP_EPP_BALANCE_POWERSAVE)
698− return 3;
699− if (epp == HWP_EPP_POWERSAVE)
700− return 4;
703+ if (epp == epp_values[EPP_INDEX_PERFORMANCE])
704+ return EPP_INDEX_PERFORMANCE;
705+ if (epp == epp_values[EPP_INDEX_BALANCE_PERFORMANCE])
706+ return EPP_INDEX_BALANCE_PERFORMANCE;
707+ if (epp == epp_values[EPP_INDEX_BALANCE_POWERSAVE])
708+ return EPP_INDEX_BALANCE_POWERSAVE;
709+ if (epp == epp_values[EPP_INDEX_POWERSAVE])
710+ return EPP_INDEX_POWERSAVE;
701711 *raw_epp = epp;
702712 return 0;
703713 } else if (boot_cpu_has(X86_FEATURE_EPB)) {
@@ -757,7 +767,7 @@ static int intel_pstate_set_energy_pref_index(struct cpudata *cpu_data, · 53 unmodified lines
757767 if (use_raw)
758768 epp = raw_epp;
759769 else if (epp == -EINVAL)
760− epp = epp_values[pref_index - 1];
770+ epp = epp_values[pref_index];
761771
762772 /*
763773 * To avoid confusion, refuse to set EPP to any values different
@@ -843,7 +853,7 @@ static ssize_t store_energy_performance_preference( · 79 unmodified lines
843853 * upfront.
844854 */
845855 if (!raw)
846− epp = ret ? epp_values[ret - 1] : cpu->epp_default;
856+ epp = ret ? epp_values[ret] : cpu->epp_default;
847857
848858 if (cpu->epp_cached != epp) {
849859 int err;
@@ -1124,19 +1134,22 @@ static void intel_pstate_update_policies(void) · 274 unmodified lines
11241134 cpufreq_update_policy(cpu);
11251135 }
11261136
1137+static void __intel_pstate_update_max_freq(struct cpudata *cpudata,
1138+ struct cpufreq_policy *policy)
1139+{
1140+ policy->cpuinfo.max_freq = global.turbo_disabled_mf ?
1141+ cpudata->pstate.max_freq : cpudata->pstate.turbo_freq;
1142+ refresh_frequency_limits(policy);
1143+}
1144+
11271145 static void intel_pstate_update_max_freq(unsigned int cpu)
11281146 {
11291147 struct cpufreq_policy *policy = cpufreq_cpu_acquire(cpu);
1130− struct cpudata *cpudata;
11311148
11321149 if (!policy)
11331150 return;
11341151
1135− cpudata = all_cpu_data[cpu];
1136− policy->cpuinfo.max_freq = global.turbo_disabled_mf ?
1137− cpudata->pstate.max_freq : cpudata->pstate.turbo_freq;
1138−
1139− refresh_frequency_limits(policy);
1152+ __intel_pstate_update_max_freq(all_cpu_data[cpu], policy);
11401153
11411154 cpufreq_cpu_release(policy);
11421155 }
@@ -1584,8 +1597,15 @@ static void intel_pstate_notify_work(struct work_struct *work) · 441 unmodified lines
15841597 {
15851598 struct cpudata *cpudata =
15861599 container_of(to_delayed_work(work), struct cpudata, hwp_notify_work);
1600+ struct cpufreq_policy *policy = cpufreq_cpu_acquire(cpudata->cpu);
1601+
1602+ if (policy) {
1603+ intel_pstate_get_hwp_cap(cpudata);
1604+ __intel_pstate_update_max_freq(cpudata, policy);
1605+
1606+ cpufreq_cpu_release(policy);
1607+ }
15871608
1588− cpufreq_update_policy(cpudata->cpu);
15891609 wrmsrl_on_cpu(cpudata->cpu, MSR_HWP_STATUS, 0);
15901610 }
15911611
@@ -1679,10 +1699,18 @@ static void intel_pstate_hwp_enable(struct cpudata *cpudata) · 87 unmodified lines
16791699 wrmsrl_on_cpu(cpudata->cpu, MSR_HWP_INTERRUPT, 0x00);
16801700
16811701 wrmsrl_on_cpu(cpudata->cpu, MSR_PM_ENABLE, 0x1);
1682− if (cpudata->epp_default == -EINVAL)
1683− cpudata->epp_default = intel_pstate_get_epp(cpudata, 0);
16841702
16851703 intel_pstate_enable_hwp_interrupt(cpudata);
1704+
1705+ if (cpudata->epp_default >= 0)
1706+ return;
1707+
1708+ if (epp_values[EPP_INDEX_BALANCE_PERFORMANCE] == HWP_EPP_BALANCE_PERFORMANCE) {
1709+ cpudata->epp_default = intel_pstate_get_epp(cpudata, 0);
1710+ } else {
1711+ cpudata->epp_default = epp_values[EPP_INDEX_BALANCE_PERFORMANCE];
1712+ intel_pstate_set_epp(cpudata, cpudata->epp_default);
1713+ }
16861714 }
16871715
16881716 static int atom_get_min_pstate(void)
@@ -2486,18 +2514,14 @@ static void intel_pstate_update_perf_limits(struct cpudata *cpu, · 797 unmodified lines
24862514 * HWP needs some special consideration, because HWP_REQUEST uses
24872515 * abstract values to represent performance rather than pure ratios.
24882516 */
2489− if (hwp_active) {
2490− intel_pstate_get_hwp_cap(cpu);
2517+ if (hwp_active && cpu->pstate.scaling != perf_ctl_scaling) {
2518+ int scaling = cpu->pstate.scaling;
2519+ int freq;
24912520
2492− if (cpu->pstate.scaling != perf_ctl_scaling) {
2493− int scaling = cpu->pstate.scaling;
2494− int freq;
2495−
2496− freq = max_policy_perf * perf_ctl_scaling;
2497− max_policy_perf = DIV_ROUND_UP(freq, scaling);
2498− freq = min_policy_perf * perf_ctl_scaling;
2499− min_policy_perf = DIV_ROUND_UP(freq, scaling);
2500− }
2521+ freq = max_policy_perf * perf_ctl_scaling;
2522+ max_policy_perf = DIV_ROUND_UP(freq, scaling);
2523+ freq = min_policy_perf * perf_ctl_scaling;
2524+ min_policy_perf = DIV_ROUND_UP(freq, scaling);
25012525 }
25022526
25032527 pr_debug("cpu:%d min_policy_perf:%d max_policy_perf:%d\n",
@@ -3349,6 +3373,16 @@ static bool intel_pstate_hwp_is_enabled(void) · 845 unmodified lines
33493373 return !!(value & 0x1);
33503374 }
33513375
3376+static const struct x86_cpu_id intel_epp_balance_perf[] = {
3377+ /*
3378+ * Set EPP value as 102, this is the max suggested EPP
3379+ * which can result in one core turbo frequency for
3380+ * AlderLake Mobile CPUs.
3381+ */
3382+ X86_MATCH_INTEL_FAM6_MODEL(ALDERLAKE_L, 102),
3383+ {}
3384+};
3385+
33523386 static int __init intel_pstate_init(void)
33533387 {
33543388 static struct cpudata **_all_cpu_data;
@@ -3438,6 +3472,13 @@ hwp_cpu_matched: · 83 unmodified lines
34383472
34393473 intel_pstate_sysfs_expose_params();
34403474
3475+ if (hwp_active) {
3476+ const struct x86_cpu_id *id = x86_match_cpu(intel_epp_balance_perf);
3477+
3478+ if (id)
3479+ epp_values[EPP_INDEX_BALANCE_PERFORMANCE] = id->driver_data;
3480+ }
3481+
34413482 mutex_lock(&intel_pstate_driver_lock);
34423483 rc = intel_pstate_register_driver(default_driver);
34433484 mutex_unlock(&intel_pstate_driver_lock);
drivers/cpufreq/mediatek-cpufreq-hw.c+30−3
@@ -36,6 +36,8 @@ enum { · 35 unmodified lines
3636 struct mtk_cpufreq_data {
3737 struct cpufreq_frequency_table *table;
3838 void __iomem *reg_bases[REG_ARRAY_SIZE];
39+ struct resource *res;
40+ void __iomem *base;
3941 int nr_opp;
4042 };
4143
@@ -156,6 +158,7 @@ static int mtk_cpu_resources_init(struct platform_device *pdev, · 114 unmodified lines
156158 {
157159 struct mtk_cpufreq_data *data;
158160 struct device *dev = &pdev->dev;
161+ struct resource *res;
159162 void __iomem *base;
160163 int ret, i;
161164 int index;
@@ -170,9 +173,26 @@ static int mtk_cpu_resources_init(struct platform_device *pdev, · 8 unmodified lines
170173 if (index < 0)
171174 return index;
172175
173− base = devm_platform_ioremap_resource(pdev, index);
174− if (IS_ERR(base))
175− return PTR_ERR(base);
176+ res = platform_get_resource(pdev, IORESOURCE_MEM, index);
177+ if (!res) {
178+ dev_err(dev, "failed to get mem resource %d\n", index);
179+ return -ENODEV;
180+ }
181+
182+ if (!request_mem_region(res->start, resource_size(res), res->name)) {
183+ dev_err(dev, "failed to request resource %pR\n", res);
184+ return -EBUSY;
185+ }
186+
187+ base = ioremap(res->start, resource_size(res));
188+ if (!base) {
189+ dev_err(dev, "failed to map resource %pR\n", res);
190+ ret = -ENOMEM;
191+ goto release_region;
192+ }
193+
194+ data->base = base;
195+ data->res = res;
176196
177197 for (i = REG_FREQ_LUT_TABLE; i < REG_ARRAY_SIZE; i++)
178198 data->reg_bases[i] = base + offsets[i];
@@ -187,6 +207,9 @@ static int mtk_cpu_resources_init(struct platform_device *pdev, · 8 unmodified lines
187207 policy->driver_data = data;
188208
189209 return 0;
210+release_region:
211+ release_mem_region(res->start, resource_size(res));
212+ return ret;
190213 }
191214
192215 static int mtk_cpufreq_hw_cpu_init(struct cpufreq_policy *policy)
@@ -233,9 +256,13 @@ static int mtk_cpufreq_hw_cpu_init(struct cpufreq_policy *policy) · 40 unmodified lines
233256 static int mtk_cpufreq_hw_cpu_exit(struct cpufreq_policy *policy)
234257 {
235258 struct mtk_cpufreq_data *data = policy->driver_data;
259+ struct resource *res = data->res;
260+ void __iomem *base = data->base;
236261
237262 /* HW should be in paused state now */
238263 writel_relaxed(0x0, data->reg_bases[REG_FREQ_ENABLE]);
264+ iounmap(base);
265+ release_mem_region(res->start, resource_size(res));
239266
240267 return 0;
241268 }
drivers/cpufreq/qcom-cpufreq-hw.c+19−20
@@ -46,6 +46,7 @@ struct qcom_cpufreq_data { · 45 unmodified lines
4646 */
4747 struct mutex throttle_lock;
4848 int throttle_irq;
49+ char irq_name[15];
4950 bool cancel_throttle;
5051 struct delayed_work throttle_work;
5152 struct cpufreq_policy *policy;
@@ -275,10 +276,10 @@ static unsigned int qcom_lmh_get_throttle_freq(struct qcom_cpufreq_data *data) · 223 unmodified lines
275276
276277 static void qcom_lmh_dcvs_notify(struct qcom_cpufreq_data *data)
277278 {
278− unsigned long max_capacity, capacity, freq_hz, throttled_freq;
279279 struct cpufreq_policy *policy = data->policy;
280280 int cpu = cpumask_first(policy->cpus);
281281 struct device *dev = get_cpu_device(cpu);
282+ unsigned long freq_hz, throttled_freq;
282283 struct dev_pm_opp *opp;
283284 unsigned int freq;
284285
@@ -295,16 +296,8 @@ static void qcom_lmh_dcvs_notify(struct qcom_cpufreq_data *data) · 10 unmodified lines
295296
296297 throttled_freq = freq_hz / HZ_PER_KHZ;
297298
298− /* Update thermal pressure */
299−
300− max_capacity = arch_scale_cpu_capacity(cpu);
301− capacity = mult_frac(max_capacity, throttled_freq, policy->cpuinfo.max_freq);
302−
303− /* Don't pass boost capacity to scheduler */
304− if (capacity > max_capacity)
305− capacity = max_capacity;
306−
307− arch_set_thermal_pressure(policy->cpus, max_capacity - capacity);
299+ /* Update thermal pressure (the boost frequencies are accepted) */
300+ arch_update_thermal_pressure(policy->related_cpus, throttled_freq);
308301
309302 /*
310303 * In the unlikely case policy is unregistered do not enable
@@ -342,9 +335,9 @@ static irqreturn_t qcom_lmh_dcvs_handle_irq(int irq, void *data) · 31 unmodified lines
342335
343336 /* Disable interrupt and enable polling */
344337 disable_irq_nosync(c_data->throttle_irq);
345− qcom_lmh_dcvs_notify(c_data);
338+ schedule_delayed_work(&c_data->throttle_work, 0);
346339
347− return 0;
340+ return IRQ_HANDLED;
348341 }
349342
350343 static const struct qcom_cpufreq_soc_data qcom_soc_data = {
@@ -375,16 +368,17 @@ static int qcom_cpufreq_hw_lmh_init(struct cpufreq_policy *policy, int index) · 24 unmodified lines
375368 {
376369 struct qcom_cpufreq_data *data = policy->driver_data;
377370 struct platform_device *pdev = cpufreq_get_driver_data();
378− char irq_name[15];
379371 int ret;
380372
381373 /*
382374 * Look for LMh interrupt. If no interrupt line is specified /
383375 * if there is an error, allow cpufreq to be enabled as usual.
384376 */
385− data->throttle_irq = platform_get_irq(pdev, index);
386− if (data->throttle_irq <= 0)
387− return data->throttle_irq == -EPROBE_DEFER ? -EPROBE_DEFER : 0;
377+ data->throttle_irq = platform_get_irq_optional(pdev, index);
378+ if (data->throttle_irq == -ENXIO)
379+ return 0;
380+ if (data->throttle_irq < 0)
381+ return data->throttle_irq;
388382
389383 data->cancel_throttle = false;
390384 data->policy = policy;
@@ -392,14 +386,19 @@ static int qcom_cpufreq_hw_lmh_init(struct cpufreq_policy *policy, int index) · 1 unmodified line
392386 mutex_init(&data->throttle_lock);
393387 INIT_DEFERRABLE_WORK(&data->throttle_work, qcom_lmh_dcvs_poll);
394388
395− snprintf(irq_name, sizeof(irq_name), "dcvsh-irq-%u", policy->cpu);
389+ snprintf(data->irq_name, sizeof(data->irq_name), "dcvsh-irq-%u", policy->cpu);
396390 ret = request_threaded_irq(data->throttle_irq, NULL, qcom_lmh_dcvs_handle_irq,
397− IRQF_ONESHOT, irq_name, data);
391+ IRQF_ONESHOT, data->irq_name, data);
398392 if (ret) {
399− dev_err(&pdev->dev, "Error registering %s: %d\n", irq_name, ret);
393+ dev_err(&pdev->dev, "Error registering %s: %d\n", data->irq_name, ret);
400394 return 0;
401395 }
402396
397+ ret = irq_set_affinity_hint(data->throttle_irq, policy->cpus);
398+ if (ret)
399+ dev_err(&pdev->dev, "Failed to set CPU affinity of %s[%d]\n",
400+ data->irq_name, data->throttle_irq);
401+
403402 return 0;
404403 }
405404
drivers/cpuidle/governors/menu.c+1−1
@@ -34,7 +34,7 @@ · 33 unmodified lines
3434 * 1) Energy break even point
3535 * 2) Performance impact
3636 * 3) Latency tolerance (from pmqos infrastructure)
37− * These these three factors are treated independently.
37+ * These three factors are treated independently.
3838 *
3939 * Energy break even point
4040 * -----------------------
drivers/cpuidle/sysfs.c+5−3
@@ -335,6 +335,7 @@ static struct attribute *cpuidle_state_default_attrs[] = { · 334 unmodified lines
335335 &attr_default_status.attr,
336336 NULL
337337 };
338+ATTRIBUTE_GROUPS(cpuidle_state_default);
338339
339340 struct cpuidle_state_kobj {
340341 struct cpuidle_state *state;
@@ -448,7 +449,7 @@ static void cpuidle_state_sysfs_release(struct kobject *kobj) · 107 unmodified lines
448449
449450 static struct kobj_type ktype_state_cpuidle = {
450451 .sysfs_ops = &cpuidle_state_sysfs_ops,
451− .default_attrs = cpuidle_state_default_attrs,
452+ .default_groups = cpuidle_state_default_groups,
452453 .release = cpuidle_state_sysfs_release,
453454 };
454455
@@ -505,7 +506,7 @@ error_state: · 50 unmodified lines
505506 }
506507
507508 /**
508− * cpuidle_remove_driver_sysfs - removes the cpuidle states sysfs attributes
509+ * cpuidle_remove_state_sysfs - removes the cpuidle states sysfs attributes
509510 * @device: the target device
510511 */
511512 static void cpuidle_remove_state_sysfs(struct cpuidle_device *device)
@@ -591,10 +592,11 @@ static struct attribute *cpuidle_driver_default_attrs[] = { · 79 unmodified lines
591592 &attr_driver_name.attr,
592593 NULL
593594 };
595+ATTRIBUTE_GROUPS(cpuidle_driver_default);
594596
595597 static struct kobj_type ktype_driver_cpuidle = {
596598 .sysfs_ops = &cpuidle_driver_sysfs_ops,
597− .default_attrs = cpuidle_driver_default_attrs,
599+ .default_groups = cpuidle_driver_default_groups,
598600 .release = cpuidle_driver_sysfs_release,
599601 };
600602
drivers/devfreq/Kconfig+9−0
@@ -132,6 +132,15 @@ config ARM_RK3399_DMC_DEVFREQ · 131 unmodified lines
132132 It sets the frequency for the memory controller and reads the usage counts
133133 from hardware.
134134
135+config ARM_SUN8I_A33_MBUS_DEVFREQ
136+ tristate "sun8i/sun50i MBUS DEVFREQ Driver"
137+ depends on ARCH_SUNXI || COMPILE_TEST
138+ depends on COMMON_CLK
139+ select DEVFREQ_GOV_SIMPLE_ONDEMAND
140+ help
141+ This adds the DEVFREQ driver for the MBUS controller in some
142+ Allwinner sun8i (A33 through H3) and sun50i (A64 and H5) SoCs.
143+
135144 source "drivers/devfreq/event/Kconfig"
136145
137146 endif # PM_DEVFREQ
drivers/devfreq/Makefile+1−0
@@ -12,6 +12,7 @@ obj-$(CONFIG_ARM_EXYNOS_BUS_DEVFREQ) += exynos-bus.o · 11 unmodified lines
1212 obj-$(CONFIG_ARM_IMX_BUS_DEVFREQ) += imx-bus.o
1313 obj-$(CONFIG_ARM_IMX8M_DDRC_DEVFREQ) += imx8m-ddrc.o
1414 obj-$(CONFIG_ARM_RK3399_DMC_DEVFREQ) += rk3399_dmc.o
15+obj-$(CONFIG_ARM_SUN8I_A33_MBUS_DEVFREQ) += sun8i-a33-mbus.o
1516 obj-$(CONFIG_ARM_TEGRA_DEVFREQ) += tegra30-devfreq.o
1617
1718 # DEVFREQ Event Drivers
drivers/devfreq/devfreq.c+2−2
@@ -382,8 +382,8 @@ static int devfreq_set_target(struct devfreq *devfreq, unsigned long new_freq, · 381 unmodified lines
382382 devfreq_notify_transition(devfreq, &freqs, DEVFREQ_POSTCHANGE);
383383
384384 if (devfreq_update_status(devfreq, new_freq))
385− dev_err(&devfreq->dev,
386− "Couldn't update frequency transition information.\n");
385+ dev_warn(&devfreq->dev,
386+ "Couldn't update frequency transition information.\n");
387387
388388 devfreq->previous_freq = new_freq;
389389
drivers/devfreq/sun8i-a33-mbus.c+511−0
@@ -0,0 +1,511 @@
1+// SPDX-License-Identifier: GPL-2.0-only
2+//
3+// Copyright (C) 2020-2021 Samuel Holland <samuel@sholland.org>
4+//
5+
6+#include <linux/clk.h>
7+#include <linux/devfreq.h>
8+#include <linux/err.h>
9+#include <linux/io.h>
10+#include <linux/iopoll.h>
11+#include <linux/module.h>
12+#include <linux/of.h>
13+#include <linux/platform_device.h>
14+#include <linux/property.h>
15+
16+#define MBUS_CR 0x0000
17+#define MBUS_CR_GET_DRAM_TYPE(x) (((x) >> 16) & 0x7)
18+#define MBUS_CR_DRAM_TYPE_DDR2 2
19+#define MBUS_CR_DRAM_TYPE_DDR3 3
20+#define MBUS_CR_DRAM_TYPE_DDR4 4
21+#define MBUS_CR_DRAM_TYPE_LPDDR2 6
22+#define MBUS_CR_DRAM_TYPE_LPDDR3 7
23+
24+#define MBUS_TMR 0x000c
25+#define MBUS_TMR_PERIOD(x) ((x) - 1)
26+
27+#define MBUS_PMU_CFG 0x009c
28+#define MBUS_PMU_CFG_PERIOD(x) (((x) - 1) << 16)
29+#define MBUS_PMU_CFG_UNIT (0x3 << 1)
30+#define MBUS_PMU_CFG_UNIT_B (0x0 << 1)
31+#define MBUS_PMU_CFG_UNIT_KB (0x1 << 1)
32+#define MBUS_PMU_CFG_UNIT_MB (0x2 << 1)
33+#define MBUS_PMU_CFG_ENABLE (0x1 << 0)
34+
35+#define MBUS_PMU_BWCR(n) (0x00a0 + (0x04 * (n)))
36+
37+#define MBUS_TOTAL_BWCR MBUS_PMU_BWCR(5)
38+#define MBUS_TOTAL_BWCR_H616 MBUS_PMU_BWCR(13)
39+
40+#define MBUS_MDFSCR 0x0100
41+#define MBUS_MDFSCR_BUFFER_TIMING (0x1 << 15)
42+#define MBUS_MDFSCR_PAD_HOLD (0x1 << 13)
43+#define MBUS_MDFSCR_BYPASS (0x1 << 4)
44+#define MBUS_MDFSCR_MODE (0x1 << 1)
45+#define MBUS_MDFSCR_MODE_DFS (0x0 << 1)
46+#define MBUS_MDFSCR_MODE_CFS (0x1 << 1)
47+#define MBUS_MDFSCR_START (0x1 << 0)
48+
49+#define MBUS_MDFSMRMR 0x0108
50+
51+#define DRAM_PWRCTL 0x0004
52+#define DRAM_PWRCTL_SELFREF_EN (0x1 << 0)
53+
54+#define DRAM_RFSHTMG 0x0090
55+#define DRAM_RFSHTMG_TREFI(x) ((x) << 16)
56+#define DRAM_RFSHTMG_TRFC(x) ((x) << 0)
57+
58+#define DRAM_VTFCR 0x00b8
59+#define DRAM_VTFCR_VTF_ENABLE (0x3 << 8)
60+
61+#define DRAM_ODTMAP 0x0120
62+
63+#define DRAM_DX_MAX 4
64+
65+#define DRAM_DXnGCR0(n) (0x0344 + 0x80 * (n))
66+#define DRAM_DXnGCR0_DXODT (0x3 << 4)
67+#define DRAM_DXnGCR0_DXODT_DYNAMIC (0x0 << 4)
68+#define DRAM_DXnGCR0_DXODT_ENABLED (0x1 << 4)
69+#define DRAM_DXnGCR0_DXODT_DISABLED (0x2 << 4)
70+#define DRAM_DXnGCR0_DXEN (0x1 << 0)
71+
72+struct sun8i_a33_mbus_variant {
73+ u32 min_dram_divider;
74+ u32 max_dram_divider;
75+ u32 odt_freq_mhz;
76+};
77+
78+struct sun8i_a33_mbus {
79+ const struct sun8i_a33_mbus_variant *variant;
80+ void __iomem *reg_dram;
81+ void __iomem *reg_mbus;
82+ struct clk *clk_bus;
83+ struct clk *clk_dram;
84+ struct clk *clk_mbus;
85+ struct devfreq *devfreq_dram;
86+ struct devfreq_simple_ondemand_data gov_data;
87+ struct devfreq_dev_profile profile;
88+ u32 data_width;
89+ u32 nominal_bw;
90+ u32 odtmap;
91+ u32 tREFI_ns;
92+ u32 tRFC_ns;
93+ unsigned long freq_table[];
94+};
95+
96+/*
97+ * The unit for this value is (MBUS clock cycles / MBUS_TMR_PERIOD). When
98+ * MBUS_TMR_PERIOD is programmed to match the MBUS clock frequency in MHz, as
99+ * it is during DRAM init and during probe, the resulting unit is microseconds.
100+ */
101+static int pmu_period = 50000;
102+module_param(pmu_period, int, 0644);
103+MODULE_PARM_DESC(pmu_period, "Bandwidth measurement period (microseconds)");
104+
105+static u32 sun8i_a33_mbus_get_peak_bw(struct sun8i_a33_mbus *priv)
106+{
107+ /* Returns the peak transfer (in KiB) during any single PMU period. */
108+ return readl_relaxed(priv->reg_mbus + MBUS_TOTAL_BWCR);
109+}
110+
111+static void sun8i_a33_mbus_restart_pmu_counters(struct sun8i_a33_mbus *priv)
112+{
113+ u32 pmu_cfg = MBUS_PMU_CFG_PERIOD(pmu_period) | MBUS_PMU_CFG_UNIT_KB;
114+
115+ /* All PMU counters are cleared on a disable->enable transition. */
116+ writel_relaxed(pmu_cfg,
117+ priv->reg_mbus + MBUS_PMU_CFG);
118+ writel_relaxed(pmu_cfg | MBUS_PMU_CFG_ENABLE,
119+ priv->reg_mbus + MBUS_PMU_CFG);
120+
121+}
122+
123+static void sun8i_a33_mbus_update_nominal_bw(struct sun8i_a33_mbus *priv,
124+ u32 ddr_freq_mhz)
125+{
126+ /*
127+ * Nominal bandwidth (KiB per PMU period):
128+ *
129+ * DDR transfers microseconds KiB
130+ * ------------- * ------------ * --------
131+ * microsecond PMU period transfer
132+ */
133+ priv->nominal_bw = ddr_freq_mhz * pmu_period * priv->data_width / 1024;
134+}
135+
136+static int sun8i_a33_mbus_set_dram_freq(struct sun8i_a33_mbus *priv,
137+ unsigned long freq)
138+{
139+ u32 ddr_freq_mhz = freq / USEC_PER_SEC; /* DDR */
140+ u32 dram_freq_mhz = ddr_freq_mhz / 2; /* SDR */
141+ u32 mctl_freq_mhz = dram_freq_mhz / 2; /* HDR */
142+ u32 dxodt, mdfscr, pwrctl, vtfcr;
143+ u32 i, tREFI_32ck, tRFC_ck;
144+ int ret;
145+
146+ /* The rate change is not effective until the MDFS process runs. */
147+ ret = clk_set_rate(priv->clk_dram, freq);
148+ if (ret)
149+ return ret;
150+
151+ /* Disable automatic self-refesh and VTF before starting MDFS. */
152+ pwrctl = readl_relaxed(priv->reg_dram + DRAM_PWRCTL) &
153+ ~DRAM_PWRCTL_SELFREF_EN;
154+ writel_relaxed(pwrctl, priv->reg_dram + DRAM_PWRCTL);
155+ vtfcr = readl_relaxed(priv->reg_dram + DRAM_VTFCR);
156+ writel_relaxed(vtfcr & ~DRAM_VTFCR_VTF_ENABLE,
157+ priv->reg_dram + DRAM_VTFCR);
158+
159+ /* Set up MDFS and enable double buffering for timing registers. */
160+ mdfscr = MBUS_MDFSCR_MODE_DFS |
161+ MBUS_MDFSCR_BYPASS |
162+ MBUS_MDFSCR_PAD_HOLD |
163+ MBUS_MDFSCR_BUFFER_TIMING;
164+ writel(mdfscr, priv->reg_mbus + MBUS_MDFSCR);
165+
166+ /* Update the buffered copy of RFSHTMG. */
167+ tREFI_32ck = priv->tREFI_ns * mctl_freq_mhz / 1000 / 32;
168+ tRFC_ck = DIV_ROUND_UP(priv->tRFC_ns * mctl_freq_mhz, 1000);
169+ writel(DRAM_RFSHTMG_TREFI(tREFI_32ck) | DRAM_RFSHTMG_TRFC(tRFC_ck),
170+ priv->reg_dram + DRAM_RFSHTMG);
171+
172+ /* Enable ODT if needed, or disable it to save power. */
173+ if (priv->odtmap && dram_freq_mhz > priv->variant->odt_freq_mhz) {
174+ dxodt = DRAM_DXnGCR0_DXODT_DYNAMIC;
175+ writel(priv->odtmap, priv->reg_dram + DRAM_ODTMAP);
176+ } else {
177+ dxodt = DRAM_DXnGCR0_DXODT_DISABLED;
178+ writel(0, priv->reg_dram + DRAM_ODTMAP);
179+ }
180+ for (i = 0; i < DRAM_DX_MAX; ++i) {
181+ void __iomem *reg = priv->reg_dram + DRAM_DXnGCR0(i);
182+
183+ writel((readl(reg) & ~DRAM_DXnGCR0_DXODT) | dxodt, reg);
184+ }
185+
186+ dev_dbg(priv->devfreq_dram->dev.parent,
187+ "Setting DRAM to %u MHz, tREFI=%u, tRFC=%u, ODT=%s\n",
188+ dram_freq_mhz, tREFI_32ck, tRFC_ck,
189+ dxodt == DRAM_DXnGCR0_DXODT_DYNAMIC ? "dynamic" : "disabled");
190+
191+ /* Trigger hardware MDFS. */
192+ writel(mdfscr | MBUS_MDFSCR_START, priv->reg_mbus + MBUS_MDFSCR);
193+ ret = readl_poll_timeout_atomic(priv->reg_mbus + MBUS_MDFSCR, mdfscr,
194+ !(mdfscr & MBUS_MDFSCR_START), 10, 1000);
195+ if (ret)
196+ return ret;
197+
198+ /* Disable double buffering. */
199+ writel(0, priv->reg_mbus + MBUS_MDFSCR);
200+
201+ /* Restore VTF configuration. */
202+ writel_relaxed(vtfcr, priv->reg_dram + DRAM_VTFCR);
203+
204+ /* Enable automatic self-refresh at the lowest frequency only. */
205+ if (freq == priv->freq_table[0])
206+ pwrctl |= DRAM_PWRCTL_SELFREF_EN;
207+ writel_relaxed(pwrctl, priv->reg_dram + DRAM_PWRCTL);
208+
209+ sun8i_a33_mbus_restart_pmu_counters(priv);
210+ sun8i_a33_mbus_update_nominal_bw(priv, ddr_freq_mhz);
211+
212+ return 0;
213+}
214+
215+static int sun8i_a33_mbus_set_dram_target(struct device *dev,
216+ unsigned long *freq, u32 flags)
217+{
218+ struct sun8i_a33_mbus *priv = dev_get_drvdata(dev);
219+ struct devfreq *devfreq = priv->devfreq_dram;
220+ struct dev_pm_opp *opp;
221+ int ret;
222+
223+ opp = devfreq_recommended_opp(dev, freq, flags);
224+ if (IS_ERR(opp))
225+ return PTR_ERR(opp);
226+
227+ dev_pm_opp_put(opp);
228+
229+ if (*freq == devfreq->previous_freq)
230+ return 0;
231+
232+ ret = sun8i_a33_mbus_set_dram_freq(priv, *freq);
233+ if (ret) {
234+ dev_warn(dev, "failed to set DRAM frequency: %d\n", ret);
235+ *freq = devfreq->previous_freq;
236+ }
237+
238+ return ret;
239+}
240+
241+static int sun8i_a33_mbus_get_dram_status(struct device *dev,
242+ struct devfreq_dev_status *stat)
243+{
244+ struct sun8i_a33_mbus *priv = dev_get_drvdata(dev);
245+
246+ stat->busy_time = sun8i_a33_mbus_get_peak_bw(priv);
247+ stat->total_time = priv->nominal_bw;
248+ stat->current_frequency = priv->devfreq_dram->previous_freq;
249+
250+ sun8i_a33_mbus_restart_pmu_counters(priv);
251+
252+ dev_dbg(dev, "Using %lu/%lu (%lu%%) at %lu MHz\n",
253+ stat->busy_time, stat->total_time,
254+ DIV_ROUND_CLOSEST(stat->busy_time * 100, stat->total_time),
255+ stat->current_frequency / USEC_PER_SEC);
256+
257+ return 0;
258+}
259+
260+static int sun8i_a33_mbus_hw_init(struct device *dev,
261+ struct sun8i_a33_mbus *priv,
262+ unsigned long ddr_freq)
263+{
264+ u32 i, mbus_cr, mbus_freq_mhz;
265+
266+ /* Choose tREFI and tRFC to match the configured DRAM type. */
267+ mbus_cr = readl_relaxed(priv->reg_mbus + MBUS_CR);
268+ switch (MBUS_CR_GET_DRAM_TYPE(mbus_cr)) {
269+ case MBUS_CR_DRAM_TYPE_DDR2:
270+ case MBUS_CR_DRAM_TYPE_DDR3:
271+ case MBUS_CR_DRAM_TYPE_DDR4:
272+ priv->tREFI_ns = 7800;
273+ priv->tRFC_ns = 350;
274+ break;
275+ case MBUS_CR_DRAM_TYPE_LPDDR2:
276+ case MBUS_CR_DRAM_TYPE_LPDDR3:
277+ priv->tREFI_ns = 3900;
278+ priv->tRFC_ns = 210;
279+ break;
280+ default:
281+ return -EINVAL;
282+ }
283+
284+ /* Save ODTMAP so it can be restored when raising the frequency. */
285+ priv->odtmap = readl_relaxed(priv->reg_dram + DRAM_ODTMAP);
286+
287+ /* Compute the DRAM data bus width by counting enabled DATx8 blocks. */
288+ for (i = 0; i < DRAM_DX_MAX; ++i) {
289+ void __iomem *reg = priv->reg_dram + DRAM_DXnGCR0(i);
290+
291+ if (!(readl_relaxed(reg) & DRAM_DXnGCR0_DXEN))
292+ break;
293+ }
294+ priv->data_width = i;
295+
296+ dev_dbg(dev, "Detected %u-bit %sDDRx with%s ODT\n",
297+ priv->data_width * 8,
298+ MBUS_CR_GET_DRAM_TYPE(mbus_cr) > 4 ? "LP" : "",
299+ priv->odtmap ? "" : "out");
300+
301+ /* Program MBUS_TMR such that the PMU period unit is microseconds. */
302+ mbus_freq_mhz = clk_get_rate(priv->clk_mbus) / USEC_PER_SEC;
303+ writel_relaxed(MBUS_TMR_PERIOD(mbus_freq_mhz),
304+ priv->reg_mbus + MBUS_TMR);
305+
306+ /* "Master Ready Mask Register" bits must be set or MDFS will block. */
307+ writel_relaxed(0xffffffff, priv->reg_mbus + MBUS_MDFSMRMR);
308+
309+ sun8i_a33_mbus_restart_pmu_counters(priv);
310+ sun8i_a33_mbus_update_nominal_bw(priv, ddr_freq / USEC_PER_SEC);
311+
312+ return 0;
313+}
314+
315+static int __maybe_unused sun8i_a33_mbus_suspend(struct device *dev)
316+{
317+ struct sun8i_a33_mbus *priv = dev_get_drvdata(dev);
318+
319+ clk_disable_unprepare(priv->clk_bus);
320+
321+ return 0;
322+}
323+
324+static int __maybe_unused sun8i_a33_mbus_resume(struct device *dev)
325+{
326+ struct sun8i_a33_mbus *priv = dev_get_drvdata(dev);
327+
328+ return clk_prepare_enable(priv->clk_bus);
329+}
330+
331+static int sun8i_a33_mbus_probe(struct platform_device *pdev)
332+{
333+ const struct sun8i_a33_mbus_variant *variant;
334+ struct device *dev = &pdev->dev;
335+ struct sun8i_a33_mbus *priv;
336+ unsigned long base_freq;
337+ unsigned int max_state;
338+ const char *err;
339+ int i, ret;
340+
341+ variant = device_get_match_data(dev);
342+ if (!variant)
343+ return -EINVAL;
344+
345+ max_state = variant->max_dram_divider - variant->min_dram_divider + 1;
346+
347+ priv = devm_kzalloc(dev, struct_size(priv, freq_table, max_state), GFP_KERNEL);
348+ if (!priv)
349+ return -ENOMEM;
350+
351+ platform_set_drvdata(pdev, priv);
352+
353+ priv->variant = variant;
354+
355+ priv->reg_dram = devm_platform_ioremap_resource_byname(pdev, "dram");
356+ if (IS_ERR(priv->reg_dram))
357+ return PTR_ERR(priv->reg_dram);
358+
359+ priv->reg_mbus = devm_platform_ioremap_resource_byname(pdev, "mbus");
360+ if (IS_ERR(priv->reg_mbus))
361+ return PTR_ERR(priv->reg_mbus);
362+
363+ priv->clk_bus = devm_clk_get(dev, "bus");
364+ if (IS_ERR(priv->clk_bus))
365+ return dev_err_probe(dev, PTR_ERR(priv->clk_bus),
366+ "failed to get bus clock\n");
367+
368+ priv->clk_dram = devm_clk_get(dev, "dram");
369+ if (IS_ERR(priv->clk_dram))
370+ return dev_err_probe(dev, PTR_ERR(priv->clk_dram),
371+ "failed to get dram clock\n");
372+
373+ priv->clk_mbus = devm_clk_get(dev, "mbus");
374+ if (IS_ERR(priv->clk_mbus))
375+ return dev_err_probe(dev, PTR_ERR(priv->clk_mbus),
376+ "failed to get mbus clock\n");
377+
378+ ret = clk_prepare_enable(priv->clk_bus);
379+ if (ret)
380+ return dev_err_probe(dev, ret,
381+ "failed to enable bus clock\n");
382+
383+ /* Lock the DRAM clock rate to keep priv->nominal_bw in sync. */
384+ ret = clk_rate_exclusive_get(priv->clk_dram);
385+ if (ret) {
386+ err = "failed to lock dram clock rate\n";
387+ goto err_disable_bus;
388+ }
389+
390+ /* Lock the MBUS clock rate to keep MBUS_TMR_PERIOD in sync. */
391+ ret = clk_rate_exclusive_get(priv->clk_mbus);
392+ if (ret) {
393+ err = "failed to lock mbus clock rate\n";
394+ goto err_unlock_dram;
395+ }
396+
397+ priv->gov_data.upthreshold = 10;
398+ priv->gov_data.downdifferential = 5;
399+
400+ priv->profile.initial_freq = clk_get_rate(priv->clk_dram);
401+ priv->profile.polling_ms = 1000;
402+ priv->profile.target = sun8i_a33_mbus_set_dram_target;
403+ priv->profile.get_dev_status = sun8i_a33_mbus_get_dram_status;
404+ priv->profile.freq_table = priv->freq_table;
405+ priv->profile.max_state = max_state;
406+
407+ ret = devm_pm_opp_set_clkname(dev, "dram");
408+ if (ret) {
409+ err = "failed to add OPP table\n";
410+ goto err_unlock_mbus;
411+ }
412+
413+ base_freq = clk_get_rate(clk_get_parent(priv->clk_dram));
414+ for (i = 0; i < max_state; ++i) {
415+ unsigned int div = variant->max_dram_divider - i;
416+
417+ priv->freq_table[i] = base_freq / div;
418+
419+ ret = dev_pm_opp_add(dev, priv->freq_table[i], 0);
420+ if (ret) {
421+ err = "failed to add OPPs\n";
422+ goto err_remove_opps;
423+ }
424+ }
425+
426+ ret = sun8i_a33_mbus_hw_init(dev, priv, priv->profile.initial_freq);
427+ if (ret) {
428+ err = "failed to init hardware\n";
429+ goto err_remove_opps;
430+ }
431+
432+ priv->devfreq_dram = devfreq_add_device(dev, &priv->profile,
433+ DEVFREQ_GOV_SIMPLE_ONDEMAND,
434+ &priv->gov_data);
435+ if (IS_ERR(priv->devfreq_dram)) {
436+ ret = PTR_ERR(priv->devfreq_dram);
437+ err = "failed to add devfreq device\n";
438+ goto err_remove_opps;
439+ }
440+
441+ /*
442+ * This must be set manually after registering the devfreq device,
443+ * because there is no way to select a dynamic OPP as the suspend OPP.
444+ */
445+ priv->devfreq_dram->suspend_freq = priv->freq_table[0];
446+
447+ return 0;
448+
449+err_remove_opps:
450+ dev_pm_opp_remove_all_dynamic(dev);
451+err_unlock_mbus:
452+ clk_rate_exclusive_put(priv->clk_mbus);
453+err_unlock_dram:
454+ clk_rate_exclusive_put(priv->clk_dram);
455+err_disable_bus:
456+ clk_disable_unprepare(priv->clk_bus);
457+
458+ return dev_err_probe(dev, ret, err);
459+}
460+
461+static int sun8i_a33_mbus_remove(struct platform_device *pdev)
462+{
463+ struct sun8i_a33_mbus *priv = platform_get_drvdata(pdev);
464+ unsigned long initial_freq = priv->profile.initial_freq;
465+ struct device *dev = &pdev->dev;
466+ int ret;
467+
468+ devfreq_remove_device(priv->devfreq_dram);
469+
470+ ret = sun8i_a33_mbus_set_dram_freq(priv, initial_freq);
471+ if (ret)
472+ dev_warn(dev, "failed to restore DRAM frequency: %d\n", ret);
473+
474+ dev_pm_opp_remove_all_dynamic(dev);
475+ clk_rate_exclusive_put(priv->clk_mbus);
476+ clk_rate_exclusive_put(priv->clk_dram);
477+ clk_disable_unprepare(priv->clk_bus);
478+
479+ return 0;
480+}
481+
482+static const struct sun8i_a33_mbus_variant sun50i_a64_mbus = {
483+ .min_dram_divider = 1,
484+ .max_dram_divider = 4,
485+ .odt_freq_mhz = 400,
486+};
487+
488+static const struct of_device_id sun8i_a33_mbus_of_match[] = {
489+ { .compatible = "allwinner,sun50i-a64-mbus", .data = &sun50i_a64_mbus },
490+ { .compatible = "allwinner,sun50i-h5-mbus", .data = &sun50i_a64_mbus },
491+ { },
492+};
493+MODULE_DEVICE_TABLE(of, sun8i_a33_mbus_of_match);
494+
495+static SIMPLE_DEV_PM_OPS(sun8i_a33_mbus_pm_ops,
496+ sun8i_a33_mbus_suspend, sun8i_a33_mbus_resume);
497+
498+static struct platform_driver sun8i_a33_mbus_driver = {
499+ .probe = sun8i_a33_mbus_probe,
500+ .remove = sun8i_a33_mbus_remove,
501+ .driver = {
502+ .name = "sun8i-a33-mbus",
503+ .of_match_table = sun8i_a33_mbus_of_match,
504+ .pm = pm_ptr(&sun8i_a33_mbus_pm_ops),
505+ },
506+};
507+module_platform_driver(sun8i_a33_mbus_driver);
508+
509+MODULE_AUTHOR("Samuel Holland <samuel@sholland.org>");
510+MODULE_DESCRIPTION("Allwinner sun8i/sun50i MBUS DEVFREQ Driver");
511+MODULE_LICENSE("GPL v2");
drivers/mmc/host/jz4740_mmc.c+4−4
@@ -1103,17 +1103,17 @@ static int jz4740_mmc_remove(struct platform_device *pdev) · 1102 unmodified lines
11031103 return 0;
11041104 }
11051105
1106−static int __maybe_unused jz4740_mmc_suspend(struct device *dev)
1106+static int jz4740_mmc_suspend(struct device *dev)
11071107 {
11081108 return pinctrl_pm_select_sleep_state(dev);
11091109 }
11101110
1111−static int __maybe_unused jz4740_mmc_resume(struct device *dev)
1111+static int jz4740_mmc_resume(struct device *dev)
11121112 {
11131113 return pinctrl_select_default_state(dev);
11141114 }
11151115
1116−static SIMPLE_DEV_PM_OPS(jz4740_mmc_pm_ops, jz4740_mmc_suspend,
1116+DEFINE_SIMPLE_DEV_PM_OPS(jz4740_mmc_pm_ops, jz4740_mmc_suspend,
11171117 jz4740_mmc_resume);
11181118
11191119 static struct platform_driver jz4740_mmc_driver = {
@@ -1123,7 +1123,7 @@ static struct platform_driver jz4740_mmc_driver = { · 3 unmodified lines
11231123 .name = "jz4740-mmc",
11241124 .probe_type = PROBE_PREFER_ASYNCHRONOUS,
11251125 .of_match_table = of_match_ptr(jz4740_mmc_of_match),
1126− .pm = pm_ptr(&jz4740_mmc_pm_ops),
1126+ .pm = pm_sleep_ptr(&jz4740_mmc_pm_ops),
11271127 },
11281128 };
11291129
drivers/mmc/host/mxcmmc.c+2−4
@@ -1183,7 +1183,6 @@ static int mxcmci_remove(struct platform_device *pdev) · 1182 unmodified lines
11831183 return 0;
11841184 }
11851185
1186−#ifdef CONFIG_PM_SLEEP
11871186 static int mxcmci_suspend(struct device *dev)
11881187 {
11891188 struct mmc_host *mmc = dev_get_drvdata(dev);
@@ -1210,9 +1209,8 @@ static int mxcmci_resume(struct device *dev) · 20 unmodified lines
12101209
12111210 return ret;
12121211 }
1213−#endif
12141212
1215−static SIMPLE_DEV_PM_OPS(mxcmci_pm_ops, mxcmci_suspend, mxcmci_resume);
1213+DEFINE_SIMPLE_DEV_PM_OPS(mxcmci_pm_ops, mxcmci_suspend, mxcmci_resume);
12161214
12171215 static struct platform_driver mxcmci_driver = {
12181216 .probe = mxcmci_probe,
@@ -1220,7 +1218,7 @@ static struct platform_driver mxcmci_driver = { · 1 unmodified line
12201218 .driver = {
12211219 .name = DRIVER_NAME,
12221220 .probe_type = PROBE_PREFER_ASYNCHRONOUS,
1223− .pm = &mxcmci_pm_ops,
1221+ .pm = pm_sleep_ptr(&mxcmci_pm_ops),
12241222 .of_match_table = mxcmci_of_match,
12251223 }
12261224 };
drivers/net/ethernet/realtek/r8169_main.c+3−1
@@ -5460,7 +5460,9 @@ static struct pci_driver rtl8169_pci_driver = { · 5459 unmodified lines
54605460 .probe = rtl_init_one,
54615461 .remove = rtl_remove_one,
54625462 .shutdown = rtl_shutdown,
5463− .driver.pm = pm_ptr(&rtl8169_pm_ops),
5463+#ifdef CONFIG_PM
5464+ .driver.pm = &rtl8169_pm_ops,
5465+#endif
54645466 };
54655467
54665468 module_pci_driver(rtl8169_pci_driver);
drivers/powercap/dtpm.c+3−3
@@ -382,7 +382,7 @@ void dtpm_unregister(struct dtpm *dtpm) · 381 unmodified lines
382382 {
383383 powercap_unregister_zone(pct, &dtpm->zone);
384384
385− pr_info("Unregistered dtpm node '%s'\n", dtpm->zone.name);
385+ pr_debug("Unregistered dtpm node '%s'\n", dtpm->zone.name);
386386 }
387387
388388 /**
@@ -453,8 +453,8 @@ int dtpm_register(const char *name, struct dtpm *dtpm, struct dtpm *parent) · 64 unmodified lines
453453 dtpm->power_limit = dtpm->power_max;
454454 }
455455
456− pr_info("Registered dtpm node '%s' / %llu-%llu uW, \n",
457− dtpm->zone.name, dtpm->power_min, dtpm->power_max);
456+ pr_debug("Registered dtpm node '%s' / %llu-%llu uW, \n",
457+ dtpm->zone.name, dtpm->power_min, dtpm->power_max);
458458
459459 mutex_unlock(&dtpm_lock);
460460
drivers/powercap/idle_inject.c+1−1
@@ -12,7 +12,7 @@ · 11 unmodified lines
1212 *
1313 * All of the kthreads used for idle injection are created at init time.
1414 *
15− * Next, the users of the the idle injection framework provide a cpumask via
15+ * Next, the users of the idle injection framework provide a cpumask via
1616 * its register function. The kthreads will be synchronized with respect to
1717 * this cpumask.
1818 *
drivers/powercap/intel_rapl_common.c+59−2
@@ -61,6 +61,20 @@ · 60 unmodified lines
6161 #define PERF_STATUS_THROTTLE_TIME_MASK 0xffffffff
6262 #define PP_POLICY_MASK 0x1F
6363
64+/*
65+ * SPR has different layout for Psys Domain PowerLimit registers.
66+ * There are 17 bits of PL1 and PL2 instead of 15 bits.
67+ * The Enable bits and TimeWindow bits are also shifted as a result.
68+ */
69+#define PSYS_POWER_LIMIT1_MASK 0x1FFFF
70+#define PSYS_POWER_LIMIT1_ENABLE BIT(17)
71+
72+#define PSYS_POWER_LIMIT2_MASK (0x1FFFFULL<<32)
73+#define PSYS_POWER_LIMIT2_ENABLE BIT_ULL(49)
74+
75+#define PSYS_TIME_WINDOW1_MASK (0x7FULL<<19)
76+#define PSYS_TIME_WINDOW2_MASK (0x7FULL<<51)
77+
6478 /* Non HW constants */
6579 #define RAPL_PRIMITIVE_DERIVED BIT(1) /* not from raw data */
6680 #define RAPL_PRIMITIVE_DUMMY BIT(2)
@@ -97,6 +111,7 @@ struct rapl_defaults { · 30 unmodified lines
97111 bool to_raw);
98112 unsigned int dram_domain_energy_unit;
99113 unsigned int psys_domain_energy_unit;
114+ bool spr_psys_bits;
100115 };
101116 static struct rapl_defaults *rapl_defaults;
102117
@@ -669,12 +684,51 @@ static struct rapl_primitive_info rpi[] = { · 566 unmodified lines
669684 RAPL_DOMAIN_REG_PERF, TIME_UNIT, 0),
670685 PRIMITIVE_INFO_INIT(PRIORITY_LEVEL, PP_POLICY_MASK, 0,
671686 RAPL_DOMAIN_REG_POLICY, ARBITRARY_UNIT, 0),
687+ PRIMITIVE_INFO_INIT(PSYS_POWER_LIMIT1, PSYS_POWER_LIMIT1_MASK, 0,
688+ RAPL_DOMAIN_REG_LIMIT, POWER_UNIT, 0),
689+ PRIMITIVE_INFO_INIT(PSYS_POWER_LIMIT2, PSYS_POWER_LIMIT2_MASK, 32,
690+ RAPL_DOMAIN_REG_LIMIT, POWER_UNIT, 0),
691+ PRIMITIVE_INFO_INIT(PSYS_PL1_ENABLE, PSYS_POWER_LIMIT1_ENABLE, 17,
692+ RAPL_DOMAIN_REG_LIMIT, ARBITRARY_UNIT, 0),
693+ PRIMITIVE_INFO_INIT(PSYS_PL2_ENABLE, PSYS_POWER_LIMIT2_ENABLE, 49,
694+ RAPL_DOMAIN_REG_LIMIT, ARBITRARY_UNIT, 0),
695+ PRIMITIVE_INFO_INIT(PSYS_TIME_WINDOW1, PSYS_TIME_WINDOW1_MASK, 19,
696+ RAPL_DOMAIN_REG_LIMIT, TIME_UNIT, 0),
697+ PRIMITIVE_INFO_INIT(PSYS_TIME_WINDOW2, PSYS_TIME_WINDOW2_MASK, 51,
698+ RAPL_DOMAIN_REG_LIMIT, TIME_UNIT, 0),
672699 /* non-hardware */
673700 PRIMITIVE_INFO_INIT(AVERAGE_POWER, 0, 0, 0, POWER_UNIT,
674701 RAPL_PRIMITIVE_DERIVED),
675702 {NULL, 0, 0, 0},
676703 };
677704
705+static enum rapl_primitives
706+prim_fixups(struct rapl_domain *rd, enum rapl_primitives prim)
707+{
708+ if (!rapl_defaults->spr_psys_bits)
709+ return prim;
710+
711+ if (rd->id != RAPL_DOMAIN_PLATFORM)
712+ return prim;
713+
714+ switch (prim) {
715+ case POWER_LIMIT1:
716+ return PSYS_POWER_LIMIT1;
717+ case POWER_LIMIT2:
718+ return PSYS_POWER_LIMIT2;
719+ case PL1_ENABLE:
720+ return PSYS_PL1_ENABLE;
721+ case PL2_ENABLE:
722+ return PSYS_PL2_ENABLE;
723+ case TIME_WINDOW1:
724+ return PSYS_TIME_WINDOW1;
725+ case TIME_WINDOW2:
726+ return PSYS_TIME_WINDOW2;
727+ default:
728+ return prim;
729+ }
730+}
731+
678732 /* Read primitive data based on its related struct rapl_primitive_info.
679733 * if xlate flag is set, return translated data based on data units, i.e.
680734 * time, energy, and power.
@@ -692,7 +746,8 @@ static int rapl_read_data_raw(struct rapl_domain *rd, · 11 unmodified lines
692746 enum rapl_primitives prim, bool xlate, u64 *data)
693747 {
694748 u64 value;
695− struct rapl_primitive_info *rp = &rpi[prim];
749+ enum rapl_primitives prim_fixed = prim_fixups(rd, prim);
750+ struct rapl_primitive_info *rp = &rpi[prim_fixed];
696751 struct reg_action ra;
697752 int cpu;
698753
@@ -738,7 +793,8 @@ static int rapl_write_data_raw(struct rapl_domain *rd, · 39 unmodified lines
738793 enum rapl_primitives prim,
739794 unsigned long long value)
740795 {
741− struct rapl_primitive_info *rp = &rpi[prim];
796+ enum rapl_primitives prim_fixed = prim_fixups(rd, prim);
797+ struct rapl_primitive_info *rp = &rpi[prim_fixed];
742798 int cpu;
743799 u64 bits;
744800 struct reg_action ra;
@@ -981,6 +1037,7 @@ static const struct rapl_defaults rapl_defaults_spr_server = { · 236 unmodified lines
9811037 .compute_time_window = rapl_compute_time_window_core,
9821038 .dram_domain_energy_unit = 15300,
9831039 .psys_domain_energy_unit = 1000000000,
1040+ .spr_psys_bits = true,
9841041 };
9851042
9861043 static const struct rapl_defaults rapl_defaults_byt = {
drivers/thermal/cpufreq_cooling.c+1−5
@@ -462,7 +462,6 @@ static int cpufreq_set_cur_state(struct thermal_cooling_device *cdev, · 461 unmodified lines
462462 struct cpufreq_cooling_device *cpufreq_cdev = cdev->devdata;
463463 struct cpumask *cpus;
464464 unsigned int frequency;
465− unsigned long max_capacity, capacity;
466465 int ret;
467466
468467 /* Request state should be less than max_level */
@@ -479,10 +478,7 @@ static int cpufreq_set_cur_state(struct thermal_cooling_device *cdev, · 10 unmodified lines
479478 if (ret >= 0) {
480479 cpufreq_cdev->cpufreq_state = state;
481480 cpus = cpufreq_cdev->policy->related_cpus;
482− max_capacity = arch_scale_cpu_capacity(cpumask_first(cpus));
483− capacity = frequency * max_capacity;
484− capacity /= cpufreq_cdev->policy->cpuinfo.max_freq;
485− arch_set_thermal_pressure(cpus, max_capacity - capacity);
481+ arch_update_thermal_pressure(cpus, frequency);
486482 ret = 0;
487483 }
488484
include/acpi/cppc_acpi.h+5−0
@@ -138,6 +138,7 @@ extern int cppc_get_desired_perf(int cpunum, u64 *desired_perf); · 137 unmodified lines
138138 extern int cppc_get_nominal_perf(int cpunum, u64 *nominal_perf);
139139 extern int cppc_get_perf_ctrs(int cpu, struct cppc_perf_fb_ctrs *perf_fb_ctrs);
140140 extern int cppc_set_perf(int cpu, struct cppc_perf_ctrls *perf_ctrls);
141+extern int cppc_set_enable(int cpu, bool enable);
141142 extern int cppc_get_perf_caps(int cpu, struct cppc_perf_caps *caps);
142143 extern bool acpi_cpc_valid(void);
143144 extern int acpi_get_psd_map(unsigned int cpu, struct cppc_cpudata *cpu_data);
@@ -162,6 +163,10 @@ static inline int cppc_set_perf(int cpu, struct cppc_perf_ctrls *perf_ctrls) · 18 unmodified lines
162163 {
163164 return -ENOTSUPP;
164165 }
166+static inline int cppc_set_enable(int cpu, bool enable)
167+{
168+ return -ENOTSUPP;
169+}
165170 static inline int cppc_get_perf_caps(int cpu, struct cppc_perf_caps *caps)
166171 {
167172 return -ENOTSUPP;
include/linux/acpi.h+1−1
@@ -506,7 +506,7 @@ acpi_status acpi_release_memory(acpi_handle handle, struct resource *res, · 505 unmodified lines
506506 int acpi_resources_are_enforced(void);
507507
508508 #ifdef CONFIG_HIBERNATION
509−void __init acpi_no_s4_hw_signature(void);
509+void __init acpi_check_s4_hw_signature(int check);
510510 #endif
511511
512512 #ifdef CONFIG_PM_SLEEP
include/linux/arch_topology.h+2−2
@@ -56,8 +56,8 @@ static inline unsigned long topology_get_thermal_pressure(int cpu) · 55 unmodified lines
5656 return per_cpu(thermal_pressure, cpu);
5757 }
5858
59−void topology_set_thermal_pressure(const struct cpumask *cpus,
60− unsigned long th_pressure);
59+void topology_update_thermal_pressure(const struct cpumask *cpus,
60+ unsigned long capped_freq);
6161
6262 struct cpu_topology {
6363 int thread_id;
include/linux/dtpm.h+0−2
@@ -70,6 +70,4 @@ void dtpm_unregister(struct dtpm *dtpm); · 69 unmodified lines
7070
7171 int dtpm_register(const char *name, struct dtpm *dtpm, struct dtpm *parent);
7272
73−int dtpm_register_cpu(struct dtpm *parent);
74−
7573 #endif
include/linux/intel_rapl.h+6−0
@@ -58,6 +58,12 @@ enum rapl_primitives { · 57 unmodified lines
5858 THROTTLED_TIME,
5959 PRIORITY_LEVEL,
6060
61+ PSYS_POWER_LIMIT1,
62+ PSYS_POWER_LIMIT2,
63+ PSYS_PL1_ENABLE,
64+ PSYS_PL2_ENABLE,
65+ PSYS_TIME_WINDOW1,
66+ PSYS_TIME_WINDOW2,
6167 /* below are not raw primitive data */
6268 AVERAGE_POWER,
6369 NR_RAPL_PRIMITIVES,
include/linux/pm.h+53−29
@@ -300,47 +300,59 @@ struct dev_pm_ops { · 299 unmodified lines
300300 int (*runtime_idle)(struct device *dev);
301301 };
302302
303+#define SYSTEM_SLEEP_PM_OPS(suspend_fn, resume_fn) \
304+ .suspend = pm_sleep_ptr(suspend_fn), \
305+ .resume = pm_sleep_ptr(resume_fn), \
306+ .freeze = pm_sleep_ptr(suspend_fn), \
307+ .thaw = pm_sleep_ptr(resume_fn), \
308+ .poweroff = pm_sleep_ptr(suspend_fn), \
309+ .restore = pm_sleep_ptr(resume_fn),
310+
311+#define LATE_SYSTEM_SLEEP_PM_OPS(suspend_fn, resume_fn) \
312+ .suspend_late = pm_sleep_ptr(suspend_fn), \
313+ .resume_early = pm_sleep_ptr(resume_fn), \
314+ .freeze_late = pm_sleep_ptr(suspend_fn), \
315+ .thaw_early = pm_sleep_ptr(resume_fn), \
316+ .poweroff_late = pm_sleep_ptr(suspend_fn), \
317+ .restore_early = pm_sleep_ptr(resume_fn),
318+
319+#define NOIRQ_SYSTEM_SLEEP_PM_OPS(suspend_fn, resume_fn) \
320+ .suspend_noirq = pm_sleep_ptr(suspend_fn), \
321+ .resume_noirq = pm_sleep_ptr(resume_fn), \
322+ .freeze_noirq = pm_sleep_ptr(suspend_fn), \
323+ .thaw_noirq = pm_sleep_ptr(resume_fn), \
324+ .poweroff_noirq = pm_sleep_ptr(suspend_fn), \
325+ .restore_noirq = pm_sleep_ptr(resume_fn),
326+
327+#define RUNTIME_PM_OPS(suspend_fn, resume_fn, idle_fn) \
328+ .runtime_suspend = suspend_fn, \
329+ .runtime_resume = resume_fn, \
330+ .runtime_idle = idle_fn,
331+
303332 #ifdef CONFIG_PM_SLEEP
304333 #define SET_SYSTEM_SLEEP_PM_OPS(suspend_fn, resume_fn) \
305− .suspend = suspend_fn, \
306− .resume = resume_fn, \
307− .freeze = suspend_fn, \
308− .thaw = resume_fn, \
309− .poweroff = suspend_fn, \
310− .restore = resume_fn,
334+ SYSTEM_SLEEP_PM_OPS(suspend_fn, resume_fn)
311335 #else
312336 #define SET_SYSTEM_SLEEP_PM_OPS(suspend_fn, resume_fn)
313337 #endif
314338
315339 #ifdef CONFIG_PM_SLEEP
316340 #define SET_LATE_SYSTEM_SLEEP_PM_OPS(suspend_fn, resume_fn) \
317− .suspend_late = suspend_fn, \
318− .resume_early = resume_fn, \
319− .freeze_late = suspend_fn, \
320− .thaw_early = resume_fn, \
321− .poweroff_late = suspend_fn, \
322− .restore_early = resume_fn,
341+ LATE_SYSTEM_SLEEP_PM_OPS(suspend_fn, resume_fn)
323342 #else
324343 #define SET_LATE_SYSTEM_SLEEP_PM_OPS(suspend_fn, resume_fn)
325344 #endif
326345
327346 #ifdef CONFIG_PM_SLEEP
328347 #define SET_NOIRQ_SYSTEM_SLEEP_PM_OPS(suspend_fn, resume_fn) \
329− .suspend_noirq = suspend_fn, \
330− .resume_noirq = resume_fn, \
331− .freeze_noirq = suspend_fn, \
332− .thaw_noirq = resume_fn, \
333− .poweroff_noirq = suspend_fn, \
334− .restore_noirq = resume_fn,
348+ NOIRQ_SYSTEM_SLEEP_PM_OPS(suspend_fn, resume_fn)
335349 #else
336350 #define SET_NOIRQ_SYSTEM_SLEEP_PM_OPS(suspend_fn, resume_fn)
337351 #endif
338352
339353 #ifdef CONFIG_PM
340354 #define SET_RUNTIME_PM_OPS(suspend_fn, resume_fn, idle_fn) \
341− .runtime_suspend = suspend_fn, \
342− .runtime_resume = resume_fn, \
343− .runtime_idle = idle_fn,
355+ RUNTIME_PM_OPS(suspend_fn, resume_fn, idle_fn)
344356 #else
345357 #define SET_RUNTIME_PM_OPS(suspend_fn, resume_fn, idle_fn)
346358 #endif
@@ -349,9 +361,9 @@ struct dev_pm_ops { · 2 unmodified lines
349361 * Use this if you want to use the same suspend and resume callbacks for suspend
350362 * to RAM and hibernation.
351363 */
352−#define SIMPLE_DEV_PM_OPS(name, suspend_fn, resume_fn) \
353−const struct dev_pm_ops __maybe_unused name = { \
354− SET_SYSTEM_SLEEP_PM_OPS(suspend_fn, resume_fn) \
364+#define DEFINE_SIMPLE_DEV_PM_OPS(name, suspend_fn, resume_fn) \
365+static const struct dev_pm_ops name = { \
366+ SYSTEM_SLEEP_PM_OPS(suspend_fn, resume_fn) \
355367 }
356368
357369 /*
@@ -367,17 +379,27 @@ const struct dev_pm_ops __maybe_unused name = { \ · 9 unmodified lines
367379 * .resume_early(), to the same routines as .runtime_suspend() and
368380 * .runtime_resume(), respectively (and analogously for hibernation).
369381 */
382+#define DEFINE_UNIVERSAL_DEV_PM_OPS(name, suspend_fn, resume_fn, idle_fn) \
383+static const struct dev_pm_ops name = { \
384+ SYSTEM_SLEEP_PM_OPS(suspend_fn, resume_fn) \
385+ RUNTIME_PM_OPS(suspend_fn, resume_fn, idle_fn) \
386+}
387+
388+/* Deprecated. Use DEFINE_SIMPLE_DEV_PM_OPS() instead. */
389+#define SIMPLE_DEV_PM_OPS(name, suspend_fn, resume_fn) \
390+const struct dev_pm_ops __maybe_unused name = { \
391+ SET_SYSTEM_SLEEP_PM_OPS(suspend_fn, resume_fn) \
392+}
393+
394+/* Deprecated. Use DEFINE_UNIVERSAL_DEV_PM_OPS() instead. */
370395 #define UNIVERSAL_DEV_PM_OPS(name, suspend_fn, resume_fn, idle_fn) \
371396 const struct dev_pm_ops __maybe_unused name = { \
372397 SET_SYSTEM_SLEEP_PM_OPS(suspend_fn, resume_fn) \
373398 SET_RUNTIME_PM_OPS(suspend_fn, resume_fn, idle_fn) \
374399 }
375400
376−#ifdef CONFIG_PM
377−#define pm_ptr(_ptr) (_ptr)
378−#else
379−#define pm_ptr(_ptr) NULL
380−#endif
401+#define pm_ptr(_ptr) PTR_IF(IS_ENABLED(CONFIG_PM), (_ptr))
402+#define pm_sleep_ptr(_ptr) PTR_IF(IS_ENABLED(CONFIG_PM_SLEEP), (_ptr))
381403
382404 /*
383405 * PM_EVENT_ messages
@@ -499,6 +521,7 @@ const struct dev_pm_ops __maybe_unused name = { \ · 115 unmodified lines
499521 */
500522
501523 enum rpm_status {
524+ RPM_INVALID = -1,
502525 RPM_ACTIVE = 0,
503526 RPM_RESUMING,
504527 RPM_SUSPENDED,
@@ -612,6 +635,7 @@ struct dev_pm_info { · 107 unmodified lines
612635 unsigned int links_count;
613636 enum rpm_request request;
614637 enum rpm_status runtime_status;
638+ enum rpm_status last_status;
615639 int runtime_error;
616640 int autosuspend_delay;
617641 u64 last_busy;
include/linux/pm_runtime.h+3−0
@@ -58,6 +58,7 @@ extern void pm_runtime_get_suppliers(struct device *dev); · 57 unmodified lines
5858 extern void pm_runtime_put_suppliers(struct device *dev);
5959 extern void pm_runtime_new_link(struct device *dev);
6060 extern void pm_runtime_drop_link(struct device_link *link);
61+extern void pm_runtime_release_supplier(struct device_link *link, bool check_idle);
6162
6263 extern int devm_pm_runtime_enable(struct device *dev);
6364
@@ -283,6 +284,8 @@ static inline void pm_runtime_get_suppliers(struct device *dev) {} · 219 unmodified lines
283284 static inline void pm_runtime_put_suppliers(struct device *dev) {}
284285 static inline void pm_runtime_new_link(struct device *dev) {}
285286 static inline void pm_runtime_drop_link(struct device_link *link) {}
287+static inline void pm_runtime_release_supplier(struct device_link *link,
288+ bool check_idle) {}
286289
287290 #endif /* !CONFIG_PM */
288291
include/linux/sched/topology.h+3−3
@@ -266,10 +266,10 @@ unsigned long arch_scale_thermal_pressure(int cpu) · 265 unmodified lines
266266 }
267267 #endif
268268
269−#ifndef arch_set_thermal_pressure
269+#ifndef arch_update_thermal_pressure
270270 static __always_inline
271−void arch_set_thermal_pressure(const struct cpumask *cpus,
272− unsigned long th_pressure)
271+void arch_update_thermal_pressure(const struct cpumask *cpus,
272+ unsigned long capped_frequency)
273273 { }
274274 #endif
275275
include/linux/suspend.h+1−0
@@ -446,6 +446,7 @@ extern unsigned long get_safe_page(gfp_t gfp_mask); · 445 unmodified lines
446446 extern asmlinkage int swsusp_arch_suspend(void);
447447 extern asmlinkage int swsusp_arch_resume(void);
448448
449+extern u32 swsusp_hardware_signature;
449450 extern void hibernation_set_ops(const struct platform_hibernation_ops *ops);
450451 extern int hibernate(void);
451452 extern bool system_entering_hibernation(void);
init/Kconfig+1−1
@@ -550,7 +550,7 @@ config SCHED_THERMAL_PRESSURE · 549 unmodified lines
550550 i.e. put less load on throttled CPUs than on non/less throttled ones.
551551
552552 This requires the architecture to implement
553− arch_set_thermal_pressure() and arch_scale_thermal_pressure().
553+ arch_update_thermal_pressure() and arch_scale_thermal_pressure().
554554
555555 config BSD_PROCESS_ACCT
556556 bool "BSD Process Accounting"
kernel/power/power.h+1−0
@@ -170,6 +170,7 @@ extern int swsusp_swap_in_use(void); · 169 unmodified lines
170170 #define SF_PLATFORM_MODE 1
171171 #define SF_NOCOMPRESS_MODE 2
172172 #define SF_CRC32_MODE 4
173+#define SF_HW_SIG 8
173174
174175 /* kernel/power/hibernate.c */
175176 extern int swsusp_check(void);
kernel/power/swap.c+14−2
@@ -36,6 +36,8 @@ · 35 unmodified lines
3636
3737 #define HIBERNATE_SIG "S1SUSPEND"
3838
39+u32 swsusp_hardware_signature;
40+
3941 /*
4042 * When reading an {un,}compressed image, we may restore pages in place,
4143 * in which case some architectures need these pages cleaning before they
@@ -104,7 +106,8 @@ struct swap_map_handle { · 62 unmodified lines
104106
105107 struct swsusp_header {
106108 char reserved[PAGE_SIZE - 20 - sizeof(sector_t) - sizeof(int) -
107− sizeof(u32)];
109+ sizeof(u32) - sizeof(u32)];
110+ u32 hw_sig;
108111 u32 crc32;
109112 sector_t image;
110113 unsigned int flags; /* Flags to pass to the "boot" kernel */
@@ -312,7 +315,6 @@ static int hib_wait_io(struct hib_bio_batch *hb) · 201 unmodified lines
312315 /*
313316 * Saving part
314317 */
315−
316318 static int mark_swapfiles(struct swap_map_handle *handle, unsigned int flags)
317319 {
318320 int error;
@@ -324,6 +326,10 @@ static int mark_swapfiles(struct swap_map_handle *handle, unsigned int flags) · 5 unmodified lines
324326 memcpy(swsusp_header->orig_sig,swsusp_header->sig, 10);
325327 memcpy(swsusp_header->sig, HIBERNATE_SIG, 10);
326328 swsusp_header->image = handle->first_sector;
329+ if (swsusp_hardware_signature) {
330+ swsusp_header->hw_sig = swsusp_hardware_signature;
331+ flags |= SF_HW_SIG;
332+ }
327333 swsusp_header->flags = flags;
328334 if (flags & SF_CRC32_MODE)
329335 swsusp_header->crc32 = handle->crc32;
@@ -1537,6 +1543,12 @@ int swsusp_check(void) · 1207 unmodified lines
15371543 } else {
15381544 error = -EINVAL;
15391545 }
1546+ if (!error && swsusp_header->flags & SF_HW_SIG &&
1547+ swsusp_header->hw_sig != swsusp_hardware_signature) {
1548+ pr_info("Suspend image hardware signature mismatch (%08x now %08x); aborting resume.\n",
1549+ swsusp_header->hw_sig, swsusp_hardware_signature);
1550+ error = -EINVAL;
1551+ }
15401552
15411553 put:
15421554 if (error)

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t