Pull power management updates from Rafael Wysocki: "The most signigicant change here is the addition of a new cpufreq 'P-state' driver for AMD processors as a better replacement for the venerable acpi-cpufreq driver. There are also other cpufreq updates (in the core, intel_pstate, ARM drivers), PM core updates (mostly related to adding new macros for declaring PM operations which should make the lives of driver developers somewhat easier), and a bunch of assorted fixes and cleanups. Summary: - Add new P-state driver for AMD processors (Huang Rui). - Fix initialization of min and max frequency QoS requests in the cpufreq core (Rafael Wysocki). - Fix EPP handling on Alder Lake in intel_pstate (Srinivas Pandruvada). - Make intel_pstate update cpuinfo.max_freq when notified of HWP capabilities changes and drop a redundant function call from that driver (Rafael Wysocki). - Improve IRQ support in the Qcom cpufreq driver (Ard Biesheuvel, Stephen Boyd, Vladimir Zapolskiy). - Fix double devm_remap() in the Mediatek cpufreq driver (Hector Yuan). - Introduce thermal pressure helpers for cpufreq CPU cooling (Lukasz Luba). - Make cpufreq use default_groups in kobj_type (Greg Kroah-Hartman). - Make cpuidle use default_groups in kobj_type (Greg Kroah-Hartman). - Fix two comments in cpuidle code (Jason Wang, Yang Li). - Allow model-specific normal EPB value to be used in the intel_epb sysfs attribute handling code (Srinivas Pandruvada). - Simplify locking in pm_runtime_put_suppliers() (Rafael Wysocki). - Add safety net to supplier device release in the runtime PM core code (Rafael Wysocki). - Capture device status before disabling runtime PM for it (Rafael Wysocki). - Add new macros for declaring PM operations to allow drivers to avoid guarding them with CONFIG_PM #ifdefs or __maybe_unused and update some drivers to use these macros (Paul Cercueil). - Allow ACPI hardware signature to be honoured during restore from hibernation (David Woodhouse). - Update outdated operating performance points (OPP) documentation (Tang Yizhou). - Reduce log severity for informative message regarding frequency transition failures in devfreq (Tzung-Bi Shih). - Add DRAM frequency controller devfreq driver for Allwinner sunXi SoCs (Samuel Holland). - Add missing COMMON_CLK dependency to sun8i devfreq driver (Arnd Bergmann). - Add support for new layout of Psys PowerLimit Register on SPR to the Intel RAPL power capping driver (Zhang Rui). - Fix typo in a comment in idle_inject.c (Jason Wang). - Remove unused function definition from the DTPM (Dynamit Thermal Power Management) power capping framework (Daniel Lezcano). - Reduce DTPM trace verbosity (Daniel Lezcano)" * tag 'pm-5.17-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm: (53 commits) x86, sched: Fix undefined reference to init_freq_invariance_cppc() build error cpufreq: amd-pstate: Fix Kconfig dependencies for AMD P-State cpufreq: amd-pstate: Fix struct amd_cpudata kernel-doc comment cpuidle: use default_groups in kobj_type x86: intel_epb: Allow model specific normal EPB value MAINTAINERS: Add AMD P-State driver maintainer entry Documentation: amd-pstate: Add AMD P-State driver introduction cpufreq: amd-pstate: Add AMD P-State performance attributes cpufreq: amd-pstate: Add AMD P-State frequencies attributes cpufreq: amd-pstate: Add boost mode support for AMD P-State cpufreq: amd-pstate: Add trace for AMD P-State module cpufreq: amd-pstate: Introduce the support for the processors with shared memory solution cpufreq: amd-pstate: Add fast switch function for AMD P-State cpufreq: amd-pstate: Introduce a new AMD P-State driver to support future processors ACPI: CPPC: Add CPPC enable register function ACPI: CPPC: Check present CPUs for determining _CPC is valid ACPI: CPPC: Implement support for SystemIO registers x86/msr: Add AMD CPPC MSR definitions x86/cpufeatures: Add AMD Collaborative Processor Performance Control feature flag cpufreq: use default_groups in kobj_type ...
Unified · Split
| @@ -4,6 +4,8 @@ · 3 unmodified lines | ||
| 4 | 4 | Collaborative Processor Performance Control (CPPC) |
| 5 | 5 | ================================================== |
| 6 | 6 | |
| 7 | +.. _cppc_sysfs: | |
| 8 | + | |
| 7 | 9 | CPPC |
| 8 | 10 | ==== |
| 9 | 11 | |
| @@ -225,14 +225,23 @@ · 224 unmodified lines | ||
| 225 | 225 | For broken nForce2 BIOS resulting in XT-PIC timer. |
| 226 | 226 | |
| 227 | 227 | acpi_sleep= [HW,ACPI] Sleep options |
| 228 | − Format: { s3_bios, s3_mode, s3_beep, s4_nohwsig, | |
| 229 | − old_ordering, nonvs, sci_force_enable, nobl } | |
| 228 | + Format: { s3_bios, s3_mode, s3_beep, s4_hwsig, | |
| 229 | + s4_nohwsig, old_ordering, nonvs, | |
| 230 | + sci_force_enable, nobl } | |
| 230 | 231 | See Documentation/power/video.rst for information on |
| 231 | 232 | s3_bios and s3_mode. |
| 232 | 233 | s3_beep is for debugging; it makes the PC's speaker beep |
| 233 | 234 | as soon as the kernel's real-mode entry point is called. |
| 235 | + s4_hwsig causes the kernel to check the ACPI hardware | |
| 236 | + signature during resume from hibernation, and gracefully | |
| 237 | + refuse to resume if it has changed. This complies with | |
| 238 | + the ACPI specification but not with reality, since | |
| 239 | + Windows does not do this and many laptops do change it | |
| 240 | + on docking. So the default behaviour is to allow resume | |
| 241 | + and simply warn when the signature changes, unless the | |
| 242 | + s4_hwsig option is enabled. | |
| 234 | 243 | s4_nohwsig prevents ACPI hardware signature from being |
| 235 | − used during resume from hibernation. | |
| 244 | + used (or even warned about) during resume. | |
| 236 | 245 | old_ordering causes the ACPI 1.0 ordering of the _PTS |
| 237 | 246 | control method, with respect to putting devices into |
| 238 | 247 | low power states, to be enforced (the ACPI 2.0 ordering |
| @@ -0,0 +1,382 @@ | ||
| 1 | +.. SPDX-License-Identifier: GPL-2.0 | |
| 2 | +.. include:: <isonum.txt> | |
| 3 | + | |
| 4 | +=============================================== | |
| 5 | +``amd-pstate`` CPU Performance Scaling Driver | |
| 6 | +=============================================== | |
| 7 | + | |
| 8 | +:Copyright: |copy| 2021 Advanced Micro Devices, Inc. | |
| 9 | + | |
| 10 | +:Author: Huang Rui <ray.huang@amd.com> | |
| 11 | + | |
| 12 | + | |
| 13 | +Introduction | |
| 14 | +=================== | |
| 15 | + | |
| 16 | +``amd-pstate`` is the AMD CPU performance scaling driver that introduces a | |
| 17 | +new CPU frequency control mechanism on modern AMD APU and CPU series in | |
| 18 | +Linux kernel. The new mechanism is based on Collaborative Processor | |
| 19 | +Performance Control (CPPC) which provides finer grain frequency management | |
| 20 | +than legacy ACPI hardware P-States. Current AMD CPU/APU platforms are using | |
| 21 | +the ACPI P-states driver to manage CPU frequency and clocks with switching | |
| 22 | +only in 3 P-states. CPPC replaces the ACPI P-states controls, allows a | |
| 23 | +flexible, low-latency interface for the Linux kernel to directly | |
| 24 | +communicate the performance hints to hardware. | |
| 25 | + | |
| 26 | +``amd-pstate`` leverages the Linux kernel governors such as ``schedutil``, | |
| 27 | +``ondemand``, etc. to manage the performance hints which are provided by | |
| 28 | +CPPC hardware functionality that internally follows the hardware | |
| 29 | +specification (for details refer to AMD64 Architecture Programmer's Manual | |
| 30 | +Volume 2: System Programming [1]_). Currently ``amd-pstate`` supports basic | |
| 31 | +frequency control function according to kernel governors on some of the | |
| 32 | +Zen2 and Zen3 processors, and we will implement more AMD specific functions | |
| 33 | +in future after we verify them on the hardware and SBIOS. | |
| 34 | + | |
| 35 | + | |
| 36 | +AMD CPPC Overview | |
| 37 | +======================= | |
| 38 | + | |
| 39 | +Collaborative Processor Performance Control (CPPC) interface enumerates a | |
| 40 | +continuous, abstract, and unit-less performance value in a scale that is | |
| 41 | +not tied to a specific performance state / frequency. This is an ACPI | |
| 42 | +standard [2]_ which software can specify application performance goals and | |
| 43 | +hints as a relative target to the infrastructure limits. AMD processors | |
| 44 | +provides the low latency register model (MSR) instead of AML code | |
| 45 | +interpreter for performance adjustments. ``amd-pstate`` will initialize a | |
| 46 | +``struct cpufreq_driver`` instance ``amd_pstate_driver`` with the callbacks | |
| 47 | +to manage each performance update behavior. :: | |
| 48 | + | |
| 49 | + Highest Perf ------>+-----------------------+ +-----------------------+ | |
| 50 | + | | | | | |
| 51 | + | | | | | |
| 52 | + | | Max Perf ---->| | | |
| 53 | + | | | | | |
| 54 | + | | | | | |
| 55 | + Nominal Perf ------>+-----------------------+ +-----------------------+ | |
| 56 | + | | | | | |
| 57 | + | | | | | |
| 58 | + | | | | | |
| 59 | + | | | | | |
| 60 | + | | | | | |
| 61 | + | | | | | |
| 62 | + | | Desired Perf ---->| | | |
| 63 | + | | | | | |
| 64 | + | | | | | |
| 65 | + | | | | | |
| 66 | + | | | | | |
| 67 | + | | | | | |
| 68 | + | | | | | |
| 69 | + | | | | | |
| 70 | + | | | | | |
| 71 | + | | | | | |
| 72 | + Lowest non- | | | | | |
| 73 | + linear perf ------>+-----------------------+ +-----------------------+ | |
| 74 | + | | | | | |
| 75 | + | | Lowest perf ---->| | | |
| 76 | + | | | | | |
| 77 | + Lowest perf ------>+-----------------------+ +-----------------------+ | |
| 78 | + | | | | | |
| 79 | + | | | | | |
| 80 | + | | | | | |
| 81 | + 0 ------>+-----------------------+ +-----------------------+ | |
| 82 | + | |
| 83 | + AMD P-States Performance Scale | |
| 84 | + | |
| 85 | + | |
| 86 | +.. _perf_cap: | |
| 87 | + | |
| 88 | +AMD CPPC Performance Capability | |
| 89 | +-------------------------------- | |
| 90 | + | |
| 91 | +Highest Performance (RO) | |
| 92 | +......................... | |
| 93 | + | |
| 94 | +It is the absolute maximum performance an individual processor may reach, | |
| 95 | +assuming ideal conditions. This performance level may not be sustainable | |
| 96 | +for long durations and may only be achievable if other platform components | |
| 97 | +are in a specific state; for example, it may require other processors be in | |
| 98 | +an idle state. This would be equivalent to the highest frequencies | |
| 99 | +supported by the processor. | |
| 100 | + | |
| 101 | +Nominal (Guaranteed) Performance (RO) | |
| 102 | +...................................... | |
| 103 | + | |
| 104 | +It is the maximum sustained performance level of the processor, assuming | |
| 105 | +ideal operating conditions. In absence of an external constraint (power, | |
| 106 | +thermal, etc.) this is the performance level the processor is expected to | |
| 107 | +be able to maintain continuously. All cores/processors are expected to be | |
| 108 | +able to sustain their nominal performance state simultaneously. | |
| 109 | + | |
| 110 | +Lowest non-linear Performance (RO) | |
| 111 | +................................... | |
| 112 | + | |
| 113 | +It is the lowest performance level at which nonlinear power savings are | |
| 114 | +achieved, for example, due to the combined effects of voltage and frequency | |
| 115 | +scaling. Above this threshold, lower performance levels should be generally | |
| 116 | +more energy efficient than higher performance levels. This register | |
| 117 | +effectively conveys the most efficient performance level to ``amd-pstate``. | |
| 118 | + | |
| 119 | +Lowest Performance (RO) | |
| 120 | +........................ | |
| 121 | + | |
| 122 | +It is the absolute lowest performance level of the processor. Selecting a | |
| 123 | +performance level lower than the lowest nonlinear performance level may | |
| 124 | +cause an efficiency penalty but should reduce the instantaneous power | |
| 125 | +consumption of the processor. | |
| 126 | + | |
| 127 | +AMD CPPC Performance Control | |
| 128 | +------------------------------ | |
| 129 | + | |
| 130 | +``amd-pstate`` passes performance goals through these registers. The | |
| 131 | +register drives the behavior of the desired performance target. | |
| 132 | + | |
| 133 | +Minimum requested performance (RW) | |
| 134 | +................................... | |
| 135 | + | |
| 136 | +``amd-pstate`` specifies the minimum allowed performance level. | |
| 137 | + | |
| 138 | +Maximum requested performance (RW) | |
| 139 | +................................... | |
| 140 | + | |
| 141 | +``amd-pstate`` specifies a limit the maximum performance that is expected | |
| 142 | +to be supplied by the hardware. | |
| 143 | + | |
| 144 | +Desired performance target (RW) | |
| 145 | +................................... | |
| 146 | + | |
| 147 | +``amd-pstate`` specifies a desired target in the CPPC performance scale as | |
| 148 | +a relative number. This can be expressed as percentage of nominal | |
| 149 | +performance (infrastructure max). Below the nominal sustained performance | |
| 150 | +level, desired performance expresses the average performance level of the | |
| 151 | +processor subject to hardware. Above the nominal performance level, | |
| 152 | +processor must provide at least nominal performance requested and go higher | |
| 153 | +if current operating conditions allow. | |
| 154 | + | |
| 155 | +Energy Performance Preference (EPP) (RW) | |
| 156 | +......................................... | |
| 157 | + | |
| 158 | +Provides a hint to the hardware if software wants to bias toward performance | |
| 159 | +(0x0) or energy efficiency (0xff). | |
| 160 | + | |
| 161 | + | |
| 162 | +Key Governors Support | |
| 163 | +======================= | |
| 164 | + | |
| 165 | +``amd-pstate`` can be used with all the (generic) scaling governors listed | |
| 166 | +by the ``scaling_available_governors`` policy attribute in ``sysfs``. Then, | |
| 167 | +it is responsible for the configuration of policy objects corresponding to | |
| 168 | +CPUs and provides the ``CPUFreq`` core (and the scaling governors attached | |
| 169 | +to the policy objects) with accurate information on the maximum and minimum | |
| 170 | +operating frequencies supported by the hardware. Users can check the | |
| 171 | +``scaling_cur_freq`` information comes from the ``CPUFreq`` core. | |
| 172 | + | |
| 173 | +``amd-pstate`` mainly supports ``schedutil`` and ``ondemand`` for dynamic | |
| 174 | +frequency control. It is to fine tune the processor configuration on | |
| 175 | +``amd-pstate`` to the ``schedutil`` with CPU CFS scheduler. ``amd-pstate`` | |
| 176 | +registers adjust_perf callback to implement the CPPC similar performance | |
| 177 | +update behavior. It is initialized by ``sugov_start`` and then populate the | |
| 178 | +CPU's update_util_data pointer to assign ``sugov_update_single_perf`` as | |
| 179 | +the utilization update callback function in CPU scheduler. CPU scheduler | |
| 180 | +will call ``cpufreq_update_util`` and assign the target performance | |
| 181 | +according to the ``struct sugov_cpu`` that utilization update belongs to. | |
| 182 | +Then ``amd-pstate`` updates the desired performance according to the CPU | |
| 183 | +scheduler assigned. | |
| 184 | + | |
| 185 | + | |
| 186 | +Processor Support | |
| 187 | +======================= | |
| 188 | + | |
| 189 | +The ``amd-pstate`` initialization will fail if the _CPC in ACPI SBIOS is | |
| 190 | +not existed at the detected processor, and it uses ``acpi_cpc_valid`` to | |
| 191 | +check the _CPC existence. All Zen based processors support legacy ACPI | |
| 192 | +hardware P-States function, so while the ``amd-pstate`` fails to be | |
| 193 | +initialized, the kernel will fall back to initialize ``acpi-cpufreq`` | |
| 194 | +driver. | |
| 195 | + | |
| 196 | +There are two types of hardware implementations for ``amd-pstate``: one is | |
| 197 | +`Full MSR Support <perf_cap_>`_ and another is `Shared Memory Support | |
| 198 | +<perf_cap_>`_. It can use :c:macro:`X86_FEATURE_CPPC` feature flag (for | |
| 199 | +details refer to Processor Programming Reference (PPR) for AMD Family | |
| 200 | +19h Model 51h, Revision A1 Processors [3]_) to indicate the different | |
| 201 | +types. ``amd-pstate`` is to register different ``static_call`` instances | |
| 202 | +for different hardware implementations. | |
| 203 | + | |
| 204 | +Currently, some of Zen2 and Zen3 processors support ``amd-pstate``. In the | |
| 205 | +future, it will be supported on more and more AMD processors. | |
| 206 | + | |
| 207 | +Full MSR Support | |
| 208 | +----------------- | |
| 209 | + | |
| 210 | +Some new Zen3 processors such as Cezanne provide the MSR registers directly | |
| 211 | +while the :c:macro:`X86_FEATURE_CPPC` CPU feature flag is set. | |
| 212 | +``amd-pstate`` can handle the MSR register to implement the fast switch | |
| 213 | +function in ``CPUFreq`` that can shrink latency of frequency control on the | |
| 214 | +interrupt context. The functions with ``pstate_xxx`` prefix represent the | |
| 215 | +operations of MSR registers. | |
| 216 | + | |
| 217 | +Shared Memory Support | |
| 218 | +---------------------- | |
| 219 | + | |
| 220 | +If :c:macro:`X86_FEATURE_CPPC` CPU feature flag is not set, that means the | |
| 221 | +processor supports shared memory solution. In this case, ``amd-pstate`` | |
| 222 | +uses the ``cppc_acpi`` helper methods to implement the callback functions | |
| 223 | +that defined on ``static_call``. The functions with ``cppc_xxx`` prefix | |
| 224 | +represent the operations of acpi cppc helpers for shared memory solution. | |
| 225 | + | |
| 226 | + | |
| 227 | +AMD P-States and ACPI hardware P-States always can be supported in one | |
| 228 | +processor. But AMD P-States has the higher priority and if it is enabled | |
| 229 | +with :c:macro:`MSR_AMD_CPPC_ENABLE` or ``cppc_set_enable``, it will respond | |
| 230 | +to the request from AMD P-States. | |
| 231 | + | |
| 232 | + | |
| 233 | +User Space Interface in ``sysfs`` | |
| 234 | +================================== | |
| 235 | + | |
| 236 | +``amd-pstate`` exposes several global attributes (files) in ``sysfs`` to | |
| 237 | +control its functionality at the system level. They located in the | |
| 238 | +``/sys/devices/system/cpu/cpufreq/policyX/`` directory and affect all CPUs. :: | |
| 239 | + | |
| 240 | + root@hr-test1:/home/ray# ls /sys/devices/system/cpu/cpufreq/policy0/*amd* | |
| 241 | + /sys/devices/system/cpu/cpufreq/policy0/amd_pstate_highest_perf | |
| 242 | + /sys/devices/system/cpu/cpufreq/policy0/amd_pstate_lowest_nonlinear_freq | |
| 243 | + /sys/devices/system/cpu/cpufreq/policy0/amd_pstate_max_freq | |
| 244 | + | |
| 245 | + | |
| 246 | +``amd_pstate_highest_perf / amd_pstate_max_freq`` | |
| 247 | + | |
| 248 | +Maximum CPPC performance and CPU frequency that the driver is allowed to | |
| 249 | +set in percent of the maximum supported CPPC performance level (the highest | |
| 250 | +performance supported in `AMD CPPC Performance Capability <perf_cap_>`_). | |
| 251 | +In some of ASICs, the highest CPPC performance is not the one in the _CPC | |
| 252 | +table, so we need to expose it to sysfs. If boost is not active but | |
| 253 | +supported, this maximum frequency will be larger than the one in | |
| 254 | +``cpuinfo``. | |
| 255 | +This attribute is read-only. | |
| 256 | + | |
| 257 | +``amd_pstate_lowest_nonlinear_freq`` | |
| 258 | + | |
| 259 | +The lowest non-linear CPPC CPU frequency that the driver is allowed to set | |
| 260 | +in percent of the maximum supported CPPC performance level (Please see the | |
| 261 | +lowest non-linear performance in `AMD CPPC Performance Capability | |
| 262 | +<perf_cap_>`_). | |
| 263 | +This attribute is read-only. | |
| 264 | + | |
| 265 | +For other performance and frequency values, we can read them back from | |
| 266 | +``/sys/devices/system/cpu/cpuX/acpi_cppc/``, see :ref:`cppc_sysfs`. | |
| 267 | + | |
| 268 | + | |
| 269 | +``amd-pstate`` vs ``acpi-cpufreq`` | |
| 270 | +====================================== | |
| 271 | + | |
| 272 | +On majority of AMD platforms supported by ``acpi-cpufreq``, the ACPI tables | |
| 273 | +provided by the platform firmware used for CPU performance scaling, but | |
| 274 | +only provides 3 P-states on AMD processors. | |
| 275 | +However, on modern AMD APU and CPU series, it provides the collaborative | |
| 276 | +processor performance control according to ACPI protocol and customize this | |
| 277 | +for AMD platforms. That is fine-grain and continuous frequency range | |
| 278 | +instead of the legacy hardware P-states. ``amd-pstate`` is the kernel | |
| 279 | +module which supports the new AMD P-States mechanism on most of future AMD | |
| 280 | +platforms. The AMD P-States mechanism will be the more performance and energy | |
| 281 | +efficiency frequency management method on AMD processors. | |
| 282 | + | |
| 283 | +Kernel Module Options for ``amd-pstate`` | |
| 284 | +========================================= | |
| 285 | + | |
| 286 | +``shared_mem`` | |
| 287 | +Use a module param (shared_mem) to enable related processors manually with | |
| 288 | +**amd_pstate.shared_mem=1**. | |
| 289 | +Due to the performance issue on the processors with `Shared Memory Support | |
| 290 | +<perf_cap_>`_, so we disable it for the moment and will enable this by default | |
| 291 | +once we address performance issue on this solution. | |
| 292 | + | |
| 293 | +The way to check whether current processor is `Full MSR Support <perf_cap_>`_ | |
| 294 | +or `Shared Memory Support <perf_cap_>`_ : :: | |
| 295 | + | |
| 296 | + ray@hr-test1:~$ lscpu | grep cppc | |
| 297 | + Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate ssbd mba ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr rdpru wbnoinvd cppc arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif v_spec_ctrl umip pku ospke vaes vpclmulqdq rdpid overflow_recov succor smca fsrm | |
| 298 | + | |
| 299 | +If CPU Flags have cppc, then this processor supports `Full MSR Support | |
| 300 | +<perf_cap_>`_. Otherwise it supports `Shared Memory Support <perf_cap_>`_. | |
| 301 | + | |
| 302 | + | |
| 303 | +``cpupower`` tool support for ``amd-pstate`` | |
| 304 | +=============================================== | |
| 305 | + | |
| 306 | +``amd-pstate`` is supported on ``cpupower`` tool that can be used to dump the frequency | |
| 307 | +information. And it is in progress to support more and more operations for new | |
| 308 | +``amd-pstate`` module with this tool. :: | |
| 309 | + | |
| 310 | + root@hr-test1:/home/ray# cpupower frequency-info | |
| 311 | + analyzing CPU 0: | |
| 312 | + driver: amd-pstate | |
| 313 | + CPUs which run at the same hardware frequency: 0 | |
| 314 | + CPUs which need to have their frequency coordinated by software: 0 | |
| 315 | + maximum transition latency: 131 us | |
| 316 | + hardware limits: 400 MHz - 4.68 GHz | |
| 317 | + available cpufreq governors: ondemand conservative powersave userspace performance schedutil | |
| 318 | + current policy: frequency should be within 400 MHz and 4.68 GHz. | |
| 319 | + The governor "schedutil" may decide which speed to use | |
| 320 | + within this range. | |
| 321 | + current CPU frequency: Unable to call hardware | |
| 322 | + current CPU frequency: 4.02 GHz (asserted by call to kernel) | |
| 323 | + boost state support: | |
| 324 | + Supported: yes | |
| 325 | + Active: yes | |
| 326 | + AMD PSTATE Highest Performance: 166. Maximum Frequency: 4.68 GHz. | |
| 327 | + AMD PSTATE Nominal Performance: 117. Nominal Frequency: 3.30 GHz. | |
| 328 | + AMD PSTATE Lowest Non-linear Performance: 39. Lowest Non-linear Frequency: 1.10 GHz. | |
| 329 | + AMD PSTATE Lowest Performance: 15. Lowest Frequency: 400 MHz. | |
| 330 | + | |
| 331 | + | |
| 332 | +Diagnostics and Tuning | |
| 333 | +======================= | |
| 334 | + | |
| 335 | +Trace Events | |
| 336 | +-------------- | |
| 337 | + | |
| 338 | +There are two static trace events that can be used for ``amd-pstate`` | |
| 339 | +diagnostics. One of them is the cpu_frequency trace event generally used | |
| 340 | +by ``CPUFreq``, and the other one is the ``amd_pstate_perf`` trace event | |
| 341 | +specific to ``amd-pstate``. The following sequence of shell commands can | |
| 342 | +be used to enable them and see their output (if the kernel is generally | |
| 343 | +configured to support event tracing). :: | |
| 344 | + | |
| 345 | + root@hr-test1:/home/ray# cd /sys/kernel/tracing/ | |
| 346 | + root@hr-test1:/sys/kernel/tracing# echo 1 > events/amd_cpu/enable | |
| 347 | + root@hr-test1:/sys/kernel/tracing# cat trace | |
| 348 | + # tracer: nop | |
| 349 | + # | |
| 350 | + # entries-in-buffer/entries-written: 47827/42233061 #P:2 | |
| 351 | + # | |
| 352 | + # _-----=> irqs-off | |
| 353 | + # / _----=> need-resched | |
| 354 | + # | / _---=> hardirq/softirq | |
| 355 | + # || / _--=> preempt-depth | |
| 356 | + # ||| / delay | |
| 357 | + # TASK-PID CPU# |||| TIMESTAMP FUNCTION | |
| 358 | + # | | | |||| | | | |
| 359 | + <idle>-0 [015] dN... 4995.979886: amd_pstate_perf: amd_min_perf=85 amd_des_perf=85 amd_max_perf=166 cpu_id=15 changed=false fast_switch=true | |
| 360 | + <idle>-0 [007] d.h.. 4995.979893: amd_pstate_perf: amd_min_perf=85 amd_des_perf=85 amd_max_perf=166 cpu_id=7 changed=false fast_switch=true | |
| 361 | + cat-2161 [000] d.... 4995.980841: amd_pstate_perf: amd_min_perf=85 amd_des_perf=85 amd_max_perf=166 cpu_id=0 changed=false fast_switch=true | |
| 362 | + sshd-2125 [004] d.s.. 4995.980968: amd_pstate_perf: amd_min_perf=85 amd_des_perf=85 amd_max_perf=166 cpu_id=4 changed=false fast_switch=true | |
| 363 | + <idle>-0 [007] d.s.. 4995.980968: amd_pstate_perf: amd_min_perf=85 amd_des_perf=85 amd_max_perf=166 cpu_id=7 changed=false fast_switch=true | |
| 364 | + <idle>-0 [003] d.s.. 4995.980971: amd_pstate_perf: amd_min_perf=85 amd_des_perf=85 amd_max_perf=166 cpu_id=3 changed=false fast_switch=true | |
| 365 | + <idle>-0 [011] d.s.. 4995.980996: amd_pstate_perf: amd_min_perf=85 amd_des_perf=85 amd_max_perf=166 cpu_id=11 changed=false fast_switch=true | |
| 366 | + | |
| 367 | +The cpu_frequency trace event will be triggered either by the ``schedutil`` scaling | |
| 368 | +governor (for the policies it is attached to), or by the ``CPUFreq`` core (for the | |
| 369 | +policies with other scaling governors). | |
| 370 | + | |
| 371 | + | |
| 372 | +Reference | |
| 373 | +=========== | |
| 374 | + | |
| 375 | +.. [1] AMD64 Architecture Programmer's Manual Volume 2: System Programming, | |
| 376 | + https://www.amd.com/system/files/TechDocs/24593.pdf | |
| 377 | + | |
| 378 | +.. [2] Advanced Configuration and Power Interface Specification, | |
| 379 | + https://uefi.org/sites/default/files/resources/ACPI_Spec_6_4_Jan22.pdf | |
| 380 | + | |
| 381 | +.. [3] Processor Programming Reference (PPR) for AMD Family 19h Model 51h, Revision A1 Processors | |
| 382 | + https://www.amd.com/system/files/TechDocs/56569-A1-PUB.zip | |
| @@ -11,6 +11,7 @@ Working-State Power Management · 10 unmodified lines | ||
| 11 | 11 | intel_idle |
| 12 | 12 | cpufreq |
| 13 | 13 | intel_pstate |
| 14 | + amd-pstate | |
| 14 | 15 | cpufreq_drivers |
| 15 | 16 | intel_epb |
| 16 | 17 | intel-speed-select |
| @@ -48,9 +48,9 @@ We can represent these as three OPPs as the following {Hz, uV} tuples: · 47 unmodified lines | ||
| 48 | 48 | OPP library provides a set of helper functions to organize and query the OPP |
| 49 | 49 | information. The library is located in drivers/opp/ directory and the header |
| 50 | 50 | is located in include/linux/pm_opp.h. OPP library can be enabled by enabling |
| 51 | −CONFIG_PM_OPP from power management menuconfig menu. OPP library depends on | |
| 52 | −CONFIG_PM as certain SoCs such as Texas Instrument's OMAP framework allows to | |
| 53 | −optionally boot at a certain OPP without needing cpufreq. | |
| 51 | +CONFIG_PM_OPP from power management menuconfig menu. Certain SoCs such as Texas | |
| 52 | +Instrument's OMAP framework allows to optionally boot at a certain OPP without | |
| 53 | +needing cpufreq. | |
| 54 | 54 | |
| 55 | 55 | Typical usage of the OPP library is as follows:: |
| 56 | 56 | |
| @@ -75,8 +75,8 @@ operations until that OPP could be re-enabled if possible. · 18 unmodified lines | ||
| 75 | 75 | |
| 76 | 76 | OPP library facilitates this concept in its implementation. The following |
| 77 | 77 | operational functions operate only on available opps: |
| 78 | −opp_find_freq_{ceil, floor}, dev_pm_opp_get_voltage, dev_pm_opp_get_freq, | |
| 79 | −dev_pm_opp_get_opp_count | |
| 78 | +dev_pm_opp_find_freq_{ceil, floor}, dev_pm_opp_get_voltage, dev_pm_opp_get_freq, | |
| 79 | +dev_pm_opp_get_opp_count. | |
| 80 | 80 | |
| 81 | 81 | dev_pm_opp_find_freq_exact is meant to be used to find the opp pointer |
| 82 | 82 | which can then be used for dev_pm_opp_enable/disable functions to make an |
| @@ -103,7 +103,7 @@ dev_pm_opp_add · 20 unmodified lines | ||
| 103 | 103 | The OPP is defined using the frequency and voltage. Once added, the OPP |
| 104 | 104 | is assumed to be available and control of its availability can be done |
| 105 | 105 | with the dev_pm_opp_enable/disable functions. OPP library |
| 106 | − internally stores and manages this information in the opp struct. | |
| 106 | + internally stores and manages this information in the dev_pm_opp struct. | |
| 107 | 107 | This function may be used by SoC framework to define a optimal list |
| 108 | 108 | as per the demands of SoC usage environment. |
| 109 | 109 | |
| @@ -247,7 +247,7 @@ dev_pm_opp_disable · 137 unmodified lines | ||
| 247 | 247 | 5. OPP Data Retrieval Functions |
| 248 | 248 | =============================== |
| 249 | 249 | Since OPP library abstracts away the OPP information, a set of functions to pull |
| 250 | −information from the OPP structure is necessary. Once an OPP pointer is | |
| 250 | +information from the dev_pm_opp structure is necessary. Once an OPP pointer is | |
| 251 | 251 | retrieved using the search functions, the following functions can be used by SoC |
| 252 | 252 | framework to retrieve the information represented inside the OPP layer. |
| 253 | 253 | |
| @@ -265,6 +265,10 @@ defined in include/linux/pm.h: · 264 unmodified lines | ||
| 265 | 265 | RPM_SUSPENDED, which means that each device is initially regarded by the |
| 266 | 266 | PM core as 'suspended', regardless of its real hardware status |
| 267 | 267 | |
| 268 | + `enum rpm_status last_status;` | |
| 269 | + - the last runtime PM status of the device captured before disabling runtime | |
| 270 | + PM for it (invalid initially and when disable_depth is 0) | |
| 271 | + | |
| 268 | 272 | `unsigned int runtime_auto;` |
| 269 | 273 | - if set, indicates that the user space has allowed the device driver to |
| 270 | 274 | power manage the device at run time via the /sys/devices/.../power/control |
| @@ -333,10 +337,12 @@ drivers/base/power/runtime.c and include/linux/pm_runtime.h: · 62 unmodified lines | ||
| 333 | 337 | |
| 334 | 338 | `int pm_runtime_resume(struct device *dev);` |
| 335 | 339 | - execute the subsystem-level resume callback for the device; returns 0 on |
| 336 | − success, 1 if the device's runtime PM status was already 'active' or | |
| 337 | − error code on failure, where -EAGAIN means it may be safe to attempt to | |
| 338 | − resume the device again in future, but 'power.runtime_error' should be | |
| 339 | − checked additionally, and -EACCES means that 'power.disable_depth' is | |
| 340 | + success, 1 if the device's runtime PM status is already 'active' (also if | |
| 341 | + 'power.disable_depth' is nonzero, but the status was 'active' when it was | |
| 342 | + changing from 0 to 1) or error code on failure, where -EAGAIN means it may | |
| 343 | + be safe to attempt to resume the device again in future, but | |
| 344 | + 'power.runtime_error' should be checked additionally, and -EACCES means | |
| 345 | + that the callback could not be run, because 'power.disable_depth' was | |
| 340 | 346 | different from 0 |
| 341 | 347 | |
| 342 | 348 | `int pm_runtime_resume_and_get(struct device *dev);` |
| @@ -994,6 +994,13 @@ S: Supported · 993 unmodified lines | ||
| 994 | 994 | T: git https://gitlab.freedesktop.org/agd5f/linux.git |
| 995 | 995 | F: drivers/gpu/drm/amd/pm/ |
| 996 | 996 | |
| 997 | +AMD PSTATE DRIVER | |
| 998 | +M: Huang Rui <ray.huang@amd.com> | |
| 999 | +L: linux-pm@vger.kernel.org | |
| 1000 | +S: Supported | |
| 1001 | +F: Documentation/admin-guide/pm/amd-pstate.rst | |
| 1002 | +F: drivers/cpufreq/amd-pstate* | |
| 1003 | + | |
| 997 | 1004 | AMD PTDMA DRIVER |
| 998 | 1005 | M: Sanjay R Mehta <sanju.mehta@amd.com> |
| 999 | 1006 | L: dmaengine@vger.kernel.org |
| @@ -23,7 +23,7 @@ · 22 unmodified lines | ||
| 23 | 23 | |
| 24 | 24 | /* Replace task scheduler's default thermal pressure API */ |
| 25 | 25 | #define arch_scale_thermal_pressure topology_get_thermal_pressure |
| 26 | −#define arch_set_thermal_pressure topology_set_thermal_pressure | |
| 26 | +#define arch_update_thermal_pressure topology_update_thermal_pressure | |
| 27 | 27 | |
| 28 | 28 | #else |
| 29 | 29 | |
| @@ -32,7 +32,7 @@ void update_freq_counters_refs(void); · 31 unmodified lines | ||
| 32 | 32 | |
| 33 | 33 | /* Replace task scheduler's default thermal pressure API */ |
| 34 | 34 | #define arch_scale_thermal_pressure topology_get_thermal_pressure |
| 35 | −#define arch_set_thermal_pressure topology_set_thermal_pressure | |
| 35 | +#define arch_update_thermal_pressure topology_update_thermal_pressure | |
| 36 | 36 | |
| 37 | 37 | #include <asm-generic/topology.h> |
| 38 | 38 | |
| @@ -315,6 +315,7 @@ · 314 unmodified lines | ||
| 315 | 315 | #define X86_FEATURE_AMD_SSBD (13*32+24) /* "" Speculative Store Bypass Disable */ |
| 316 | 316 | #define X86_FEATURE_VIRT_SSBD (13*32+25) /* Virtualized Speculative Store Bypass Disable */ |
| 317 | 317 | #define X86_FEATURE_AMD_SSB_NO (13*32+26) /* "" Speculative Store Bypass is fixed in hardware. */ |
| 318 | +#define X86_FEATURE_CPPC (13*32+27) /* Collaborative Processor Performance Control */ | |
| 318 | 319 | |
| 319 | 320 | /* Thermal and Power Management Leaf, CPUID level 0x00000006 (EAX), word 14 */ |
| 320 | 321 | #define X86_FEATURE_DTHERM (14*32+ 0) /* Digital Thermal Sensor */ |
| @@ -486,6 +486,23 @@ · 485 unmodified lines | ||
| 486 | 486 | |
| 487 | 487 | #define MSR_AMD64_VIRT_SPEC_CTRL 0xc001011f |
| 488 | 488 | |
| 489 | +/* AMD Collaborative Processor Performance Control MSRs */ | |
| 490 | +#define MSR_AMD_CPPC_CAP1 0xc00102b0 | |
| 491 | +#define MSR_AMD_CPPC_ENABLE 0xc00102b1 | |
| 492 | +#define MSR_AMD_CPPC_CAP2 0xc00102b2 | |
| 493 | +#define MSR_AMD_CPPC_REQ 0xc00102b3 | |
| 494 | +#define MSR_AMD_CPPC_STATUS 0xc00102b4 | |
| 495 | + | |
| 496 | +#define AMD_CPPC_LOWEST_PERF(x) (((x) >> 0) & 0xff) | |
| 497 | +#define AMD_CPPC_LOWNONLIN_PERF(x) (((x) >> 8) & 0xff) | |
| 498 | +#define AMD_CPPC_NOMINAL_PERF(x) (((x) >> 16) & 0xff) | |
| 499 | +#define AMD_CPPC_HIGHEST_PERF(x) (((x) >> 24) & 0xff) | |
| 500 | + | |
| 501 | +#define AMD_CPPC_MAX_PERF(x) (((x) & 0xff) << 0) | |
| 502 | +#define AMD_CPPC_MIN_PERF(x) (((x) & 0xff) << 8) | |
| 503 | +#define AMD_CPPC_DES_PERF(x) (((x) & 0xff) << 16) | |
| 504 | +#define AMD_CPPC_ENERGY_PERF_PREF(x) (((x) & 0xff) << 24) | |
| 505 | + | |
| 489 | 506 | /* Fam 17h MSRs */ |
| 490 | 507 | #define MSR_F17H_IRPERF 0xc00000e9 |
| 491 | 508 | |
| @@ -221,7 +221,7 @@ static inline void arch_set_max_freq_ratio(bool turbo_disabled) · 220 unmodified lines | ||
| 221 | 221 | } |
| 222 | 222 | #endif |
| 223 | 223 | |
| 224 | −#ifdef CONFIG_ACPI_CPPC_LIB | |
| 224 | +#if defined(CONFIG_ACPI_CPPC_LIB) && defined(CONFIG_SMP) | |
| 225 | 225 | void init_freq_invariance_cppc(void); |
| 226 | 226 | #define init_freq_invariance_cppc init_freq_invariance_cppc |
| 227 | 227 | #endif |
| @@ -139,8 +139,10 @@ static int __init acpi_sleep_setup(char *str) · 138 unmodified lines | ||
| 139 | 139 | if (strncmp(str, "s3_beep", 7) == 0) |
| 140 | 140 | acpi_realmode_flags |= 4; |
| 141 | 141 | #ifdef CONFIG_HIBERNATION |
| 142 | + if (strncmp(str, "s4_hwsig", 8) == 0) | |
| 143 | + acpi_check_s4_hw_signature(1); | |
| 142 | 144 | if (strncmp(str, "s4_nohwsig", 10) == 0) |
| 143 | − acpi_no_s4_hw_signature(); | |
| 145 | + acpi_check_s4_hw_signature(0); | |
| 144 | 146 | #endif |
| 145 | 147 | if (strncmp(str, "nonvs", 5) == 0) |
| 146 | 148 | acpi_nvs_nosave(); |
| @@ -16,6 +16,7 @@ · 15 unmodified lines | ||
| 16 | 16 | #include <linux/syscore_ops.h> |
| 17 | 17 | #include <linux/pm.h> |
| 18 | 18 | |
| 19 | +#include <asm/cpu_device_id.h> | |
| 19 | 20 | #include <asm/cpufeature.h> |
| 20 | 21 | #include <asm/msr.h> |
| 21 | 22 | |
| @@ -58,6 +59,22 @@ static DEFINE_PER_CPU(u8, saved_epb); · 36 unmodified lines | ||
| 58 | 59 | #define EPB_SAVED 0x10ULL |
| 59 | 60 | #define MAX_EPB EPB_MASK |
| 60 | 61 | |
| 62 | +enum energy_perf_value_index { | |
| 63 | + EPB_INDEX_PERFORMANCE, | |
| 64 | + EPB_INDEX_BALANCE_PERFORMANCE, | |
| 65 | + EPB_INDEX_NORMAL, | |
| 66 | + EPB_INDEX_BALANCE_POWERSAVE, | |
| 67 | + EPB_INDEX_POWERSAVE, | |
| 68 | +}; | |
| 69 | + | |
| 70 | +static u8 energ_perf_values[] = { | |
| 71 | + [EPB_INDEX_PERFORMANCE] = ENERGY_PERF_BIAS_PERFORMANCE, | |
| 72 | + [EPB_INDEX_BALANCE_PERFORMANCE] = ENERGY_PERF_BIAS_BALANCE_PERFORMANCE, | |
| 73 | + [EPB_INDEX_NORMAL] = ENERGY_PERF_BIAS_NORMAL, | |
| 74 | + [EPB_INDEX_BALANCE_POWERSAVE] = ENERGY_PERF_BIAS_BALANCE_POWERSAVE, | |
| 75 | + [EPB_INDEX_POWERSAVE] = ENERGY_PERF_BIAS_POWERSAVE, | |
| 76 | +}; | |
| 77 | + | |
| 61 | 78 | static int intel_epb_save(void) |
| 62 | 79 | { |
| 63 | 80 | u64 epb; |
| @@ -90,7 +107,7 @@ static void intel_epb_restore(void) · 26 unmodified lines | ||
| 90 | 107 | */ |
| 91 | 108 | val = epb & EPB_MASK; |
| 92 | 109 | if (val == ENERGY_PERF_BIAS_PERFORMANCE) { |
| 93 | − val = ENERGY_PERF_BIAS_NORMAL; | |
| 110 | + val = energ_perf_values[EPB_INDEX_NORMAL]; | |
| 94 | 111 | pr_warn_once("ENERGY_PERF_BIAS: Set to 'normal', was 'performance'\n"); |
| 95 | 112 | } |
| 96 | 113 | } |
| @@ -103,18 +120,11 @@ static struct syscore_ops intel_epb_syscore_ops = { · 6 unmodified lines | ||
| 103 | 120 | }; |
| 104 | 121 | |
| 105 | 122 | static const char * const energy_perf_strings[] = { |
| 106 | − "performance", | |
| 107 | − "balance-performance", | |
| 108 | − "normal", | |
| 109 | − "balance-power", | |
| 110 | − "power" | |
| 111 | −}; | |
| 112 | −static const u8 energ_perf_values[] = { | |
| 113 | − ENERGY_PERF_BIAS_PERFORMANCE, | |
| 114 | − ENERGY_PERF_BIAS_BALANCE_PERFORMANCE, | |
| 115 | − ENERGY_PERF_BIAS_NORMAL, | |
| 116 | − ENERGY_PERF_BIAS_BALANCE_POWERSAVE, | |
| 117 | − ENERGY_PERF_BIAS_POWERSAVE | |
| 123 | + [EPB_INDEX_PERFORMANCE] = "performance", | |
| 124 | + [EPB_INDEX_BALANCE_PERFORMANCE] = "balance-performance", | |
| 125 | + [EPB_INDEX_NORMAL] = "normal", | |
| 126 | + [EPB_INDEX_BALANCE_POWERSAVE] = "balance-power", | |
| 127 | + [EPB_INDEX_POWERSAVE] = "power", | |
| 118 | 128 | }; |
| 119 | 129 | |
| 120 | 130 | static ssize_t energy_perf_bias_show(struct device *dev, |
| @@ -193,13 +203,22 @@ static int intel_epb_offline(unsigned int cpu) · 72 unmodified lines | ||
| 193 | 203 | return 0; |
| 194 | 204 | } |
| 195 | 205 | |
| 206 | +static const struct x86_cpu_id intel_epb_normal[] = { | |
| 207 | + X86_MATCH_INTEL_FAM6_MODEL(ALDERLAKE_L, 7), | |
| 208 | + {} | |
| 209 | +}; | |
| 210 | + | |
| 196 | 211 | static __init int intel_epb_init(void) |
| 197 | 212 | { |
| 213 | + const struct x86_cpu_id *id = x86_match_cpu(intel_epb_normal); | |
| 198 | 214 | int ret; |
| 199 | 215 | |
| 200 | 216 | if (!boot_cpu_has(X86_FEATURE_EPB)) |
| 201 | 217 | return -ENODEV; |
| 202 | 218 | |
| 219 | + if (id) | |
| 220 | + energ_perf_values[EPB_INDEX_NORMAL] = id->driver_data; | |
| 221 | + | |
| 203 | 222 | ret = cpuhp_setup_state(CPUHP_AP_X86_INTEL_EPB_ONLINE, |
| 204 | 223 | "x86/intel/epb:online", intel_epb_online, |
| 205 | 224 | intel_epb_offline); |
| @@ -118,6 +118,8 @@ static DEFINE_PER_CPU(struct cpc_desc *, cpc_desc_ptr); · 117 unmodified lines | ||
| 118 | 118 | */ |
| 119 | 119 | #define NUM_RETRIES 500ULL |
| 120 | 120 | |
| 121 | +#define OVER_16BTS_MASK ~0xFFFFULL | |
| 122 | + | |
| 121 | 123 | #define define_one_cppc_ro(_name) \ |
| 122 | 124 | static struct kobj_attribute _name = \ |
| 123 | 125 | __ATTR(_name, 0444, show_##_name, NULL) |
| @@ -412,7 +414,7 @@ bool acpi_cpc_valid(void) · 288 unmodified lines | ||
| 412 | 414 | struct cpc_desc *cpc_ptr; |
| 413 | 415 | int cpu; |
| 414 | 416 | |
| 415 | − for_each_possible_cpu(cpu) { | |
| 417 | + for_each_present_cpu(cpu) { | |
| 416 | 418 | cpc_ptr = per_cpu(cpc_desc_ptr, cpu); |
| 417 | 419 | if (!cpc_ptr) |
| 418 | 420 | return false; |
| @@ -730,9 +732,26 @@ int acpi_cppc_processor_probe(struct acpi_processor *pr) · 311 unmodified lines | ||
| 730 | 732 | goto out_free; |
| 731 | 733 | cpc_ptr->cpc_regs[i-2].sys_mem_vaddr = addr; |
| 732 | 734 | } |
| 735 | + } else if (gas_t->space_id == ACPI_ADR_SPACE_SYSTEM_IO) { | |
| 736 | + if (gas_t->access_width < 1 || gas_t->access_width > 3) { | |
| 737 | + /* | |
| 738 | + * 1 = 8-bit, 2 = 16-bit, and 3 = 32-bit. | |
| 739 | + * SystemIO doesn't implement 64-bit | |
| 740 | + * registers. | |
| 741 | + */ | |
| 742 | + pr_debug("Invalid access width %d for SystemIO register\n", | |
| 743 | + gas_t->access_width); | |
| 744 | + goto out_free; | |
| 745 | + } | |
| 746 | + if (gas_t->address & OVER_16BTS_MASK) { | |
| 747 | + /* SystemIO registers use 16-bit integer addresses */ | |
| 748 | + pr_debug("Invalid IO port %llu for SystemIO register\n", | |
| 749 | + gas_t->address); | |
| 750 | + goto out_free; | |
| 751 | + } | |
| 733 | 752 | } else { |
| 734 | 753 | if (gas_t->space_id != ACPI_ADR_SPACE_FIXED_HARDWARE || !cpc_ffh_supported()) { |
| 735 | − /* Support only PCC ,SYS MEM and FFH type regs */ | |
| 754 | + /* Support only PCC, SystemMemory, SystemIO, and FFH type regs. */ | |
| 736 | 755 | pr_debug("Unsupported register type: %d\n", gas_t->space_id); |
| 737 | 756 | goto out_free; |
| 738 | 757 | } |
| @@ -907,7 +926,21 @@ static int cpc_read(int cpu, struct cpc_register_resource *reg_res, u64 *val) · 168 unmodified lines | ||
| 907 | 926 | } |
| 908 | 927 | |
| 909 | 928 | *val = 0; |
| 910 | − if (reg->space_id == ACPI_ADR_SPACE_PLATFORM_COMM && pcc_ss_id >= 0) | |
| 929 | + | |
| 930 | + if (reg->space_id == ACPI_ADR_SPACE_SYSTEM_IO) { | |
| 931 | + u32 width = 8 << (reg->access_width - 1); | |
| 932 | + acpi_status status; | |
| 933 | + | |
| 934 | + status = acpi_os_read_port((acpi_io_address)reg->address, | |
| 935 | + (u32 *)val, width); | |
| 936 | + if (ACPI_FAILURE(status)) { | |
| 937 | + pr_debug("Error: Failed to read SystemIO port %llx\n", | |
| 938 | + reg->address); | |
| 939 | + return -EFAULT; | |
| 940 | + } | |
| 941 | + | |
| 942 | + return 0; | |
| 943 | + } else if (reg->space_id == ACPI_ADR_SPACE_PLATFORM_COMM && pcc_ss_id >= 0) | |
| 911 | 944 | vaddr = GET_PCC_VADDR(reg->address, pcc_ss_id); |
| 912 | 945 | else if (reg->space_id == ACPI_ADR_SPACE_SYSTEM_MEMORY) |
| 913 | 946 | vaddr = reg_res->sys_mem_vaddr; |
| @@ -946,7 +979,20 @@ static int cpc_write(int cpu, struct cpc_register_resource *reg_res, u64 val) · 32 unmodified lines | ||
| 946 | 979 | int pcc_ss_id = per_cpu(cpu_pcc_subspace_idx, cpu); |
| 947 | 980 | struct cpc_reg *reg = ®_res->cpc_entry.reg; |
| 948 | 981 | |
| 949 | − if (reg->space_id == ACPI_ADR_SPACE_PLATFORM_COMM && pcc_ss_id >= 0) | |
| 982 | + if (reg->space_id == ACPI_ADR_SPACE_SYSTEM_IO) { | |
| 983 | + u32 width = 8 << (reg->access_width - 1); | |
| 984 | + acpi_status status; | |
| 985 | + | |
| 986 | + status = acpi_os_write_port((acpi_io_address)reg->address, | |
| 987 | + (u32)val, width); | |
| 988 | + if (ACPI_FAILURE(status)) { | |
| 989 | + pr_debug("Error: Failed to write SystemIO port %llx\n", | |
| 990 | + reg->address); | |
| 991 | + return -EFAULT; | |
| 992 | + } | |
| 993 | + | |
| 994 | + return 0; | |
| 995 | + } else if (reg->space_id == ACPI_ADR_SPACE_PLATFORM_COMM && pcc_ss_id >= 0) | |
| 950 | 996 | vaddr = GET_PCC_VADDR(reg->address, pcc_ss_id); |
| 951 | 997 | else if (reg->space_id == ACPI_ADR_SPACE_SYSTEM_MEMORY) |
| 952 | 998 | vaddr = reg_res->sys_mem_vaddr; |
| @@ -1213,6 +1259,51 @@ out_err: · 260 unmodified lines | ||
| 1213 | 1259 | } |
| 1214 | 1260 | EXPORT_SYMBOL_GPL(cppc_get_perf_ctrs); |
| 1215 | 1261 | |
| 1262 | +/** | |
| 1263 | + * cppc_set_enable - Set to enable CPPC on the processor by writing the | |
| 1264 | + * Continuous Performance Control package EnableRegister field. | |
| 1265 | + * @cpu: CPU for which to enable CPPC register. | |
| 1266 | + * @enable: 0 - disable, 1 - enable CPPC feature on the processor. | |
| 1267 | + * | |
| 1268 | + * Return: 0 for success, -ERRNO or -EIO otherwise. | |
| 1269 | + */ | |
| 1270 | +int cppc_set_enable(int cpu, bool enable) | |
| 1271 | +{ | |
| 1272 | + int pcc_ss_id = per_cpu(cpu_pcc_subspace_idx, cpu); | |
| 1273 | + struct cpc_register_resource *enable_reg; | |
| 1274 | + struct cpc_desc *cpc_desc = per_cpu(cpc_desc_ptr, cpu); | |
| 1275 | + struct cppc_pcc_data *pcc_ss_data = NULL; | |
| 1276 | + int ret = -EINVAL; | |
| 1277 | + | |
| 1278 | + if (!cpc_desc) { | |
| 1279 | + pr_debug("No CPC descriptor for CPU:%d\n", cpu); | |
| 1280 | + return -EINVAL; | |
| 1281 | + } | |
| 1282 | + | |
| 1283 | + enable_reg = &cpc_desc->cpc_regs[ENABLE]; | |
| 1284 | + | |
| 1285 | + if (CPC_IN_PCC(enable_reg)) { | |
| 1286 | + | |
| 1287 | + if (pcc_ss_id < 0) | |
| 1288 | + return -EIO; | |
| 1289 | + | |
| 1290 | + ret = cpc_write(cpu, enable_reg, enable); | |
| 1291 | + if (ret) | |
| 1292 | + return ret; | |
| 1293 | + | |
| 1294 | + pcc_ss_data = pcc_data[pcc_ss_id]; | |
| 1295 | + | |
| 1296 | + down_write(&pcc_ss_data->pcc_lock); | |
| 1297 | + /* after writing CPC, transfer the ownership of PCC to platfrom */ | |
| 1298 | + ret = send_pcc_cmd(pcc_ss_id, CMD_WRITE); | |
| 1299 | + up_write(&pcc_ss_data->pcc_lock); | |
| 1300 | + return ret; | |
| 1301 | + } | |
| 1302 | + | |
| 1303 | + return cpc_write(cpu, enable_reg, enable); | |
| 1304 | +} | |
| 1305 | +EXPORT_SYMBOL_GPL(cppc_set_enable); | |
| 1306 | + | |
| 1216 | 1307 | /** |
| 1217 | 1308 | * cppc_set_perf - Set a CPU's performance controls. |
| 1218 | 1309 | * @cpu: CPU for which to set performance controls. |
| @@ -874,11 +874,11 @@ static inline void acpi_sleep_syscore_init(void) {} · 873 unmodified lines | ||
| 874 | 874 | #ifdef CONFIG_HIBERNATION |
| 875 | 875 | static unsigned long s4_hardware_signature; |
| 876 | 876 | static struct acpi_table_facs *facs; |
| 877 | −static bool nosigcheck; | |
| 877 | +static int sigcheck = -1; /* Default behaviour is just to warn */ | |
| 878 | 878 | |
| 879 | −void __init acpi_no_s4_hw_signature(void) | |
| 879 | +void __init acpi_check_s4_hw_signature(int check) | |
| 880 | 880 | { |
| 881 | − nosigcheck = true; | |
| 881 | + sigcheck = check; | |
| 882 | 882 | } |
| 883 | 883 | |
| 884 | 884 | static int acpi_hibernation_begin(pm_message_t stage) |
| @@ -1004,12 +1004,28 @@ static void acpi_sleep_hibernate_setup(void) · 119 unmodified lines | ||
| 1004 | 1004 | hibernation_set_ops(old_suspend_ordering ? |
| 1005 | 1005 | &acpi_hibernation_ops_old : &acpi_hibernation_ops); |
| 1006 | 1006 | sleep_states[ACPI_STATE_S4] = 1; |
| 1007 | − if (nosigcheck) | |
| 1007 | + if (!sigcheck) | |
| 1008 | 1008 | return; |
| 1009 | 1009 | |
| 1010 | 1010 | acpi_get_table(ACPI_SIG_FACS, 1, (struct acpi_table_header **)&facs); |
| 1011 | − if (facs) | |
| 1011 | + if (facs) { | |
| 1012 | + /* | |
| 1013 | + * s4_hardware_signature is the local variable which is just | |
| 1014 | + * used to warn about mismatch after we're attempting to | |
| 1015 | + * resume (in violation of the ACPI specification.) | |
| 1016 | + */ | |
| 1012 | 1017 | s4_hardware_signature = facs->hardware_signature; |
| 1018 | + | |
| 1019 | + if (sigcheck > 0) { | |
| 1020 | + /* | |
| 1021 | + * If we're actually obeying the ACPI specification | |
| 1022 | + * then the signature is written out as part of the | |
| 1023 | + * swsusp header, in order to allow the boot kernel | |
| 1024 | + * to gracefully decline to resume. | |
| 1025 | + */ | |
| 1026 | + swsusp_hardware_signature = facs->hardware_signature; | |
| 1027 | + } | |
| 1028 | + } | |
| 1013 | 1029 | } |
| 1014 | 1030 | #else /* !CONFIG_HIBERNATION */ |
| 1015 | 1031 | static inline void acpi_sleep_hibernate_setup(void) {} |
| @@ -22,6 +22,7 @@ · 21 unmodified lines | ||
| 22 | 22 | static DEFINE_PER_CPU(struct scale_freq_data __rcu *, sft_data); |
| 23 | 23 | static struct cpumask scale_freq_counters_mask; |
| 24 | 24 | static bool scale_freq_invariant; |
| 25 | +static DEFINE_PER_CPU(u32, freq_factor) = 1; | |
| 25 | 26 | |
| 26 | 27 | static bool supports_scale_freq_counters(const struct cpumask *cpus) |
| 27 | 28 | { |
| @@ -155,15 +156,49 @@ void topology_set_cpu_scale(unsigned int cpu, unsigned long capacity) · 127 unmodified lines | ||
| 155 | 156 | |
| 156 | 157 | DEFINE_PER_CPU(unsigned long, thermal_pressure); |
| 157 | 158 | |
| 158 | −void topology_set_thermal_pressure(const struct cpumask *cpus, | |
| 159 | − unsigned long th_pressure) | |
| 159 | +/** | |
| 160 | + * topology_update_thermal_pressure() - Update thermal pressure for CPUs | |
| 161 | + * @cpus : The related CPUs for which capacity has been reduced | |
| 162 | + * @capped_freq : The maximum allowed frequency that CPUs can run at | |
| 163 | + * | |
| 164 | + * Update the value of thermal pressure for all @cpus in the mask. The | |
| 165 | + * cpumask should include all (online+offline) affected CPUs, to avoid | |
| 166 | + * operating on stale data when hot-plug is used for some CPUs. The | |
| 167 | + * @capped_freq reflects the currently allowed max CPUs frequency due to | |
| 168 | + * thermal capping. It might be also a boost frequency value, which is bigger | |
| 169 | + * than the internal 'freq_factor' max frequency. In such case the pressure | |
| 170 | + * value should simply be removed, since this is an indication that there is | |
| 171 | + * no thermal throttling. The @capped_freq must be provided in kHz. | |
| 172 | + */ | |
| 173 | +void topology_update_thermal_pressure(const struct cpumask *cpus, | |
| 174 | + unsigned long capped_freq) | |
| 160 | 175 | { |
| 176 | + unsigned long max_capacity, capacity, th_pressure; | |
| 177 | + u32 max_freq; | |
| 161 | 178 | int cpu; |
| 162 | 179 | |
| 180 | + cpu = cpumask_first(cpus); | |
| 181 | + max_capacity = arch_scale_cpu_capacity(cpu); | |
| 182 | + max_freq = per_cpu(freq_factor, cpu); | |
| 183 | + | |
| 184 | + /* Convert to MHz scale which is used in 'freq_factor' */ | |
| 185 | + capped_freq /= 1000; | |
| 186 | + | |
| 187 | + /* | |
| 188 | + * Handle properly the boost frequencies, which should simply clean | |
| 189 | + * the thermal pressure value. | |
| 190 | + */ | |
| 191 | + if (max_freq <= capped_freq) | |
| 192 | + capacity = max_capacity; | |
| 193 | + else | |
| 194 | + capacity = mult_frac(max_capacity, capped_freq, max_freq); | |
| 195 | + | |
| 196 | + th_pressure = max_capacity - capacity; | |
| 197 | + | |
| 163 | 198 | for_each_cpu(cpu, cpus) |
| 164 | 199 | WRITE_ONCE(per_cpu(thermal_pressure, cpu), th_pressure); |
| 165 | 200 | } |
| 166 | −EXPORT_SYMBOL_GPL(topology_set_thermal_pressure); | |
| 201 | +EXPORT_SYMBOL_GPL(topology_update_thermal_pressure); | |
| 167 | 202 | |
| 168 | 203 | static ssize_t cpu_capacity_show(struct device *dev, |
| 169 | 204 | struct device_attribute *attr, |
| @@ -217,7 +252,6 @@ static void update_topology_flags_workfn(struct work_struct *work) · 47 unmodified lines | ||
| 217 | 252 | update_topology = 0; |
| 218 | 253 | } |
| 219 | 254 | |
| 220 | −static DEFINE_PER_CPU(u32, freq_factor) = 1; | |
| 221 | 255 | static u32 *raw_capacity; |
| 222 | 256 | |
| 223 | 257 | static int free_raw_capacity(void) |
| @@ -485,8 +485,7 @@ static void device_link_release_fn(struct work_struct *work) · 484 unmodified lines | ||
| 485 | 485 | /* Ensure that all references to the link object have been dropped. */ |
| 486 | 486 | device_link_synchronize_removal(); |
| 487 | 487 | |
| 488 | − while (refcount_dec_not_one(&link->rpm_active)) | |
| 489 | − pm_runtime_put(link->supplier); | |
| 488 | + pm_runtime_release_supplier(link, true); | |
| 490 | 489 | |
| 491 | 490 | put_device(link->consumer); |
| 492 | 491 | put_device(link->supplier); |
| @@ -305,19 +305,40 @@ static int rpm_get_suppliers(struct device *dev) · 304 unmodified lines | ||
| 305 | 305 | return 0; |
| 306 | 306 | } |
| 307 | 307 | |
| 308 | +/** | |
| 309 | + * pm_runtime_release_supplier - Drop references to device link's supplier. | |
| 310 | + * @link: Target device link. | |
| 311 | + * @check_idle: Whether or not to check if the supplier device is idle. | |
| 312 | + * | |
| 313 | + * Drop all runtime PM references associated with @link to its supplier device | |
| 314 | + * and if @check_idle is set, check if that device is idle (and so it can be | |
| 315 | + * suspended). | |
| 316 | + */ | |
| 317 | +void pm_runtime_release_supplier(struct device_link *link, bool check_idle) | |
| 318 | +{ | |
| 319 | + struct device *supplier = link->supplier; | |
| 320 | + | |
| 321 | + /* | |
| 322 | + * The additional power.usage_count check is a safety net in case | |
| 323 | + * the rpm_active refcount becomes saturated, in which case | |
| 324 | + * refcount_dec_not_one() would return true forever, but it is not | |
| 325 | + * strictly necessary. | |
| 326 | + */ | |
| 327 | + while (refcount_dec_not_one(&link->rpm_active) && | |
| 328 | + atomic_read(&supplier->power.usage_count) > 0) | |
| 329 | + pm_runtime_put_noidle(supplier); | |
| 330 | + | |
| 331 | + if (check_idle) | |
| 332 | + pm_request_idle(supplier); | |
| 333 | +} | |
| 334 | + | |
| 308 | 335 | static void __rpm_put_suppliers(struct device *dev, bool try_to_suspend) |
| 309 | 336 | { |
| 310 | 337 | struct device_link *link; |
| 311 | 338 | |
| 312 | 339 | list_for_each_entry_rcu(link, &dev->links.suppliers, c_node, |
| 313 | − device_links_read_lock_held()) { | |
| 314 | − | |
| 315 | − while (refcount_dec_not_one(&link->rpm_active)) | |
| 316 | − pm_runtime_put_noidle(link->supplier); | |
| 317 | − | |
| 318 | − if (try_to_suspend) | |
| 319 | − pm_request_idle(link->supplier); | |
| 320 | − } | |
| 340 | + device_links_read_lock_held()) | |
| 341 | + pm_runtime_release_supplier(link, try_to_suspend); | |
| 321 | 342 | } |
| 322 | 343 | |
| 323 | 344 | static void rpm_put_suppliers(struct device *dev) |
| @@ -742,13 +763,15 @@ static int rpm_resume(struct device *dev, int rpmflags) · 418 unmodified lines | ||
| 742 | 763 | trace_rpm_resume_rcuidle(dev, rpmflags); |
| 743 | 764 | |
| 744 | 765 | repeat: |
| 745 | − if (dev->power.runtime_error) | |
| 766 | + if (dev->power.runtime_error) { | |
| 746 | 767 | retval = -EINVAL; |
| 747 | − else if (dev->power.disable_depth == 1 && dev->power.is_suspended | |
| 748 | − && dev->power.runtime_status == RPM_ACTIVE) | |
| 749 | − retval = 1; | |
| 750 | − else if (dev->power.disable_depth > 0) | |
| 751 | − retval = -EACCES; | |
| 768 | + } else if (dev->power.disable_depth > 0) { | |
| 769 | + if (dev->power.runtime_status == RPM_ACTIVE && | |
| 770 | + dev->power.last_status == RPM_ACTIVE) | |
| 771 | + retval = 1; | |
| 772 | + else | |
| 773 | + retval = -EACCES; | |
| 774 | + } | |
| 752 | 775 | if (retval) |
| 753 | 776 | goto out; |
| 754 | 777 | |
| @@ -1410,8 +1433,10 @@ void __pm_runtime_disable(struct device *dev, bool check_resume) · 655 unmodified lines | ||
| 1410 | 1433 | /* Update time accounting before disabling PM-runtime. */ |
| 1411 | 1434 | update_pm_runtime_accounting(dev); |
| 1412 | 1435 | |
| 1413 | − if (!dev->power.disable_depth++) | |
| 1436 | + if (!dev->power.disable_depth++) { | |
| 1414 | 1437 | __pm_runtime_barrier(dev); |
| 1438 | + dev->power.last_status = dev->power.runtime_status; | |
| 1439 | + } | |
| 1415 | 1440 | |
| 1416 | 1441 | out: |
| 1417 | 1442 | spin_unlock_irq(&dev->power.lock); |
| @@ -1428,23 +1453,23 @@ void pm_runtime_enable(struct device *dev) · 10 unmodified lines | ||
| 1428 | 1453 | |
| 1429 | 1454 | spin_lock_irqsave(&dev->power.lock, flags); |
| 1430 | 1455 | |
| 1431 | − if (dev->power.disable_depth > 0) { | |
| 1432 | − dev->power.disable_depth--; | |
| 1433 | − | |
| 1434 | − /* About to enable runtime pm, set accounting_timestamp to now */ | |
| 1435 | − if (!dev->power.disable_depth) | |
| 1436 | − dev->power.accounting_timestamp = ktime_get_mono_fast_ns(); | |
| 1437 | − } else { | |
| 1456 | + if (!dev->power.disable_depth) { | |
| 1438 | 1457 | dev_warn(dev, "Unbalanced %s!\n", __func__); |
| 1458 | + goto out; | |
| 1439 | 1459 | } |
| 1440 | 1460 | |
| 1441 | − WARN(!dev->power.disable_depth && | |
| 1442 | − dev->power.runtime_status == RPM_SUSPENDED && | |
| 1443 | − !dev->power.ignore_children && | |
| 1444 | − atomic_read(&dev->power.child_count) > 0, | |
| 1445 | − "Enabling runtime PM for inactive device (%s) with active children\n", | |
| 1446 | − dev_name(dev)); | |
| 1461 | + if (--dev->power.disable_depth > 0) | |
| 1462 | + goto out; | |
| 1463 | + | |
| 1464 | + dev->power.last_status = RPM_INVALID; | |
| 1465 | + dev->power.accounting_timestamp = ktime_get_mono_fast_ns(); | |
| 1466 | + | |
| 1467 | + if (dev->power.runtime_status == RPM_SUSPENDED && | |
| 1468 | + !dev->power.ignore_children && | |
| 1469 | + atomic_read(&dev->power.child_count) > 0) | |
| 1470 | + dev_warn(dev, "Enabling runtime PM for inactive device with active children\n"); | |
| 1447 | 1471 | |
| 1472 | +out: | |
| 1448 | 1473 | spin_unlock_irqrestore(&dev->power.lock, flags); |
| 1449 | 1474 | } |
| 1450 | 1475 | EXPORT_SYMBOL_GPL(pm_runtime_enable); |
| @@ -1640,6 +1665,7 @@ EXPORT_SYMBOL_GPL(__pm_runtime_use_autosuspend); · 189 unmodified lines | ||
| 1640 | 1665 | void pm_runtime_init(struct device *dev) |
| 1641 | 1666 | { |
| 1642 | 1667 | dev->power.runtime_status = RPM_SUSPENDED; |
| 1668 | + dev->power.last_status = RPM_INVALID; | |
| 1643 | 1669 | dev->power.idle_notification = false; |
| 1644 | 1670 | |
| 1645 | 1671 | dev->power.disable_depth = 1; |
| @@ -1722,8 +1748,6 @@ void pm_runtime_get_suppliers(struct device *dev) · 76 unmodified lines | ||
| 1722 | 1748 | void pm_runtime_put_suppliers(struct device *dev) |
| 1723 | 1749 | { |
| 1724 | 1750 | struct device_link *link; |
| 1725 | − unsigned long flags; | |
| 1726 | − bool put; | |
| 1727 | 1751 | int idx; |
| 1728 | 1752 | |
| 1729 | 1753 | idx = device_links_read_lock(); |
| @@ -1731,11 +1755,17 @@ void pm_runtime_put_suppliers(struct device *dev) · 1 unmodified line | ||
| 1731 | 1755 | list_for_each_entry_rcu(link, &dev->links.suppliers, c_node, |
| 1732 | 1756 | device_links_read_lock_held()) |
| 1733 | 1757 | if (link->supplier_preactivated) { |
| 1758 | + bool put; | |
| 1759 | + | |
| 1734 | 1760 | link->supplier_preactivated = false; |
| 1735 | − spin_lock_irqsave(&dev->power.lock, flags); | |
| 1761 | + | |
| 1762 | + spin_lock_irq(&dev->power.lock); | |
| 1763 | + | |
| 1736 | 1764 | put = pm_runtime_status_suspended(dev) && |
| 1737 | 1765 | refcount_dec_not_one(&link->rpm_active); |
| 1738 | − spin_unlock_irqrestore(&dev->power.lock, flags); | |
| 1766 | + | |
| 1767 | + spin_unlock_irq(&dev->power.lock); | |
| 1768 | + | |
| 1739 | 1769 | if (put) |
| 1740 | 1770 | pm_runtime_put(link->supplier); |
| 1741 | 1771 | } |
| @@ -1772,9 +1802,7 @@ void pm_runtime_drop_link(struct device_link *link) · 30 unmodified lines | ||
| 1772 | 1802 | return; |
| 1773 | 1803 | |
| 1774 | 1804 | pm_runtime_drop_link_count(link->consumer); |
| 1775 | − | |
| 1776 | − while (refcount_dec_not_one(&link->rpm_active)) | |
| 1777 | − pm_runtime_put(link->supplier); | |
| 1805 | + pm_runtime_release_supplier(link, true); | |
| 1778 | 1806 | } |
| 1779 | 1807 | |
| 1780 | 1808 | static bool pm_runtime_need_not_resume(struct device *dev) |
| @@ -34,6 +34,23 @@ config X86_PCC_CPUFREQ · 33 unmodified lines | ||
| 34 | 34 | |
| 35 | 35 | If in doubt, say N. |
| 36 | 36 | |
| 37 | +config X86_AMD_PSTATE | |
| 38 | + tristate "AMD Processor P-State driver" | |
| 39 | + depends on X86 && ACPI | |
| 40 | + select ACPI_PROCESSOR | |
| 41 | + select ACPI_CPPC_LIB if X86_64 | |
| 42 | + select CPU_FREQ_GOV_SCHEDUTIL if SMP | |
| 43 | + help | |
| 44 | + This driver adds a CPUFreq driver which utilizes a fine grain | |
| 45 | + processor performance frequency control range instead of legacy | |
| 46 | + performance levels. _CPC needs to be present in the ACPI tables | |
| 47 | + of the system. | |
| 48 | + | |
| 49 | + For details, take a look at: | |
| 50 | + <file:Documentation/admin-guide/pm/amd-pstate.rst>. | |
| 51 | + | |
| 52 | + If in doubt, say N. | |
| 53 | + | |
| 37 | 54 | config X86_ACPI_CPUFREQ |
| 38 | 55 | tristate "ACPI Processor P-States driver" |
| 39 | 56 | depends on ACPI_PROCESSOR |
| @@ -17,6 +17,10 @@ obj-$(CONFIG_CPU_FREQ_GOV_ATTR_SET) += cpufreq_governor_attr_set.o · 16 unmodified lines | ||
| 17 | 17 | obj-$(CONFIG_CPUFREQ_DT) += cpufreq-dt.o |
| 18 | 18 | obj-$(CONFIG_CPUFREQ_DT_PLATDEV) += cpufreq-dt-platdev.o |
| 19 | 19 | |
| 20 | +# Traces | |
| 21 | +CFLAGS_amd-pstate-trace.o := -I$(src) | |
| 22 | +amd_pstate-y := amd-pstate.o amd-pstate-trace.o | |
| 23 | + | |
| 20 | 24 | ################################################################################## |
| 21 | 25 | # x86 drivers. |
| 22 | 26 | # Link order matters. K8 is preferred to ACPI because of firmware bugs in early |
| @@ -25,6 +29,7 @@ obj-$(CONFIG_CPUFREQ_DT_PLATDEV) += cpufreq-dt-platdev.o · 2 unmodified lines | ||
| 25 | 29 | # speedstep-* is preferred over p4-clockmod. |
| 26 | 30 | |
| 27 | 31 | obj-$(CONFIG_X86_ACPI_CPUFREQ) += acpi-cpufreq.o |
| 32 | +obj-$(CONFIG_X86_AMD_PSTATE) += amd_pstate.o | |
| 28 | 33 | obj-$(CONFIG_X86_POWERNOW_K8) += powernow-k8.o |
| 29 | 34 | obj-$(CONFIG_X86_PCC_CPUFREQ) += pcc-cpufreq.o |
| 30 | 35 | obj-$(CONFIG_X86_POWERNOW_K6) += powernow-k6.o |
| @@ -0,0 +1,2 @@ | ||
| 1 | +#define CREATE_TRACE_POINTS | |
| 2 | +#include "amd-pstate-trace.h" | |
| @@ -0,0 +1,77 @@ | ||
| 1 | +/* SPDX-License-Identifier: GPL-2.0 */ | |
| 2 | +/* | |
| 3 | + * amd-pstate-trace.h - AMD Processor P-state Frequency Driver Tracer | |
| 4 | + * | |
| 5 | + * Copyright (C) 2021 Advanced Micro Devices, Inc. All Rights Reserved. | |
| 6 | + * | |
| 7 | + * Author: Huang Rui <ray.huang@amd.com> | |
| 8 | + */ | |
| 9 | + | |
| 10 | +#if !defined(_AMD_PSTATE_TRACE_H) || defined(TRACE_HEADER_MULTI_READ) | |
| 11 | +#define _AMD_PSTATE_TRACE_H | |
| 12 | + | |
| 13 | +#include <linux/cpufreq.h> | |
| 14 | +#include <linux/tracepoint.h> | |
| 15 | +#include <linux/trace_events.h> | |
| 16 | + | |
| 17 | +#undef TRACE_SYSTEM | |
| 18 | +#define TRACE_SYSTEM amd_cpu | |
| 19 | + | |
| 20 | +#undef TRACE_INCLUDE_FILE | |
| 21 | +#define TRACE_INCLUDE_FILE amd-pstate-trace | |
| 22 | + | |
| 23 | +#define TPS(x) tracepoint_string(x) | |
| 24 | + | |
| 25 | +TRACE_EVENT(amd_pstate_perf, | |
| 26 | + | |
| 27 | + TP_PROTO(unsigned long min_perf, | |
| 28 | + unsigned long target_perf, | |
| 29 | + unsigned long capacity, | |
| 30 | + unsigned int cpu_id, | |
| 31 | + bool changed, | |
| 32 | + bool fast_switch | |
| 33 | + ), | |
| 34 | + | |
| 35 | + TP_ARGS(min_perf, | |
| 36 | + target_perf, | |
| 37 | + capacity, | |
| 38 | + cpu_id, | |
| 39 | + changed, | |
| 40 | + fast_switch | |
| 41 | + ), | |
| 42 | + | |
| 43 | + TP_STRUCT__entry( | |
| 44 | + __field(unsigned long, min_perf) | |
| 45 | + __field(unsigned long, target_perf) | |
| 46 | + __field(unsigned long, capacity) | |
| 47 | + __field(unsigned int, cpu_id) | |
| 48 | + __field(bool, changed) | |
| 49 | + __field(bool, fast_switch) | |
| 50 | + ), | |
| 51 | + | |
| 52 | + TP_fast_assign( | |
| 53 | + __entry->min_perf = min_perf; | |
| 54 | + __entry->target_perf = target_perf; | |
| 55 | + __entry->capacity = capacity; | |
| 56 | + __entry->cpu_id = cpu_id; | |
| 57 | + __entry->changed = changed; | |
| 58 | + __entry->fast_switch = fast_switch; | |
| 59 | + ), | |
| 60 | + | |
| 61 | + TP_printk("amd_min_perf=%lu amd_des_perf=%lu amd_max_perf=%lu cpu_id=%u changed=%s fast_switch=%s", | |
| 62 | + (unsigned long)__entry->min_perf, | |
| 63 | + (unsigned long)__entry->target_perf, | |
| 64 | + (unsigned long)__entry->capacity, | |
| 65 | + (unsigned int)__entry->cpu_id, | |
| 66 | + (__entry->changed) ? "true" : "false", | |
| 67 | + (__entry->fast_switch) ? "true" : "false" | |
| 68 | + ) | |
| 69 | +); | |
| 70 | + | |
| 71 | +#endif /* _AMD_PSTATE_TRACE_H */ | |
| 72 | + | |
| 73 | +/* This part must be outside protection */ | |
| 74 | +#undef TRACE_INCLUDE_PATH | |
| 75 | +#define TRACE_INCLUDE_PATH . | |
| 76 | + | |
| 77 | +#include <trace/define_trace.h> | |
| @@ -0,0 +1,645 @@ | ||
| 1 | +// SPDX-License-Identifier: GPL-2.0-or-later | |
| 2 | +/* | |
| 3 | + * amd-pstate.c - AMD Processor P-state Frequency Driver | |
| 4 | + * | |
| 5 | + * Copyright (C) 2021 Advanced Micro Devices, Inc. All Rights Reserved. | |
| 6 | + * | |
| 7 | + * Author: Huang Rui <ray.huang@amd.com> | |
| 8 | + * | |
| 9 | + * AMD P-State introduces a new CPU performance scaling design for AMD | |
| 10 | + * processors using the ACPI Collaborative Performance and Power Control (CPPC) | |
| 11 | + * feature which works with the AMD SMU firmware providing a finer grained | |
| 12 | + * frequency control range. It is to replace the legacy ACPI P-States control, | |
| 13 | + * allows a flexible, low-latency interface for the Linux kernel to directly | |
| 14 | + * communicate the performance hints to hardware. | |
| 15 | + * | |
| 16 | + * AMD P-State is supported on recent AMD Zen base CPU series include some of | |
| 17 | + * Zen2 and Zen3 processors. _CPC needs to be present in the ACPI tables of AMD | |
| 18 | + * P-State supported system. And there are two types of hardware implementations | |
| 19 | + * for AMD P-State: 1) Full MSR Solution and 2) Shared Memory Solution. | |
| 20 | + * X86_FEATURE_CPPC CPU feature flag is used to distinguish the different types. | |
| 21 | + */ | |
| 22 | + | |
| 23 | +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt | |
| 24 | + | |
| 25 | +#include <linux/kernel.h> | |
| 26 | +#include <linux/module.h> | |
| 27 | +#include <linux/init.h> | |
| 28 | +#include <linux/smp.h> | |
| 29 | +#include <linux/sched.h> | |
| 30 | +#include <linux/cpufreq.h> | |
| 31 | +#include <linux/compiler.h> | |
| 32 | +#include <linux/dmi.h> | |
| 33 | +#include <linux/slab.h> | |
| 34 | +#include <linux/acpi.h> | |
| 35 | +#include <linux/io.h> | |
| 36 | +#include <linux/delay.h> | |
| 37 | +#include <linux/uaccess.h> | |
| 38 | +#include <linux/static_call.h> | |
| 39 | + | |
| 40 | +#include <acpi/processor.h> | |
| 41 | +#include <acpi/cppc_acpi.h> | |
| 42 | + | |
| 43 | +#include <asm/msr.h> | |
| 44 | +#include <asm/processor.h> | |
| 45 | +#include <asm/cpufeature.h> | |
| 46 | +#include <asm/cpu_device_id.h> | |
| 47 | +#include "amd-pstate-trace.h" | |
| 48 | + | |
| 49 | +#define AMD_PSTATE_TRANSITION_LATENCY 0x20000 | |
| 50 | +#define AMD_PSTATE_TRANSITION_DELAY 500 | |
| 51 | + | |
| 52 | +/* | |
| 53 | + * TODO: We need more time to fine tune processors with shared memory solution | |
| 54 | + * with community together. | |
| 55 | + * | |
| 56 | + * There are some performance drops on the CPU benchmarks which reports from | |
| 57 | + * Suse. We are co-working with them to fine tune the shared memory solution. So | |
| 58 | + * we disable it by default to go acpi-cpufreq on these processors and add a | |
| 59 | + * module parameter to be able to enable it manually for debugging. | |
| 60 | + */ | |
| 61 | +static bool shared_mem = false; | |
| 62 | +module_param(shared_mem, bool, 0444); | |
| 63 | +MODULE_PARM_DESC(shared_mem, | |
| 64 | + "enable amd-pstate on processors with shared memory solution (false = disabled (default), true = enabled)"); | |
| 65 | + | |
| 66 | +static struct cpufreq_driver amd_pstate_driver; | |
| 67 | + | |
| 68 | +/** | |
| 69 | + * struct amd_cpudata - private CPU data for AMD P-State | |
| 70 | + * @cpu: CPU number | |
| 71 | + * @req: constraint request to apply | |
| 72 | + * @cppc_req_cached: cached performance request hints | |
| 73 | + * @highest_perf: the maximum performance an individual processor may reach, | |
| 74 | + * assuming ideal conditions | |
| 75 | + * @nominal_perf: the maximum sustained performance level of the processor, | |
| 76 | + * assuming ideal operating conditions | |
| 77 | + * @lowest_nonlinear_perf: the lowest performance level at which nonlinear power | |
| 78 | + * savings are achieved | |
| 79 | + * @lowest_perf: the absolute lowest performance level of the processor | |
| 80 | + * @max_freq: the frequency that mapped to highest_perf | |
| 81 | + * @min_freq: the frequency that mapped to lowest_perf | |
| 82 | + * @nominal_freq: the frequency that mapped to nominal_perf | |
| 83 | + * @lowest_nonlinear_freq: the frequency that mapped to lowest_nonlinear_perf | |
| 84 | + * @boost_supported: check whether the Processor or SBIOS supports boost mode | |
| 85 | + * | |
| 86 | + * The amd_cpudata is key private data for each CPU thread in AMD P-State, and | |
| 87 | + * represents all the attributes and goals that AMD P-State requests at runtime. | |
| 88 | + */ | |
| 89 | +struct amd_cpudata { | |
| 90 | + int cpu; | |
| 91 | + | |
| 92 | + struct freq_qos_request req[2]; | |
| 93 | + u64 cppc_req_cached; | |
| 94 | + | |
| 95 | + u32 highest_perf; | |
| 96 | + u32 nominal_perf; | |
| 97 | + u32 lowest_nonlinear_perf; | |
| 98 | + u32 lowest_perf; | |
| 99 | + | |
| 100 | + u32 max_freq; | |
| 101 | + u32 min_freq; | |
| 102 | + u32 nominal_freq; | |
| 103 | + u32 lowest_nonlinear_freq; | |
| 104 | + | |
| 105 | + bool boost_supported; | |
| 106 | +}; | |
| 107 | + | |
| 108 | +static inline int pstate_enable(bool enable) | |
| 109 | +{ | |
| 110 | + return wrmsrl_safe(MSR_AMD_CPPC_ENABLE, enable); | |
| 111 | +} | |
| 112 | + | |
| 113 | +static int cppc_enable(bool enable) | |
| 114 | +{ | |
| 115 | + int cpu, ret = 0; | |
| 116 | + | |
| 117 | + for_each_present_cpu(cpu) { | |
| 118 | + ret = cppc_set_enable(cpu, enable); | |
| 119 | + if (ret) | |
| 120 | + return ret; | |
| 121 | + } | |
| 122 | + | |
| 123 | + return ret; | |
| 124 | +} | |
| 125 | + | |
| 126 | +DEFINE_STATIC_CALL(amd_pstate_enable, pstate_enable); | |
| 127 | + | |
| 128 | +static inline int amd_pstate_enable(bool enable) | |
| 129 | +{ | |
| 130 | + return static_call(amd_pstate_enable)(enable); | |
| 131 | +} | |
| 132 | + | |
| 133 | +static int pstate_init_perf(struct amd_cpudata *cpudata) | |
| 134 | +{ | |
| 135 | + u64 cap1; | |
| 136 | + | |
| 137 | + int ret = rdmsrl_safe_on_cpu(cpudata->cpu, MSR_AMD_CPPC_CAP1, | |
| 138 | + &cap1); | |
| 139 | + if (ret) | |
| 140 | + return ret; | |
| 141 | + | |
| 142 | + /* | |
| 143 | + * TODO: Introduce AMD specific power feature. | |
| 144 | + * | |
| 145 | + * CPPC entry doesn't indicate the highest performance in some ASICs. | |
| 146 | + */ | |
| 147 | + WRITE_ONCE(cpudata->highest_perf, amd_get_highest_perf()); | |
| 148 | + | |
| 149 | + WRITE_ONCE(cpudata->nominal_perf, AMD_CPPC_NOMINAL_PERF(cap1)); | |
| 150 | + WRITE_ONCE(cpudata->lowest_nonlinear_perf, AMD_CPPC_LOWNONLIN_PERF(cap1)); | |
| 151 | + WRITE_ONCE(cpudata->lowest_perf, AMD_CPPC_LOWEST_PERF(cap1)); | |
| 152 | + | |
| 153 | + return 0; | |
| 154 | +} | |
| 155 | + | |
| 156 | +static int cppc_init_perf(struct amd_cpudata *cpudata) | |
| 157 | +{ | |
| 158 | + struct cppc_perf_caps cppc_perf; | |
| 159 | + | |
| 160 | + int ret = cppc_get_perf_caps(cpudata->cpu, &cppc_perf); | |
| 161 | + if (ret) | |
| 162 | + return ret; | |
| 163 | + | |
| 164 | + WRITE_ONCE(cpudata->highest_perf, amd_get_highest_perf()); | |
| 165 | + | |
| 166 | + WRITE_ONCE(cpudata->nominal_perf, cppc_perf.nominal_perf); | |
| 167 | + WRITE_ONCE(cpudata->lowest_nonlinear_perf, | |
| 168 | + cppc_perf.lowest_nonlinear_perf); | |
| 169 | + WRITE_ONCE(cpudata->lowest_perf, cppc_perf.lowest_perf); | |
| 170 | + | |
| 171 | + return 0; | |
| 172 | +} | |
| 173 | + | |
| 174 | +DEFINE_STATIC_CALL(amd_pstate_init_perf, pstate_init_perf); | |
| 175 | + | |
| 176 | +static inline int amd_pstate_init_perf(struct amd_cpudata *cpudata) | |
| 177 | +{ | |
| 178 | + return static_call(amd_pstate_init_perf)(cpudata); | |
| 179 | +} | |
| 180 | + | |
| 181 | +static void pstate_update_perf(struct amd_cpudata *cpudata, u32 min_perf, | |
| 182 | + u32 des_perf, u32 max_perf, bool fast_switch) | |
| 183 | +{ | |
| 184 | + if (fast_switch) | |
| 185 | + wrmsrl(MSR_AMD_CPPC_REQ, READ_ONCE(cpudata->cppc_req_cached)); | |
| 186 | + else | |
| 187 | + wrmsrl_on_cpu(cpudata->cpu, MSR_AMD_CPPC_REQ, | |
| 188 | + READ_ONCE(cpudata->cppc_req_cached)); | |
| 189 | +} | |
| 190 | + | |
| 191 | +static void cppc_update_perf(struct amd_cpudata *cpudata, | |
| 192 | + u32 min_perf, u32 des_perf, | |
| 193 | + u32 max_perf, bool fast_switch) | |
| 194 | +{ | |
| 195 | + struct cppc_perf_ctrls perf_ctrls; | |
| 196 | + | |
| 197 | + perf_ctrls.max_perf = max_perf; | |
| 198 | + perf_ctrls.min_perf = min_perf; | |
| 199 | + perf_ctrls.desired_perf = des_perf; | |
| 200 | + | |
| 201 | + cppc_set_perf(cpudata->cpu, &perf_ctrls); | |
| 202 | +} | |
| 203 | + | |
| 204 | +DEFINE_STATIC_CALL(amd_pstate_update_perf, pstate_update_perf); | |
| 205 | + | |
| 206 | +static inline void amd_pstate_update_perf(struct amd_cpudata *cpudata, | |
| 207 | + u32 min_perf, u32 des_perf, | |
| 208 | + u32 max_perf, bool fast_switch) | |
| 209 | +{ | |
| 210 | + static_call(amd_pstate_update_perf)(cpudata, min_perf, des_perf, | |
| 211 | + max_perf, fast_switch); | |
| 212 | +} | |
| 213 | + | |
| 214 | +static void amd_pstate_update(struct amd_cpudata *cpudata, u32 min_perf, | |
| 215 | + u32 des_perf, u32 max_perf, bool fast_switch) | |
| 216 | +{ | |
| 217 | + u64 prev = READ_ONCE(cpudata->cppc_req_cached); | |
| 218 | + u64 value = prev; | |
| 219 | + | |
| 220 | + value &= ~AMD_CPPC_MIN_PERF(~0L); | |
| 221 | + value |= AMD_CPPC_MIN_PERF(min_perf); | |
| 222 | + | |
| 223 | + value &= ~AMD_CPPC_DES_PERF(~0L); | |
| 224 | + value |= AMD_CPPC_DES_PERF(des_perf); | |
| 225 | + | |
| 226 | + value &= ~AMD_CPPC_MAX_PERF(~0L); | |
| 227 | + value |= AMD_CPPC_MAX_PERF(max_perf); | |
| 228 | + | |
| 229 | + trace_amd_pstate_perf(min_perf, des_perf, max_perf, | |
| 230 | + cpudata->cpu, (value != prev), fast_switch); | |
| 231 | + | |
| 232 | + if (value == prev) | |
| 233 | + return; | |
| 234 | + | |
| 235 | + WRITE_ONCE(cpudata->cppc_req_cached, value); | |
| 236 | + | |
| 237 | + amd_pstate_update_perf(cpudata, min_perf, des_perf, | |
| 238 | + max_perf, fast_switch); | |
| 239 | +} | |
| 240 | + | |
| 241 | +static int amd_pstate_verify(struct cpufreq_policy_data *policy) | |
| 242 | +{ | |
| 243 | + cpufreq_verify_within_cpu_limits(policy); | |
| 244 | + | |
| 245 | + return 0; | |
| 246 | +} | |
| 247 | + | |
| 248 | +static int amd_pstate_target(struct cpufreq_policy *policy, | |
| 249 | + unsigned int target_freq, | |
| 250 | + unsigned int relation) | |
| 251 | +{ | |
| 252 | + struct cpufreq_freqs freqs; | |
| 253 | + struct amd_cpudata *cpudata = policy->driver_data; | |
| 254 | + unsigned long max_perf, min_perf, des_perf, cap_perf; | |
| 255 | + | |
| 256 | + if (!cpudata->max_freq) | |
| 257 | + return -ENODEV; | |
| 258 | + | |
| 259 | + cap_perf = READ_ONCE(cpudata->highest_perf); | |
| 260 | + min_perf = READ_ONCE(cpudata->lowest_nonlinear_perf); | |
| 261 | + max_perf = cap_perf; | |
| 262 | + | |
| 263 | + freqs.old = policy->cur; | |
| 264 | + freqs.new = target_freq; | |
| 265 | + | |
| 266 | + des_perf = DIV_ROUND_CLOSEST(target_freq * cap_perf, | |
| 267 | + cpudata->max_freq); | |
| 268 | + | |
| 269 | + cpufreq_freq_transition_begin(policy, &freqs); | |
| 270 | + amd_pstate_update(cpudata, min_perf, des_perf, | |
| 271 | + max_perf, false); | |
| 272 | + cpufreq_freq_transition_end(policy, &freqs, false); | |
| 273 | + | |
| 274 | + return 0; | |
| 275 | +} | |
| 276 | + | |
| 277 | +static void amd_pstate_adjust_perf(unsigned int cpu, | |
| 278 | + unsigned long _min_perf, | |
| 279 | + unsigned long target_perf, | |
| 280 | + unsigned long capacity) | |
| 281 | +{ | |
| 282 | + unsigned long max_perf, min_perf, des_perf, | |
| 283 | + cap_perf, lowest_nonlinear_perf; | |
| 284 | + struct cpufreq_policy *policy = cpufreq_cpu_get(cpu); | |
| 285 | + struct amd_cpudata *cpudata = policy->driver_data; | |
| 286 | + | |
| 287 | + cap_perf = READ_ONCE(cpudata->highest_perf); | |
| 288 | + lowest_nonlinear_perf = READ_ONCE(cpudata->lowest_nonlinear_perf); | |
| 289 | + | |
| 290 | + des_perf = cap_perf; | |
| 291 | + if (target_perf < capacity) | |
| 292 | + des_perf = DIV_ROUND_UP(cap_perf * target_perf, capacity); | |
| 293 | + | |
| 294 | + min_perf = READ_ONCE(cpudata->highest_perf); | |
| 295 | + if (_min_perf < capacity) | |
| 296 | + min_perf = DIV_ROUND_UP(cap_perf * _min_perf, capacity); | |
| 297 | + | |
| 298 | + if (min_perf < lowest_nonlinear_perf) | |
| 299 | + min_perf = lowest_nonlinear_perf; | |
| 300 | + | |
| 301 | + max_perf = cap_perf; | |
| 302 | + if (max_perf < min_perf) | |
| 303 | + max_perf = min_perf; | |
| 304 | + | |
| 305 | + des_perf = clamp_t(unsigned long, des_perf, min_perf, max_perf); | |
| 306 | + | |
| 307 | + amd_pstate_update(cpudata, min_perf, des_perf, max_perf, true); | |
| 308 | +} | |
| 309 | + | |
| 310 | +static int amd_get_min_freq(struct amd_cpudata *cpudata) | |
| 311 | +{ | |
| 312 | + struct cppc_perf_caps cppc_perf; | |
| 313 | + | |
| 314 | + int ret = cppc_get_perf_caps(cpudata->cpu, &cppc_perf); | |
| 315 | + if (ret) | |
| 316 | + return ret; | |
| 317 | + | |
| 318 | + /* Switch to khz */ | |
| 319 | + return cppc_perf.lowest_freq * 1000; | |
| 320 | +} | |
| 321 | + | |
| 322 | +static int amd_get_max_freq(struct amd_cpudata *cpudata) | |
| 323 | +{ | |
| 324 | + struct cppc_perf_caps cppc_perf; | |
| 325 | + u32 max_perf, max_freq, nominal_freq, nominal_perf; | |
| 326 | + u64 boost_ratio; | |
| 327 | + | |
| 328 | + int ret = cppc_get_perf_caps(cpudata->cpu, &cppc_perf); | |
| 329 | + if (ret) | |
| 330 | + return ret; | |
| 331 | + | |
| 332 | + nominal_freq = cppc_perf.nominal_freq; | |
| 333 | + nominal_perf = READ_ONCE(cpudata->nominal_perf); | |
| 334 | + max_perf = READ_ONCE(cpudata->highest_perf); | |
| 335 | + | |
| 336 | + boost_ratio = div_u64(max_perf << SCHED_CAPACITY_SHIFT, | |
| 337 | + nominal_perf); | |
| 338 | + | |
| 339 | + max_freq = nominal_freq * boost_ratio >> SCHED_CAPACITY_SHIFT; | |
| 340 | + | |
| 341 | + /* Switch to khz */ | |
| 342 | + return max_freq * 1000; | |
| 343 | +} | |
| 344 | + | |
| 345 | +static int amd_get_nominal_freq(struct amd_cpudata *cpudata) | |
| 346 | +{ | |
| 347 | + struct cppc_perf_caps cppc_perf; | |
| 348 | + | |
| 349 | + int ret = cppc_get_perf_caps(cpudata->cpu, &cppc_perf); | |
| 350 | + if (ret) | |
| 351 | + return ret; | |
| 352 | + | |
| 353 | + /* Switch to khz */ | |
| 354 | + return cppc_perf.nominal_freq * 1000; | |
| 355 | +} | |
| 356 | + | |
| 357 | +static int amd_get_lowest_nonlinear_freq(struct amd_cpudata *cpudata) | |
| 358 | +{ | |
| 359 | + struct cppc_perf_caps cppc_perf; | |
| 360 | + u32 lowest_nonlinear_freq, lowest_nonlinear_perf, | |
| 361 | + nominal_freq, nominal_perf; | |
| 362 | + u64 lowest_nonlinear_ratio; | |
| 363 | + | |
| 364 | + int ret = cppc_get_perf_caps(cpudata->cpu, &cppc_perf); | |
| 365 | + if (ret) | |
| 366 | + return ret; | |
| 367 | + | |
| 368 | + nominal_freq = cppc_perf.nominal_freq; | |
| 369 | + nominal_perf = READ_ONCE(cpudata->nominal_perf); | |
| 370 | + | |
| 371 | + lowest_nonlinear_perf = cppc_perf.lowest_nonlinear_perf; | |
| 372 | + | |
| 373 | + lowest_nonlinear_ratio = div_u64(lowest_nonlinear_perf << SCHED_CAPACITY_SHIFT, | |
| 374 | + nominal_perf); | |
| 375 | + | |
| 376 | + lowest_nonlinear_freq = nominal_freq * lowest_nonlinear_ratio >> SCHED_CAPACITY_SHIFT; | |
| 377 | + | |
| 378 | + /* Switch to khz */ | |
| 379 | + return lowest_nonlinear_freq * 1000; | |
| 380 | +} | |
| 381 | + | |
| 382 | +static int amd_pstate_set_boost(struct cpufreq_policy *policy, int state) | |
| 383 | +{ | |
| 384 | + struct amd_cpudata *cpudata = policy->driver_data; | |
| 385 | + int ret; | |
| 386 | + | |
| 387 | + if (!cpudata->boost_supported) { | |
| 388 | + pr_err("Boost mode is not supported by this processor or SBIOS\n"); | |
| 389 | + return -EINVAL; | |
| 390 | + } | |
| 391 | + | |
| 392 | + if (state) | |
| 393 | + policy->cpuinfo.max_freq = cpudata->max_freq; | |
| 394 | + else | |
| 395 | + policy->cpuinfo.max_freq = cpudata->nominal_freq; | |
| 396 | + | |
| 397 | + policy->max = policy->cpuinfo.max_freq; | |
| 398 | + | |
| 399 | + ret = freq_qos_update_request(&cpudata->req[1], | |
| 400 | + policy->cpuinfo.max_freq); | |
| 401 | + if (ret < 0) | |
| 402 | + return ret; | |
| 403 | + | |
| 404 | + return 0; | |
| 405 | +} | |
| 406 | + | |
| 407 | +static void amd_pstate_boost_init(struct amd_cpudata *cpudata) | |
| 408 | +{ | |
| 409 | + u32 highest_perf, nominal_perf; | |
| 410 | + | |
| 411 | + highest_perf = READ_ONCE(cpudata->highest_perf); | |
| 412 | + nominal_perf = READ_ONCE(cpudata->nominal_perf); | |
| 413 | + | |
| 414 | + if (highest_perf <= nominal_perf) | |
| 415 | + return; | |
| 416 | + | |
| 417 | + cpudata->boost_supported = true; | |
| 418 | + amd_pstate_driver.boost_enabled = true; | |
| 419 | +} | |
| 420 | + | |
| 421 | +static int amd_pstate_cpu_init(struct cpufreq_policy *policy) | |
| 422 | +{ | |
| 423 | + int min_freq, max_freq, nominal_freq, lowest_nonlinear_freq, ret; | |
| 424 | + struct device *dev; | |
| 425 | + struct amd_cpudata *cpudata; | |
| 426 | + | |
| 427 | + dev = get_cpu_device(policy->cpu); | |
| 428 | + if (!dev) | |
| 429 | + return -ENODEV; | |
| 430 | + | |
| 431 | + cpudata = kzalloc(sizeof(*cpudata), GFP_KERNEL); | |
| 432 | + if (!cpudata) | |
| 433 | + return -ENOMEM; | |
| 434 | + | |
| 435 | + cpudata->cpu = policy->cpu; | |
| 436 | + | |
| 437 | + ret = amd_pstate_init_perf(cpudata); | |
| 438 | + if (ret) | |
| 439 | + goto free_cpudata1; | |
| 440 | + | |
| 441 | + min_freq = amd_get_min_freq(cpudata); | |
| 442 | + max_freq = amd_get_max_freq(cpudata); | |
| 443 | + nominal_freq = amd_get_nominal_freq(cpudata); | |
| 444 | + lowest_nonlinear_freq = amd_get_lowest_nonlinear_freq(cpudata); | |
| 445 | + | |
| 446 | + if (min_freq < 0 || max_freq < 0 || min_freq > max_freq) { | |
| 447 | + dev_err(dev, "min_freq(%d) or max_freq(%d) value is incorrect\n", | |
| 448 | + min_freq, max_freq); | |
| 449 | + ret = -EINVAL; | |
| 450 | + goto free_cpudata1; | |
| 451 | + } | |
| 452 | + | |
| 453 | + policy->cpuinfo.transition_latency = AMD_PSTATE_TRANSITION_LATENCY; | |
| 454 | + policy->transition_delay_us = AMD_PSTATE_TRANSITION_DELAY; | |
| 455 | + | |
| 456 | + policy->min = min_freq; | |
| 457 | + policy->max = max_freq; | |
| 458 | + | |
| 459 | + policy->cpuinfo.min_freq = min_freq; | |
| 460 | + policy->cpuinfo.max_freq = max_freq; | |
| 461 | + | |
| 462 | + /* It will be updated by governor */ | |
| 463 | + policy->cur = policy->cpuinfo.min_freq; | |
| 464 | + | |
| 465 | + if (boot_cpu_has(X86_FEATURE_CPPC)) | |
| 466 | + policy->fast_switch_possible = true; | |
| 467 | + | |
| 468 | + ret = freq_qos_add_request(&policy->constraints, &cpudata->req[0], | |
| 469 | + FREQ_QOS_MIN, policy->cpuinfo.min_freq); | |
| 470 | + if (ret < 0) { | |
| 471 | + dev_err(dev, "Failed to add min-freq constraint (%d)\n", ret); | |
| 472 | + goto free_cpudata1; | |
| 473 | + } | |
| 474 | + | |
| 475 | + ret = freq_qos_add_request(&policy->constraints, &cpudata->req[1], | |
| 476 | + FREQ_QOS_MAX, policy->cpuinfo.max_freq); | |
| 477 | + if (ret < 0) { | |
| 478 | + dev_err(dev, "Failed to add max-freq constraint (%d)\n", ret); | |
| 479 | + goto free_cpudata2; | |
| 480 | + } | |
| 481 | + | |
| 482 | + /* Initial processor data capability frequencies */ | |
| 483 | + cpudata->max_freq = max_freq; | |
| 484 | + cpudata->min_freq = min_freq; | |
| 485 | + cpudata->nominal_freq = nominal_freq; | |
| 486 | + cpudata->lowest_nonlinear_freq = lowest_nonlinear_freq; | |
| 487 | + | |
| 488 | + policy->driver_data = cpudata; | |
| 489 | + | |
| 490 | + amd_pstate_boost_init(cpudata); | |
| 491 | + | |
| 492 | + return 0; | |
| 493 | + | |
| 494 | +free_cpudata2: | |
| 495 | + freq_qos_remove_request(&cpudata->req[0]); | |
| 496 | +free_cpudata1: | |
| 497 | + kfree(cpudata); | |
| 498 | + return ret; | |
| 499 | +} | |
| 500 | + | |
| 501 | +static int amd_pstate_cpu_exit(struct cpufreq_policy *policy) | |
| 502 | +{ | |
| 503 | + struct amd_cpudata *cpudata; | |
| 504 | + | |
| 505 | + cpudata = policy->driver_data; | |
| 506 | + | |
| 507 | + freq_qos_remove_request(&cpudata->req[1]); | |
| 508 | + freq_qos_remove_request(&cpudata->req[0]); | |
| 509 | + kfree(cpudata); | |
| 510 | + | |
| 511 | + return 0; | |
| 512 | +} | |
| 513 | + | |
| 514 | +/* Sysfs attributes */ | |
| 515 | + | |
| 516 | +/* | |
| 517 | + * This frequency is to indicate the maximum hardware frequency. | |
| 518 | + * If boost is not active but supported, the frequency will be larger than the | |
| 519 | + * one in cpuinfo. | |
| 520 | + */ | |
| 521 | +static ssize_t show_amd_pstate_max_freq(struct cpufreq_policy *policy, | |
| 522 | + char *buf) | |
| 523 | +{ | |
| 524 | + int max_freq; | |
| 525 | + struct amd_cpudata *cpudata; | |
| 526 | + | |
| 527 | + cpudata = policy->driver_data; | |
| 528 | + | |
| 529 | + max_freq = amd_get_max_freq(cpudata); | |
| 530 | + if (max_freq < 0) | |
| 531 | + return max_freq; | |
| 532 | + | |
| 533 | + return sprintf(&buf[0], "%u\n", max_freq); | |
| 534 | +} | |
| 535 | + | |
| 536 | +static ssize_t show_amd_pstate_lowest_nonlinear_freq(struct cpufreq_policy *policy, | |
| 537 | + char *buf) | |
| 538 | +{ | |
| 539 | + int freq; | |
| 540 | + struct amd_cpudata *cpudata; | |
| 541 | + | |
| 542 | + cpudata = policy->driver_data; | |
| 543 | + | |
| 544 | + freq = amd_get_lowest_nonlinear_freq(cpudata); | |
| 545 | + if (freq < 0) | |
| 546 | + return freq; | |
| 547 | + | |
| 548 | + return sprintf(&buf[0], "%u\n", freq); | |
| 549 | +} | |
| 550 | + | |
| 551 | +/* | |
| 552 | + * In some of ASICs, the highest_perf is not the one in the _CPC table, so we | |
| 553 | + * need to expose it to sysfs. | |
| 554 | + */ | |
| 555 | +static ssize_t show_amd_pstate_highest_perf(struct cpufreq_policy *policy, | |
| 556 | + char *buf) | |
| 557 | +{ | |
| 558 | + u32 perf; | |
| 559 | + struct amd_cpudata *cpudata = policy->driver_data; | |
| 560 | + | |
| 561 | + perf = READ_ONCE(cpudata->highest_perf); | |
| 562 | + | |
| 563 | + return sprintf(&buf[0], "%u\n", perf); | |
| 564 | +} | |
| 565 | + | |
| 566 | +cpufreq_freq_attr_ro(amd_pstate_max_freq); | |
| 567 | +cpufreq_freq_attr_ro(amd_pstate_lowest_nonlinear_freq); | |
| 568 | + | |
| 569 | +cpufreq_freq_attr_ro(amd_pstate_highest_perf); | |
| 570 | + | |
| 571 | +static struct freq_attr *amd_pstate_attr[] = { | |
| 572 | + &amd_pstate_max_freq, | |
| 573 | + &amd_pstate_lowest_nonlinear_freq, | |
| 574 | + &amd_pstate_highest_perf, | |
| 575 | + NULL, | |
| 576 | +}; | |
| 577 | + | |
| 578 | +static struct cpufreq_driver amd_pstate_driver = { | |
| 579 | + .flags = CPUFREQ_CONST_LOOPS | CPUFREQ_NEED_UPDATE_LIMITS, | |
| 580 | + .verify = amd_pstate_verify, | |
| 581 | + .target = amd_pstate_target, | |
| 582 | + .init = amd_pstate_cpu_init, | |
| 583 | + .exit = amd_pstate_cpu_exit, | |
| 584 | + .set_boost = amd_pstate_set_boost, | |
| 585 | + .name = "amd-pstate", | |
| 586 | + .attr = amd_pstate_attr, | |
| 587 | +}; | |
| 588 | + | |
| 589 | +static int __init amd_pstate_init(void) | |
| 590 | +{ | |
| 591 | + int ret; | |
| 592 | + | |
| 593 | + if (boot_cpu_data.x86_vendor != X86_VENDOR_AMD) | |
| 594 | + return -ENODEV; | |
| 595 | + | |
| 596 | + if (!acpi_cpc_valid()) { | |
| 597 | + pr_debug("the _CPC object is not present in SBIOS\n"); | |
| 598 | + return -ENODEV; | |
| 599 | + } | |
| 600 | + | |
| 601 | + /* don't keep reloading if cpufreq_driver exists */ | |
| 602 | + if (cpufreq_get_current_driver()) | |
| 603 | + return -EEXIST; | |
| 604 | + | |
| 605 | + /* capability check */ | |
| 606 | + if (boot_cpu_has(X86_FEATURE_CPPC)) { | |
| 607 | + pr_debug("AMD CPPC MSR based functionality is supported\n"); | |
| 608 | + amd_pstate_driver.adjust_perf = amd_pstate_adjust_perf; | |
| 609 | + } else if (shared_mem) { | |
| 610 | + static_call_update(amd_pstate_enable, cppc_enable); | |
| 611 | + static_call_update(amd_pstate_init_perf, cppc_init_perf); | |
| 612 | + static_call_update(amd_pstate_update_perf, cppc_update_perf); | |
| 613 | + } else { | |
| 614 | + pr_info("This processor supports shared memory solution, you can enable it with amd_pstate.shared_mem=1\n"); | |
| 615 | + return -ENODEV; | |
| 616 | + } | |
| 617 | + | |
| 618 | + /* enable amd pstate feature */ | |
| 619 | + ret = amd_pstate_enable(true); | |
| 620 | + if (ret) { | |
| 621 | + pr_err("failed to enable amd-pstate with return %d\n", ret); | |
| 622 | + return ret; | |
| 623 | + } | |
| 624 | + | |
| 625 | + ret = cpufreq_register_driver(&amd_pstate_driver); | |
| 626 | + if (ret) | |
| 627 | + pr_err("failed to register amd_pstate_driver with return %d\n", | |
| 628 | + ret); | |
| 629 | + | |
| 630 | + return ret; | |
| 631 | +} | |
| 632 | + | |
| 633 | +static void __exit amd_pstate_exit(void) | |
| 634 | +{ | |
| 635 | + cpufreq_unregister_driver(&amd_pstate_driver); | |
| 636 | + | |
| 637 | + amd_pstate_enable(false); | |
| 638 | +} | |
| 639 | + | |
| 640 | +module_init(amd_pstate_init); | |
| 641 | +module_exit(amd_pstate_exit); | |
| 642 | + | |
| 643 | +MODULE_AUTHOR("Huang Rui <ray.huang@amd.com>"); | |
| 644 | +MODULE_DESCRIPTION("AMD Processor P-state Frequency Driver"); | |
| 645 | +MODULE_LICENSE("GPL"); | |
| @@ -924,7 +924,7 @@ cpufreq_freq_attr_rw(scaling_max_freq); · 923 unmodified lines | ||
| 924 | 924 | cpufreq_freq_attr_rw(scaling_governor); |
| 925 | 925 | cpufreq_freq_attr_rw(scaling_setspeed); |
| 926 | 926 | |
| 927 | −static struct attribute *default_attrs[] = { | |
| 927 | +static struct attribute *cpufreq_attrs[] = { | |
| 928 | 928 | &cpuinfo_min_freq.attr, |
| 929 | 929 | &cpuinfo_max_freq.attr, |
| 930 | 930 | &cpuinfo_transition_latency.attr, |
| @@ -938,6 +938,7 @@ static struct attribute *default_attrs[] = { · 7 unmodified lines | ||
| 938 | 938 | &scaling_setspeed.attr, |
| 939 | 939 | NULL |
| 940 | 940 | }; |
| 941 | +ATTRIBUTE_GROUPS(cpufreq); | |
| 941 | 942 | |
| 942 | 943 | #define to_policy(k) container_of(k, struct cpufreq_policy, kobj) |
| 943 | 944 | #define to_attr(a) container_of(a, struct freq_attr, attr) |
| @@ -1000,7 +1001,7 @@ static const struct sysfs_ops sysfs_ops = { · 56 unmodified lines | ||
| 1000 | 1001 | |
| 1001 | 1002 | static struct kobj_type ktype_cpufreq = { |
| 1002 | 1003 | .sysfs_ops = &sysfs_ops, |
| 1003 | − .default_attrs = default_attrs, | |
| 1004 | + .default_groups = cpufreq_groups, | |
| 1004 | 1005 | .release = cpufreq_sysfs_release, |
| 1005 | 1006 | }; |
| 1006 | 1007 | |
| @@ -1403,7 +1404,7 @@ static int cpufreq_online(unsigned int cpu) · 396 unmodified lines | ||
| 1403 | 1404 | |
| 1404 | 1405 | ret = freq_qos_add_request(&policy->constraints, |
| 1405 | 1406 | policy->min_freq_req, FREQ_QOS_MIN, |
| 1406 | − policy->min); | |
| 1407 | + FREQ_QOS_MIN_DEFAULT_VALUE); | |
| 1407 | 1408 | if (ret < 0) { |
| 1408 | 1409 | /* |
| 1409 | 1410 | * So we don't call freq_qos_remove_request() for an |
| @@ -1423,7 +1424,7 @@ static int cpufreq_online(unsigned int cpu) · 13 unmodified lines | ||
| 1423 | 1424 | |
| 1424 | 1425 | ret = freq_qos_add_request(&policy->constraints, |
| 1425 | 1426 | policy->max_freq_req, FREQ_QOS_MAX, |
| 1426 | − policy->max); | |
| 1427 | + FREQ_QOS_MAX_DEFAULT_VALUE); | |
| 1427 | 1428 | if (ret < 0) { |
| 1428 | 1429 | policy->max_freq_req = NULL; |
| 1429 | 1430 | goto out_destroy_policy; |
| @@ -257,7 +257,7 @@ gov_attr_rw(ignore_nice_load); · 256 unmodified lines | ||
| 257 | 257 | gov_attr_rw(down_threshold); |
| 258 | 258 | gov_attr_rw(freq_step); |
| 259 | 259 | |
| 260 | −static struct attribute *cs_attributes[] = { | |
| 260 | +static struct attribute *cs_attrs[] = { | |
| 261 | 261 | &sampling_rate.attr, |
| 262 | 262 | &sampling_down_factor.attr, |
| 263 | 263 | &up_threshold.attr, |
| @@ -266,6 +266,7 @@ static struct attribute *cs_attributes[] = { · 2 unmodified lines | ||
| 266 | 266 | &freq_step.attr, |
| 267 | 267 | NULL |
| 268 | 268 | }; |
| 269 | +ATTRIBUTE_GROUPS(cs); | |
| 269 | 270 | |
| 270 | 271 | /************************** sysfs end ************************/ |
| 271 | 272 | |
| @@ -315,7 +316,7 @@ static void cs_start(struct cpufreq_policy *policy) · 43 unmodified lines | ||
| 315 | 316 | |
| 316 | 317 | static struct dbs_governor cs_governor = { |
| 317 | 318 | .gov = CPUFREQ_DBS_GOVERNOR_INITIALIZER("conservative"), |
| 318 | − .kobj_type = { .default_attrs = cs_attributes }, | |
| 319 | + .kobj_type = { .default_groups = cs_groups }, | |
| 319 | 320 | .gov_dbs_update = cs_dbs_update, |
| 320 | 321 | .alloc = cs_alloc, |
| 321 | 322 | .free = cs_free, |
| @@ -328,7 +328,7 @@ gov_attr_rw(sampling_down_factor); · 327 unmodified lines | ||
| 328 | 328 | gov_attr_rw(ignore_nice_load); |
| 329 | 329 | gov_attr_rw(powersave_bias); |
| 330 | 330 | |
| 331 | −static struct attribute *od_attributes[] = { | |
| 331 | +static struct attribute *od_attrs[] = { | |
| 332 | 332 | &sampling_rate.attr, |
| 333 | 333 | &up_threshold.attr, |
| 334 | 334 | &sampling_down_factor.attr, |
| @@ -337,6 +337,7 @@ static struct attribute *od_attributes[] = { · 2 unmodified lines | ||
| 337 | 337 | &io_is_busy.attr, |
| 338 | 338 | NULL |
| 339 | 339 | }; |
| 340 | +ATTRIBUTE_GROUPS(od); | |
| 340 | 341 | |
| 341 | 342 | /************************** sysfs end ************************/ |
| 342 | 343 | |
| @@ -401,7 +402,7 @@ static struct od_ops od_ops = { · 58 unmodified lines | ||
| 401 | 402 | |
| 402 | 403 | static struct dbs_governor od_dbs_gov = { |
| 403 | 404 | .gov = CPUFREQ_DBS_GOVERNOR_INITIALIZER("ondemand"), |
| 404 | − .kobj_type = { .default_attrs = od_attributes }, | |
| 405 | + .kobj_type = { .default_groups = od_groups }, | |
| 405 | 406 | .gov_dbs_update = od_dbs_update, |
| 406 | 407 | .alloc = od_alloc, |
| 407 | 408 | .free = od_free, |
| @@ -664,19 +664,29 @@ static int intel_pstate_set_epb(int cpu, s16 pref) · 663 unmodified lines | ||
| 664 | 664 | * 3 balance_power |
| 665 | 665 | * 4 power |
| 666 | 666 | */ |
| 667 | + | |
| 668 | +enum energy_perf_value_index { | |
| 669 | + EPP_INDEX_DEFAULT = 0, | |
| 670 | + EPP_INDEX_PERFORMANCE, | |
| 671 | + EPP_INDEX_BALANCE_PERFORMANCE, | |
| 672 | + EPP_INDEX_BALANCE_POWERSAVE, | |
| 673 | + EPP_INDEX_POWERSAVE, | |
| 674 | +}; | |
| 675 | + | |
| 667 | 676 | static const char * const energy_perf_strings[] = { |
| 668 | − "default", | |
| 669 | − "performance", | |
| 670 | − "balance_performance", | |
| 671 | − "balance_power", | |
| 672 | − "power", | |
| 677 | + [EPP_INDEX_DEFAULT] = "default", | |
| 678 | + [EPP_INDEX_PERFORMANCE] = "performance", | |
| 679 | + [EPP_INDEX_BALANCE_PERFORMANCE] = "balance_performance", | |
| 680 | + [EPP_INDEX_BALANCE_POWERSAVE] = "balance_power", | |
| 681 | + [EPP_INDEX_POWERSAVE] = "power", | |
| 673 | 682 | NULL |
| 674 | 683 | }; |
| 675 | −static const unsigned int epp_values[] = { | |
| 676 | − HWP_EPP_PERFORMANCE, | |
| 677 | − HWP_EPP_BALANCE_PERFORMANCE, | |
| 678 | − HWP_EPP_BALANCE_POWERSAVE, | |
| 679 | − HWP_EPP_POWERSAVE | |
| 684 | +static unsigned int epp_values[] = { | |
| 685 | + [EPP_INDEX_DEFAULT] = 0, /* Unused index */ | |
| 686 | + [EPP_INDEX_PERFORMANCE] = HWP_EPP_PERFORMANCE, | |
| 687 | + [EPP_INDEX_BALANCE_PERFORMANCE] = HWP_EPP_BALANCE_PERFORMANCE, | |
| 688 | + [EPP_INDEX_BALANCE_POWERSAVE] = HWP_EPP_BALANCE_POWERSAVE, | |
| 689 | + [EPP_INDEX_POWERSAVE] = HWP_EPP_POWERSAVE, | |
| 680 | 690 | }; |
| 681 | 691 | |
| 682 | 692 | static int intel_pstate_get_energy_pref_index(struct cpudata *cpu_data, int *raw_epp) |
| @@ -690,14 +700,14 @@ static int intel_pstate_get_energy_pref_index(struct cpudata *cpu_data, int *raw · 7 unmodified lines | ||
| 690 | 700 | return epp; |
| 691 | 701 | |
| 692 | 702 | if (boot_cpu_has(X86_FEATURE_HWP_EPP)) { |
| 693 | − if (epp == HWP_EPP_PERFORMANCE) | |
| 694 | − return 1; | |
| 695 | − if (epp == HWP_EPP_BALANCE_PERFORMANCE) | |
| 696 | − return 2; | |
| 697 | − if (epp == HWP_EPP_BALANCE_POWERSAVE) | |
| 698 | − return 3; | |
| 699 | − if (epp == HWP_EPP_POWERSAVE) | |
| 700 | − return 4; | |
| 703 | + if (epp == epp_values[EPP_INDEX_PERFORMANCE]) | |
| 704 | + return EPP_INDEX_PERFORMANCE; | |
| 705 | + if (epp == epp_values[EPP_INDEX_BALANCE_PERFORMANCE]) | |
| 706 | + return EPP_INDEX_BALANCE_PERFORMANCE; | |
| 707 | + if (epp == epp_values[EPP_INDEX_BALANCE_POWERSAVE]) | |
| 708 | + return EPP_INDEX_BALANCE_POWERSAVE; | |
| 709 | + if (epp == epp_values[EPP_INDEX_POWERSAVE]) | |
| 710 | + return EPP_INDEX_POWERSAVE; | |
| 701 | 711 | *raw_epp = epp; |
| 702 | 712 | return 0; |
| 703 | 713 | } else if (boot_cpu_has(X86_FEATURE_EPB)) { |
| @@ -757,7 +767,7 @@ static int intel_pstate_set_energy_pref_index(struct cpudata *cpu_data, · 53 unmodified lines | ||
| 757 | 767 | if (use_raw) |
| 758 | 768 | epp = raw_epp; |
| 759 | 769 | else if (epp == -EINVAL) |
| 760 | − epp = epp_values[pref_index - 1]; | |
| 770 | + epp = epp_values[pref_index]; | |
| 761 | 771 | |
| 762 | 772 | /* |
| 763 | 773 | * To avoid confusion, refuse to set EPP to any values different |
| @@ -843,7 +853,7 @@ static ssize_t store_energy_performance_preference( · 79 unmodified lines | ||
| 843 | 853 | * upfront. |
| 844 | 854 | */ |
| 845 | 855 | if (!raw) |
| 846 | − epp = ret ? epp_values[ret - 1] : cpu->epp_default; | |
| 856 | + epp = ret ? epp_values[ret] : cpu->epp_default; | |
| 847 | 857 | |
| 848 | 858 | if (cpu->epp_cached != epp) { |
| 849 | 859 | int err; |
| @@ -1124,19 +1134,22 @@ static void intel_pstate_update_policies(void) · 274 unmodified lines | ||
| 1124 | 1134 | cpufreq_update_policy(cpu); |
| 1125 | 1135 | } |
| 1126 | 1136 | |
| 1137 | +static void __intel_pstate_update_max_freq(struct cpudata *cpudata, | |
| 1138 | + struct cpufreq_policy *policy) | |
| 1139 | +{ | |
| 1140 | + policy->cpuinfo.max_freq = global.turbo_disabled_mf ? | |
| 1141 | + cpudata->pstate.max_freq : cpudata->pstate.turbo_freq; | |
| 1142 | + refresh_frequency_limits(policy); | |
| 1143 | +} | |
| 1144 | + | |
| 1127 | 1145 | static void intel_pstate_update_max_freq(unsigned int cpu) |
| 1128 | 1146 | { |
| 1129 | 1147 | struct cpufreq_policy *policy = cpufreq_cpu_acquire(cpu); |
| 1130 | − struct cpudata *cpudata; | |
| 1131 | 1148 | |
| 1132 | 1149 | if (!policy) |
| 1133 | 1150 | return; |
| 1134 | 1151 | |
| 1135 | − cpudata = all_cpu_data[cpu]; | |
| 1136 | − policy->cpuinfo.max_freq = global.turbo_disabled_mf ? | |
| 1137 | − cpudata->pstate.max_freq : cpudata->pstate.turbo_freq; | |
| 1138 | − | |
| 1139 | − refresh_frequency_limits(policy); | |
| 1152 | + __intel_pstate_update_max_freq(all_cpu_data[cpu], policy); | |
| 1140 | 1153 | |
| 1141 | 1154 | cpufreq_cpu_release(policy); |
| 1142 | 1155 | } |
| @@ -1584,8 +1597,15 @@ static void intel_pstate_notify_work(struct work_struct *work) · 441 unmodified lines | ||
| 1584 | 1597 | { |
| 1585 | 1598 | struct cpudata *cpudata = |
| 1586 | 1599 | container_of(to_delayed_work(work), struct cpudata, hwp_notify_work); |
| 1600 | + struct cpufreq_policy *policy = cpufreq_cpu_acquire(cpudata->cpu); | |
| 1601 | + | |
| 1602 | + if (policy) { | |
| 1603 | + intel_pstate_get_hwp_cap(cpudata); | |
| 1604 | + __intel_pstate_update_max_freq(cpudata, policy); | |
| 1605 | + | |
| 1606 | + cpufreq_cpu_release(policy); | |
| 1607 | + } | |
| 1587 | 1608 | |
| 1588 | − cpufreq_update_policy(cpudata->cpu); | |
| 1589 | 1609 | wrmsrl_on_cpu(cpudata->cpu, MSR_HWP_STATUS, 0); |
| 1590 | 1610 | } |
| 1591 | 1611 | |
| @@ -1679,10 +1699,18 @@ static void intel_pstate_hwp_enable(struct cpudata *cpudata) · 87 unmodified lines | ||
| 1679 | 1699 | wrmsrl_on_cpu(cpudata->cpu, MSR_HWP_INTERRUPT, 0x00); |
| 1680 | 1700 | |
| 1681 | 1701 | wrmsrl_on_cpu(cpudata->cpu, MSR_PM_ENABLE, 0x1); |
| 1682 | − if (cpudata->epp_default == -EINVAL) | |
| 1683 | − cpudata->epp_default = intel_pstate_get_epp(cpudata, 0); | |
| 1684 | 1702 | |
| 1685 | 1703 | intel_pstate_enable_hwp_interrupt(cpudata); |
| 1704 | + | |
| 1705 | + if (cpudata->epp_default >= 0) | |
| 1706 | + return; | |
| 1707 | + | |
| 1708 | + if (epp_values[EPP_INDEX_BALANCE_PERFORMANCE] == HWP_EPP_BALANCE_PERFORMANCE) { | |
| 1709 | + cpudata->epp_default = intel_pstate_get_epp(cpudata, 0); | |
| 1710 | + } else { | |
| 1711 | + cpudata->epp_default = epp_values[EPP_INDEX_BALANCE_PERFORMANCE]; | |
| 1712 | + intel_pstate_set_epp(cpudata, cpudata->epp_default); | |
| 1713 | + } | |
| 1686 | 1714 | } |
| 1687 | 1715 | |
| 1688 | 1716 | static int atom_get_min_pstate(void) |
| @@ -2486,18 +2514,14 @@ static void intel_pstate_update_perf_limits(struct cpudata *cpu, · 797 unmodified lines | ||
| 2486 | 2514 | * HWP needs some special consideration, because HWP_REQUEST uses |
| 2487 | 2515 | * abstract values to represent performance rather than pure ratios. |
| 2488 | 2516 | */ |
| 2489 | − if (hwp_active) { | |
| 2490 | − intel_pstate_get_hwp_cap(cpu); | |
| 2517 | + if (hwp_active && cpu->pstate.scaling != perf_ctl_scaling) { | |
| 2518 | + int scaling = cpu->pstate.scaling; | |
| 2519 | + int freq; | |
| 2491 | 2520 | |
| 2492 | − if (cpu->pstate.scaling != perf_ctl_scaling) { | |
| 2493 | − int scaling = cpu->pstate.scaling; | |
| 2494 | − int freq; | |
| 2495 | − | |
| 2496 | − freq = max_policy_perf * perf_ctl_scaling; | |
| 2497 | − max_policy_perf = DIV_ROUND_UP(freq, scaling); | |
| 2498 | − freq = min_policy_perf * perf_ctl_scaling; | |
| 2499 | − min_policy_perf = DIV_ROUND_UP(freq, scaling); | |
| 2500 | − } | |
| 2521 | + freq = max_policy_perf * perf_ctl_scaling; | |
| 2522 | + max_policy_perf = DIV_ROUND_UP(freq, scaling); | |
| 2523 | + freq = min_policy_perf * perf_ctl_scaling; | |
| 2524 | + min_policy_perf = DIV_ROUND_UP(freq, scaling); | |
| 2501 | 2525 | } |
| 2502 | 2526 | |
| 2503 | 2527 | pr_debug("cpu:%d min_policy_perf:%d max_policy_perf:%d\n", |
| @@ -3349,6 +3373,16 @@ static bool intel_pstate_hwp_is_enabled(void) · 845 unmodified lines | ||
| 3349 | 3373 | return !!(value & 0x1); |
| 3350 | 3374 | } |
| 3351 | 3375 | |
| 3376 | +static const struct x86_cpu_id intel_epp_balance_perf[] = { | |
| 3377 | + /* | |
| 3378 | + * Set EPP value as 102, this is the max suggested EPP | |
| 3379 | + * which can result in one core turbo frequency for | |
| 3380 | + * AlderLake Mobile CPUs. | |
| 3381 | + */ | |
| 3382 | + X86_MATCH_INTEL_FAM6_MODEL(ALDERLAKE_L, 102), | |
| 3383 | + {} | |
| 3384 | +}; | |
| 3385 | + | |
| 3352 | 3386 | static int __init intel_pstate_init(void) |
| 3353 | 3387 | { |
| 3354 | 3388 | static struct cpudata **_all_cpu_data; |
| @@ -3438,6 +3472,13 @@ hwp_cpu_matched: · 83 unmodified lines | ||
| 3438 | 3472 | |
| 3439 | 3473 | intel_pstate_sysfs_expose_params(); |
| 3440 | 3474 | |
| 3475 | + if (hwp_active) { | |
| 3476 | + const struct x86_cpu_id *id = x86_match_cpu(intel_epp_balance_perf); | |
| 3477 | + | |
| 3478 | + if (id) | |
| 3479 | + epp_values[EPP_INDEX_BALANCE_PERFORMANCE] = id->driver_data; | |
| 3480 | + } | |
| 3481 | + | |
| 3441 | 3482 | mutex_lock(&intel_pstate_driver_lock); |
| 3442 | 3483 | rc = intel_pstate_register_driver(default_driver); |
| 3443 | 3484 | mutex_unlock(&intel_pstate_driver_lock); |
| @@ -36,6 +36,8 @@ enum { · 35 unmodified lines | ||
| 36 | 36 | struct mtk_cpufreq_data { |
| 37 | 37 | struct cpufreq_frequency_table *table; |
| 38 | 38 | void __iomem *reg_bases[REG_ARRAY_SIZE]; |
| 39 | + struct resource *res; | |
| 40 | + void __iomem *base; | |
| 39 | 41 | int nr_opp; |
| 40 | 42 | }; |
| 41 | 43 | |
| @@ -156,6 +158,7 @@ static int mtk_cpu_resources_init(struct platform_device *pdev, · 114 unmodified lines | ||
| 156 | 158 | { |
| 157 | 159 | struct mtk_cpufreq_data *data; |
| 158 | 160 | struct device *dev = &pdev->dev; |
| 161 | + struct resource *res; | |
| 159 | 162 | void __iomem *base; |
| 160 | 163 | int ret, i; |
| 161 | 164 | int index; |
| @@ -170,9 +173,26 @@ static int mtk_cpu_resources_init(struct platform_device *pdev, · 8 unmodified lines | ||
| 170 | 173 | if (index < 0) |
| 171 | 174 | return index; |
| 172 | 175 | |
| 173 | − base = devm_platform_ioremap_resource(pdev, index); | |
| 174 | − if (IS_ERR(base)) | |
| 175 | − return PTR_ERR(base); | |
| 176 | + res = platform_get_resource(pdev, IORESOURCE_MEM, index); | |
| 177 | + if (!res) { | |
| 178 | + dev_err(dev, "failed to get mem resource %d\n", index); | |
| 179 | + return -ENODEV; | |
| 180 | + } | |
| 181 | + | |
| 182 | + if (!request_mem_region(res->start, resource_size(res), res->name)) { | |
| 183 | + dev_err(dev, "failed to request resource %pR\n", res); | |
| 184 | + return -EBUSY; | |
| 185 | + } | |
| 186 | + | |
| 187 | + base = ioremap(res->start, resource_size(res)); | |
| 188 | + if (!base) { | |
| 189 | + dev_err(dev, "failed to map resource %pR\n", res); | |
| 190 | + ret = -ENOMEM; | |
| 191 | + goto release_region; | |
| 192 | + } | |
| 193 | + | |
| 194 | + data->base = base; | |
| 195 | + data->res = res; | |
| 176 | 196 | |
| 177 | 197 | for (i = REG_FREQ_LUT_TABLE; i < REG_ARRAY_SIZE; i++) |
| 178 | 198 | data->reg_bases[i] = base + offsets[i]; |
| @@ -187,6 +207,9 @@ static int mtk_cpu_resources_init(struct platform_device *pdev, · 8 unmodified lines | ||
| 187 | 207 | policy->driver_data = data; |
| 188 | 208 | |
| 189 | 209 | return 0; |
| 210 | +release_region: | |
| 211 | + release_mem_region(res->start, resource_size(res)); | |
| 212 | + return ret; | |
| 190 | 213 | } |
| 191 | 214 | |
| 192 | 215 | static int mtk_cpufreq_hw_cpu_init(struct cpufreq_policy *policy) |
| @@ -233,9 +256,13 @@ static int mtk_cpufreq_hw_cpu_init(struct cpufreq_policy *policy) · 40 unmodified lines | ||
| 233 | 256 | static int mtk_cpufreq_hw_cpu_exit(struct cpufreq_policy *policy) |
| 234 | 257 | { |
| 235 | 258 | struct mtk_cpufreq_data *data = policy->driver_data; |
| 259 | + struct resource *res = data->res; | |
| 260 | + void __iomem *base = data->base; | |
| 236 | 261 | |
| 237 | 262 | /* HW should be in paused state now */ |
| 238 | 263 | writel_relaxed(0x0, data->reg_bases[REG_FREQ_ENABLE]); |
| 264 | + iounmap(base); | |
| 265 | + release_mem_region(res->start, resource_size(res)); | |
| 239 | 266 | |
| 240 | 267 | return 0; |
| 241 | 268 | } |
| @@ -46,6 +46,7 @@ struct qcom_cpufreq_data { · 45 unmodified lines | ||
| 46 | 46 | */ |
| 47 | 47 | struct mutex throttle_lock; |
| 48 | 48 | int throttle_irq; |
| 49 | + char irq_name[15]; | |
| 49 | 50 | bool cancel_throttle; |
| 50 | 51 | struct delayed_work throttle_work; |
| 51 | 52 | struct cpufreq_policy *policy; |
| @@ -275,10 +276,10 @@ static unsigned int qcom_lmh_get_throttle_freq(struct qcom_cpufreq_data *data) · 223 unmodified lines | ||
| 275 | 276 | |
| 276 | 277 | static void qcom_lmh_dcvs_notify(struct qcom_cpufreq_data *data) |
| 277 | 278 | { |
| 278 | − unsigned long max_capacity, capacity, freq_hz, throttled_freq; | |
| 279 | 279 | struct cpufreq_policy *policy = data->policy; |
| 280 | 280 | int cpu = cpumask_first(policy->cpus); |
| 281 | 281 | struct device *dev = get_cpu_device(cpu); |
| 282 | + unsigned long freq_hz, throttled_freq; | |
| 282 | 283 | struct dev_pm_opp *opp; |
| 283 | 284 | unsigned int freq; |
| 284 | 285 | |
| @@ -295,16 +296,8 @@ static void qcom_lmh_dcvs_notify(struct qcom_cpufreq_data *data) · 10 unmodified lines | ||
| 295 | 296 | |
| 296 | 297 | throttled_freq = freq_hz / HZ_PER_KHZ; |
| 297 | 298 | |
| 298 | − /* Update thermal pressure */ | |
| 299 | − | |
| 300 | − max_capacity = arch_scale_cpu_capacity(cpu); | |
| 301 | − capacity = mult_frac(max_capacity, throttled_freq, policy->cpuinfo.max_freq); | |
| 302 | − | |
| 303 | − /* Don't pass boost capacity to scheduler */ | |
| 304 | − if (capacity > max_capacity) | |
| 305 | − capacity = max_capacity; | |
| 306 | − | |
| 307 | − arch_set_thermal_pressure(policy->cpus, max_capacity - capacity); | |
| 299 | + /* Update thermal pressure (the boost frequencies are accepted) */ | |
| 300 | + arch_update_thermal_pressure(policy->related_cpus, throttled_freq); | |
| 308 | 301 | |
| 309 | 302 | /* |
| 310 | 303 | * In the unlikely case policy is unregistered do not enable |
| @@ -342,9 +335,9 @@ static irqreturn_t qcom_lmh_dcvs_handle_irq(int irq, void *data) · 31 unmodified lines | ||
| 342 | 335 | |
| 343 | 336 | /* Disable interrupt and enable polling */ |
| 344 | 337 | disable_irq_nosync(c_data->throttle_irq); |
| 345 | − qcom_lmh_dcvs_notify(c_data); | |
| 338 | + schedule_delayed_work(&c_data->throttle_work, 0); | |
| 346 | 339 | |
| 347 | − return 0; | |
| 340 | + return IRQ_HANDLED; | |
| 348 | 341 | } |
| 349 | 342 | |
| 350 | 343 | static const struct qcom_cpufreq_soc_data qcom_soc_data = { |
| @@ -375,16 +368,17 @@ static int qcom_cpufreq_hw_lmh_init(struct cpufreq_policy *policy, int index) · 24 unmodified lines | ||
| 375 | 368 | { |
| 376 | 369 | struct qcom_cpufreq_data *data = policy->driver_data; |
| 377 | 370 | struct platform_device *pdev = cpufreq_get_driver_data(); |
| 378 | − char irq_name[15]; | |
| 379 | 371 | int ret; |
| 380 | 372 | |
| 381 | 373 | /* |
| 382 | 374 | * Look for LMh interrupt. If no interrupt line is specified / |
| 383 | 375 | * if there is an error, allow cpufreq to be enabled as usual. |
| 384 | 376 | */ |
| 385 | − data->throttle_irq = platform_get_irq(pdev, index); | |
| 386 | − if (data->throttle_irq <= 0) | |
| 387 | − return data->throttle_irq == -EPROBE_DEFER ? -EPROBE_DEFER : 0; | |
| 377 | + data->throttle_irq = platform_get_irq_optional(pdev, index); | |
| 378 | + if (data->throttle_irq == -ENXIO) | |
| 379 | + return 0; | |
| 380 | + if (data->throttle_irq < 0) | |
| 381 | + return data->throttle_irq; | |
| 388 | 382 | |
| 389 | 383 | data->cancel_throttle = false; |
| 390 | 384 | data->policy = policy; |
| @@ -392,14 +386,19 @@ static int qcom_cpufreq_hw_lmh_init(struct cpufreq_policy *policy, int index) · 1 unmodified line | ||
| 392 | 386 | mutex_init(&data->throttle_lock); |
| 393 | 387 | INIT_DEFERRABLE_WORK(&data->throttle_work, qcom_lmh_dcvs_poll); |
| 394 | 388 | |
| 395 | − snprintf(irq_name, sizeof(irq_name), "dcvsh-irq-%u", policy->cpu); | |
| 389 | + snprintf(data->irq_name, sizeof(data->irq_name), "dcvsh-irq-%u", policy->cpu); | |
| 396 | 390 | ret = request_threaded_irq(data->throttle_irq, NULL, qcom_lmh_dcvs_handle_irq, |
| 397 | − IRQF_ONESHOT, irq_name, data); | |
| 391 | + IRQF_ONESHOT, data->irq_name, data); | |
| 398 | 392 | if (ret) { |
| 399 | − dev_err(&pdev->dev, "Error registering %s: %d\n", irq_name, ret); | |
| 393 | + dev_err(&pdev->dev, "Error registering %s: %d\n", data->irq_name, ret); | |
| 400 | 394 | return 0; |
| 401 | 395 | } |
| 402 | 396 | |
| 397 | + ret = irq_set_affinity_hint(data->throttle_irq, policy->cpus); | |
| 398 | + if (ret) | |
| 399 | + dev_err(&pdev->dev, "Failed to set CPU affinity of %s[%d]\n", | |
| 400 | + data->irq_name, data->throttle_irq); | |
| 401 | + | |
| 403 | 402 | return 0; |
| 404 | 403 | } |
| 405 | 404 | |
| @@ -34,7 +34,7 @@ · 33 unmodified lines | ||
| 34 | 34 | * 1) Energy break even point |
| 35 | 35 | * 2) Performance impact |
| 36 | 36 | * 3) Latency tolerance (from pmqos infrastructure) |
| 37 | − * These these three factors are treated independently. | |
| 37 | + * These three factors are treated independently. | |
| 38 | 38 | * |
| 39 | 39 | * Energy break even point |
| 40 | 40 | * ----------------------- |
| @@ -335,6 +335,7 @@ static struct attribute *cpuidle_state_default_attrs[] = { · 334 unmodified lines | ||
| 335 | 335 | &attr_default_status.attr, |
| 336 | 336 | NULL |
| 337 | 337 | }; |
| 338 | +ATTRIBUTE_GROUPS(cpuidle_state_default); | |
| 338 | 339 | |
| 339 | 340 | struct cpuidle_state_kobj { |
| 340 | 341 | struct cpuidle_state *state; |
| @@ -448,7 +449,7 @@ static void cpuidle_state_sysfs_release(struct kobject *kobj) · 107 unmodified lines | ||
| 448 | 449 | |
| 449 | 450 | static struct kobj_type ktype_state_cpuidle = { |
| 450 | 451 | .sysfs_ops = &cpuidle_state_sysfs_ops, |
| 451 | − .default_attrs = cpuidle_state_default_attrs, | |
| 452 | + .default_groups = cpuidle_state_default_groups, | |
| 452 | 453 | .release = cpuidle_state_sysfs_release, |
| 453 | 454 | }; |
| 454 | 455 | |
| @@ -505,7 +506,7 @@ error_state: · 50 unmodified lines | ||
| 505 | 506 | } |
| 506 | 507 | |
| 507 | 508 | /** |
| 508 | − * cpuidle_remove_driver_sysfs - removes the cpuidle states sysfs attributes | |
| 509 | + * cpuidle_remove_state_sysfs - removes the cpuidle states sysfs attributes | |
| 509 | 510 | * @device: the target device |
| 510 | 511 | */ |
| 511 | 512 | static void cpuidle_remove_state_sysfs(struct cpuidle_device *device) |
| @@ -591,10 +592,11 @@ static struct attribute *cpuidle_driver_default_attrs[] = { · 79 unmodified lines | ||
| 591 | 592 | &attr_driver_name.attr, |
| 592 | 593 | NULL |
| 593 | 594 | }; |
| 595 | +ATTRIBUTE_GROUPS(cpuidle_driver_default); | |
| 594 | 596 | |
| 595 | 597 | static struct kobj_type ktype_driver_cpuidle = { |
| 596 | 598 | .sysfs_ops = &cpuidle_driver_sysfs_ops, |
| 597 | − .default_attrs = cpuidle_driver_default_attrs, | |
| 599 | + .default_groups = cpuidle_driver_default_groups, | |
| 598 | 600 | .release = cpuidle_driver_sysfs_release, |
| 599 | 601 | }; |
| 600 | 602 | |
| @@ -132,6 +132,15 @@ config ARM_RK3399_DMC_DEVFREQ · 131 unmodified lines | ||
| 132 | 132 | It sets the frequency for the memory controller and reads the usage counts |
| 133 | 133 | from hardware. |
| 134 | 134 | |
| 135 | +config ARM_SUN8I_A33_MBUS_DEVFREQ | |
| 136 | + tristate "sun8i/sun50i MBUS DEVFREQ Driver" | |
| 137 | + depends on ARCH_SUNXI || COMPILE_TEST | |
| 138 | + depends on COMMON_CLK | |
| 139 | + select DEVFREQ_GOV_SIMPLE_ONDEMAND | |
| 140 | + help | |
| 141 | + This adds the DEVFREQ driver for the MBUS controller in some | |
| 142 | + Allwinner sun8i (A33 through H3) and sun50i (A64 and H5) SoCs. | |
| 143 | + | |
| 135 | 144 | source "drivers/devfreq/event/Kconfig" |
| 136 | 145 | |
| 137 | 146 | endif # PM_DEVFREQ |
| @@ -12,6 +12,7 @@ obj-$(CONFIG_ARM_EXYNOS_BUS_DEVFREQ) += exynos-bus.o · 11 unmodified lines | ||
| 12 | 12 | obj-$(CONFIG_ARM_IMX_BUS_DEVFREQ) += imx-bus.o |
| 13 | 13 | obj-$(CONFIG_ARM_IMX8M_DDRC_DEVFREQ) += imx8m-ddrc.o |
| 14 | 14 | obj-$(CONFIG_ARM_RK3399_DMC_DEVFREQ) += rk3399_dmc.o |
| 15 | +obj-$(CONFIG_ARM_SUN8I_A33_MBUS_DEVFREQ) += sun8i-a33-mbus.o | |
| 15 | 16 | obj-$(CONFIG_ARM_TEGRA_DEVFREQ) += tegra30-devfreq.o |
| 16 | 17 | |
| 17 | 18 | # DEVFREQ Event Drivers |
| @@ -382,8 +382,8 @@ static int devfreq_set_target(struct devfreq *devfreq, unsigned long new_freq, · 381 unmodified lines | ||
| 382 | 382 | devfreq_notify_transition(devfreq, &freqs, DEVFREQ_POSTCHANGE); |
| 383 | 383 | |
| 384 | 384 | if (devfreq_update_status(devfreq, new_freq)) |
| 385 | − dev_err(&devfreq->dev, | |
| 386 | − "Couldn't update frequency transition information.\n"); | |
| 385 | + dev_warn(&devfreq->dev, | |
| 386 | + "Couldn't update frequency transition information.\n"); | |
| 387 | 387 | |
| 388 | 388 | devfreq->previous_freq = new_freq; |
| 389 | 389 | |
| @@ -0,0 +1,511 @@ | ||
| 1 | +// SPDX-License-Identifier: GPL-2.0-only | |
| 2 | +// | |
| 3 | +// Copyright (C) 2020-2021 Samuel Holland <samuel@sholland.org> | |
| 4 | +// | |
| 5 | + | |
| 6 | +#include <linux/clk.h> | |
| 7 | +#include <linux/devfreq.h> | |
| 8 | +#include <linux/err.h> | |
| 9 | +#include <linux/io.h> | |
| 10 | +#include <linux/iopoll.h> | |
| 11 | +#include <linux/module.h> | |
| 12 | +#include <linux/of.h> | |
| 13 | +#include <linux/platform_device.h> | |
| 14 | +#include <linux/property.h> | |
| 15 | + | |
| 16 | +#define MBUS_CR 0x0000 | |
| 17 | +#define MBUS_CR_GET_DRAM_TYPE(x) (((x) >> 16) & 0x7) | |
| 18 | +#define MBUS_CR_DRAM_TYPE_DDR2 2 | |
| 19 | +#define MBUS_CR_DRAM_TYPE_DDR3 3 | |
| 20 | +#define MBUS_CR_DRAM_TYPE_DDR4 4 | |
| 21 | +#define MBUS_CR_DRAM_TYPE_LPDDR2 6 | |
| 22 | +#define MBUS_CR_DRAM_TYPE_LPDDR3 7 | |
| 23 | + | |
| 24 | +#define MBUS_TMR 0x000c | |
| 25 | +#define MBUS_TMR_PERIOD(x) ((x) - 1) | |
| 26 | + | |
| 27 | +#define MBUS_PMU_CFG 0x009c | |
| 28 | +#define MBUS_PMU_CFG_PERIOD(x) (((x) - 1) << 16) | |
| 29 | +#define MBUS_PMU_CFG_UNIT (0x3 << 1) | |
| 30 | +#define MBUS_PMU_CFG_UNIT_B (0x0 << 1) | |
| 31 | +#define MBUS_PMU_CFG_UNIT_KB (0x1 << 1) | |
| 32 | +#define MBUS_PMU_CFG_UNIT_MB (0x2 << 1) | |
| 33 | +#define MBUS_PMU_CFG_ENABLE (0x1 << 0) | |
| 34 | + | |
| 35 | +#define MBUS_PMU_BWCR(n) (0x00a0 + (0x04 * (n))) | |
| 36 | + | |
| 37 | +#define MBUS_TOTAL_BWCR MBUS_PMU_BWCR(5) | |
| 38 | +#define MBUS_TOTAL_BWCR_H616 MBUS_PMU_BWCR(13) | |
| 39 | + | |
| 40 | +#define MBUS_MDFSCR 0x0100 | |
| 41 | +#define MBUS_MDFSCR_BUFFER_TIMING (0x1 << 15) | |
| 42 | +#define MBUS_MDFSCR_PAD_HOLD (0x1 << 13) | |
| 43 | +#define MBUS_MDFSCR_BYPASS (0x1 << 4) | |
| 44 | +#define MBUS_MDFSCR_MODE (0x1 << 1) | |
| 45 | +#define MBUS_MDFSCR_MODE_DFS (0x0 << 1) | |
| 46 | +#define MBUS_MDFSCR_MODE_CFS (0x1 << 1) | |
| 47 | +#define MBUS_MDFSCR_START (0x1 << 0) | |
| 48 | + | |
| 49 | +#define MBUS_MDFSMRMR 0x0108 | |
| 50 | + | |
| 51 | +#define DRAM_PWRCTL 0x0004 | |
| 52 | +#define DRAM_PWRCTL_SELFREF_EN (0x1 << 0) | |
| 53 | + | |
| 54 | +#define DRAM_RFSHTMG 0x0090 | |
| 55 | +#define DRAM_RFSHTMG_TREFI(x) ((x) << 16) | |
| 56 | +#define DRAM_RFSHTMG_TRFC(x) ((x) << 0) | |
| 57 | + | |
| 58 | +#define DRAM_VTFCR 0x00b8 | |
| 59 | +#define DRAM_VTFCR_VTF_ENABLE (0x3 << 8) | |
| 60 | + | |
| 61 | +#define DRAM_ODTMAP 0x0120 | |
| 62 | + | |
| 63 | +#define DRAM_DX_MAX 4 | |
| 64 | + | |
| 65 | +#define DRAM_DXnGCR0(n) (0x0344 + 0x80 * (n)) | |
| 66 | +#define DRAM_DXnGCR0_DXODT (0x3 << 4) | |
| 67 | +#define DRAM_DXnGCR0_DXODT_DYNAMIC (0x0 << 4) | |
| 68 | +#define DRAM_DXnGCR0_DXODT_ENABLED (0x1 << 4) | |
| 69 | +#define DRAM_DXnGCR0_DXODT_DISABLED (0x2 << 4) | |
| 70 | +#define DRAM_DXnGCR0_DXEN (0x1 << 0) | |
| 71 | + | |
| 72 | +struct sun8i_a33_mbus_variant { | |
| 73 | + u32 min_dram_divider; | |
| 74 | + u32 max_dram_divider; | |
| 75 | + u32 odt_freq_mhz; | |
| 76 | +}; | |
| 77 | + | |
| 78 | +struct sun8i_a33_mbus { | |
| 79 | + const struct sun8i_a33_mbus_variant *variant; | |
| 80 | + void __iomem *reg_dram; | |
| 81 | + void __iomem *reg_mbus; | |
| 82 | + struct clk *clk_bus; | |
| 83 | + struct clk *clk_dram; | |
| 84 | + struct clk *clk_mbus; | |
| 85 | + struct devfreq *devfreq_dram; | |
| 86 | + struct devfreq_simple_ondemand_data gov_data; | |
| 87 | + struct devfreq_dev_profile profile; | |
| 88 | + u32 data_width; | |
| 89 | + u32 nominal_bw; | |
| 90 | + u32 odtmap; | |
| 91 | + u32 tREFI_ns; | |
| 92 | + u32 tRFC_ns; | |
| 93 | + unsigned long freq_table[]; | |
| 94 | +}; | |
| 95 | + | |
| 96 | +/* | |
| 97 | + * The unit for this value is (MBUS clock cycles / MBUS_TMR_PERIOD). When | |
| 98 | + * MBUS_TMR_PERIOD is programmed to match the MBUS clock frequency in MHz, as | |
| 99 | + * it is during DRAM init and during probe, the resulting unit is microseconds. | |
| 100 | + */ | |
| 101 | +static int pmu_period = 50000; | |
| 102 | +module_param(pmu_period, int, 0644); | |
| 103 | +MODULE_PARM_DESC(pmu_period, "Bandwidth measurement period (microseconds)"); | |
| 104 | + | |
| 105 | +static u32 sun8i_a33_mbus_get_peak_bw(struct sun8i_a33_mbus *priv) | |
| 106 | +{ | |
| 107 | + /* Returns the peak transfer (in KiB) during any single PMU period. */ | |
| 108 | + return readl_relaxed(priv->reg_mbus + MBUS_TOTAL_BWCR); | |
| 109 | +} | |
| 110 | + | |
| 111 | +static void sun8i_a33_mbus_restart_pmu_counters(struct sun8i_a33_mbus *priv) | |
| 112 | +{ | |
| 113 | + u32 pmu_cfg = MBUS_PMU_CFG_PERIOD(pmu_period) | MBUS_PMU_CFG_UNIT_KB; | |
| 114 | + | |
| 115 | + /* All PMU counters are cleared on a disable->enable transition. */ | |
| 116 | + writel_relaxed(pmu_cfg, | |
| 117 | + priv->reg_mbus + MBUS_PMU_CFG); | |
| 118 | + writel_relaxed(pmu_cfg | MBUS_PMU_CFG_ENABLE, | |
| 119 | + priv->reg_mbus + MBUS_PMU_CFG); | |
| 120 | + | |
| 121 | +} | |
| 122 | + | |
| 123 | +static void sun8i_a33_mbus_update_nominal_bw(struct sun8i_a33_mbus *priv, | |
| 124 | + u32 ddr_freq_mhz) | |
| 125 | +{ | |
| 126 | + /* | |
| 127 | + * Nominal bandwidth (KiB per PMU period): | |
| 128 | + * | |
| 129 | + * DDR transfers microseconds KiB | |
| 130 | + * ------------- * ------------ * -------- | |
| 131 | + * microsecond PMU period transfer | |
| 132 | + */ | |
| 133 | + priv->nominal_bw = ddr_freq_mhz * pmu_period * priv->data_width / 1024; | |
| 134 | +} | |
| 135 | + | |
| 136 | +static int sun8i_a33_mbus_set_dram_freq(struct sun8i_a33_mbus *priv, | |
| 137 | + unsigned long freq) | |
| 138 | +{ | |
| 139 | + u32 ddr_freq_mhz = freq / USEC_PER_SEC; /* DDR */ | |
| 140 | + u32 dram_freq_mhz = ddr_freq_mhz / 2; /* SDR */ | |
| 141 | + u32 mctl_freq_mhz = dram_freq_mhz / 2; /* HDR */ | |
| 142 | + u32 dxodt, mdfscr, pwrctl, vtfcr; | |
| 143 | + u32 i, tREFI_32ck, tRFC_ck; | |
| 144 | + int ret; | |
| 145 | + | |
| 146 | + /* The rate change is not effective until the MDFS process runs. */ | |
| 147 | + ret = clk_set_rate(priv->clk_dram, freq); | |
| 148 | + if (ret) | |
| 149 | + return ret; | |
| 150 | + | |
| 151 | + /* Disable automatic self-refesh and VTF before starting MDFS. */ | |
| 152 | + pwrctl = readl_relaxed(priv->reg_dram + DRAM_PWRCTL) & | |
| 153 | + ~DRAM_PWRCTL_SELFREF_EN; | |
| 154 | + writel_relaxed(pwrctl, priv->reg_dram + DRAM_PWRCTL); | |
| 155 | + vtfcr = readl_relaxed(priv->reg_dram + DRAM_VTFCR); | |
| 156 | + writel_relaxed(vtfcr & ~DRAM_VTFCR_VTF_ENABLE, | |
| 157 | + priv->reg_dram + DRAM_VTFCR); | |
| 158 | + | |
| 159 | + /* Set up MDFS and enable double buffering for timing registers. */ | |
| 160 | + mdfscr = MBUS_MDFSCR_MODE_DFS | | |
| 161 | + MBUS_MDFSCR_BYPASS | | |
| 162 | + MBUS_MDFSCR_PAD_HOLD | | |
| 163 | + MBUS_MDFSCR_BUFFER_TIMING; | |
| 164 | + writel(mdfscr, priv->reg_mbus + MBUS_MDFSCR); | |
| 165 | + | |
| 166 | + /* Update the buffered copy of RFSHTMG. */ | |
| 167 | + tREFI_32ck = priv->tREFI_ns * mctl_freq_mhz / 1000 / 32; | |
| 168 | + tRFC_ck = DIV_ROUND_UP(priv->tRFC_ns * mctl_freq_mhz, 1000); | |
| 169 | + writel(DRAM_RFSHTMG_TREFI(tREFI_32ck) | DRAM_RFSHTMG_TRFC(tRFC_ck), | |
| 170 | + priv->reg_dram + DRAM_RFSHTMG); | |
| 171 | + | |
| 172 | + /* Enable ODT if needed, or disable it to save power. */ | |
| 173 | + if (priv->odtmap && dram_freq_mhz > priv->variant->odt_freq_mhz) { | |
| 174 | + dxodt = DRAM_DXnGCR0_DXODT_DYNAMIC; | |
| 175 | + writel(priv->odtmap, priv->reg_dram + DRAM_ODTMAP); | |
| 176 | + } else { | |
| 177 | + dxodt = DRAM_DXnGCR0_DXODT_DISABLED; | |
| 178 | + writel(0, priv->reg_dram + DRAM_ODTMAP); | |
| 179 | + } | |
| 180 | + for (i = 0; i < DRAM_DX_MAX; ++i) { | |
| 181 | + void __iomem *reg = priv->reg_dram + DRAM_DXnGCR0(i); | |
| 182 | + | |
| 183 | + writel((readl(reg) & ~DRAM_DXnGCR0_DXODT) | dxodt, reg); | |
| 184 | + } | |
| 185 | + | |
| 186 | + dev_dbg(priv->devfreq_dram->dev.parent, | |
| 187 | + "Setting DRAM to %u MHz, tREFI=%u, tRFC=%u, ODT=%s\n", | |
| 188 | + dram_freq_mhz, tREFI_32ck, tRFC_ck, | |
| 189 | + dxodt == DRAM_DXnGCR0_DXODT_DYNAMIC ? "dynamic" : "disabled"); | |
| 190 | + | |
| 191 | + /* Trigger hardware MDFS. */ | |
| 192 | + writel(mdfscr | MBUS_MDFSCR_START, priv->reg_mbus + MBUS_MDFSCR); | |
| 193 | + ret = readl_poll_timeout_atomic(priv->reg_mbus + MBUS_MDFSCR, mdfscr, | |
| 194 | + !(mdfscr & MBUS_MDFSCR_START), 10, 1000); | |
| 195 | + if (ret) | |
| 196 | + return ret; | |
| 197 | + | |
| 198 | + /* Disable double buffering. */ | |
| 199 | + writel(0, priv->reg_mbus + MBUS_MDFSCR); | |
| 200 | + | |
| 201 | + /* Restore VTF configuration. */ | |
| 202 | + writel_relaxed(vtfcr, priv->reg_dram + DRAM_VTFCR); | |
| 203 | + | |
| 204 | + /* Enable automatic self-refresh at the lowest frequency only. */ | |
| 205 | + if (freq == priv->freq_table[0]) | |
| 206 | + pwrctl |= DRAM_PWRCTL_SELFREF_EN; | |
| 207 | + writel_relaxed(pwrctl, priv->reg_dram + DRAM_PWRCTL); | |
| 208 | + | |
| 209 | + sun8i_a33_mbus_restart_pmu_counters(priv); | |
| 210 | + sun8i_a33_mbus_update_nominal_bw(priv, ddr_freq_mhz); | |
| 211 | + | |
| 212 | + return 0; | |
| 213 | +} | |
| 214 | + | |
| 215 | +static int sun8i_a33_mbus_set_dram_target(struct device *dev, | |
| 216 | + unsigned long *freq, u32 flags) | |
| 217 | +{ | |
| 218 | + struct sun8i_a33_mbus *priv = dev_get_drvdata(dev); | |
| 219 | + struct devfreq *devfreq = priv->devfreq_dram; | |
| 220 | + struct dev_pm_opp *opp; | |
| 221 | + int ret; | |
| 222 | + | |
| 223 | + opp = devfreq_recommended_opp(dev, freq, flags); | |
| 224 | + if (IS_ERR(opp)) | |
| 225 | + return PTR_ERR(opp); | |
| 226 | + | |
| 227 | + dev_pm_opp_put(opp); | |
| 228 | + | |
| 229 | + if (*freq == devfreq->previous_freq) | |
| 230 | + return 0; | |
| 231 | + | |
| 232 | + ret = sun8i_a33_mbus_set_dram_freq(priv, *freq); | |
| 233 | + if (ret) { | |
| 234 | + dev_warn(dev, "failed to set DRAM frequency: %d\n", ret); | |
| 235 | + *freq = devfreq->previous_freq; | |
| 236 | + } | |
| 237 | + | |
| 238 | + return ret; | |
| 239 | +} | |
| 240 | + | |
| 241 | +static int sun8i_a33_mbus_get_dram_status(struct device *dev, | |
| 242 | + struct devfreq_dev_status *stat) | |
| 243 | +{ | |
| 244 | + struct sun8i_a33_mbus *priv = dev_get_drvdata(dev); | |
| 245 | + | |
| 246 | + stat->busy_time = sun8i_a33_mbus_get_peak_bw(priv); | |
| 247 | + stat->total_time = priv->nominal_bw; | |
| 248 | + stat->current_frequency = priv->devfreq_dram->previous_freq; | |
| 249 | + | |
| 250 | + sun8i_a33_mbus_restart_pmu_counters(priv); | |
| 251 | + | |
| 252 | + dev_dbg(dev, "Using %lu/%lu (%lu%%) at %lu MHz\n", | |
| 253 | + stat->busy_time, stat->total_time, | |
| 254 | + DIV_ROUND_CLOSEST(stat->busy_time * 100, stat->total_time), | |
| 255 | + stat->current_frequency / USEC_PER_SEC); | |
| 256 | + | |
| 257 | + return 0; | |
| 258 | +} | |
| 259 | + | |
| 260 | +static int sun8i_a33_mbus_hw_init(struct device *dev, | |
| 261 | + struct sun8i_a33_mbus *priv, | |
| 262 | + unsigned long ddr_freq) | |
| 263 | +{ | |
| 264 | + u32 i, mbus_cr, mbus_freq_mhz; | |
| 265 | + | |
| 266 | + /* Choose tREFI and tRFC to match the configured DRAM type. */ | |
| 267 | + mbus_cr = readl_relaxed(priv->reg_mbus + MBUS_CR); | |
| 268 | + switch (MBUS_CR_GET_DRAM_TYPE(mbus_cr)) { | |
| 269 | + case MBUS_CR_DRAM_TYPE_DDR2: | |
| 270 | + case MBUS_CR_DRAM_TYPE_DDR3: | |
| 271 | + case MBUS_CR_DRAM_TYPE_DDR4: | |
| 272 | + priv->tREFI_ns = 7800; | |
| 273 | + priv->tRFC_ns = 350; | |
| 274 | + break; | |
| 275 | + case MBUS_CR_DRAM_TYPE_LPDDR2: | |
| 276 | + case MBUS_CR_DRAM_TYPE_LPDDR3: | |
| 277 | + priv->tREFI_ns = 3900; | |
| 278 | + priv->tRFC_ns = 210; | |
| 279 | + break; | |
| 280 | + default: | |
| 281 | + return -EINVAL; | |
| 282 | + } | |
| 283 | + | |
| 284 | + /* Save ODTMAP so it can be restored when raising the frequency. */ | |
| 285 | + priv->odtmap = readl_relaxed(priv->reg_dram + DRAM_ODTMAP); | |
| 286 | + | |
| 287 | + /* Compute the DRAM data bus width by counting enabled DATx8 blocks. */ | |
| 288 | + for (i = 0; i < DRAM_DX_MAX; ++i) { | |
| 289 | + void __iomem *reg = priv->reg_dram + DRAM_DXnGCR0(i); | |
| 290 | + | |
| 291 | + if (!(readl_relaxed(reg) & DRAM_DXnGCR0_DXEN)) | |
| 292 | + break; | |
| 293 | + } | |
| 294 | + priv->data_width = i; | |
| 295 | + | |
| 296 | + dev_dbg(dev, "Detected %u-bit %sDDRx with%s ODT\n", | |
| 297 | + priv->data_width * 8, | |
| 298 | + MBUS_CR_GET_DRAM_TYPE(mbus_cr) > 4 ? "LP" : "", | |
| 299 | + priv->odtmap ? "" : "out"); | |
| 300 | + | |
| 301 | + /* Program MBUS_TMR such that the PMU period unit is microseconds. */ | |
| 302 | + mbus_freq_mhz = clk_get_rate(priv->clk_mbus) / USEC_PER_SEC; | |
| 303 | + writel_relaxed(MBUS_TMR_PERIOD(mbus_freq_mhz), | |
| 304 | + priv->reg_mbus + MBUS_TMR); | |
| 305 | + | |
| 306 | + /* "Master Ready Mask Register" bits must be set or MDFS will block. */ | |
| 307 | + writel_relaxed(0xffffffff, priv->reg_mbus + MBUS_MDFSMRMR); | |
| 308 | + | |
| 309 | + sun8i_a33_mbus_restart_pmu_counters(priv); | |
| 310 | + sun8i_a33_mbus_update_nominal_bw(priv, ddr_freq / USEC_PER_SEC); | |
| 311 | + | |
| 312 | + return 0; | |
| 313 | +} | |
| 314 | + | |
| 315 | +static int __maybe_unused sun8i_a33_mbus_suspend(struct device *dev) | |
| 316 | +{ | |
| 317 | + struct sun8i_a33_mbus *priv = dev_get_drvdata(dev); | |
| 318 | + | |
| 319 | + clk_disable_unprepare(priv->clk_bus); | |
| 320 | + | |
| 321 | + return 0; | |
| 322 | +} | |
| 323 | + | |
| 324 | +static int __maybe_unused sun8i_a33_mbus_resume(struct device *dev) | |
| 325 | +{ | |
| 326 | + struct sun8i_a33_mbus *priv = dev_get_drvdata(dev); | |
| 327 | + | |
| 328 | + return clk_prepare_enable(priv->clk_bus); | |
| 329 | +} | |
| 330 | + | |
| 331 | +static int sun8i_a33_mbus_probe(struct platform_device *pdev) | |
| 332 | +{ | |
| 333 | + const struct sun8i_a33_mbus_variant *variant; | |
| 334 | + struct device *dev = &pdev->dev; | |
| 335 | + struct sun8i_a33_mbus *priv; | |
| 336 | + unsigned long base_freq; | |
| 337 | + unsigned int max_state; | |
| 338 | + const char *err; | |
| 339 | + int i, ret; | |
| 340 | + | |
| 341 | + variant = device_get_match_data(dev); | |
| 342 | + if (!variant) | |
| 343 | + return -EINVAL; | |
| 344 | + | |
| 345 | + max_state = variant->max_dram_divider - variant->min_dram_divider + 1; | |
| 346 | + | |
| 347 | + priv = devm_kzalloc(dev, struct_size(priv, freq_table, max_state), GFP_KERNEL); | |
| 348 | + if (!priv) | |
| 349 | + return -ENOMEM; | |
| 350 | + | |
| 351 | + platform_set_drvdata(pdev, priv); | |
| 352 | + | |
| 353 | + priv->variant = variant; | |
| 354 | + | |
| 355 | + priv->reg_dram = devm_platform_ioremap_resource_byname(pdev, "dram"); | |
| 356 | + if (IS_ERR(priv->reg_dram)) | |
| 357 | + return PTR_ERR(priv->reg_dram); | |
| 358 | + | |
| 359 | + priv->reg_mbus = devm_platform_ioremap_resource_byname(pdev, "mbus"); | |
| 360 | + if (IS_ERR(priv->reg_mbus)) | |
| 361 | + return PTR_ERR(priv->reg_mbus); | |
| 362 | + | |
| 363 | + priv->clk_bus = devm_clk_get(dev, "bus"); | |
| 364 | + if (IS_ERR(priv->clk_bus)) | |
| 365 | + return dev_err_probe(dev, PTR_ERR(priv->clk_bus), | |
| 366 | + "failed to get bus clock\n"); | |
| 367 | + | |
| 368 | + priv->clk_dram = devm_clk_get(dev, "dram"); | |
| 369 | + if (IS_ERR(priv->clk_dram)) | |
| 370 | + return dev_err_probe(dev, PTR_ERR(priv->clk_dram), | |
| 371 | + "failed to get dram clock\n"); | |
| 372 | + | |
| 373 | + priv->clk_mbus = devm_clk_get(dev, "mbus"); | |
| 374 | + if (IS_ERR(priv->clk_mbus)) | |
| 375 | + return dev_err_probe(dev, PTR_ERR(priv->clk_mbus), | |
| 376 | + "failed to get mbus clock\n"); | |
| 377 | + | |
| 378 | + ret = clk_prepare_enable(priv->clk_bus); | |
| 379 | + if (ret) | |
| 380 | + return dev_err_probe(dev, ret, | |
| 381 | + "failed to enable bus clock\n"); | |
| 382 | + | |
| 383 | + /* Lock the DRAM clock rate to keep priv->nominal_bw in sync. */ | |
| 384 | + ret = clk_rate_exclusive_get(priv->clk_dram); | |
| 385 | + if (ret) { | |
| 386 | + err = "failed to lock dram clock rate\n"; | |
| 387 | + goto err_disable_bus; | |
| 388 | + } | |
| 389 | + | |
| 390 | + /* Lock the MBUS clock rate to keep MBUS_TMR_PERIOD in sync. */ | |
| 391 | + ret = clk_rate_exclusive_get(priv->clk_mbus); | |
| 392 | + if (ret) { | |
| 393 | + err = "failed to lock mbus clock rate\n"; | |
| 394 | + goto err_unlock_dram; | |
| 395 | + } | |
| 396 | + | |
| 397 | + priv->gov_data.upthreshold = 10; | |
| 398 | + priv->gov_data.downdifferential = 5; | |
| 399 | + | |
| 400 | + priv->profile.initial_freq = clk_get_rate(priv->clk_dram); | |
| 401 | + priv->profile.polling_ms = 1000; | |
| 402 | + priv->profile.target = sun8i_a33_mbus_set_dram_target; | |
| 403 | + priv->profile.get_dev_status = sun8i_a33_mbus_get_dram_status; | |
| 404 | + priv->profile.freq_table = priv->freq_table; | |
| 405 | + priv->profile.max_state = max_state; | |
| 406 | + | |
| 407 | + ret = devm_pm_opp_set_clkname(dev, "dram"); | |
| 408 | + if (ret) { | |
| 409 | + err = "failed to add OPP table\n"; | |
| 410 | + goto err_unlock_mbus; | |
| 411 | + } | |
| 412 | + | |
| 413 | + base_freq = clk_get_rate(clk_get_parent(priv->clk_dram)); | |
| 414 | + for (i = 0; i < max_state; ++i) { | |
| 415 | + unsigned int div = variant->max_dram_divider - i; | |
| 416 | + | |
| 417 | + priv->freq_table[i] = base_freq / div; | |
| 418 | + | |
| 419 | + ret = dev_pm_opp_add(dev, priv->freq_table[i], 0); | |
| 420 | + if (ret) { | |
| 421 | + err = "failed to add OPPs\n"; | |
| 422 | + goto err_remove_opps; | |
| 423 | + } | |
| 424 | + } | |
| 425 | + | |
| 426 | + ret = sun8i_a33_mbus_hw_init(dev, priv, priv->profile.initial_freq); | |
| 427 | + if (ret) { | |
| 428 | + err = "failed to init hardware\n"; | |
| 429 | + goto err_remove_opps; | |
| 430 | + } | |
| 431 | + | |
| 432 | + priv->devfreq_dram = devfreq_add_device(dev, &priv->profile, | |
| 433 | + DEVFREQ_GOV_SIMPLE_ONDEMAND, | |
| 434 | + &priv->gov_data); | |
| 435 | + if (IS_ERR(priv->devfreq_dram)) { | |
| 436 | + ret = PTR_ERR(priv->devfreq_dram); | |
| 437 | + err = "failed to add devfreq device\n"; | |
| 438 | + goto err_remove_opps; | |
| 439 | + } | |
| 440 | + | |
| 441 | + /* | |
| 442 | + * This must be set manually after registering the devfreq device, | |
| 443 | + * because there is no way to select a dynamic OPP as the suspend OPP. | |
| 444 | + */ | |
| 445 | + priv->devfreq_dram->suspend_freq = priv->freq_table[0]; | |
| 446 | + | |
| 447 | + return 0; | |
| 448 | + | |
| 449 | +err_remove_opps: | |
| 450 | + dev_pm_opp_remove_all_dynamic(dev); | |
| 451 | +err_unlock_mbus: | |
| 452 | + clk_rate_exclusive_put(priv->clk_mbus); | |
| 453 | +err_unlock_dram: | |
| 454 | + clk_rate_exclusive_put(priv->clk_dram); | |
| 455 | +err_disable_bus: | |
| 456 | + clk_disable_unprepare(priv->clk_bus); | |
| 457 | + | |
| 458 | + return dev_err_probe(dev, ret, err); | |
| 459 | +} | |
| 460 | + | |
| 461 | +static int sun8i_a33_mbus_remove(struct platform_device *pdev) | |
| 462 | +{ | |
| 463 | + struct sun8i_a33_mbus *priv = platform_get_drvdata(pdev); | |
| 464 | + unsigned long initial_freq = priv->profile.initial_freq; | |
| 465 | + struct device *dev = &pdev->dev; | |
| 466 | + int ret; | |
| 467 | + | |
| 468 | + devfreq_remove_device(priv->devfreq_dram); | |
| 469 | + | |
| 470 | + ret = sun8i_a33_mbus_set_dram_freq(priv, initial_freq); | |
| 471 | + if (ret) | |
| 472 | + dev_warn(dev, "failed to restore DRAM frequency: %d\n", ret); | |
| 473 | + | |
| 474 | + dev_pm_opp_remove_all_dynamic(dev); | |
| 475 | + clk_rate_exclusive_put(priv->clk_mbus); | |
| 476 | + clk_rate_exclusive_put(priv->clk_dram); | |
| 477 | + clk_disable_unprepare(priv->clk_bus); | |
| 478 | + | |
| 479 | + return 0; | |
| 480 | +} | |
| 481 | + | |
| 482 | +static const struct sun8i_a33_mbus_variant sun50i_a64_mbus = { | |
| 483 | + .min_dram_divider = 1, | |
| 484 | + .max_dram_divider = 4, | |
| 485 | + .odt_freq_mhz = 400, | |
| 486 | +}; | |
| 487 | + | |
| 488 | +static const struct of_device_id sun8i_a33_mbus_of_match[] = { | |
| 489 | + { .compatible = "allwinner,sun50i-a64-mbus", .data = &sun50i_a64_mbus }, | |
| 490 | + { .compatible = "allwinner,sun50i-h5-mbus", .data = &sun50i_a64_mbus }, | |
| 491 | + { }, | |
| 492 | +}; | |
| 493 | +MODULE_DEVICE_TABLE(of, sun8i_a33_mbus_of_match); | |
| 494 | + | |
| 495 | +static SIMPLE_DEV_PM_OPS(sun8i_a33_mbus_pm_ops, | |
| 496 | + sun8i_a33_mbus_suspend, sun8i_a33_mbus_resume); | |
| 497 | + | |
| 498 | +static struct platform_driver sun8i_a33_mbus_driver = { | |
| 499 | + .probe = sun8i_a33_mbus_probe, | |
| 500 | + .remove = sun8i_a33_mbus_remove, | |
| 501 | + .driver = { | |
| 502 | + .name = "sun8i-a33-mbus", | |
| 503 | + .of_match_table = sun8i_a33_mbus_of_match, | |
| 504 | + .pm = pm_ptr(&sun8i_a33_mbus_pm_ops), | |
| 505 | + }, | |
| 506 | +}; | |
| 507 | +module_platform_driver(sun8i_a33_mbus_driver); | |
| 508 | + | |
| 509 | +MODULE_AUTHOR("Samuel Holland <samuel@sholland.org>"); | |
| 510 | +MODULE_DESCRIPTION("Allwinner sun8i/sun50i MBUS DEVFREQ Driver"); | |
| 511 | +MODULE_LICENSE("GPL v2"); | |
| @@ -1103,17 +1103,17 @@ static int jz4740_mmc_remove(struct platform_device *pdev) · 1102 unmodified lines | ||
| 1103 | 1103 | return 0; |
| 1104 | 1104 | } |
| 1105 | 1105 | |
| 1106 | −static int __maybe_unused jz4740_mmc_suspend(struct device *dev) | |
| 1106 | +static int jz4740_mmc_suspend(struct device *dev) | |
| 1107 | 1107 | { |
| 1108 | 1108 | return pinctrl_pm_select_sleep_state(dev); |
| 1109 | 1109 | } |
| 1110 | 1110 | |
| 1111 | −static int __maybe_unused jz4740_mmc_resume(struct device *dev) | |
| 1111 | +static int jz4740_mmc_resume(struct device *dev) | |
| 1112 | 1112 | { |
| 1113 | 1113 | return pinctrl_select_default_state(dev); |
| 1114 | 1114 | } |
| 1115 | 1115 | |
| 1116 | −static SIMPLE_DEV_PM_OPS(jz4740_mmc_pm_ops, jz4740_mmc_suspend, | |
| 1116 | +DEFINE_SIMPLE_DEV_PM_OPS(jz4740_mmc_pm_ops, jz4740_mmc_suspend, | |
| 1117 | 1117 | jz4740_mmc_resume); |
| 1118 | 1118 | |
| 1119 | 1119 | static struct platform_driver jz4740_mmc_driver = { |
| @@ -1123,7 +1123,7 @@ static struct platform_driver jz4740_mmc_driver = { · 3 unmodified lines | ||
| 1123 | 1123 | .name = "jz4740-mmc", |
| 1124 | 1124 | .probe_type = PROBE_PREFER_ASYNCHRONOUS, |
| 1125 | 1125 | .of_match_table = of_match_ptr(jz4740_mmc_of_match), |
| 1126 | − .pm = pm_ptr(&jz4740_mmc_pm_ops), | |
| 1126 | + .pm = pm_sleep_ptr(&jz4740_mmc_pm_ops), | |
| 1127 | 1127 | }, |
| 1128 | 1128 | }; |
| 1129 | 1129 | |
| @@ -1183,7 +1183,6 @@ static int mxcmci_remove(struct platform_device *pdev) · 1182 unmodified lines | ||
| 1183 | 1183 | return 0; |
| 1184 | 1184 | } |
| 1185 | 1185 | |
| 1186 | −#ifdef CONFIG_PM_SLEEP | |
| 1187 | 1186 | static int mxcmci_suspend(struct device *dev) |
| 1188 | 1187 | { |
| 1189 | 1188 | struct mmc_host *mmc = dev_get_drvdata(dev); |
| @@ -1210,9 +1209,8 @@ static int mxcmci_resume(struct device *dev) · 20 unmodified lines | ||
| 1210 | 1209 | |
| 1211 | 1210 | return ret; |
| 1212 | 1211 | } |
| 1213 | −#endif | |
| 1214 | 1212 | |
| 1215 | −static SIMPLE_DEV_PM_OPS(mxcmci_pm_ops, mxcmci_suspend, mxcmci_resume); | |
| 1213 | +DEFINE_SIMPLE_DEV_PM_OPS(mxcmci_pm_ops, mxcmci_suspend, mxcmci_resume); | |
| 1216 | 1214 | |
| 1217 | 1215 | static struct platform_driver mxcmci_driver = { |
| 1218 | 1216 | .probe = mxcmci_probe, |
| @@ -1220,7 +1218,7 @@ static struct platform_driver mxcmci_driver = { · 1 unmodified line | ||
| 1220 | 1218 | .driver = { |
| 1221 | 1219 | .name = DRIVER_NAME, |
| 1222 | 1220 | .probe_type = PROBE_PREFER_ASYNCHRONOUS, |
| 1223 | − .pm = &mxcmci_pm_ops, | |
| 1221 | + .pm = pm_sleep_ptr(&mxcmci_pm_ops), | |
| 1224 | 1222 | .of_match_table = mxcmci_of_match, |
| 1225 | 1223 | } |
| 1226 | 1224 | }; |
| @@ -5460,7 +5460,9 @@ static struct pci_driver rtl8169_pci_driver = { · 5459 unmodified lines | ||
| 5460 | 5460 | .probe = rtl_init_one, |
| 5461 | 5461 | .remove = rtl_remove_one, |
| 5462 | 5462 | .shutdown = rtl_shutdown, |
| 5463 | − .driver.pm = pm_ptr(&rtl8169_pm_ops), | |
| 5463 | +#ifdef CONFIG_PM | |
| 5464 | + .driver.pm = &rtl8169_pm_ops, | |
| 5465 | +#endif | |
| 5464 | 5466 | }; |
| 5465 | 5467 | |
| 5466 | 5468 | module_pci_driver(rtl8169_pci_driver); |
| @@ -382,7 +382,7 @@ void dtpm_unregister(struct dtpm *dtpm) · 381 unmodified lines | ||
| 382 | 382 | { |
| 383 | 383 | powercap_unregister_zone(pct, &dtpm->zone); |
| 384 | 384 | |
| 385 | − pr_info("Unregistered dtpm node '%s'\n", dtpm->zone.name); | |
| 385 | + pr_debug("Unregistered dtpm node '%s'\n", dtpm->zone.name); | |
| 386 | 386 | } |
| 387 | 387 | |
| 388 | 388 | /** |
| @@ -453,8 +453,8 @@ int dtpm_register(const char *name, struct dtpm *dtpm, struct dtpm *parent) · 64 unmodified lines | ||
| 453 | 453 | dtpm->power_limit = dtpm->power_max; |
| 454 | 454 | } |
| 455 | 455 | |
| 456 | − pr_info("Registered dtpm node '%s' / %llu-%llu uW, \n", | |
| 457 | − dtpm->zone.name, dtpm->power_min, dtpm->power_max); | |
| 456 | + pr_debug("Registered dtpm node '%s' / %llu-%llu uW, \n", | |
| 457 | + dtpm->zone.name, dtpm->power_min, dtpm->power_max); | |
| 458 | 458 | |
| 459 | 459 | mutex_unlock(&dtpm_lock); |
| 460 | 460 | |
| @@ -12,7 +12,7 @@ · 11 unmodified lines | ||
| 12 | 12 | * |
| 13 | 13 | * All of the kthreads used for idle injection are created at init time. |
| 14 | 14 | * |
| 15 | − * Next, the users of the the idle injection framework provide a cpumask via | |
| 15 | + * Next, the users of the idle injection framework provide a cpumask via | |
| 16 | 16 | * its register function. The kthreads will be synchronized with respect to |
| 17 | 17 | * this cpumask. |
| 18 | 18 | * |
| @@ -61,6 +61,20 @@ · 60 unmodified lines | ||
| 61 | 61 | #define PERF_STATUS_THROTTLE_TIME_MASK 0xffffffff |
| 62 | 62 | #define PP_POLICY_MASK 0x1F |
| 63 | 63 | |
| 64 | +/* | |
| 65 | + * SPR has different layout for Psys Domain PowerLimit registers. | |
| 66 | + * There are 17 bits of PL1 and PL2 instead of 15 bits. | |
| 67 | + * The Enable bits and TimeWindow bits are also shifted as a result. | |
| 68 | + */ | |
| 69 | +#define PSYS_POWER_LIMIT1_MASK 0x1FFFF | |
| 70 | +#define PSYS_POWER_LIMIT1_ENABLE BIT(17) | |
| 71 | + | |
| 72 | +#define PSYS_POWER_LIMIT2_MASK (0x1FFFFULL<<32) | |
| 73 | +#define PSYS_POWER_LIMIT2_ENABLE BIT_ULL(49) | |
| 74 | + | |
| 75 | +#define PSYS_TIME_WINDOW1_MASK (0x7FULL<<19) | |
| 76 | +#define PSYS_TIME_WINDOW2_MASK (0x7FULL<<51) | |
| 77 | + | |
| 64 | 78 | /* Non HW constants */ |
| 65 | 79 | #define RAPL_PRIMITIVE_DERIVED BIT(1) /* not from raw data */ |
| 66 | 80 | #define RAPL_PRIMITIVE_DUMMY BIT(2) |
| @@ -97,6 +111,7 @@ struct rapl_defaults { · 30 unmodified lines | ||
| 97 | 111 | bool to_raw); |
| 98 | 112 | unsigned int dram_domain_energy_unit; |
| 99 | 113 | unsigned int psys_domain_energy_unit; |
| 114 | + bool spr_psys_bits; | |
| 100 | 115 | }; |
| 101 | 116 | static struct rapl_defaults *rapl_defaults; |
| 102 | 117 | |
| @@ -669,12 +684,51 @@ static struct rapl_primitive_info rpi[] = { · 566 unmodified lines | ||
| 669 | 684 | RAPL_DOMAIN_REG_PERF, TIME_UNIT, 0), |
| 670 | 685 | PRIMITIVE_INFO_INIT(PRIORITY_LEVEL, PP_POLICY_MASK, 0, |
| 671 | 686 | RAPL_DOMAIN_REG_POLICY, ARBITRARY_UNIT, 0), |
| 687 | + PRIMITIVE_INFO_INIT(PSYS_POWER_LIMIT1, PSYS_POWER_LIMIT1_MASK, 0, | |
| 688 | + RAPL_DOMAIN_REG_LIMIT, POWER_UNIT, 0), | |
| 689 | + PRIMITIVE_INFO_INIT(PSYS_POWER_LIMIT2, PSYS_POWER_LIMIT2_MASK, 32, | |
| 690 | + RAPL_DOMAIN_REG_LIMIT, POWER_UNIT, 0), | |
| 691 | + PRIMITIVE_INFO_INIT(PSYS_PL1_ENABLE, PSYS_POWER_LIMIT1_ENABLE, 17, | |
| 692 | + RAPL_DOMAIN_REG_LIMIT, ARBITRARY_UNIT, 0), | |
| 693 | + PRIMITIVE_INFO_INIT(PSYS_PL2_ENABLE, PSYS_POWER_LIMIT2_ENABLE, 49, | |
| 694 | + RAPL_DOMAIN_REG_LIMIT, ARBITRARY_UNIT, 0), | |
| 695 | + PRIMITIVE_INFO_INIT(PSYS_TIME_WINDOW1, PSYS_TIME_WINDOW1_MASK, 19, | |
| 696 | + RAPL_DOMAIN_REG_LIMIT, TIME_UNIT, 0), | |
| 697 | + PRIMITIVE_INFO_INIT(PSYS_TIME_WINDOW2, PSYS_TIME_WINDOW2_MASK, 51, | |
| 698 | + RAPL_DOMAIN_REG_LIMIT, TIME_UNIT, 0), | |
| 672 | 699 | /* non-hardware */ |
| 673 | 700 | PRIMITIVE_INFO_INIT(AVERAGE_POWER, 0, 0, 0, POWER_UNIT, |
| 674 | 701 | RAPL_PRIMITIVE_DERIVED), |
| 675 | 702 | {NULL, 0, 0, 0}, |
| 676 | 703 | }; |
| 677 | 704 | |
| 705 | +static enum rapl_primitives | |
| 706 | +prim_fixups(struct rapl_domain *rd, enum rapl_primitives prim) | |
| 707 | +{ | |
| 708 | + if (!rapl_defaults->spr_psys_bits) | |
| 709 | + return prim; | |
| 710 | + | |
| 711 | + if (rd->id != RAPL_DOMAIN_PLATFORM) | |
| 712 | + return prim; | |
| 713 | + | |
| 714 | + switch (prim) { | |
| 715 | + case POWER_LIMIT1: | |
| 716 | + return PSYS_POWER_LIMIT1; | |
| 717 | + case POWER_LIMIT2: | |
| 718 | + return PSYS_POWER_LIMIT2; | |
| 719 | + case PL1_ENABLE: | |
| 720 | + return PSYS_PL1_ENABLE; | |
| 721 | + case PL2_ENABLE: | |
| 722 | + return PSYS_PL2_ENABLE; | |
| 723 | + case TIME_WINDOW1: | |
| 724 | + return PSYS_TIME_WINDOW1; | |
| 725 | + case TIME_WINDOW2: | |
| 726 | + return PSYS_TIME_WINDOW2; | |
| 727 | + default: | |
| 728 | + return prim; | |
| 729 | + } | |
| 730 | +} | |
| 731 | + | |
| 678 | 732 | /* Read primitive data based on its related struct rapl_primitive_info. |
| 679 | 733 | * if xlate flag is set, return translated data based on data units, i.e. |
| 680 | 734 | * time, energy, and power. |
| @@ -692,7 +746,8 @@ static int rapl_read_data_raw(struct rapl_domain *rd, · 11 unmodified lines | ||
| 692 | 746 | enum rapl_primitives prim, bool xlate, u64 *data) |
| 693 | 747 | { |
| 694 | 748 | u64 value; |
| 695 | − struct rapl_primitive_info *rp = &rpi[prim]; | |
| 749 | + enum rapl_primitives prim_fixed = prim_fixups(rd, prim); | |
| 750 | + struct rapl_primitive_info *rp = &rpi[prim_fixed]; | |
| 696 | 751 | struct reg_action ra; |
| 697 | 752 | int cpu; |
| 698 | 753 | |
| @@ -738,7 +793,8 @@ static int rapl_write_data_raw(struct rapl_domain *rd, · 39 unmodified lines | ||
| 738 | 793 | enum rapl_primitives prim, |
| 739 | 794 | unsigned long long value) |
| 740 | 795 | { |
| 741 | − struct rapl_primitive_info *rp = &rpi[prim]; | |
| 796 | + enum rapl_primitives prim_fixed = prim_fixups(rd, prim); | |
| 797 | + struct rapl_primitive_info *rp = &rpi[prim_fixed]; | |
| 742 | 798 | int cpu; |
| 743 | 799 | u64 bits; |
| 744 | 800 | struct reg_action ra; |
| @@ -981,6 +1037,7 @@ static const struct rapl_defaults rapl_defaults_spr_server = { · 236 unmodified lines | ||
| 981 | 1037 | .compute_time_window = rapl_compute_time_window_core, |
| 982 | 1038 | .dram_domain_energy_unit = 15300, |
| 983 | 1039 | .psys_domain_energy_unit = 1000000000, |
| 1040 | + .spr_psys_bits = true, | |
| 984 | 1041 | }; |
| 985 | 1042 | |
| 986 | 1043 | static const struct rapl_defaults rapl_defaults_byt = { |
| @@ -462,7 +462,6 @@ static int cpufreq_set_cur_state(struct thermal_cooling_device *cdev, · 461 unmodified lines | ||
| 462 | 462 | struct cpufreq_cooling_device *cpufreq_cdev = cdev->devdata; |
| 463 | 463 | struct cpumask *cpus; |
| 464 | 464 | unsigned int frequency; |
| 465 | − unsigned long max_capacity, capacity; | |
| 466 | 465 | int ret; |
| 467 | 466 | |
| 468 | 467 | /* Request state should be less than max_level */ |
| @@ -479,10 +478,7 @@ static int cpufreq_set_cur_state(struct thermal_cooling_device *cdev, · 10 unmodified lines | ||
| 479 | 478 | if (ret >= 0) { |
| 480 | 479 | cpufreq_cdev->cpufreq_state = state; |
| 481 | 480 | cpus = cpufreq_cdev->policy->related_cpus; |
| 482 | − max_capacity = arch_scale_cpu_capacity(cpumask_first(cpus)); | |
| 483 | − capacity = frequency * max_capacity; | |
| 484 | − capacity /= cpufreq_cdev->policy->cpuinfo.max_freq; | |
| 485 | − arch_set_thermal_pressure(cpus, max_capacity - capacity); | |
| 481 | + arch_update_thermal_pressure(cpus, frequency); | |
| 486 | 482 | ret = 0; |
| 487 | 483 | } |
| 488 | 484 | |
| @@ -138,6 +138,7 @@ extern int cppc_get_desired_perf(int cpunum, u64 *desired_perf); · 137 unmodified lines | ||
| 138 | 138 | extern int cppc_get_nominal_perf(int cpunum, u64 *nominal_perf); |
| 139 | 139 | extern int cppc_get_perf_ctrs(int cpu, struct cppc_perf_fb_ctrs *perf_fb_ctrs); |
| 140 | 140 | extern int cppc_set_perf(int cpu, struct cppc_perf_ctrls *perf_ctrls); |
| 141 | +extern int cppc_set_enable(int cpu, bool enable); | |
| 141 | 142 | extern int cppc_get_perf_caps(int cpu, struct cppc_perf_caps *caps); |
| 142 | 143 | extern bool acpi_cpc_valid(void); |
| 143 | 144 | extern int acpi_get_psd_map(unsigned int cpu, struct cppc_cpudata *cpu_data); |
| @@ -162,6 +163,10 @@ static inline int cppc_set_perf(int cpu, struct cppc_perf_ctrls *perf_ctrls) · 18 unmodified lines | ||
| 162 | 163 | { |
| 163 | 164 | return -ENOTSUPP; |
| 164 | 165 | } |
| 166 | +static inline int cppc_set_enable(int cpu, bool enable) | |
| 167 | +{ | |
| 168 | + return -ENOTSUPP; | |
| 169 | +} | |
| 165 | 170 | static inline int cppc_get_perf_caps(int cpu, struct cppc_perf_caps *caps) |
| 166 | 171 | { |
| 167 | 172 | return -ENOTSUPP; |
| @@ -506,7 +506,7 @@ acpi_status acpi_release_memory(acpi_handle handle, struct resource *res, · 505 unmodified lines | ||
| 506 | 506 | int acpi_resources_are_enforced(void); |
| 507 | 507 | |
| 508 | 508 | #ifdef CONFIG_HIBERNATION |
| 509 | −void __init acpi_no_s4_hw_signature(void); | |
| 509 | +void __init acpi_check_s4_hw_signature(int check); | |
| 510 | 510 | #endif |
| 511 | 511 | |
| 512 | 512 | #ifdef CONFIG_PM_SLEEP |
| @@ -56,8 +56,8 @@ static inline unsigned long topology_get_thermal_pressure(int cpu) · 55 unmodified lines | ||
| 56 | 56 | return per_cpu(thermal_pressure, cpu); |
| 57 | 57 | } |
| 58 | 58 | |
| 59 | −void topology_set_thermal_pressure(const struct cpumask *cpus, | |
| 60 | − unsigned long th_pressure); | |
| 59 | +void topology_update_thermal_pressure(const struct cpumask *cpus, | |
| 60 | + unsigned long capped_freq); | |
| 61 | 61 | |
| 62 | 62 | struct cpu_topology { |
| 63 | 63 | int thread_id; |
| @@ -70,6 +70,4 @@ void dtpm_unregister(struct dtpm *dtpm); · 69 unmodified lines | ||
| 70 | 70 | |
| 71 | 71 | int dtpm_register(const char *name, struct dtpm *dtpm, struct dtpm *parent); |
| 72 | 72 | |
| 73 | −int dtpm_register_cpu(struct dtpm *parent); | |
| 74 | − | |
| 75 | 73 | #endif |
| @@ -58,6 +58,12 @@ enum rapl_primitives { · 57 unmodified lines | ||
| 58 | 58 | THROTTLED_TIME, |
| 59 | 59 | PRIORITY_LEVEL, |
| 60 | 60 | |
| 61 | + PSYS_POWER_LIMIT1, | |
| 62 | + PSYS_POWER_LIMIT2, | |
| 63 | + PSYS_PL1_ENABLE, | |
| 64 | + PSYS_PL2_ENABLE, | |
| 65 | + PSYS_TIME_WINDOW1, | |
| 66 | + PSYS_TIME_WINDOW2, | |
| 61 | 67 | /* below are not raw primitive data */ |
| 62 | 68 | AVERAGE_POWER, |
| 63 | 69 | NR_RAPL_PRIMITIVES, |
| @@ -300,47 +300,59 @@ struct dev_pm_ops { · 299 unmodified lines | ||
| 300 | 300 | int (*runtime_idle)(struct device *dev); |
| 301 | 301 | }; |
| 302 | 302 | |
| 303 | +#define SYSTEM_SLEEP_PM_OPS(suspend_fn, resume_fn) \ | |
| 304 | + .suspend = pm_sleep_ptr(suspend_fn), \ | |
| 305 | + .resume = pm_sleep_ptr(resume_fn), \ | |
| 306 | + .freeze = pm_sleep_ptr(suspend_fn), \ | |
| 307 | + .thaw = pm_sleep_ptr(resume_fn), \ | |
| 308 | + .poweroff = pm_sleep_ptr(suspend_fn), \ | |
| 309 | + .restore = pm_sleep_ptr(resume_fn), | |
| 310 | + | |
| 311 | +#define LATE_SYSTEM_SLEEP_PM_OPS(suspend_fn, resume_fn) \ | |
| 312 | + .suspend_late = pm_sleep_ptr(suspend_fn), \ | |
| 313 | + .resume_early = pm_sleep_ptr(resume_fn), \ | |
| 314 | + .freeze_late = pm_sleep_ptr(suspend_fn), \ | |
| 315 | + .thaw_early = pm_sleep_ptr(resume_fn), \ | |
| 316 | + .poweroff_late = pm_sleep_ptr(suspend_fn), \ | |
| 317 | + .restore_early = pm_sleep_ptr(resume_fn), | |
| 318 | + | |
| 319 | +#define NOIRQ_SYSTEM_SLEEP_PM_OPS(suspend_fn, resume_fn) \ | |
| 320 | + .suspend_noirq = pm_sleep_ptr(suspend_fn), \ | |
| 321 | + .resume_noirq = pm_sleep_ptr(resume_fn), \ | |
| 322 | + .freeze_noirq = pm_sleep_ptr(suspend_fn), \ | |
| 323 | + .thaw_noirq = pm_sleep_ptr(resume_fn), \ | |
| 324 | + .poweroff_noirq = pm_sleep_ptr(suspend_fn), \ | |
| 325 | + .restore_noirq = pm_sleep_ptr(resume_fn), | |
| 326 | + | |
| 327 | +#define RUNTIME_PM_OPS(suspend_fn, resume_fn, idle_fn) \ | |
| 328 | + .runtime_suspend = suspend_fn, \ | |
| 329 | + .runtime_resume = resume_fn, \ | |
| 330 | + .runtime_idle = idle_fn, | |
| 331 | + | |
| 303 | 332 | #ifdef CONFIG_PM_SLEEP |
| 304 | 333 | #define SET_SYSTEM_SLEEP_PM_OPS(suspend_fn, resume_fn) \ |
| 305 | − .suspend = suspend_fn, \ | |
| 306 | − .resume = resume_fn, \ | |
| 307 | − .freeze = suspend_fn, \ | |
| 308 | − .thaw = resume_fn, \ | |
| 309 | − .poweroff = suspend_fn, \ | |
| 310 | − .restore = resume_fn, | |
| 334 | + SYSTEM_SLEEP_PM_OPS(suspend_fn, resume_fn) | |
| 311 | 335 | #else |
| 312 | 336 | #define SET_SYSTEM_SLEEP_PM_OPS(suspend_fn, resume_fn) |
| 313 | 337 | #endif |
| 314 | 338 | |
| 315 | 339 | #ifdef CONFIG_PM_SLEEP |
| 316 | 340 | #define SET_LATE_SYSTEM_SLEEP_PM_OPS(suspend_fn, resume_fn) \ |
| 317 | − .suspend_late = suspend_fn, \ | |
| 318 | − .resume_early = resume_fn, \ | |
| 319 | − .freeze_late = suspend_fn, \ | |
| 320 | − .thaw_early = resume_fn, \ | |
| 321 | − .poweroff_late = suspend_fn, \ | |
| 322 | − .restore_early = resume_fn, | |
| 341 | + LATE_SYSTEM_SLEEP_PM_OPS(suspend_fn, resume_fn) | |
| 323 | 342 | #else |
| 324 | 343 | #define SET_LATE_SYSTEM_SLEEP_PM_OPS(suspend_fn, resume_fn) |
| 325 | 344 | #endif |
| 326 | 345 | |
| 327 | 346 | #ifdef CONFIG_PM_SLEEP |
| 328 | 347 | #define SET_NOIRQ_SYSTEM_SLEEP_PM_OPS(suspend_fn, resume_fn) \ |
| 329 | − .suspend_noirq = suspend_fn, \ | |
| 330 | − .resume_noirq = resume_fn, \ | |
| 331 | − .freeze_noirq = suspend_fn, \ | |
| 332 | − .thaw_noirq = resume_fn, \ | |
| 333 | − .poweroff_noirq = suspend_fn, \ | |
| 334 | − .restore_noirq = resume_fn, | |
| 348 | + NOIRQ_SYSTEM_SLEEP_PM_OPS(suspend_fn, resume_fn) | |
| 335 | 349 | #else |
| 336 | 350 | #define SET_NOIRQ_SYSTEM_SLEEP_PM_OPS(suspend_fn, resume_fn) |
| 337 | 351 | #endif |
| 338 | 352 | |
| 339 | 353 | #ifdef CONFIG_PM |
| 340 | 354 | #define SET_RUNTIME_PM_OPS(suspend_fn, resume_fn, idle_fn) \ |
| 341 | − .runtime_suspend = suspend_fn, \ | |
| 342 | − .runtime_resume = resume_fn, \ | |
| 343 | − .runtime_idle = idle_fn, | |
| 355 | + RUNTIME_PM_OPS(suspend_fn, resume_fn, idle_fn) | |
| 344 | 356 | #else |
| 345 | 357 | #define SET_RUNTIME_PM_OPS(suspend_fn, resume_fn, idle_fn) |
| 346 | 358 | #endif |
| @@ -349,9 +361,9 @@ struct dev_pm_ops { · 2 unmodified lines | ||
| 349 | 361 | * Use this if you want to use the same suspend and resume callbacks for suspend |
| 350 | 362 | * to RAM and hibernation. |
| 351 | 363 | */ |
| 352 | −#define SIMPLE_DEV_PM_OPS(name, suspend_fn, resume_fn) \ | |
| 353 | −const struct dev_pm_ops __maybe_unused name = { \ | |
| 354 | − SET_SYSTEM_SLEEP_PM_OPS(suspend_fn, resume_fn) \ | |
| 364 | +#define DEFINE_SIMPLE_DEV_PM_OPS(name, suspend_fn, resume_fn) \ | |
| 365 | +static const struct dev_pm_ops name = { \ | |
| 366 | + SYSTEM_SLEEP_PM_OPS(suspend_fn, resume_fn) \ | |
| 355 | 367 | } |
| 356 | 368 | |
| 357 | 369 | /* |
| @@ -367,17 +379,27 @@ const struct dev_pm_ops __maybe_unused name = { \ · 9 unmodified lines | ||
| 367 | 379 | * .resume_early(), to the same routines as .runtime_suspend() and |
| 368 | 380 | * .runtime_resume(), respectively (and analogously for hibernation). |
| 369 | 381 | */ |
| 382 | +#define DEFINE_UNIVERSAL_DEV_PM_OPS(name, suspend_fn, resume_fn, idle_fn) \ | |
| 383 | +static const struct dev_pm_ops name = { \ | |
| 384 | + SYSTEM_SLEEP_PM_OPS(suspend_fn, resume_fn) \ | |
| 385 | + RUNTIME_PM_OPS(suspend_fn, resume_fn, idle_fn) \ | |
| 386 | +} | |
| 387 | + | |
| 388 | +/* Deprecated. Use DEFINE_SIMPLE_DEV_PM_OPS() instead. */ | |
| 389 | +#define SIMPLE_DEV_PM_OPS(name, suspend_fn, resume_fn) \ | |
| 390 | +const struct dev_pm_ops __maybe_unused name = { \ | |
| 391 | + SET_SYSTEM_SLEEP_PM_OPS(suspend_fn, resume_fn) \ | |
| 392 | +} | |
| 393 | + | |
| 394 | +/* Deprecated. Use DEFINE_UNIVERSAL_DEV_PM_OPS() instead. */ | |
| 370 | 395 | #define UNIVERSAL_DEV_PM_OPS(name, suspend_fn, resume_fn, idle_fn) \ |
| 371 | 396 | const struct dev_pm_ops __maybe_unused name = { \ |
| 372 | 397 | SET_SYSTEM_SLEEP_PM_OPS(suspend_fn, resume_fn) \ |
| 373 | 398 | SET_RUNTIME_PM_OPS(suspend_fn, resume_fn, idle_fn) \ |
| 374 | 399 | } |
| 375 | 400 | |
| 376 | −#ifdef CONFIG_PM | |
| 377 | −#define pm_ptr(_ptr) (_ptr) | |
| 378 | −#else | |
| 379 | −#define pm_ptr(_ptr) NULL | |
| 380 | −#endif | |
| 401 | +#define pm_ptr(_ptr) PTR_IF(IS_ENABLED(CONFIG_PM), (_ptr)) | |
| 402 | +#define pm_sleep_ptr(_ptr) PTR_IF(IS_ENABLED(CONFIG_PM_SLEEP), (_ptr)) | |
| 381 | 403 | |
| 382 | 404 | /* |
| 383 | 405 | * PM_EVENT_ messages |
| @@ -499,6 +521,7 @@ const struct dev_pm_ops __maybe_unused name = { \ · 115 unmodified lines | ||
| 499 | 521 | */ |
| 500 | 522 | |
| 501 | 523 | enum rpm_status { |
| 524 | + RPM_INVALID = -1, | |
| 502 | 525 | RPM_ACTIVE = 0, |
| 503 | 526 | RPM_RESUMING, |
| 504 | 527 | RPM_SUSPENDED, |
| @@ -612,6 +635,7 @@ struct dev_pm_info { · 107 unmodified lines | ||
| 612 | 635 | unsigned int links_count; |
| 613 | 636 | enum rpm_request request; |
| 614 | 637 | enum rpm_status runtime_status; |
| 638 | + enum rpm_status last_status; | |
| 615 | 639 | int runtime_error; |
| 616 | 640 | int autosuspend_delay; |
| 617 | 641 | u64 last_busy; |
| @@ -58,6 +58,7 @@ extern void pm_runtime_get_suppliers(struct device *dev); · 57 unmodified lines | ||
| 58 | 58 | extern void pm_runtime_put_suppliers(struct device *dev); |
| 59 | 59 | extern void pm_runtime_new_link(struct device *dev); |
| 60 | 60 | extern void pm_runtime_drop_link(struct device_link *link); |
| 61 | +extern void pm_runtime_release_supplier(struct device_link *link, bool check_idle); | |
| 61 | 62 | |
| 62 | 63 | extern int devm_pm_runtime_enable(struct device *dev); |
| 63 | 64 | |
| @@ -283,6 +284,8 @@ static inline void pm_runtime_get_suppliers(struct device *dev) {} · 219 unmodified lines | ||
| 283 | 284 | static inline void pm_runtime_put_suppliers(struct device *dev) {} |
| 284 | 285 | static inline void pm_runtime_new_link(struct device *dev) {} |
| 285 | 286 | static inline void pm_runtime_drop_link(struct device_link *link) {} |
| 287 | +static inline void pm_runtime_release_supplier(struct device_link *link, | |
| 288 | + bool check_idle) {} | |
| 286 | 289 | |
| 287 | 290 | #endif /* !CONFIG_PM */ |
| 288 | 291 | |
| @@ -266,10 +266,10 @@ unsigned long arch_scale_thermal_pressure(int cpu) · 265 unmodified lines | ||
| 266 | 266 | } |
| 267 | 267 | #endif |
| 268 | 268 | |
| 269 | −#ifndef arch_set_thermal_pressure | |
| 269 | +#ifndef arch_update_thermal_pressure | |
| 270 | 270 | static __always_inline |
| 271 | −void arch_set_thermal_pressure(const struct cpumask *cpus, | |
| 272 | − unsigned long th_pressure) | |
| 271 | +void arch_update_thermal_pressure(const struct cpumask *cpus, | |
| 272 | + unsigned long capped_frequency) | |
| 273 | 273 | { } |
| 274 | 274 | #endif |
| 275 | 275 | |
| @@ -446,6 +446,7 @@ extern unsigned long get_safe_page(gfp_t gfp_mask); · 445 unmodified lines | ||
| 446 | 446 | extern asmlinkage int swsusp_arch_suspend(void); |
| 447 | 447 | extern asmlinkage int swsusp_arch_resume(void); |
| 448 | 448 | |
| 449 | +extern u32 swsusp_hardware_signature; | |
| 449 | 450 | extern void hibernation_set_ops(const struct platform_hibernation_ops *ops); |
| 450 | 451 | extern int hibernate(void); |
| 451 | 452 | extern bool system_entering_hibernation(void); |
| @@ -550,7 +550,7 @@ config SCHED_THERMAL_PRESSURE · 549 unmodified lines | ||
| 550 | 550 | i.e. put less load on throttled CPUs than on non/less throttled ones. |
| 551 | 551 | |
| 552 | 552 | This requires the architecture to implement |
| 553 | − arch_set_thermal_pressure() and arch_scale_thermal_pressure(). | |
| 553 | + arch_update_thermal_pressure() and arch_scale_thermal_pressure(). | |
| 554 | 554 | |
| 555 | 555 | config BSD_PROCESS_ACCT |
| 556 | 556 | bool "BSD Process Accounting" |
| @@ -170,6 +170,7 @@ extern int swsusp_swap_in_use(void); · 169 unmodified lines | ||
| 170 | 170 | #define SF_PLATFORM_MODE 1 |
| 171 | 171 | #define SF_NOCOMPRESS_MODE 2 |
| 172 | 172 | #define SF_CRC32_MODE 4 |
| 173 | +#define SF_HW_SIG 8 | |
| 173 | 174 | |
| 174 | 175 | /* kernel/power/hibernate.c */ |
| 175 | 176 | extern int swsusp_check(void); |
| @@ -36,6 +36,8 @@ · 35 unmodified lines | ||
| 36 | 36 | |
| 37 | 37 | #define HIBERNATE_SIG "S1SUSPEND" |
| 38 | 38 | |
| 39 | +u32 swsusp_hardware_signature; | |
| 40 | + | |
| 39 | 41 | /* |
| 40 | 42 | * When reading an {un,}compressed image, we may restore pages in place, |
| 41 | 43 | * in which case some architectures need these pages cleaning before they |
| @@ -104,7 +106,8 @@ struct swap_map_handle { · 62 unmodified lines | ||
| 104 | 106 | |
| 105 | 107 | struct swsusp_header { |
| 106 | 108 | char reserved[PAGE_SIZE - 20 - sizeof(sector_t) - sizeof(int) - |
| 107 | − sizeof(u32)]; | |
| 109 | + sizeof(u32) - sizeof(u32)]; | |
| 110 | + u32 hw_sig; | |
| 108 | 111 | u32 crc32; |
| 109 | 112 | sector_t image; |
| 110 | 113 | unsigned int flags; /* Flags to pass to the "boot" kernel */ |
| @@ -312,7 +315,6 @@ static int hib_wait_io(struct hib_bio_batch *hb) · 201 unmodified lines | ||
| 312 | 315 | /* |
| 313 | 316 | * Saving part |
| 314 | 317 | */ |
| 315 | − | |
| 316 | 318 | static int mark_swapfiles(struct swap_map_handle *handle, unsigned int flags) |
| 317 | 319 | { |
| 318 | 320 | int error; |
| @@ -324,6 +326,10 @@ static int mark_swapfiles(struct swap_map_handle *handle, unsigned int flags) · 5 unmodified lines | ||
| 324 | 326 | memcpy(swsusp_header->orig_sig,swsusp_header->sig, 10); |
| 325 | 327 | memcpy(swsusp_header->sig, HIBERNATE_SIG, 10); |
| 326 | 328 | swsusp_header->image = handle->first_sector; |
| 329 | + if (swsusp_hardware_signature) { | |
| 330 | + swsusp_header->hw_sig = swsusp_hardware_signature; | |
| 331 | + flags |= SF_HW_SIG; | |
| 332 | + } | |
| 327 | 333 | swsusp_header->flags = flags; |
| 328 | 334 | if (flags & SF_CRC32_MODE) |
| 329 | 335 | swsusp_header->crc32 = handle->crc32; |
| @@ -1537,6 +1543,12 @@ int swsusp_check(void) · 1207 unmodified lines | ||
| 1537 | 1543 | } else { |
| 1538 | 1544 | error = -EINVAL; |
| 1539 | 1545 | } |
| 1546 | + if (!error && swsusp_header->flags & SF_HW_SIG && | |
| 1547 | + swsusp_header->hw_sig != swsusp_hardware_signature) { | |
| 1548 | + pr_info("Suspend image hardware signature mismatch (%08x now %08x); aborting resume.\n", | |
| 1549 | + swsusp_header->hw_sig, swsusp_hardware_signature); | |
| 1550 | + error = -EINVAL; | |
| 1551 | + } | |
| 1540 | 1552 | |
| 1541 | 1553 | put: |
| 1542 | 1554 | if (error) |