* pm-cpuidle: (27 commits) intel_idle: Clean up irtl_2_usec() intel_idle: Move 3 functions closer to their callers intel_idle: Annotate initialization code and data structures intel_idle: Move and clean up intel_idle_cpuidle_devices_uninit() intel_idle: Rearrange intel_idle_cpuidle_driver_init() intel_idle: Clean up NULL pointer check in intel_idle_init() intel_idle: Fold intel_idle_probe() into intel_idle_init() intel_idle: Eliminate __setup_broadcast_timer() cpuidle: fix cpuidle_find_deepest_state() kerneldoc warnings cpuidle: sysfs: fix warnings when compiling with W=1 cpuidle: coupled: fix warnings when compiling with W=1 Documentation: admin-guide: PM: Add intel_idle document cpuidle: arm: Enable compile testing for some of drivers cpuidle: Drop unused cpuidle_driver_ref/unref() functions intel_idle: Use ACPI _CST on server systems intel_idle: Add module parameter to prevent ACPI _CST from being used intel_idle: Allow ACPI _CST to be used for selected known processors cpuidle: Allow idle states to be disabled by default intel_idle: Use ACPI _CST for processor models without C-state tables intel_idle: Refactor intel_idle_cpuidle_driver_init() ...
Unified · Split
| @@ -196,6 +196,12 @@ Description: · 195 unmodified lines | |||
| 196 | does not reflect it. Likewise, if one enables a deep state but a | 196 | does not reflect it. Likewise, if one enables a deep state but a |
| 197 | lighter state still is disabled, then this has no effect. | 197 | lighter state still is disabled, then this has no effect. |
| 198 | 198 | ||
| 199 | What: /sys/devices/system/cpu/cpuX/cpuidle/stateN/default_status | ||
| 200 | Date: December 2019 | ||
| 201 | KernelVersion: v5.6 | ||
| 202 | Contact: Linux power management list <linux-pm@vger.kernel.org> | ||
| 203 | Description: | ||
| 204 | (RO) The default status of this state, "enabled" or "disabled". | ||
| 199 | 205 | ||
| 200 | What: /sys/devices/system/cpu/cpuX/cpuidle/stateN/residency | 206 | What: /sys/devices/system/cpu/cpuX/cpuidle/stateN/residency |
| 201 | Date: March 2014 | 207 | Date: March 2014 |
| @@ -506,6 +506,9 @@ object corresponding to it, as follows: · 505 unmodified lines | |||
| 506 | ``disable`` | 506 | ``disable`` |
| 507 | Whether or not this idle state is disabled. | 507 | Whether or not this idle state is disabled. |
| 508 | 508 | ||
| 509 | ``default_status`` | ||
| 510 | The default status of this state, "enabled" or "disabled". | ||
| 511 | |||
| 509 | ``latency`` | 512 | ``latency`` |
| 510 | Exit latency of the idle state in microseconds. | 513 | Exit latency of the idle state in microseconds. |
| 511 | 514 | ||
| @@ -0,0 +1,246 @@ | |||
| 1 | .. SPDX-License-Identifier: GPL-2.0 | ||
| 2 | .. include:: <isonum.txt> | ||
| 3 | |||
| 4 | ============================================== | ||
| 5 | ``intel_idle`` CPU Idle Time Management Driver | ||
| 6 | ============================================== | ||
| 7 | |||
| 8 | :Copyright: |copy| 2020 Intel Corporation | ||
| 9 | |||
| 10 | :Author: Rafael J. Wysocki <rafael.j.wysocki@intel.com> | ||
| 11 | |||
| 12 | |||
| 13 | General Information | ||
| 14 | =================== | ||
| 15 | |||
| 16 | ``intel_idle`` is a part of the | ||
| 17 | :doc:`CPU idle time management subsystem <cpuidle>` in the Linux kernel | ||
| 18 | (``CPUIdle``). It is the default CPU idle time management driver for the | ||
| 19 | Nehalem and later generations of Intel processors, but the level of support for | ||
| 20 | a particular processor model in it depends on whether or not it recognizes that | ||
| 21 | processor model and may also depend on information coming from the platform | ||
| 22 | firmware. [To understand ``intel_idle`` it is necessary to know how ``CPUIdle`` | ||
| 23 | works in general, so this is the time to get familiar with :doc:`cpuidle` if you | ||
| 24 | have not done that yet.] | ||
| 25 | |||
| 26 | ``intel_idle`` uses the ``MWAIT`` instruction to inform the processor that the | ||
| 27 | logical CPU executing it is idle and so it may be possible to put some of the | ||
| 28 | processor's functional blocks into low-power states. That instruction takes two | ||
| 29 | arguments (passed in the ``EAX`` and ``ECX`` registers of the target CPU), the | ||
| 30 | first of which, referred to as a *hint*, can be used by the processor to | ||
| 31 | determine what can be done (for details refer to Intel Software Developer’s | ||
| 32 | Manual [1]_). Accordingly, ``intel_idle`` refuses to work with processors in | ||
| 33 | which the support for the ``MWAIT`` instruction has been disabled (for example, | ||
| 34 | via the platform firmware configuration menu) or which do not support that | ||
| 35 | instruction at all. | ||
| 36 | |||
| 37 | ``intel_idle`` is not modular, so it cannot be unloaded, which means that the | ||
| 38 | only way to pass early-configuration-time parameters to it is via the kernel | ||
| 39 | command line. | ||
| 40 | |||
| 41 | |||
| 42 | .. _intel-idle-enumeration-of-states: | ||
| 43 | |||
| 44 | Enumeration of Idle States | ||
| 45 | ========================== | ||
| 46 | |||
| 47 | Each ``MWAIT`` hint value is interpreted by the processor as a license to | ||
| 48 | reconfigure itself in a certain way in order to save energy. The processor | ||
| 49 | configurations (with reduced power draw) resulting from that are referred to | ||
| 50 | as C-states (in the ACPI terminology) or idle states. The list of meaningful | ||
| 51 | ``MWAIT`` hint values and idle states (i.e. low-power configurations of the | ||
| 52 | processor) corresponding to them depends on the processor model and it may also | ||
| 53 | depend on the configuration of the platform. | ||
| 54 | |||
| 55 | In order to create a list of available idle states required by the ``CPUIdle`` | ||
| 56 | subsystem (see :ref:`idle-states-representation` in :doc:`cpuidle`), | ||
| 57 | ``intel_idle`` can use two sources of information: static tables of idle states | ||
| 58 | for different processor models included in the driver itself and the ACPI tables | ||
| 59 | of the system. The former are always used if the processor model at hand is | ||
| 60 | recognized by ``intel_idle`` and the latter are used if that is required for | ||
| 61 | the given processor model (which is the case for all server processor models | ||
| 62 | recognized by ``intel_idle``) or if the processor model is not recognized. | ||
| 63 | |||
| 64 | If the ACPI tables are going to be used for building the list of available idle | ||
| 65 | states, ``intel_idle`` first looks for a ``_CST`` object under one of the ACPI | ||
| 66 | objects corresponding to the CPUs in the system (refer to the ACPI specification | ||
| 67 | [2]_ for the description of ``_CST`` and its output package). Because the | ||
| 68 | ``CPUIdle`` subsystem expects that the list of idle states supplied by the | ||
| 69 | driver will be suitable for all of the CPUs handled by it and ``intel_idle`` is | ||
| 70 | registered as the ``CPUIdle`` driver for all of the CPUs in the system, the | ||
| 71 | driver looks for the first ``_CST`` object returning at least one valid idle | ||
| 72 | state description and such that all of the idle states included in its return | ||
| 73 | package are of the FFH (Functional Fixed Hardware) type, which means that the | ||
| 74 | ``MWAIT`` instruction is expected to be used to tell the processor that it can | ||
| 75 | enter one of them. The return package of that ``_CST`` is then assumed to be | ||
| 76 | applicable to all of the other CPUs in the system and the idle state | ||
| 77 | descriptions extracted from it are stored in a preliminary list of idle states | ||
| 78 | coming from the ACPI tables. [This step is skipped if ``intel_idle`` is | ||
| 79 | configured to ignore the ACPI tables; see `below <intel-idle-parameters_>`_.] | ||
| 80 | |||
| 81 | Next, the first (index 0) entry in the list of available idle states is | ||
| 82 | initialized to represent a "polling idle state" (a pseudo-idle state in which | ||
| 83 | the target CPU continuously fetches and executes instructions), and the | ||
| 84 | subsequent (real) idle state entries are populated as follows. | ||
| 85 | |||
| 86 | If the processor model at hand is recognized by ``intel_idle``, there is a | ||
| 87 | (static) table of idle state descriptions for it in the driver. In that case, | ||
| 88 | the "internal" table is the primary source of information on idle states and the | ||
| 89 | information from it is copied to the final list of available idle states. If | ||
| 90 | using the ACPI tables for the enumeration of idle states is not required | ||
| 91 | (depending on the processor model), all of the listed idle state are enabled by | ||
| 92 | default (so all of them will be taken into consideration by ``CPUIdle`` | ||
| 93 | governors during CPU idle state selection). Otherwise, some of the listed idle | ||
| 94 | states may not be enabled by default if there are no matching entries in the | ||
| 95 | preliminary list of idle states coming from the ACPI tables. In that case user | ||
| 96 | space still can enable them later (on a per-CPU basis) with the help of | ||
| 97 | the ``disable`` idle state attribute in ``sysfs`` (see | ||
| 98 | :ref:`idle-states-representation` in :doc:`cpuidle`). This basically means that | ||
| 99 | the idle states "known" to the driver may not be enabled by default if they have | ||
| 100 | not been exposed by the platform firmware (through the ACPI tables). | ||
| 101 | |||
| 102 | If the given processor model is not recognized by ``intel_idle``, but it | ||
| 103 | supports ``MWAIT``, the preliminary list of idle states coming from the ACPI | ||
| 104 | tables is used for building the final list that will be supplied to the | ||
| 105 | ``CPUIdle`` core during driver registration. For each idle state in that list, | ||
| 106 | the description, ``MWAIT`` hint and exit latency are copied to the corresponding | ||
| 107 | entry in the final list of idle states. The name of the idle state represented | ||
| 108 | by it (to be returned by the ``name`` idle state attribute in ``sysfs``) is | ||
| 109 | "CX_ACPI", where X is the index of that idle state in the final list (note that | ||
| 110 | the minimum value of X is 1, because 0 is reserved for the "polling" state), and | ||
| 111 | its target residency is based on the exit latency value. Specifically, for | ||
| 112 | C1-type idle states the exit latency value is also used as the target residency | ||
| 113 | (for compatibility with the majority of the "internal" tables of idle states for | ||
| 114 | various processor models recognized by ``intel_idle``) and for the other idle | ||
| 115 | state types (C2 and C3) the target residency value is 3 times the exit latency | ||
| 116 | (again, that is because it reflects the target residency to exit latency ratio | ||
| 117 | in the majority of cases for the processor models recognized by ``intel_idle``). | ||
| 118 | All of the idle states in the final list are enabled by default in this case. | ||
| 119 | |||
| 120 | |||
| 121 | .. _intel-idle-initialization: | ||
| 122 | |||
| 123 | Initialization | ||
| 124 | ============== | ||
| 125 | |||
| 126 | The initialization of ``intel_idle`` starts with checking if the kernel command | ||
| 127 | line options forbid the use of the ``MWAIT`` instruction. If that is the case, | ||
| 128 | an error code is returned right away. | ||
| 129 | |||
| 130 | The next step is to check whether or not the processor model is known to the | ||
| 131 | driver, which determines the idle states enumeration method (see | ||
| 132 | `above <intel-idle-enumeration-of-states_>`_), and whether or not the processor | ||
| 133 | supports ``MWAIT`` (the initialization fails if that is not the case). Then, | ||
| 134 | the ``MWAIT`` support in the processor is enumerated through ``CPUID`` and the | ||
| 135 | driver initialization fails if the level of support is not as expected (for | ||
| 136 | example, if the total number of ``MWAIT`` substates returned is 0). | ||
| 137 | |||
| 138 | Next, if the driver is not configured to ignore the ACPI tables (see | ||
| 139 | `below <intel-idle-parameters_>`_), the idle states information provided by the | ||
| 140 | platform firmware is extracted from them. | ||
| 141 | |||
| 142 | Then, ``CPUIdle`` device objects are allocated for all CPUs and the list of | ||
| 143 | available idle states is created as explained | ||
| 144 | `above <intel-idle-enumeration-of-states_>`_. | ||
| 145 | |||
| 146 | Finally, ``intel_idle`` is registered with the help of cpuidle_register_driver() | ||
| 147 | as the ``CPUIdle`` driver for all CPUs in the system and a CPU online callback | ||
| 148 | for configuring individual CPUs is registered via cpuhp_setup_state(), which | ||
| 149 | (among other things) causes the callback routine to be invoked for all of the | ||
| 150 | CPUs present in the system at that time (each CPU executes its own instance of | ||
| 151 | the callback routine). That routine registers a ``CPUIdle`` device for the CPU | ||
| 152 | running it (which enables the ``CPUIdle`` subsystem to operate that CPU) and | ||
| 153 | optionally performs some CPU-specific initialization actions that may be | ||
| 154 | required for the given processor model. | ||
| 155 | |||
| 156 | |||
| 157 | .. _intel-idle-parameters: | ||
| 158 | |||
| 159 | Kernel Command Line Options and Module Parameters | ||
| 160 | ================================================= | ||
| 161 | |||
| 162 | The *x86* architecture support code recognizes three kernel command line | ||
| 163 | options related to CPU idle time management: ``idle=poll``, ``idle=halt``, | ||
| 164 | and ``idle=nomwait``. If any of them is present in the kernel command line, the | ||
| 165 | ``MWAIT`` instruction is not allowed to be used, so the initialization of | ||
| 166 | ``intel_idle`` will fail. | ||
| 167 | |||
| 168 | Apart from that there are two module parameters recognized by ``intel_idle`` | ||
| 169 | itself that can be set via the kernel command line (they cannot be updated via | ||
| 170 | sysfs, so that is the only way to change their values). | ||
| 171 | |||
| 172 | The ``max_cstate`` parameter value is the maximum idle state index in the list | ||
| 173 | of idle states supplied to the ``CPUIdle`` core during the registration of the | ||
| 174 | driver. It is also the maximum number of regular (non-polling) idle states that | ||
| 175 | can be used by ``intel_idle``, so the enumeration of idle states is terminated | ||
| 176 | after finding that number of usable idle states (the other idle states that | ||
| 177 | potentially might have been used if ``max_cstate`` had been greater are not | ||
| 178 | taken into consideration at all). Setting ``max_cstate`` can prevent | ||
| 179 | ``intel_idle`` from exposing idle states that are regarded as "too deep" for | ||
| 180 | some reason to the ``CPUIdle`` core, but it does so by making them effectively | ||
| 181 | invisible until the system is shut down and started again which may not always | ||
| 182 | be desirable. In practice, it is only really necessary to do that if the idle | ||
| 183 | states in question cannot be enabled during system startup, because in the | ||
| 184 | working state of the system the CPU power management quality of service (PM | ||
| 185 | QoS) feature can be used to prevent ``CPUIdle`` from touching those idle states | ||
| 186 | even if they have been enumerated (see :ref:`cpu-pm-qos` in :doc:`cpuidle`). | ||
| 187 | Setting ``max_cstate`` to 0 causes the ``intel_idle`` initialization to fail. | ||
| 188 | |||
| 189 | The ``noacpi`` module parameter (which is recognized by ``intel_idle`` if the | ||
| 190 | kernel has been configured with ACPI support), can be set to make the driver | ||
| 191 | ignore the system's ACPI tables entirely (it is unset by default). | ||
| 192 | |||
| 193 | |||
| 194 | .. _intel-idle-core-and-package-idle-states: | ||
| 195 | |||
| 196 | Core and Package Levels of Idle States | ||
| 197 | ====================================== | ||
| 198 | |||
| 199 | Typically, in a processor supporting the ``MWAIT`` instruction there are (at | ||
| 200 | least) two levels of idle states (or C-states). One level, referred to as | ||
| 201 | "core C-states", covers individual cores in the processor, whereas the other | ||
| 202 | level, referred to as "package C-states", covers the entire processor package | ||
| 203 | and it may also involve other components of the system (GPUs, memory | ||
| 204 | controllers, I/O hubs etc.). | ||
| 205 | |||
| 206 | Some of the ``MWAIT`` hint values allow the processor to use core C-states only | ||
| 207 | (most importantly, that is the case for the ``MWAIT`` hint value corresponding | ||
| 208 | to the ``C1`` idle state), but the majority of them give it a license to put | ||
| 209 | the target core (i.e. the core containing the logical CPU executing ``MWAIT`` | ||
| 210 | with the given hint value) into a specific core C-state and then (if possible) | ||
| 211 | to enter a specific package C-state at the deeper level. For example, the | ||
| 212 | ``MWAIT`` hint value representing the ``C3`` idle state allows the processor to | ||
| 213 | put the target core into the low-power state referred to as "core ``C3``" (or | ||
| 214 | ``CC3``), which happens if all of the logical CPUs (SMT siblings) in that core | ||
| 215 | have executed ``MWAIT`` with the ``C3`` hint value (or with a hint value | ||
| 216 | representing a deeper idle state), and in addition to that (in the majority of | ||
| 217 | cases) it gives the processor a license to put the entire package (possibly | ||
| 218 | including some non-CPU components such as a GPU or a memory controller) into the | ||
| 219 | low-power state referred to as "package ``C3``" (or ``PC3``), which happens if | ||
| 220 | all of the cores have gone into the ``CC3`` state and (possibly) some additional | ||
| 221 | conditions are satisfied (for instance, if the GPU is covered by ``PC3``, it may | ||
| 222 | be required to be in a certain GPU-specific low-power state for ``PC3`` to be | ||
| 223 | reachable). | ||
| 224 | |||
| 225 | As a rule, there is no simple way to make the processor use core C-states only | ||
| 226 | if the conditions for entering the corresponding package C-states are met, so | ||
| 227 | the logical CPU executing ``MWAIT`` with a hint value that is not core-level | ||
| 228 | only (like for ``C1``) must always assume that this may cause the processor to | ||
| 229 | enter a package C-state. [That is why the exit latency and target residency | ||
| 230 | values corresponding to the majority of ``MWAIT`` hint values in the "internal" | ||
| 231 | tables of idle states in ``intel_idle`` reflect the properties of package | ||
| 232 | C-states.] If using package C-states is not desirable at all, either | ||
| 233 | :ref:`PM QoS <cpu-pm-qos>` or the ``max_cstate`` module parameter of | ||
| 234 | ``intel_idle`` described `above <intel-idle-parameters_>`_ must be used to | ||
| 235 | restrict the range of permissible idle states to the ones with core-level only | ||
| 236 | ``MWAIT`` hint values (like ``C1``). | ||
| 237 | |||
| 238 | |||
| 239 | References | ||
| 240 | ========== | ||
| 241 | |||
| 242 | .. [1] *Intel® 64 and IA-32 Architectures Software Developer’s Manual Volume 2B*, | ||
| 243 | https://www.intel.com/content/www/us/en/architecture-and-technology/64-ia-32-architectures-software-developer-vol-2b-manual.html | ||
| 244 | |||
| 245 | .. [2] *Advanced Configuration and Power Interface (ACPI) Specification*, | ||
| 246 | https://uefi.org/specifications | ||
| @@ -8,6 +8,7 @@ Working-State Power Management · 7 unmodified lines | |||
| 8 | :maxdepth: 2 | 8 | :maxdepth: 2 |
| 9 | 9 | ||
| 10 | cpuidle | 10 | cpuidle |
| 11 | intel_idle | ||
| 11 | cpufreq | 12 | cpufreq |
| 12 | intel_pstate | 13 | intel_pstate |
| 13 | intel_epb | 14 | intel_epb |
| @@ -241,6 +241,7 @@ config ACPI_CPU_FREQ_PSS · 240 unmodified lines | |||
| 241 | 241 | ||
| 242 | config ACPI_PROCESSOR_CSTATE | 242 | config ACPI_PROCESSOR_CSTATE |
| 243 | def_bool y | 243 | def_bool y |
| 244 | depends on ACPI_PROCESSOR | ||
| 244 | depends on IA64 || X86 | 245 | depends on IA64 || X86 |
| 245 | 246 | ||
| 246 | config ACPI_PROCESSOR_IDLE | 247 | config ACPI_PROCESSOR_IDLE |
| @@ -705,3 +705,185 @@ void __init acpi_processor_init(void) · 704 unmodified lines | |||
| 705 | acpi_scan_add_handler_with_hotplug(&processor_handler, "processor"); | 705 | acpi_scan_add_handler_with_hotplug(&processor_handler, "processor"); |
| 706 | acpi_scan_add_handler(&processor_container_handler); | 706 | acpi_scan_add_handler(&processor_container_handler); |
| 707 | } | 707 | } |
| 708 | |||
| 709 | #ifdef CONFIG_ACPI_PROCESSOR_CSTATE | ||
| 710 | /** | ||
| 711 | * acpi_processor_claim_cst_control - Request _CST control from the platform. | ||
| 712 | */ | ||
| 713 | bool acpi_processor_claim_cst_control(void) | ||
| 714 | { | ||
| 715 | static bool cst_control_claimed; | ||
| 716 | acpi_status status; | ||
| 717 | |||
| 718 | if (!acpi_gbl_FADT.cst_control || cst_control_claimed) | ||
| 719 | return true; | ||
| 720 | |||
| 721 | status = acpi_os_write_port(acpi_gbl_FADT.smi_command, | ||
| 722 | acpi_gbl_FADT.cst_control, 8); | ||
| 723 | if (ACPI_FAILURE(status)) { | ||
| 724 | pr_warn("ACPI: Failed to claim processor _CST control\n"); | ||
| 725 | return false; | ||
| 726 | } | ||
| 727 | |||
| 728 | cst_control_claimed = true; | ||
| 729 | return true; | ||
| 730 | } | ||
| 731 | EXPORT_SYMBOL_GPL(acpi_processor_claim_cst_control); | ||
| 732 | |||
| 733 | /** | ||
| 734 | * acpi_processor_evaluate_cst - Evaluate the processor _CST control method. | ||
| 735 | * @handle: ACPI handle of the processor object containing the _CST. | ||
| 736 | * @cpu: The numeric ID of the target CPU. | ||
| 737 | * @info: Object write the C-states information into. | ||
| 738 | * | ||
| 739 | * Extract the C-state information for the given CPU from the output of the _CST | ||
| 740 | * control method under the corresponding ACPI processor object (or processor | ||
| 741 | * device object) and populate @info with it. | ||
| 742 | * | ||
| 743 | * If any ACPI_ADR_SPACE_FIXED_HARDWARE C-states are found, invoke | ||
| 744 | * acpi_processor_ffh_cstate_probe() to verify them and update the | ||
| 745 | * cpu_cstate_entry data for @cpu. | ||
| 746 | */ | ||
| 747 | int acpi_processor_evaluate_cst(acpi_handle handle, u32 cpu, | ||
| 748 | struct acpi_processor_power *info) | ||
| 749 | { | ||
| 750 | struct acpi_buffer buffer = { ACPI_ALLOCATE_BUFFER, NULL }; | ||
| 751 | union acpi_object *cst; | ||
| 752 | acpi_status status; | ||
| 753 | u64 count; | ||
| 754 | int last_index = 0; | ||
| 755 | int i, ret = 0; | ||
| 756 | |||
| 757 | status = acpi_evaluate_object(handle, "_CST", NULL, &buffer); | ||
| 758 | if (ACPI_FAILURE(status)) { | ||
| 759 | acpi_handle_debug(handle, "No _CST\n"); | ||
| 760 | return -ENODEV; | ||
| 761 | } | ||
| 762 | |||
| 763 | cst = buffer.pointer; | ||
| 764 | |||
| 765 | /* There must be at least 2 elements. */ | ||
| 766 | if (!cst || cst->type != ACPI_TYPE_PACKAGE || cst->package.count < 2) { | ||
| 767 | acpi_handle_warn(handle, "Invalid _CST output\n"); | ||
| 768 | ret = -EFAULT; | ||
| 769 | goto end; | ||
| 770 | } | ||
| 771 | |||
| 772 | count = cst->package.elements[0].integer.value; | ||
| 773 | |||
| 774 | /* Validate the number of C-states. */ | ||
| 775 | if (count < 1 || count != cst->package.count - 1) { | ||
| 776 | acpi_handle_warn(handle, "Inconsistent _CST data\n"); | ||
| 777 | ret = -EFAULT; | ||
| 778 | goto end; | ||
| 779 | } | ||
| 780 | |||
| 781 | for (i = 1; i <= count; i++) { | ||
| 782 | union acpi_object *element; | ||
| 783 | union acpi_object *obj; | ||
| 784 | struct acpi_power_register *reg; | ||
| 785 | struct acpi_processor_cx cx; | ||
| 786 | |||
| 787 | /* | ||
| 788 | * If there is not enough space for all C-states, skip the | ||
| 789 | * excess ones and log a warning. | ||
| 790 | */ | ||
| 791 | if (last_index >= ACPI_PROCESSOR_MAX_POWER - 1) { | ||
| 792 | acpi_handle_warn(handle, | ||
| 793 | "No room for more idle states (limit: %d)\n", | ||
| 794 | ACPI_PROCESSOR_MAX_POWER - 1); | ||
| 795 | break; | ||
| 796 | } | ||
| 797 | |||
| 798 | memset(&cx, 0, sizeof(cx)); | ||
| 799 | |||
| 800 | element = &cst->package.elements[i]; | ||
| 801 | if (element->type != ACPI_TYPE_PACKAGE) | ||
| 802 | continue; | ||
| 803 | |||
| 804 | if (element->package.count != 4) | ||
| 805 | continue; | ||
| 806 | |||
| 807 | obj = &element->package.elements[0]; | ||
| 808 | |||
| 809 | if (obj->type != ACPI_TYPE_BUFFER) | ||
| 810 | continue; | ||
| 811 | |||
| 812 | reg = (struct acpi_power_register *)obj->buffer.pointer; | ||
| 813 | |||
| 814 | obj = &element->package.elements[1]; | ||
| 815 | if (obj->type != ACPI_TYPE_INTEGER) | ||
| 816 | continue; | ||
| 817 | |||
| 818 | cx.type = obj->integer.value; | ||
| 819 | /* | ||
| 820 | * There are known cases in which the _CST output does not | ||
| 821 | * contain C1, so if the type of the first state found is not | ||
| 822 | * C1, leave an empty slot for C1 to be filled in later. | ||
| 823 | */ | ||
| 824 | if (i == 1 && cx.type != ACPI_STATE_C1) | ||
| 825 | last_index = 1; | ||
| 826 | |||
| 827 | cx.address = reg->address; | ||
| 828 | cx.index = last_index + 1; | ||
| 829 | |||
| 830 | if (reg->space_id == ACPI_ADR_SPACE_FIXED_HARDWARE) { | ||
| 831 | if (!acpi_processor_ffh_cstate_probe(cpu, &cx, reg)) { | ||
| 832 | /* | ||
| 833 | * In the majority of cases _CST describes C1 as | ||
| 834 | * a FIXED_HARDWARE C-state, but if the command | ||
| 835 | * line forbids using MWAIT, use CSTATE_HALT for | ||
| 836 | * C1 regardless. | ||
| 837 | */ | ||
| 838 | if (cx.type == ACPI_STATE_C1 && | ||
| 839 | boot_option_idle_override == IDLE_NOMWAIT) { | ||
| 840 | cx.entry_method = ACPI_CSTATE_HALT; | ||
| 841 | snprintf(cx.desc, ACPI_CX_DESC_LEN, "ACPI HLT"); | ||
| 842 | } else { | ||
| 843 | cx.entry_method = ACPI_CSTATE_FFH; | ||
| 844 | } | ||
| 845 | } else if (cx.type == ACPI_STATE_C1) { | ||
| 846 | /* | ||
| 847 | * In the special case of C1, FIXED_HARDWARE can | ||
| 848 | * be handled by executing the HLT instruction. | ||
| 849 | */ | ||
| 850 | cx.entry_method = ACPI_CSTATE_HALT; | ||
| 851 | snprintf(cx.desc, ACPI_CX_DESC_LEN, "ACPI HLT"); | ||
| 852 | } else { | ||
| 853 | continue; | ||
| 854 | } | ||
| 855 | } else if (reg->space_id == ACPI_ADR_SPACE_SYSTEM_IO) { | ||
| 856 | cx.entry_method = ACPI_CSTATE_SYSTEMIO; | ||
| 857 | snprintf(cx.desc, ACPI_CX_DESC_LEN, "ACPI IOPORT 0x%x", | ||
| 858 | cx.address); | ||
| 859 | } else { | ||
| 860 | continue; | ||
| 861 | } | ||
| 862 | |||
| 863 | if (cx.type == ACPI_STATE_C1) | ||
| 864 | cx.valid = 1; | ||
| 865 | |||
| 866 | obj = &element->package.elements[2]; | ||
| 867 | if (obj->type != ACPI_TYPE_INTEGER) | ||
| 868 | continue; | ||
| 869 | |||
| 870 | cx.latency = obj->integer.value; | ||
| 871 | |||
| 872 | obj = &element->package.elements[3]; | ||
| 873 | if (obj->type != ACPI_TYPE_INTEGER) | ||
| 874 | continue; | ||
| 875 | |||
| 876 | memcpy(&info->states[++last_index], &cx, sizeof(cx)); | ||
| 877 | } | ||
| 878 | |||
| 879 | acpi_handle_info(handle, "Found %d idle states\n", last_index); | ||
| 880 | |||
| 881 | info->count = last_index; | ||
| 882 | |||
| 883 | end: | ||
| 884 | kfree(buffer.pointer); | ||
| 885 | |||
| 886 | return ret; | ||
| 887 | } | ||
| 888 | EXPORT_SYMBOL_GPL(acpi_processor_evaluate_cst); | ||
| 889 | #endif /* CONFIG_ACPI_PROCESSOR_CSTATE */ | ||
| @@ -299,164 +299,24 @@ static int acpi_processor_get_power_info_default(struct acpi_processor *pr) · 298 unmodified lines | |||
| 299 | 299 | ||
| 300 | static int acpi_processor_get_power_info_cst(struct acpi_processor *pr) | 300 | static int acpi_processor_get_power_info_cst(struct acpi_processor *pr) |
| 301 | { | 301 | { |
| 302 | acpi_status status; | 302 | int ret; |
| 303 | u64 count; | ||
| 304 | int current_count; | ||
| 305 | int i, ret = 0; | ||
| 306 | struct acpi_buffer buffer = { ACPI_ALLOCATE_BUFFER, NULL }; | ||
| 307 | union acpi_object *cst; | ||
| 308 | 303 | ||
| 309 | if (nocst) | 304 | if (nocst) |
| 310 | return -ENODEV; | 305 | return -ENODEV; |
| 311 | 306 | ||
| 312 | current_count = 0; | 307 | ret = acpi_processor_evaluate_cst(pr->handle, pr->id, &pr->power); |
| 313 | 308 | if (ret) | |
| 314 | status = acpi_evaluate_object(pr->handle, "_CST", NULL, &buffer); | 309 | return ret; |
| 315 | if (ACPI_FAILURE(status)) { | ||
| 316 | ACPI_DEBUG_PRINT((ACPI_DB_INFO, "No _CST, giving up\n")); | ||
| 317 | return -ENODEV; | ||
| 318 | } | ||
| 319 | |||
| 320 | cst = buffer.pointer; | ||
| 321 | |||
| 322 | /* There must be at least 2 elements */ | ||
| 323 | if (!cst || (cst->type != ACPI_TYPE_PACKAGE) || cst->package.count < 2) { | ||
| 324 | pr_err("not enough elements in _CST\n"); | ||
| 325 | ret = -EFAULT; | ||
| 326 | goto end; | ||
| 327 | } | ||
| 328 | |||
| 329 | count = cst->package.elements[0].integer.value; | ||
| 330 | 310 | ||
| 331 | /* Validate number of power states. */ | 311 | /* |
| 332 | if (count < 1 || count != cst->package.count - 1) { | 312 | * It is expected that there will be at least 2 states, C1 and |
| 333 | pr_err("count given by _CST is not valid\n"); | 313 | * something else (C2 or C3), so fail if that is not the case. |
| 334 | ret = -EFAULT; | 314 | */ |
| 335 | goto end; | 315 | if (pr->power.count < 2) |
| 336 | } | 316 | return -EFAULT; |
| 337 | 317 | ||
| 338 | /* Tell driver that at least _CST is supported. */ | ||
| 339 | pr->flags.has_cst = 1; | 318 | pr->flags.has_cst = 1; |
| 340 | 319 | return 0; | |
| 341 | for (i = 1; i <= count; i++) { | ||
| 342 | union acpi_object *element; | ||
| 343 | union acpi_object *obj; | ||
| 344 | struct acpi_power_register *reg; | ||
| 345 | struct acpi_processor_cx cx; | ||
| 346 | |||
| 347 | memset(&cx, 0, sizeof(cx)); | ||
| 348 | |||
| 349 | element = &(cst->package.elements[i]); | ||
| 350 | if (element->type != ACPI_TYPE_PACKAGE) | ||
| 351 | continue; | ||
| 352 | |||
| 353 | if (element->package.count != 4) | ||
| 354 | continue; | ||
| 355 | |||
| 356 | obj = &(element->package.elements[0]); | ||
| 357 | |||
| 358 | if (obj->type != ACPI_TYPE_BUFFER) | ||
| 359 | continue; | ||
| 360 | |||
| 361 | reg = (struct acpi_power_register *)obj->buffer.pointer; | ||
| 362 | |||
| 363 | if (reg->space_id != ACPI_ADR_SPACE_SYSTEM_IO && | ||
| 364 | (reg->space_id != ACPI_ADR_SPACE_FIXED_HARDWARE)) | ||
| 365 | continue; | ||
| 366 | |||
| 367 | /* There should be an easy way to extract an integer... */ | ||
| 368 | obj = &(element->package.elements[1]); | ||
| 369 | if (obj->type != ACPI_TYPE_INTEGER) | ||
| 370 | continue; | ||
| 371 | |||
| 372 | cx.type = obj->integer.value; | ||
| 373 | /* | ||
| 374 | * Some buggy BIOSes won't list C1 in _CST - | ||
| 375 | * Let acpi_processor_get_power_info_default() handle them later | ||
| 376 | */ | ||
| 377 | if (i == 1 && cx.type != ACPI_STATE_C1) | ||
| 378 | current_count++; | ||
| 379 | |||
| 380 | cx.address = reg->address; | ||
| 381 | cx.index = current_count + 1; | ||
| 382 | |||
| 383 | cx.entry_method = ACPI_CSTATE_SYSTEMIO; | ||
| 384 | if (reg->space_id == ACPI_ADR_SPACE_FIXED_HARDWARE) { | ||
| 385 | if (acpi_processor_ffh_cstate_probe | ||
| 386 | (pr->id, &cx, reg) == 0) { | ||
| 387 | cx.entry_method = ACPI_CSTATE_FFH; | ||
| 388 | } else if (cx.type == ACPI_STATE_C1) { | ||
| 389 | /* | ||
| 390 | * C1 is a special case where FIXED_HARDWARE | ||
| 391 | * can be handled in non-MWAIT way as well. | ||
| 392 | * In that case, save this _CST entry info. | ||
| 393 | * Otherwise, ignore this info and continue. | ||
| 394 | */ | ||
| 395 | cx.entry_method = ACPI_CSTATE_HALT; | ||
| 396 | snprintf(cx.desc, ACPI_CX_DESC_LEN, "ACPI HLT"); | ||
| 397 | } else { | ||
| 398 | continue; | ||
| 399 | } | ||
| 400 | if (cx.type == ACPI_STATE_C1 && | ||
| 401 | (boot_option_idle_override == IDLE_NOMWAIT)) { | ||
| 402 | /* | ||
| 403 | * In most cases the C1 space_id obtained from | ||
| 404 | * _CST object is FIXED_HARDWARE access mode. | ||
| 405 | * But when the option of idle=halt is added, | ||
| 406 | * the entry_method type should be changed from | ||
| 407 | * CSTATE_FFH to CSTATE_HALT. | ||
| 408 | * When the option of idle=nomwait is added, | ||
| 409 | * the C1 entry_method type should be | ||
| 410 | * CSTATE_HALT. | ||
| 411 | */ | ||
| 412 | cx.entry_method = ACPI_CSTATE_HALT; | ||
| 413 | snprintf(cx.desc, ACPI_CX_DESC_LEN, "ACPI HLT"); | ||
| 414 | } | ||
| 415 | } else { | ||
| 416 | snprintf(cx.desc, ACPI_CX_DESC_LEN, "ACPI IOPORT 0x%x", | ||
| 417 | cx.address); | ||
| 418 | } | ||
| 419 | |||
| 420 | if (cx.type == ACPI_STATE_C1) { | ||
| 421 | cx.valid = 1; | ||
| 422 | } | ||
| 423 | |||
| 424 | obj = &(element->package.elements[2]); | ||
| 425 | if (obj->type != ACPI_TYPE_INTEGER) | ||
| 426 | continue; | ||
| 427 | |||
| 428 | cx.latency = obj->integer.value; | ||
| 429 | |||
| 430 | obj = &(element->package.elements[3]); | ||
| 431 | if (obj->type != ACPI_TYPE_INTEGER) | ||
| 432 | continue; | ||
| 433 | |||
| 434 | current_count++; | ||
| 435 | memcpy(&(pr->power.states[current_count]), &cx, sizeof(cx)); | ||
| 436 | |||
| 437 | /* | ||
| 438 | * We support total ACPI_PROCESSOR_MAX_POWER - 1 | ||
| 439 | * (From 1 through ACPI_PROCESSOR_MAX_POWER - 1) | ||
| 440 | */ | ||
| 441 | if (current_count >= (ACPI_PROCESSOR_MAX_POWER - 1)) { | ||
| 442 | pr_warn("Limiting number of power states to max (%d)\n", | ||
| 443 | ACPI_PROCESSOR_MAX_POWER); | ||
| 444 | pr_warn("Please increase ACPI_PROCESSOR_MAX_POWER if needed.\n"); | ||
| 445 | break; | ||
| 446 | } | ||
| 447 | } | ||
| 448 | |||
| 449 | ACPI_DEBUG_PRINT((ACPI_DB_INFO, "Found %d power states\n", | ||
| 450 | current_count)); | ||
| 451 | |||
| 452 | /* Validate number of power states discovered */ | ||
| 453 | if (current_count < 2) | ||
| 454 | ret = -EFAULT; | ||
| 455 | |||
| 456 | end: | ||
| 457 | kfree(buffer.pointer); | ||
| 458 | |||
| 459 | return ret; | ||
| 460 | } | 320 | } |
| 461 | 321 | ||
| 462 | static void acpi_processor_power_verify_c3(struct acpi_processor *pr, | 322 | static void acpi_processor_power_verify_c3(struct acpi_processor *pr, |
| @@ -909,7 +769,6 @@ static int acpi_processor_setup_cstates(struct acpi_processor *pr) · 446 unmodified lines | |||
| 909 | 769 | ||
| 910 | static inline void acpi_processor_cstate_first_run_checks(void) | 770 | static inline void acpi_processor_cstate_first_run_checks(void) |
| 911 | { | 771 | { |
| 912 | acpi_status status; | ||
| 913 | static int first_run; | 772 | static int first_run; |
| 914 | 773 | ||
| 915 | if (first_run) | 774 | if (first_run) |
| @@ -921,13 +780,10 @@ static inline void acpi_processor_cstate_first_run_checks(void) · 5 unmodified lines | |||
| 921 | max_cstate); | 780 | max_cstate); |
| 922 | first_run++; | 781 | first_run++; |
| 923 | 782 | ||
| 924 | if (acpi_gbl_FADT.cst_control && !nocst) { | 783 | if (nocst) |
| 925 | status = acpi_os_write_port(acpi_gbl_FADT.smi_command, | 784 | return; |
| 926 | acpi_gbl_FADT.cst_control, 8); | 785 | |
| 927 | if (ACPI_FAILURE(status)) | 786 | acpi_processor_claim_cst_control(); |
| 928 | ACPI_EXCEPTION((AE_INFO, status, | ||
| 929 | "Notifying BIOS of _CST ability failed")); | ||
| 930 | } | ||
| 931 | } | 787 | } |
| 932 | #else | 788 | #else |
| 933 | 789 | ||
| @@ -25,7 +25,7 @@ config ARM_PSCI_CPUIDLE · 24 unmodified lines | |||
| 25 | 25 | ||
| 26 | config ARM_BIG_LITTLE_CPUIDLE | 26 | config ARM_BIG_LITTLE_CPUIDLE |
| 27 | bool "Support for ARM big.LITTLE processors" | 27 | bool "Support for ARM big.LITTLE processors" |
| 28 | depends on ARCH_VEXPRESS_TC2_PM || ARCH_EXYNOS | 28 | depends on ARCH_VEXPRESS_TC2_PM || ARCH_EXYNOS || COMPILE_TEST |
| 29 | depends on MCPM && !ARM64 | 29 | depends on MCPM && !ARM64 |
| 30 | select ARM_CPU_SUSPEND | 30 | select ARM_CPU_SUSPEND |
| 31 | select CPU_IDLE_MULTIPLE_DRIVERS | 31 | select CPU_IDLE_MULTIPLE_DRIVERS |
| @@ -51,13 +51,13 @@ config ARM_HIGHBANK_CPUIDLE · 19 unmodified lines | |||
| 51 | 51 | ||
| 52 | config ARM_KIRKWOOD_CPUIDLE | 52 | config ARM_KIRKWOOD_CPUIDLE |
| 53 | bool "CPU Idle Driver for Marvell Kirkwood SoCs" | 53 | bool "CPU Idle Driver for Marvell Kirkwood SoCs" |
| 54 | depends on MACH_KIRKWOOD && !ARM64 | 54 | depends on (MACH_KIRKWOOD || COMPILE_TEST) && !ARM64 |
| 55 | help | 55 | help |
| 56 | This adds the CPU Idle driver for Marvell Kirkwood SoCs. | 56 | This adds the CPU Idle driver for Marvell Kirkwood SoCs. |
| 57 | 57 | ||
| 58 | config ARM_ZYNQ_CPUIDLE | 58 | config ARM_ZYNQ_CPUIDLE |
| 59 | bool "CPU Idle Driver for Xilinx Zynq processors" | 59 | bool "CPU Idle Driver for Xilinx Zynq processors" |
| 60 | depends on ARCH_ZYNQ && !ARM64 | 60 | depends on (ARCH_ZYNQ || COMPILE_TEST) && !ARM64 |
| 61 | help | 61 | help |
| 62 | Select this to enable cpuidle on Xilinx Zynq processors. | 62 | Select this to enable cpuidle on Xilinx Zynq processors. |
| 63 | 63 | ||
| @@ -70,19 +70,19 @@ config ARM_U8500_CPUIDLE · 6 unmodified lines | |||
| 70 | config ARM_AT91_CPUIDLE | 70 | config ARM_AT91_CPUIDLE |
| 71 | bool "Cpu Idle Driver for the AT91 processors" | 71 | bool "Cpu Idle Driver for the AT91 processors" |
| 72 | default y | 72 | default y |
| 73 | depends on ARCH_AT91 && !ARM64 | 73 | depends on (ARCH_AT91 || COMPILE_TEST) && !ARM64 |
| 74 | help | 74 | help |
| 75 | Select this to enable cpuidle for AT91 processors. | 75 | Select this to enable cpuidle for AT91 processors. |
| 76 | 76 | ||
| 77 | config ARM_EXYNOS_CPUIDLE | 77 | config ARM_EXYNOS_CPUIDLE |
| 78 | bool "Cpu Idle Driver for the Exynos processors" | 78 | bool "Cpu Idle Driver for the Exynos processors" |
| 79 | depends on ARCH_EXYNOS && !ARM64 | 79 | depends on (ARCH_EXYNOS || COMPILE_TEST) && !ARM64 |
| 80 | select ARCH_NEEDS_CPU_IDLE_COUPLED if SMP | 80 | select ARCH_NEEDS_CPU_IDLE_COUPLED if SMP |
| 81 | help | 81 | help |
| 82 | Select this to enable cpuidle for Exynos processors. | 82 | Select this to enable cpuidle for Exynos processors. |
| 83 | 83 | ||
| 84 | config ARM_MVEBU_V7_CPUIDLE | 84 | config ARM_MVEBU_V7_CPUIDLE |
| 85 | bool "CPU Idle Driver for mvebu v7 family processors" | 85 | bool "CPU Idle Driver for mvebu v7 family processors" |
| 86 | depends on ARCH_MVEBU && !ARM64 | 86 | depends on (ARCH_MVEBU || COMPILE_TEST) && !ARM64 |
| 87 | help | 87 | help |
| 88 | Select this to enable cpuidle on Armada 370, 38x and XP processors. | 88 | Select this to enable cpuidle on Armada 370, 38x and XP processors. |
| @@ -89,6 +89,7 @@ · 88 unmodified lines | |||
| 89 | * @coupled_cpus: mask of cpus that are part of the coupled set | 89 | * @coupled_cpus: mask of cpus that are part of the coupled set |
| 90 | * @requested_state: array of requested states for cpus in the coupled set | 90 | * @requested_state: array of requested states for cpus in the coupled set |
| 91 | * @ready_waiting_counts: combined count of cpus in ready or waiting loops | 91 | * @ready_waiting_counts: combined count of cpus in ready or waiting loops |
| 92 | * @abort_barrier: synchronisation point for abort cases | ||
| 92 | * @online_count: count of cpus that are online | 93 | * @online_count: count of cpus that are online |
| 93 | * @refcnt: reference count of cpuidle devices that are using this struct | 94 | * @refcnt: reference count of cpuidle devices that are using this struct |
| 94 | * @prevent: flag to prevent coupled idle while a cpu is hotplugging | 95 | * @prevent: flag to prevent coupled idle while a cpu is hotplugging |
| @@ -338,7 +339,7 @@ static void cpuidle_coupled_poke(int cpu) · 243 unmodified lines | |||
| 338 | 339 | ||
| 339 | /** | 340 | /** |
| 340 | * cpuidle_coupled_poke_others - wake up all other cpus that may be waiting | 341 | * cpuidle_coupled_poke_others - wake up all other cpus that may be waiting |
| 341 | * @dev: struct cpuidle_device for this cpu | 342 | * @this_cpu: target cpu |
| 342 | * @coupled: the struct coupled that contains the current cpu | 343 | * @coupled: the struct coupled that contains the current cpu |
| 343 | * | 344 | * |
| 344 | * Calls cpuidle_coupled_poke on all other online cpus. | 345 | * Calls cpuidle_coupled_poke on all other online cpus. |
| @@ -355,7 +356,7 @@ static void cpuidle_coupled_poke_others(int this_cpu, · 10 unmodified lines | |||
| 355 | 356 | ||
| 356 | /** | 357 | /** |
| 357 | * cpuidle_coupled_set_waiting - mark this cpu as in the wait loop | 358 | * cpuidle_coupled_set_waiting - mark this cpu as in the wait loop |
| 358 | * @dev: struct cpuidle_device for this cpu | 359 | * @cpu: target cpu |
| 359 | * @coupled: the struct coupled that contains the current cpu | 360 | * @coupled: the struct coupled that contains the current cpu |
| 360 | * @next_state: the index in drv->states of the requested state for this cpu | 361 | * @next_state: the index in drv->states of the requested state for this cpu |
| 361 | * | 362 | * |
| @@ -376,7 +377,7 @@ static int cpuidle_coupled_set_waiting(int cpu, · 14 unmodified lines | |||
| 376 | 377 | ||
| 377 | /** | 378 | /** |
| 378 | * cpuidle_coupled_set_not_waiting - mark this cpu as leaving the wait loop | 379 | * cpuidle_coupled_set_not_waiting - mark this cpu as leaving the wait loop |
| 379 | * @dev: struct cpuidle_device for this cpu | 380 | * @cpu: target cpu |
| 380 | * @coupled: the struct coupled that contains the current cpu | 381 | * @coupled: the struct coupled that contains the current cpu |
| 381 | * | 382 | * |
| 382 | * Removes the requested idle state for the specified cpuidle device. | 383 | * Removes the requested idle state for the specified cpuidle device. |
| @@ -412,7 +413,7 @@ static void cpuidle_coupled_set_done(int cpu, struct cpuidle_coupled *coupled) · 29 unmodified lines | |||
| 412 | 413 | ||
| 413 | /** | 414 | /** |
| 414 | * cpuidle_coupled_clear_pokes - spin until the poke interrupt is processed | 415 | * cpuidle_coupled_clear_pokes - spin until the poke interrupt is processed |
| 415 | * @cpu - this cpu | 416 | * @cpu: this cpu |
| 416 | * | 417 | * |
| 417 | * Turns on interrupts and spins until any outstanding poke interrupts have | 418 | * Turns on interrupts and spins until any outstanding poke interrupts have |
| 418 | * been processed and the poke bit has been cleared. | 419 | * been processed and the poke bit has been cleared. |
| @@ -37,10 +37,7 @@ static struct cpuidle_driver clps711x_idle_driver = { · 36 unmodified lines | |||
| 37 | 37 | ||
| 38 | static int __init clps711x_cpuidle_probe(struct platform_device *pdev) | 38 | static int __init clps711x_cpuidle_probe(struct platform_device *pdev) |
| 39 | { | 39 | { |
| 40 | struct resource *res; | 40 | clps711x_halt = devm_platform_ioremap_resource(pdev, 0); |
| 41 | |||
| 42 | res = platform_get_resource(pdev, IORESOURCE_MEM, 0); | ||
| 43 | clps711x_halt = devm_ioremap_resource(&pdev->dev, res); | ||
| 44 | if (IS_ERR(clps711x_halt)) | 41 | if (IS_ERR(clps711x_halt)) |
| 45 | return PTR_ERR(clps711x_halt); | 42 | return PTR_ERR(clps711x_halt); |
| 46 | 43 | ||
| @@ -55,10 +55,7 @@ static struct cpuidle_driver kirkwood_idle_driver = { · 54 unmodified lines | |||
| 55 | /* Initialize CPU idle by registering the idle states */ | 55 | /* Initialize CPU idle by registering the idle states */ |
| 56 | static int kirkwood_cpuidle_probe(struct platform_device *pdev) | 56 | static int kirkwood_cpuidle_probe(struct platform_device *pdev) |
| 57 | { | 57 | { |
| 58 | struct resource *res; | 58 | ddr_operation_base = devm_platform_ioremap_resource(pdev, 0); |
| 59 | |||
| 60 | res = platform_get_resource(pdev, IORESOURCE_MEM, 0); | ||
| 61 | ddr_operation_base = devm_ioremap_resource(&pdev->dev, res); | ||
| 62 | if (IS_ERR(ddr_operation_base)) | 59 | if (IS_ERR(ddr_operation_base)) |
| 63 | return PTR_ERR(ddr_operation_base); | 60 | return PTR_ERR(ddr_operation_base); |
| 64 | 61 | ||
| @@ -121,6 +121,9 @@ void cpuidle_use_deepest_state(u64 latency_limit_ns) · 120 unmodified lines | |||
| 121 | * cpuidle_find_deepest_state - Find the deepest available idle state. | 121 | * cpuidle_find_deepest_state - Find the deepest available idle state. |
| 122 | * @drv: cpuidle driver for the given CPU. | 122 | * @drv: cpuidle driver for the given CPU. |
| 123 | * @dev: cpuidle device for the given CPU. | 123 | * @dev: cpuidle device for the given CPU. |
| 124 | * @latency_limit_ns: Idle state exit latency limit | ||
| 125 | * | ||
| 126 | * Return: the index of the deepest available idle state. | ||
| 124 | */ | 127 | */ |
| 125 | int cpuidle_find_deepest_state(struct cpuidle_driver *drv, | 128 | int cpuidle_find_deepest_state(struct cpuidle_driver *drv, |
| 126 | struct cpuidle_device *dev, | 129 | struct cpuidle_device *dev, |
| @@ -572,10 +575,14 @@ static int __cpuidle_register_device(struct cpuidle_device *dev) · 445 unmodified lines | |||
| 572 | if (!try_module_get(drv->owner)) | 575 | if (!try_module_get(drv->owner)) |
| 573 | return -EINVAL; | 576 | return -EINVAL; |
| 574 | 577 | ||
| 575 | for (i = 0; i < drv->state_count; i++) | 578 | for (i = 0; i < drv->state_count; i++) { |
| 576 | if (drv->states[i].flags & CPUIDLE_FLAG_UNUSABLE) | 579 | if (drv->states[i].flags & CPUIDLE_FLAG_UNUSABLE) |
| 577 | dev->states_usage[i].disable |= CPUIDLE_STATE_DISABLED_BY_DRIVER; | 580 | dev->states_usage[i].disable |= CPUIDLE_STATE_DISABLED_BY_DRIVER; |
| 578 | 581 | ||
| 582 | if (drv->states[i].flags & CPUIDLE_FLAG_OFF) | ||
| 583 | dev->states_usage[i].disable |= CPUIDLE_STATE_DISABLED_BY_USER; | ||
| 584 | } | ||
| 585 | |||
| 579 | per_cpu(cpuidle_devices, dev->cpu) = dev; | 586 | per_cpu(cpuidle_devices, dev->cpu) = dev; |
| 580 | list_add(&dev->device_list, &cpuidle_detected_devices); | 587 | list_add(&dev->device_list, &cpuidle_detected_devices); |
| 581 | 588 | ||
| @@ -155,8 +155,6 @@ static void __cpuidle_driver_init(struct cpuidle_driver *drv) · 154 unmodified lines | |||
| 155 | { | 155 | { |
| 156 | int i; | 156 | int i; |
| 157 | 157 | ||
| 158 | drv->refcnt = 0; | ||
| 159 | |||
| 160 | /* | 158 | /* |
| 161 | * Use all possible CPUs as the default, because if the kernel boots | 159 | * Use all possible CPUs as the default, because if the kernel boots |
| 162 | * with some CPUs offline and then we online one of them, the CPU | 160 | * with some CPUs offline and then we online one of them, the CPU |
| @@ -240,9 +238,6 @@ static int __cpuidle_register_driver(struct cpuidle_driver *drv) · 77 unmodified lines | |||
| 240 | */ | 238 | */ |
| 241 | static void __cpuidle_unregister_driver(struct cpuidle_driver *drv) | 239 | static void __cpuidle_unregister_driver(struct cpuidle_driver *drv) |
| 242 | { | 240 | { |
| 243 | if (WARN_ON(drv->refcnt > 0)) | ||
| 244 | return; | ||
| 245 | |||
| 246 | if (drv->bctimer) { | 241 | if (drv->bctimer) { |
| 247 | drv->bctimer = 0; | 242 | drv->bctimer = 0; |
| 248 | on_each_cpu_mask(drv->cpumask, cpuidle_setup_broadcast_timer, | 243 | on_each_cpu_mask(drv->cpumask, cpuidle_setup_broadcast_timer, |
| @@ -349,47 +344,6 @@ struct cpuidle_driver *cpuidle_get_cpu_driver(struct cpuidle_device *dev) · 100 unmodified lines | |||
| 349 | } | 344 | } |
| 350 | EXPORT_SYMBOL_GPL(cpuidle_get_cpu_driver); | 345 | EXPORT_SYMBOL_GPL(cpuidle_get_cpu_driver); |
| 351 | 346 | ||
| 352 | /** | ||
| 353 | * cpuidle_driver_ref - get a reference to the driver. | ||
| 354 | * | ||
| 355 | * Increment the reference counter of the cpuidle driver associated with | ||
| 356 | * the current CPU. | ||
| 357 | * | ||
| 358 | * Returns a pointer to the driver, or NULL if the current CPU has no driver. | ||
| 359 | */ | ||
| 360 | struct cpuidle_driver *cpuidle_driver_ref(void) | ||
| 361 | { | ||
| 362 | struct cpuidle_driver *drv; | ||
| 363 | |||
| 364 | spin_lock(&cpuidle_driver_lock); | ||
| 365 | |||
| 366 | drv = cpuidle_get_driver(); | ||
| 367 | if (drv) | ||
| 368 | drv->refcnt++; | ||
| 369 | |||
| 370 | spin_unlock(&cpuidle_driver_lock); | ||
| 371 | return drv; | ||
| 372 | } | ||
| 373 | |||
| 374 | /** | ||
| 375 | * cpuidle_driver_unref - puts down the refcount for the driver | ||
| 376 | * | ||
| 377 | * Decrement the reference counter of the cpuidle driver associated with | ||
| 378 | * the current CPU. | ||
| 379 | */ | ||
| 380 | void cpuidle_driver_unref(void) | ||
| 381 | { | ||
| 382 | struct cpuidle_driver *drv; | ||
| 383 | |||
| 384 | spin_lock(&cpuidle_driver_lock); | ||
| 385 | |||
| 386 | drv = cpuidle_get_driver(); | ||
| 387 | if (drv && !WARN_ON(drv->refcnt <= 0)) | ||
| 388 | drv->refcnt--; | ||
| 389 | |||
| 390 | spin_unlock(&cpuidle_driver_lock); | ||
| 391 | } | ||
| 392 | |||
| 393 | /** | 347 | /** |
| 394 | * cpuidle_driver_state_disabled - Disable or enable an idle state | 348 | * cpuidle_driver_state_disabled - Disable or enable an idle state |
| 395 | * @drv: cpuidle driver owning the state | 349 | * @drv: cpuidle driver owning the state |
| @@ -142,6 +142,7 @@ static struct attribute_group cpuidle_attr_group = { · 141 unmodified lines | |||
| 142 | 142 | ||
| 143 | /** | 143 | /** |
| 144 | * cpuidle_add_interface - add CPU global sysfs attributes | 144 | * cpuidle_add_interface - add CPU global sysfs attributes |
| 145 | * @dev: the target device | ||
| 145 | */ | 146 | */ |
| 146 | int cpuidle_add_interface(struct device *dev) | 147 | int cpuidle_add_interface(struct device *dev) |
| 147 | { | 148 | { |
| @@ -153,6 +154,7 @@ int cpuidle_add_interface(struct device *dev) · 5 unmodified lines | |||
| 153 | 154 | ||
| 154 | /** | 155 | /** |
| 155 | * cpuidle_remove_interface - remove CPU global sysfs attributes | 156 | * cpuidle_remove_interface - remove CPU global sysfs attributes |
| 157 | * @dev: the target device | ||
| 156 | */ | 158 | */ |
| 157 | void cpuidle_remove_interface(struct device *dev) | 159 | void cpuidle_remove_interface(struct device *dev) |
| 158 | { | 160 | { |
| @@ -327,6 +329,14 @@ static ssize_t store_state_disable(struct cpuidle_state *state, · 168 unmodified lines | |||
| 327 | return size; | 329 | return size; |
| 328 | } | 330 | } |
| 329 | 331 | ||
| 332 | static ssize_t show_state_default_status(struct cpuidle_state *state, | ||
| 333 | struct cpuidle_state_usage *state_usage, | ||
| 334 | char *buf) | ||
| 335 | { | ||
| 336 | return sprintf(buf, "%s\n", | ||
| 337 | state->flags & CPUIDLE_FLAG_OFF ? "disabled" : "enabled"); | ||
| 338 | } | ||
| 339 | |||
| 330 | define_one_state_ro(name, show_state_name); | 340 | define_one_state_ro(name, show_state_name); |
| 331 | define_one_state_ro(desc, show_state_desc); | 341 | define_one_state_ro(desc, show_state_desc); |
| 332 | define_one_state_ro(latency, show_state_exit_latency); | 342 | define_one_state_ro(latency, show_state_exit_latency); |
| @@ -337,6 +347,7 @@ define_one_state_ro(time, show_state_time); · 4 unmodified lines | |||
| 337 | define_one_state_rw(disable, show_state_disable, store_state_disable); | 347 | define_one_state_rw(disable, show_state_disable, store_state_disable); |
| 338 | define_one_state_ro(above, show_state_above); | 348 | define_one_state_ro(above, show_state_above); |
| 339 | define_one_state_ro(below, show_state_below); | 349 | define_one_state_ro(below, show_state_below); |
| 350 | define_one_state_ro(default_status, show_state_default_status); | ||
| 340 | 351 | ||
| 341 | static struct attribute *cpuidle_state_default_attrs[] = { | 352 | static struct attribute *cpuidle_state_default_attrs[] = { |
| 342 | &attr_name.attr, | 353 | &attr_name.attr, |
| @@ -349,6 +360,7 @@ static struct attribute *cpuidle_state_default_attrs[] = { · 6 unmodified lines | |||
| 349 | &attr_disable.attr, | 360 | &attr_disable.attr, |
| 350 | &attr_above.attr, | 361 | &attr_above.attr, |
| 351 | &attr_below.attr, | 362 | &attr_below.attr, |
| 363 | &attr_default_status.attr, | ||
| 352 | NULL | 364 | NULL |
| 353 | }; | 365 | }; |
| 354 | 366 | ||
| @@ -615,7 +627,7 @@ static struct kobj_type ktype_driver_cpuidle = { · 260 unmodified lines | |||
| 615 | 627 | ||
| 616 | /** | 628 | /** |
| 617 | * cpuidle_add_driver_sysfs - adds the driver name sysfs attribute | 629 | * cpuidle_add_driver_sysfs - adds the driver name sysfs attribute |
| 618 | * @device: the target device | 630 | * @dev: the target device |
| 619 | */ | 631 | */ |
| 620 | static int cpuidle_add_driver_sysfs(struct cpuidle_device *dev) | 632 | static int cpuidle_add_driver_sysfs(struct cpuidle_device *dev) |
| 621 | { | 633 | { |
| @@ -646,7 +658,7 @@ static int cpuidle_add_driver_sysfs(struct cpuidle_device *dev) · 24 unmodified lines | |||
| 646 | 658 | ||
| 647 | /** | 659 | /** |
| 648 | * cpuidle_remove_driver_sysfs - removes the driver name sysfs attribute | 660 | * cpuidle_remove_driver_sysfs - removes the driver name sysfs attribute |
| 649 | * @device: the target device | 661 | * @dev: the target device |
| 650 | */ | 662 | */ |
| 651 | static void cpuidle_remove_driver_sysfs(struct cpuidle_device *dev) | 663 | static void cpuidle_remove_driver_sysfs(struct cpuidle_device *dev) |
| 652 | { | 664 | { |
| @@ -41,6 +41,7 @@ · 40 unmodified lines | |||
| 41 | 41 | ||
| 42 | #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt | 42 | #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt |
| 43 | 43 | ||
| 44 | #include <linux/acpi.h> | ||
| 44 | #include <linux/kernel.h> | 45 | #include <linux/kernel.h> |
| 45 | #include <linux/cpuidle.h> | 46 | #include <linux/cpuidle.h> |
| 46 | #include <linux/tick.h> | 47 | #include <linux/tick.h> |
| @@ -79,6 +80,7 @@ struct idle_cpu { · 32 unmodified lines | |||
| 79 | unsigned long auto_demotion_disable_flags; | 80 | unsigned long auto_demotion_disable_flags; |
| 80 | bool byt_auto_demotion_disable_flag; | 81 | bool byt_auto_demotion_disable_flag; |
| 81 | bool disable_promotion_to_c1e; | 82 | bool disable_promotion_to_c1e; |
| 83 | bool use_acpi; | ||
| 82 | }; | 84 | }; |
| 83 | 85 | ||
| 84 | static const struct idle_cpu *icpu; | 86 | static const struct idle_cpu *icpu; |
| @@ -89,6 +91,11 @@ static void intel_idle_s2idle(struct cpuidle_device *dev, · 4 unmodified lines | |||
| 89 | struct cpuidle_driver *drv, int index); | 91 | struct cpuidle_driver *drv, int index); |
| 90 | static struct cpuidle_state *cpuidle_state_table; | 92 | static struct cpuidle_state *cpuidle_state_table; |
| 91 | 93 | ||
| 94 | /* | ||
| 95 | * Enable this state by default even if the ACPI _CST does not list it. | ||
| 96 | */ | ||
| 97 | #define CPUIDLE_FLAG_ALWAYS_ENABLE BIT(15) | ||
| 98 | |||
| 92 | /* | 99 | /* |
| 93 | * Set this flag for states where the HW flushes the TLB for us | 100 | * Set this flag for states where the HW flushes the TLB for us |
| 94 | * and so we don't need cross-calls to keep it consistent. | 101 | * and so we don't need cross-calls to keep it consistent. |
| @@ -124,7 +131,7 @@ static struct cpuidle_state nehalem_cstates[] = { · 29 unmodified lines | |||
| 124 | { | 131 | { |
| 125 | .name = "C1E", | 132 | .name = "C1E", |
| 126 | .desc = "MWAIT 0x01", | 133 | .desc = "MWAIT 0x01", |
| 127 | .flags = MWAIT2flg(0x01), | 134 | .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE, |
| 128 | .exit_latency = 10, | 135 | .exit_latency = 10, |
| 129 | .target_residency = 20, | 136 | .target_residency = 20, |
| 130 | .enter = &intel_idle, | 137 | .enter = &intel_idle, |
| @@ -161,7 +168,7 @@ static struct cpuidle_state snb_cstates[] = { · 30 unmodified lines | |||
| 161 | { | 168 | { |
| 162 | .name = "C1E", | 169 | .name = "C1E", |
| 163 | .desc = "MWAIT 0x01", | 170 | .desc = "MWAIT 0x01", |
| 164 | .flags = MWAIT2flg(0x01), | 171 | .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE, |
| 165 | .exit_latency = 10, | 172 | .exit_latency = 10, |
| 166 | .target_residency = 20, | 173 | .target_residency = 20, |
| 167 | .enter = &intel_idle, | 174 | .enter = &intel_idle, |
| @@ -296,7 +303,7 @@ static struct cpuidle_state ivb_cstates[] = { · 128 unmodified lines | |||
| 296 | { | 303 | { |
| 297 | .name = "C1E", | 304 | .name = "C1E", |
| 298 | .desc = "MWAIT 0x01", | 305 | .desc = "MWAIT 0x01", |
| 299 | .flags = MWAIT2flg(0x01), | 306 | .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE, |
| 300 | .exit_latency = 10, | 307 | .exit_latency = 10, |
| 301 | .target_residency = 20, | 308 | .target_residency = 20, |
| 302 | .enter = &intel_idle, | 309 | .enter = &intel_idle, |
| @@ -341,7 +348,7 @@ static struct cpuidle_state ivt_cstates[] = { · 38 unmodified lines | |||
| 341 | { | 348 | { |
| 342 | .name = "C1E", | 349 | .name = "C1E", |
| 343 | .desc = "MWAIT 0x01", | 350 | .desc = "MWAIT 0x01", |
| 344 | .flags = MWAIT2flg(0x01), | 351 | .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE, |
| 345 | .exit_latency = 10, | 352 | .exit_latency = 10, |
| 346 | .target_residency = 80, | 353 | .target_residency = 80, |
| 347 | .enter = &intel_idle, | 354 | .enter = &intel_idle, |
| @@ -378,7 +385,7 @@ static struct cpuidle_state ivt_cstates_4s[] = { · 30 unmodified lines | |||
| 378 | { | 385 | { |
| 379 | .name = "C1E", | 386 | .name = "C1E", |
| 380 | .desc = "MWAIT 0x01", | 387 | .desc = "MWAIT 0x01", |
| 381 | .flags = MWAIT2flg(0x01), | 388 | .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE, |
| 382 | .exit_latency = 10, | 389 | .exit_latency = 10, |
| 383 | .target_residency = 250, | 390 | .target_residency = 250, |
| 384 | .enter = &intel_idle, | 391 | .enter = &intel_idle, |
| @@ -415,7 +422,7 @@ static struct cpuidle_state ivt_cstates_8s[] = { · 30 unmodified lines | |||
| 415 | { | 422 | { |
| 416 | .name = "C1E", | 423 | .name = "C1E", |
| 417 | .desc = "MWAIT 0x01", | 424 | .desc = "MWAIT 0x01", |
| 418 | .flags = MWAIT2flg(0x01), | 425 | .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE, |
| 419 | .exit_latency = 10, | 426 | .exit_latency = 10, |
| 420 | .target_residency = 500, | 427 | .target_residency = 500, |
| 421 | .enter = &intel_idle, | 428 | .enter = &intel_idle, |
| @@ -452,7 +459,7 @@ static struct cpuidle_state hsw_cstates[] = { · 30 unmodified lines | |||
| 452 | { | 459 | { |
| 453 | .name = "C1E", | 460 | .name = "C1E", |
| 454 | .desc = "MWAIT 0x01", | 461 | .desc = "MWAIT 0x01", |
| 455 | .flags = MWAIT2flg(0x01), | 462 | .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE, |
| 456 | .exit_latency = 10, | 463 | .exit_latency = 10, |
| 457 | .target_residency = 20, | 464 | .target_residency = 20, |
| 458 | .enter = &intel_idle, | 465 | .enter = &intel_idle, |
| @@ -520,7 +527,7 @@ static struct cpuidle_state bdw_cstates[] = { · 61 unmodified lines | |||
| 520 | { | 527 | { |
| 521 | .name = "C1E", | 528 | .name = "C1E", |
| 522 | .desc = "MWAIT 0x01", | 529 | .desc = "MWAIT 0x01", |
| 523 | .flags = MWAIT2flg(0x01), | 530 | .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE, |
| 524 | .exit_latency = 10, | 531 | .exit_latency = 10, |
| 525 | .target_residency = 20, | 532 | .target_residency = 20, |
| 526 | .enter = &intel_idle, | 533 | .enter = &intel_idle, |
| @@ -589,7 +596,7 @@ static struct cpuidle_state skl_cstates[] = { · 62 unmodified lines | |||
| 589 | { | 596 | { |
| 590 | .name = "C1E", | 597 | .name = "C1E", |
| 591 | .desc = "MWAIT 0x01", | 598 | .desc = "MWAIT 0x01", |
| 592 | .flags = MWAIT2flg(0x01), | 599 | .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE, |
| 593 | .exit_latency = 10, | 600 | .exit_latency = 10, |
| 594 | .target_residency = 20, | 601 | .target_residency = 20, |
| 595 | .enter = &intel_idle, | 602 | .enter = &intel_idle, |
| @@ -658,7 +665,7 @@ static struct cpuidle_state skx_cstates[] = { · 62 unmodified lines | |||
| 658 | { | 665 | { |
| 659 | .name = "C1E", | 666 | .name = "C1E", |
| 660 | .desc = "MWAIT 0x01", | 667 | .desc = "MWAIT 0x01", |
| 661 | .flags = MWAIT2flg(0x01), | 668 | .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE, |
| 662 | .exit_latency = 10, | 669 | .exit_latency = 10, |
| 663 | .target_residency = 20, | 670 | .target_residency = 20, |
| 664 | .enter = &intel_idle, | 671 | .enter = &intel_idle, |
| @@ -808,7 +815,7 @@ static struct cpuidle_state bxt_cstates[] = { · 143 unmodified lines | |||
| 808 | { | 815 | { |
| 809 | .name = "C1E", | 816 | .name = "C1E", |
| 810 | .desc = "MWAIT 0x01", | 817 | .desc = "MWAIT 0x01", |
| 811 | .flags = MWAIT2flg(0x01), | 818 | .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE, |
| 812 | .exit_latency = 10, | 819 | .exit_latency = 10, |
| 813 | .target_residency = 20, | 820 | .target_residency = 20, |
| 814 | .enter = &intel_idle, | 821 | .enter = &intel_idle, |
| @@ -869,7 +876,7 @@ static struct cpuidle_state dnv_cstates[] = { · 54 unmodified lines | |||
| 869 | { | 876 | { |
| 870 | .name = "C1E", | 877 | .name = "C1E", |
| 871 | .desc = "MWAIT 0x01", | 878 | .desc = "MWAIT 0x01", |
| 872 | .flags = MWAIT2flg(0x01), | 879 | .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE, |
| 873 | .exit_latency = 10, | 880 | .exit_latency = 10, |
| 874 | .target_residency = 20, | 881 | .target_residency = 20, |
| 875 | .enter = &intel_idle, | 882 | .enter = &intel_idle, |
| @@ -944,37 +951,19 @@ static void intel_idle_s2idle(struct cpuidle_device *dev, · 68 unmodified lines | |||
| 944 | mwait_idle_with_hints(eax, ecx); | 951 | mwait_idle_with_hints(eax, ecx); |
| 945 | } | 952 | } |
| 946 | 953 | ||
| 947 | static void __setup_broadcast_timer(bool on) | ||
| 948 | { | ||
| 949 | if (on) | ||
| 950 | tick_broadcast_enable(); | ||
| 951 | else | ||
| 952 | tick_broadcast_disable(); | ||
| 953 | } | ||
| 954 | |||
| 955 | static void auto_demotion_disable(void) | ||
| 956 | { | ||
| 957 | unsigned long long msr_bits; | ||
| 958 | |||
| 959 | rdmsrl(MSR_PKG_CST_CONFIG_CONTROL, msr_bits); | ||
| 960 | msr_bits &= ~(icpu->auto_demotion_disable_flags); | ||
| 961 | wrmsrl(MSR_PKG_CST_CONFIG_CONTROL, msr_bits); | ||
| 962 | } | ||
| 963 | static void c1e_promotion_disable(void) | ||
| 964 | { | ||
| 965 | unsigned long long msr_bits; | ||
| 966 | |||
| 967 | rdmsrl(MSR_IA32_POWER_CTL, msr_bits); | ||
| 968 | msr_bits &= ~0x2; | ||
| 969 | wrmsrl(MSR_IA32_POWER_CTL, msr_bits); | ||
| 970 | } | ||
| 971 | |||
| 972 | static const struct idle_cpu idle_cpu_nehalem = { | 954 | static const struct idle_cpu idle_cpu_nehalem = { |
| 973 | .state_table = nehalem_cstates, | 955 | .state_table = nehalem_cstates, |
| 974 | .auto_demotion_disable_flags = NHM_C1_AUTO_DEMOTE | NHM_C3_AUTO_DEMOTE, | 956 | .auto_demotion_disable_flags = NHM_C1_AUTO_DEMOTE | NHM_C3_AUTO_DEMOTE, |
| 975 | .disable_promotion_to_c1e = true, | 957 | .disable_promotion_to_c1e = true, |
| 976 | }; | 958 | }; |
| 977 | 959 | ||
| 960 | static const struct idle_cpu idle_cpu_nhx = { | ||
| 961 | .state_table = nehalem_cstates, | ||
| 962 | .auto_demotion_disable_flags = NHM_C1_AUTO_DEMOTE | NHM_C3_AUTO_DEMOTE, | ||
| 963 | .disable_promotion_to_c1e = true, | ||
| 964 | .use_acpi = true, | ||
| 965 | }; | ||
| 966 | |||
| 978 | static const struct idle_cpu idle_cpu_atom = { | 967 | static const struct idle_cpu idle_cpu_atom = { |
| 979 | .state_table = atom_cstates, | 968 | .state_table = atom_cstates, |
| 980 | }; | 969 | }; |
| @@ -993,6 +982,12 @@ static const struct idle_cpu idle_cpu_snb = { · 12 unmodified lines | |||
| 993 | .disable_promotion_to_c1e = true, | 982 | .disable_promotion_to_c1e = true, |
| 994 | }; | 983 | }; |
| 995 | 984 | ||
| 985 | static const struct idle_cpu idle_cpu_snx = { | ||
| 986 | .state_table = snb_cstates, | ||
| 987 | .disable_promotion_to_c1e = true, | ||
| 988 | .use_acpi = true, | ||
| 989 | }; | ||
| 990 | |||
| 996 | static const struct idle_cpu idle_cpu_byt = { | 991 | static const struct idle_cpu idle_cpu_byt = { |
| 997 | .state_table = byt_cstates, | 992 | .state_table = byt_cstates, |
| 998 | .disable_promotion_to_c1e = true, | 993 | .disable_promotion_to_c1e = true, |
| @@ -1013,6 +1008,7 @@ static const struct idle_cpu idle_cpu_ivb = { · 14 unmodified lines | |||
| 1013 | static const struct idle_cpu idle_cpu_ivt = { | 1008 | static const struct idle_cpu idle_cpu_ivt = { |
| 1014 | .state_table = ivt_cstates, | 1009 | .state_table = ivt_cstates, |
| 1015 | .disable_promotion_to_c1e = true, | 1010 | .disable_promotion_to_c1e = true, |
| 1011 | .use_acpi = true, | ||
| 1016 | }; | 1012 | }; |
| 1017 | 1013 | ||
| 1018 | static const struct idle_cpu idle_cpu_hsw = { | 1014 | static const struct idle_cpu idle_cpu_hsw = { |
| @@ -1020,11 +1016,23 @@ static const struct idle_cpu idle_cpu_hsw = { · 1 unmodified line | |||
| 1020 | .disable_promotion_to_c1e = true, | 1016 | .disable_promotion_to_c1e = true, |
| 1021 | }; | 1017 | }; |
| 1022 | 1018 | ||
| 1019 | static const struct idle_cpu idle_cpu_hsx = { | ||
| 1020 | .state_table = hsw_cstates, | ||
| 1021 | .disable_promotion_to_c1e = true, | ||
| 1022 | .use_acpi = true, | ||
| 1023 | }; | ||
| 1024 | |||
| 1023 | static const struct idle_cpu idle_cpu_bdw = { | 1025 | static const struct idle_cpu idle_cpu_bdw = { |
| 1024 | .state_table = bdw_cstates, | 1026 | .state_table = bdw_cstates, |
| 1025 | .disable_promotion_to_c1e = true, | 1027 | .disable_promotion_to_c1e = true, |
| 1026 | }; | 1028 | }; |
| 1027 | 1029 | ||
| 1030 | static const struct idle_cpu idle_cpu_bdx = { | ||
| 1031 | .state_table = bdw_cstates, | ||
| 1032 | .disable_promotion_to_c1e = true, | ||
| 1033 | .use_acpi = true, | ||
| 1034 | }; | ||
| 1035 | |||
| 1028 | static const struct idle_cpu idle_cpu_skl = { | 1036 | static const struct idle_cpu idle_cpu_skl = { |
| 1029 | .state_table = skl_cstates, | 1037 | .state_table = skl_cstates, |
| 1030 | .disable_promotion_to_c1e = true, | 1038 | .disable_promotion_to_c1e = true, |
| @@ -1033,15 +1041,18 @@ static const struct idle_cpu idle_cpu_skl = { · 2 unmodified lines | |||
| 1033 | static const struct idle_cpu idle_cpu_skx = { | 1041 | static const struct idle_cpu idle_cpu_skx = { |
| 1034 | .state_table = skx_cstates, | 1042 | .state_table = skx_cstates, |
| 1035 | .disable_promotion_to_c1e = true, | 1043 | .disable_promotion_to_c1e = true, |
| 1044 | .use_acpi = true, | ||
| 1036 | }; | 1045 | }; |
| 1037 | 1046 | ||
| 1038 | static const struct idle_cpu idle_cpu_avn = { | 1047 | static const struct idle_cpu idle_cpu_avn = { |
| 1039 | .state_table = avn_cstates, | 1048 | .state_table = avn_cstates, |
| 1040 | .disable_promotion_to_c1e = true, | 1049 | .disable_promotion_to_c1e = true, |
| 1050 | .use_acpi = true, | ||
| 1041 | }; | 1051 | }; |
| 1042 | 1052 | ||
| 1043 | static const struct idle_cpu idle_cpu_knl = { | 1053 | static const struct idle_cpu idle_cpu_knl = { |
| 1044 | .state_table = knl_cstates, | 1054 | .state_table = knl_cstates, |
| 1055 | .use_acpi = true, | ||
| 1045 | }; | 1056 | }; |
| 1046 | 1057 | ||
| 1047 | static const struct idle_cpu idle_cpu_bxt = { | 1058 | static const struct idle_cpu idle_cpu_bxt = { |
| @@ -1052,20 +1063,21 @@ static const struct idle_cpu idle_cpu_bxt = { · 4 unmodified lines | |||
| 1052 | static const struct idle_cpu idle_cpu_dnv = { | 1063 | static const struct idle_cpu idle_cpu_dnv = { |
| 1053 | .state_table = dnv_cstates, | 1064 | .state_table = dnv_cstates, |
| 1054 | .disable_promotion_to_c1e = true, | 1065 | .disable_promotion_to_c1e = true, |
| 1066 | .use_acpi = true, | ||
| 1055 | }; | 1067 | }; |
| 1056 | 1068 | ||
| 1057 | static const struct x86_cpu_id intel_idle_ids[] __initconst = { | 1069 | static const struct x86_cpu_id intel_idle_ids[] __initconst = { |
| 1058 | INTEL_CPU_FAM6(NEHALEM_EP, idle_cpu_nehalem), | 1070 | INTEL_CPU_FAM6(NEHALEM_EP, idle_cpu_nhx), |
| 1059 | INTEL_CPU_FAM6(NEHALEM, idle_cpu_nehalem), | 1071 | INTEL_CPU_FAM6(NEHALEM, idle_cpu_nehalem), |
| 1060 | INTEL_CPU_FAM6(NEHALEM_G, idle_cpu_nehalem), | 1072 | INTEL_CPU_FAM6(NEHALEM_G, idle_cpu_nehalem), |
| 1061 | INTEL_CPU_FAM6(WESTMERE, idle_cpu_nehalem), | 1073 | INTEL_CPU_FAM6(WESTMERE, idle_cpu_nehalem), |
| 1062 | INTEL_CPU_FAM6(WESTMERE_EP, idle_cpu_nehalem), | 1074 | INTEL_CPU_FAM6(WESTMERE_EP, idle_cpu_nhx), |
| 1063 | INTEL_CPU_FAM6(NEHALEM_EX, idle_cpu_nehalem), | 1075 | INTEL_CPU_FAM6(NEHALEM_EX, idle_cpu_nhx), |
| 1064 | INTEL_CPU_FAM6(ATOM_BONNELL, idle_cpu_atom), | 1076 | INTEL_CPU_FAM6(ATOM_BONNELL, idle_cpu_atom), |
| 1065 | INTEL_CPU_FAM6(ATOM_BONNELL_MID, idle_cpu_lincroft), | 1077 | INTEL_CPU_FAM6(ATOM_BONNELL_MID, idle_cpu_lincroft), |
| 1066 | INTEL_CPU_FAM6(WESTMERE_EX, idle_cpu_nehalem), | 1078 | INTEL_CPU_FAM6(WESTMERE_EX, idle_cpu_nhx), |
| 1067 | INTEL_CPU_FAM6(SANDYBRIDGE, idle_cpu_snb), | 1079 | INTEL_CPU_FAM6(SANDYBRIDGE, idle_cpu_snb), |
| 1068 | INTEL_CPU_FAM6(SANDYBRIDGE_X, idle_cpu_snb), | 1080 | INTEL_CPU_FAM6(SANDYBRIDGE_X, idle_cpu_snx), |
| 1069 | INTEL_CPU_FAM6(ATOM_SALTWELL, idle_cpu_atom), | 1081 | INTEL_CPU_FAM6(ATOM_SALTWELL, idle_cpu_atom), |
| 1070 | INTEL_CPU_FAM6(ATOM_SILVERMONT, idle_cpu_byt), | 1082 | INTEL_CPU_FAM6(ATOM_SILVERMONT, idle_cpu_byt), |
| 1071 | INTEL_CPU_FAM6(ATOM_SILVERMONT_MID, idle_cpu_tangier), | 1083 | INTEL_CPU_FAM6(ATOM_SILVERMONT_MID, idle_cpu_tangier), |
| @@ -1073,14 +1085,14 @@ static const struct x86_cpu_id intel_idle_ids[] __initconst = { · 1 unmodified line | |||
| 1073 | INTEL_CPU_FAM6(IVYBRIDGE, idle_cpu_ivb), | 1085 | INTEL_CPU_FAM6(IVYBRIDGE, idle_cpu_ivb), |
| 1074 | INTEL_CPU_FAM6(IVYBRIDGE_X, idle_cpu_ivt), | 1086 | INTEL_CPU_FAM6(IVYBRIDGE_X, idle_cpu_ivt), |
| 1075 | INTEL_CPU_FAM6(HASWELL, idle_cpu_hsw), | 1087 | INTEL_CPU_FAM6(HASWELL, idle_cpu_hsw), |
| 1076 | INTEL_CPU_FAM6(HASWELL_X, idle_cpu_hsw), | 1088 | INTEL_CPU_FAM6(HASWELL_X, idle_cpu_hsx), |
| 1077 | INTEL_CPU_FAM6(HASWELL_L, idle_cpu_hsw), | 1089 | INTEL_CPU_FAM6(HASWELL_L, idle_cpu_hsw), |
| 1078 | INTEL_CPU_FAM6(HASWELL_G, idle_cpu_hsw), | 1090 | INTEL_CPU_FAM6(HASWELL_G, idle_cpu_hsw), |
| 1079 | INTEL_CPU_FAM6(ATOM_SILVERMONT_D, idle_cpu_avn), | 1091 | INTEL_CPU_FAM6(ATOM_SILVERMONT_D, idle_cpu_avn), |
| 1080 | INTEL_CPU_FAM6(BROADWELL, idle_cpu_bdw), | 1092 | INTEL_CPU_FAM6(BROADWELL, idle_cpu_bdw), |
| 1081 | INTEL_CPU_FAM6(BROADWELL_G, idle_cpu_bdw), | 1093 | INTEL_CPU_FAM6(BROADWELL_G, idle_cpu_bdw), |
| 1082 | INTEL_CPU_FAM6(BROADWELL_X, idle_cpu_bdw), | 1094 | INTEL_CPU_FAM6(BROADWELL_X, idle_cpu_bdx), |
| 1083 | INTEL_CPU_FAM6(BROADWELL_D, idle_cpu_bdw), | 1095 | INTEL_CPU_FAM6(BROADWELL_D, idle_cpu_bdx), |
| 1084 | INTEL_CPU_FAM6(SKYLAKE_L, idle_cpu_skl), | 1096 | INTEL_CPU_FAM6(SKYLAKE_L, idle_cpu_skl), |
| 1085 | INTEL_CPU_FAM6(SKYLAKE, idle_cpu_skl), | 1097 | INTEL_CPU_FAM6(SKYLAKE, idle_cpu_skl), |
| 1086 | INTEL_CPU_FAM6(KABYLAKE_L, idle_cpu_skl), | 1098 | INTEL_CPU_FAM6(KABYLAKE_L, idle_cpu_skl), |
| @@ -1095,76 +1107,169 @@ static const struct x86_cpu_id intel_idle_ids[] __initconst = { · 8 unmodified lines | |||
| 1095 | {} | 1107 | {} |
| 1096 | }; | 1108 | }; |
| 1097 | 1109 | ||
| 1098 | /* | 1110 | #define INTEL_CPU_FAM6_MWAIT \ |
| 1099 | * intel_idle_probe() | 1111 | { X86_VENDOR_INTEL, 6, X86_MODEL_ANY, X86_FEATURE_MWAIT, 0 } |
| 1112 | |||
| 1113 | static const struct x86_cpu_id intel_mwait_ids[] __initconst = { | ||
| 1114 | INTEL_CPU_FAM6_MWAIT, | ||
| 1115 | {} | ||
| 1116 | }; | ||
| 1117 | |||
| 1118 | static bool __init intel_idle_max_cstate_reached(int cstate) | ||
| 1119 | { | ||
| 1120 | if (cstate + 1 > max_cstate) { | ||
| 1121 | pr_info("max_cstate %d reached\n", max_cstate); | ||
| 1122 | return true; | ||
| 1123 | } | ||
| 1124 | return false; | ||
| 1125 | } | ||
| 1126 | |||
| 1127 | #ifdef CONFIG_ACPI_PROCESSOR_CSTATE | ||
| 1128 | #include <acpi/processor.h> | ||
| 1129 | |||
| 1130 | static bool no_acpi __read_mostly; | ||
| 1131 | module_param(no_acpi, bool, 0444); | ||
| 1132 | MODULE_PARM_DESC(no_acpi, "Do not use ACPI _CST for building the idle states list"); | ||
| 1133 | |||
| 1134 | static struct acpi_processor_power acpi_state_table __initdata; | ||
| 1135 | |||
| 1136 | /** | ||
| 1137 | * intel_idle_cst_usable - Check if the _CST information can be used. | ||
| 1138 | * | ||
| 1139 | * Check if all of the C-states listed by _CST in the max_cstate range are | ||
| 1140 | * ACPI_CSTATE_FFH, which means that they should be entered via MWAIT. | ||
| 1100 | */ | 1141 | */ |
| 1101 | static int __init intel_idle_probe(void) | 1142 | static bool __init intel_idle_cst_usable(void) |
| 1102 | { | 1143 | { |
| 1103 | unsigned int eax, ebx, ecx; | 1144 | int cstate, limit; |
| 1104 | const struct x86_cpu_id *id; | ||
| 1105 | 1145 | ||
| 1106 | if (max_cstate == 0) { | 1146 | limit = min_t(int, min_t(int, CPUIDLE_STATE_MAX, max_cstate + 1), |
| 1107 | pr_debug("disabled\n"); | 1147 | acpi_state_table.count); |
| 1108 | return -EPERM; | ||
| 1109 | } | ||
| 1110 | 1148 | ||
| 1111 | id = x86_match_cpu(intel_idle_ids); | 1149 | for (cstate = 1; cstate < limit; cstate++) { |
| 1112 | if (!id) { | 1150 | struct acpi_processor_cx *cx = &acpi_state_table.states[cstate]; |
| 1113 | if (boot_cpu_data.x86_vendor == X86_VENDOR_INTEL && | 1151 | |
| 1114 | boot_cpu_data.x86 == 6) | 1152 | if (cx->entry_method != ACPI_CSTATE_FFH) |
| 1115 | pr_debug("does not run on family %d model %d\n", | 1153 | return false; |
| 1116 | boot_cpu_data.x86, boot_cpu_data.x86_model); | ||
| 1117 | return -ENODEV; | ||
| 1118 | } | 1154 | } |
| 1119 | 1155 | ||
| 1120 | if (!boot_cpu_has(X86_FEATURE_MWAIT)) { | 1156 | return true; |
| 1121 | pr_debug("Please enable MWAIT in BIOS SETUP\n"); | 1157 | } |
| 1122 | return -ENODEV; | 1158 | |
| 1159 | static bool __init intel_idle_acpi_cst_extract(void) | ||
| 1160 | { | ||
| 1161 | unsigned int cpu; | ||
| 1162 | |||
| 1163 | if (no_acpi) { | ||
| 1164 | pr_debug("Not allowed to use ACPI _CST\n"); | ||
| 1165 | return false; | ||
| 1123 | } | 1166 | } |
| 1124 | 1167 | ||
| 1125 | if (boot_cpu_data.cpuid_level < CPUID_MWAIT_LEAF) | 1168 | for_each_possible_cpu(cpu) { |
| 1126 | return -ENODEV; | 1169 | struct acpi_processor *pr = per_cpu(processors, cpu); |
| 1127 | 1170 | ||
| 1128 | cpuid(CPUID_MWAIT_LEAF, &eax, &ebx, &ecx, &mwait_substates); | 1171 | if (!pr) |
| 1172 | continue; | ||
| 1129 | 1173 | ||
| 1130 | if (!(ecx & CPUID5_ECX_EXTENSIONS_SUPPORTED) || | 1174 | if (acpi_processor_evaluate_cst(pr->handle, cpu, &acpi_state_table)) |
| 1131 | !(ecx & CPUID5_ECX_INTERRUPT_BREAK) || | 1175 | continue; |
| 1132 | !mwait_substates) | ||
| 1133 | return -ENODEV; | ||
| 1134 | 1176 | ||
| 1135 | pr_debug("MWAIT substates: 0x%x\n", mwait_substates); | 1177 | acpi_state_table.count++; |
| 1136 | 1178 | ||
| 1137 | icpu = (const struct idle_cpu *)id->driver_data; | 1179 | if (!intel_idle_cst_usable()) |
| 1138 | cpuidle_state_table = icpu->state_table; | 1180 | continue; |
| 1139 | 1181 | ||
| 1140 | pr_debug("v" INTEL_IDLE_VERSION " model 0x%X\n", | 1182 | if (!acpi_processor_claim_cst_control()) { |
| 1141 | boot_cpu_data.x86_model); | 1183 | acpi_state_table.count = 0; |
| 1184 | return false; | ||
| 1185 | } | ||
| 1142 | 1186 | ||
| 1143 | return 0; | 1187 | return true; |
| 1188 | } | ||
| 1189 | |||
| 1190 | pr_debug("ACPI _CST not found or not usable\n"); | ||
| 1191 | return false; | ||
| 1144 | } | 1192 | } |
| 1145 | 1193 | ||
| 1146 | /* | 1194 | static void __init intel_idle_init_cstates_acpi(struct cpuidle_driver *drv) |
| 1147 | * intel_idle_cpuidle_devices_uninit() | ||
| 1148 | * Unregisters the cpuidle devices. | ||
| 1149 | */ | ||
| 1150 | static void intel_idle_cpuidle_devices_uninit(void) | ||
| 1151 | { | 1195 | { |
| 1152 | int i; | 1196 | int cstate, limit = min_t(int, CPUIDLE_STATE_MAX, acpi_state_table.count); |
| 1153 | struct cpuidle_device *dev; | 1197 | |
| 1198 | /* | ||
| 1199 | * If limit > 0, intel_idle_cst_usable() has returned 'true', so all of | ||
| 1200 | * the interesting states are ACPI_CSTATE_FFH. | ||
| 1201 | */ | ||
| 1202 | for (cstate = 1; cstate < limit; cstate++) { | ||
| 1203 | struct acpi_processor_cx *cx; | ||
| 1204 | struct cpuidle_state *state; | ||
| 1154 | 1205 | ||
| 1155 | for_each_online_cpu(i) { | 1206 | if (intel_idle_max_cstate_reached(cstate)) |
| 1156 | dev = per_cpu_ptr(intel_idle_cpuidle_devices, i); | 1207 | break; |
| 1157 | cpuidle_unregister_device(dev); | 1208 | |
| 1209 | cx = &acpi_state_table.states[cstate]; | ||
| 1210 | |||
| 1211 | state = &drv->states[drv->state_count++]; | ||
| 1212 | |||
| 1213 | snprintf(state->name, CPUIDLE_NAME_LEN, "C%d_ACPI", cstate); | ||
| 1214 | strlcpy(state->desc, cx->desc, CPUIDLE_DESC_LEN); | ||
| 1215 | state->exit_latency = cx->latency; | ||
| 1216 | /* | ||
| 1217 | * For C1-type C-states use the same number for both the exit | ||
| 1218 | * latency and target residency, because that is the case for | ||
| 1219 | * C1 in the majority of the static C-states tables above. | ||
| 1220 | * For the other types of C-states, however, set the target | ||
| 1221 | * residency to 3 times the exit latency which should lead to | ||
| 1222 | * a reasonable balance between energy-efficiency and | ||
| 1223 | * performance in the majority of interesting cases. | ||
| 1224 | */ | ||
| 1225 | state->target_residency = cx->latency; | ||
| 1226 | if (cx->type > ACPI_STATE_C1) | ||
| 1227 | state->target_residency *= 3; | ||
| 1228 | |||
| 1229 | state->flags = MWAIT2flg(cx->address); | ||
| 1230 | if (cx->type > ACPI_STATE_C2) | ||
| 1231 | state->flags |= CPUIDLE_FLAG_TLB_FLUSHED; | ||
| 1232 | |||
| 1233 | state->enter = intel_idle; | ||
| 1234 | state->enter_s2idle = intel_idle_s2idle; | ||
| 1158 | } | 1235 | } |
| 1159 | } | 1236 | } |
| 1160 | 1237 | ||
| 1238 | static bool __init intel_idle_off_by_default(u32 mwait_hint) | ||
| 1239 | { | ||
| 1240 | int cstate, limit; | ||
| 1241 | |||
| 1242 | /* | ||
| 1243 | * If there are no _CST C-states, do not disable any C-states by | ||
| 1244 | * default. | ||
| 1245 | */ | ||
| 1246 | if (!acpi_state_table.count) | ||
| 1247 | return false; | ||
| 1248 | |||
| 1249 | limit = min_t(int, CPUIDLE_STATE_MAX, acpi_state_table.count); | ||
| 1250 | /* | ||
| 1251 | * If limit > 0, intel_idle_cst_usable() has returned 'true', so all of | ||
| 1252 | * the interesting states are ACPI_CSTATE_FFH. | ||
| 1253 | */ | ||
| 1254 | for (cstate = 1; cstate < limit; cstate++) { | ||
| 1255 | if (acpi_state_table.states[cstate].address == mwait_hint) | ||
| 1256 | return false; | ||
| 1257 | } | ||
| 1258 | return true; | ||
| 1259 | } | ||
| 1260 | #else /* !CONFIG_ACPI_PROCESSOR_CSTATE */ | ||
| 1261 | static inline bool intel_idle_acpi_cst_extract(void) { return false; } | ||
| 1262 | static inline void intel_idle_init_cstates_acpi(struct cpuidle_driver *drv) { } | ||
| 1263 | static inline bool intel_idle_off_by_default(u32 mwait_hint) { return false; } | ||
| 1264 | #endif /* !CONFIG_ACPI_PROCESSOR_CSTATE */ | ||
| 1265 | |||
| 1161 | /* | 1266 | /* |
| 1162 | * ivt_idle_state_table_update(void) | 1267 | * ivt_idle_state_table_update(void) |
| 1163 | * | 1268 | * |
| 1164 | * Tune IVT multi-socket targets | 1269 | * Tune IVT multi-socket targets |
| 1165 | * Assumption: num_sockets == (max_package_num + 1) | 1270 | * Assumption: num_sockets == (max_package_num + 1) |
| 1166 | */ | 1271 | */ |
| 1167 | static void ivt_idle_state_table_update(void) | 1272 | static void __init ivt_idle_state_table_update(void) |
| 1168 | { | 1273 | { |
| 1169 | /* IVT uses a different table for 1-2, 3-4, and > 4 sockets */ | 1274 | /* IVT uses a different table for 1-2, 3-4, and > 4 sockets */ |
| 1170 | int cpu, package_num, num_sockets = 1; | 1275 | int cpu, package_num, num_sockets = 1; |
| @@ -1187,15 +1292,17 @@ static void ivt_idle_state_table_update(void) · 16 unmodified lines | |||
| 1187 | /* else, 1 and 2 socket systems use default ivt_cstates */ | 1292 | /* else, 1 and 2 socket systems use default ivt_cstates */ |
| 1188 | } | 1293 | } |
| 1189 | 1294 | ||
| 1190 | /* | 1295 | /** |
| 1191 | * Translate IRTL (Interrupt Response Time Limit) MSR to usec | 1296 | * irtl_2_usec - IRTL to microseconds conversion. |
| 1297 | * @irtl: IRTL MSR value. | ||
| 1298 | * | ||
| 1299 | * Translate the IRTL (Interrupt Response Time Limit) MSR value to microseconds. | ||
| 1192 | */ | 1300 | */ |
| 1193 | 1301 | static unsigned long long __init irtl_2_usec(unsigned long long irtl) | |
| 1194 | static unsigned int irtl_ns_units[] = { | ||
| 1195 | 1, 32, 1024, 32768, 1048576, 33554432, 0, 0 }; | ||
| 1196 | |||
| 1197 | static unsigned long long irtl_2_usec(unsigned long long irtl) | ||
| 1198 | { | 1302 | { |
| 1303 | static const unsigned int irtl_ns_units[] __initconst = { | ||
| 1304 | 1, 32, 1024, 32768, 1048576, 33554432, 0, 0 | ||
| 1305 | }; | ||
| 1199 | unsigned long long ns; | 1306 | unsigned long long ns; |
| 1200 | 1307 | ||
| 1201 | if (!irtl) | 1308 | if (!irtl) |
| @@ -1203,15 +1310,16 @@ static unsigned long long irtl_2_usec(unsigned long long irtl) · 1 unmodified line | |||
| 1203 | 1310 | ||
| 1204 | ns = irtl_ns_units[(irtl >> 10) & 0x7]; | 1311 | ns = irtl_ns_units[(irtl >> 10) & 0x7]; |
| 1205 | 1312 | ||
| 1206 | return div64_u64((irtl & 0x3FF) * ns, 1000); | 1313 | return div_u64((irtl & 0x3FF) * ns, NSEC_PER_USEC); |
| 1207 | } | 1314 | } |
| 1315 | |||
| 1208 | /* | 1316 | /* |
| 1209 | * bxt_idle_state_table_update(void) | 1317 | * bxt_idle_state_table_update(void) |
| 1210 | * | 1318 | * |
| 1211 | * On BXT, we trust the IRTL to show the definitive maximum latency | 1319 | * On BXT, we trust the IRTL to show the definitive maximum latency |
| 1212 | * We use the same value for target_residency. | 1320 | * We use the same value for target_residency. |
| 1213 | */ | 1321 | */ |
| 1214 | static void bxt_idle_state_table_update(void) | 1322 | static void __init bxt_idle_state_table_update(void) |
| 1215 | { | 1323 | { |
| 1216 | unsigned long long msr; | 1324 | unsigned long long msr; |
| 1217 | unsigned int usec; | 1325 | unsigned int usec; |
| @@ -1258,7 +1366,7 @@ static void bxt_idle_state_table_update(void) · 40 unmodified lines | |||
| 1258 | * On SKL-H (model 0x5e) disable C8 and C9 if: | 1366 | * On SKL-H (model 0x5e) disable C8 and C9 if: |
| 1259 | * C10 is enabled and SGX disabled | 1367 | * C10 is enabled and SGX disabled |
| 1260 | */ | 1368 | */ |
| 1261 | static void sklh_idle_state_table_update(void) | 1369 | static void __init sklh_idle_state_table_update(void) |
| 1262 | { | 1370 | { |
| 1263 | unsigned long long msr; | 1371 | unsigned long long msr; |
| 1264 | unsigned int eax, ebx, ecx, edx; | 1372 | unsigned int eax, ebx, ecx, edx; |
| @@ -1294,16 +1402,28 @@ static void sklh_idle_state_table_update(void) · 29 unmodified lines | |||
| 1294 | skl_cstates[5].flags |= CPUIDLE_FLAG_UNUSABLE; /* C8-SKL */ | 1402 | skl_cstates[5].flags |= CPUIDLE_FLAG_UNUSABLE; /* C8-SKL */ |
| 1295 | skl_cstates[6].flags |= CPUIDLE_FLAG_UNUSABLE; /* C9-SKL */ | 1403 | skl_cstates[6].flags |= CPUIDLE_FLAG_UNUSABLE; /* C9-SKL */ |
| 1296 | } | 1404 | } |
| 1297 | /* | ||
| 1298 | * intel_idle_state_table_update() | ||
| 1299 | * | ||
| 1300 | * Update the default state_table for this CPU-id | ||
| 1301 | */ | ||
| 1302 | 1405 | ||
| 1303 | static void intel_idle_state_table_update(void) | 1406 | static bool __init intel_idle_verify_cstate(unsigned int mwait_hint) |
| 1304 | { | 1407 | { |
| 1305 | switch (boot_cpu_data.x86_model) { | 1408 | unsigned int mwait_cstate = MWAIT_HINT2CSTATE(mwait_hint) + 1; |
| 1409 | unsigned int num_substates = (mwait_substates >> mwait_cstate * 4) & | ||
| 1410 | MWAIT_SUBSTATE_MASK; | ||
| 1411 | |||
| 1412 | /* Ignore the C-state if there are NO sub-states in CPUID for it. */ | ||
| 1413 | if (num_substates == 0) | ||
| 1414 | return false; | ||
| 1415 | |||
| 1416 | if (mwait_cstate > 2 && !boot_cpu_has(X86_FEATURE_NONSTOP_TSC)) | ||
| 1417 | mark_tsc_unstable("TSC halts in idle states deeper than C2"); | ||
| 1306 | 1418 | ||
| 1419 | return true; | ||
| 1420 | } | ||
| 1421 | |||
| 1422 | static void __init intel_idle_init_cstates_icpu(struct cpuidle_driver *drv) | ||
| 1423 | { | ||
| 1424 | int cstate; | ||
| 1425 | |||
| 1426 | switch (boot_cpu_data.x86_model) { | ||
| 1307 | case INTEL_FAM6_IVYBRIDGE_X: | 1427 | case INTEL_FAM6_IVYBRIDGE_X: |
| 1308 | ivt_idle_state_table_update(); | 1428 | ivt_idle_state_table_update(); |
| 1309 | break; | 1429 | break; |
| @@ -1315,62 +1435,36 @@ static void intel_idle_state_table_update(void) · 5 unmodified lines | |||
| 1315 | sklh_idle_state_table_update(); | 1435 | sklh_idle_state_table_update(); |
| 1316 | break; | 1436 | break; |
| 1317 | } | 1437 | } |
| 1318 | } | ||
| 1319 | |||
| 1320 | /* | ||
| 1321 | * intel_idle_cpuidle_driver_init() | ||
| 1322 | * allocate, initialize cpuidle_states | ||
| 1323 | */ | ||
| 1324 | static void __init intel_idle_cpuidle_driver_init(void) | ||
| 1325 | { | ||
| 1326 | int cstate; | ||
| 1327 | struct cpuidle_driver *drv = &intel_idle_driver; | ||
| 1328 | |||
| 1329 | intel_idle_state_table_update(); | ||
| 1330 | |||
| 1331 | cpuidle_poll_state_init(drv); | ||
| 1332 | drv->state_count = 1; | ||
| 1333 | 1438 | ||
| 1334 | for (cstate = 0; cstate < CPUIDLE_STATE_MAX; ++cstate) { | 1439 | for (cstate = 0; cstate < CPUIDLE_STATE_MAX; ++cstate) { |
| 1335 | int num_substates, mwait_hint, mwait_cstate; | 1440 | unsigned int mwait_hint; |
| 1336 | 1441 | ||
| 1337 | if ((cpuidle_state_table[cstate].enter == NULL) && | 1442 | if (intel_idle_max_cstate_reached(cstate)) |
| 1338 | (cpuidle_state_table[cstate].enter_s2idle == NULL)) | ||
| 1339 | break; | 1443 | break; |
| 1340 | 1444 | ||
| 1341 | if (cstate + 1 > max_cstate) { | 1445 | if (!cpuidle_state_table[cstate].enter && |
| 1342 | pr_info("max_cstate %d reached\n", max_cstate); | 1446 | !cpuidle_state_table[cstate].enter_s2idle) |
| 1343 | break; | 1447 | break; |
| 1344 | } | ||
| 1345 | |||
| 1346 | mwait_hint = flg2MWAIT(cpuidle_state_table[cstate].flags); | ||
| 1347 | mwait_cstate = MWAIT_HINT2CSTATE(mwait_hint); | ||
| 1348 | |||
| 1349 | /* number of sub-states for this state in CPUID.MWAIT */ | ||
| 1350 | num_substates = (mwait_substates >> ((mwait_cstate + 1) * 4)) | ||
| 1351 | & MWAIT_SUBSTATE_MASK; | ||
| 1352 | |||
| 1353 | /* if NO sub-states for this state in CPUID, skip it */ | ||
| 1354 | if (num_substates == 0) | ||
| 1355 | continue; | ||
| 1356 | 1448 | ||
| 1357 | /* if state marked as disabled, skip it */ | 1449 | /* If marked as unusable, skip this state. */ |
| 1358 | if (cpuidle_state_table[cstate].flags & CPUIDLE_FLAG_UNUSABLE) { | 1450 | if (cpuidle_state_table[cstate].flags & CPUIDLE_FLAG_UNUSABLE) { |
| 1359 | pr_debug("state %s is disabled\n", | 1451 | pr_debug("state %s is disabled\n", |
| 1360 | cpuidle_state_table[cstate].name); | 1452 | cpuidle_state_table[cstate].name); |
| 1361 | continue; | 1453 | continue; |
| 1362 | } | 1454 | } |
| 1363 | 1455 | ||
| 1456 | mwait_hint = flg2MWAIT(cpuidle_state_table[cstate].flags); | ||
| 1457 | if (!intel_idle_verify_cstate(mwait_hint)) | ||
| 1458 | continue; | ||
| 1364 | 1459 | ||
| 1365 | if (((mwait_cstate + 1) > 2) && | 1460 | /* Structure copy. */ |
| 1366 | !boot_cpu_has(X86_FEATURE_NONSTOP_TSC)) | 1461 | drv->states[drv->state_count] = cpuidle_state_table[cstate]; |
| 1367 | mark_tsc_unstable("TSC halts in idle" | ||
| 1368 | " states deeper than C2"); | ||
| 1369 | 1462 | ||
| 1370 | drv->states[drv->state_count] = /* structure copy */ | 1463 | if (icpu->use_acpi && intel_idle_off_by_default(mwait_hint) && |
| 1371 | cpuidle_state_table[cstate]; | 1464 | !(cpuidle_state_table[cstate].flags & CPUIDLE_FLAG_ALWAYS_ENABLE)) |
| 1465 | drv->states[drv->state_count].flags |= CPUIDLE_FLAG_OFF; | ||
| 1372 | 1466 | ||
| 1373 | drv->state_count += 1; | 1467 | drv->state_count++; |
| 1374 | } | 1468 | } |
| 1375 | 1469 | ||
| 1376 | if (icpu->byt_auto_demotion_disable_flag) { | 1470 | if (icpu->byt_auto_demotion_disable_flag) { |
| @@ -1379,6 +1473,38 @@ static void __init intel_idle_cpuidle_driver_init(void) · 2 unmodified lines | |||
| 1379 | } | 1473 | } |
| 1380 | } | 1474 | } |
| 1381 | 1475 | ||
| 1476 | /* | ||
| 1477 | * intel_idle_cpuidle_driver_init() | ||
| 1478 | * allocate, initialize cpuidle_states | ||
| 1479 | */ | ||
| 1480 | static void __init intel_idle_cpuidle_driver_init(struct cpuidle_driver *drv) | ||
| 1481 | { | ||
| 1482 | cpuidle_poll_state_init(drv); | ||
| 1483 | drv->state_count = 1; | ||
| 1484 | |||
| 1485 | if (icpu) | ||
| 1486 | intel_idle_init_cstates_icpu(drv); | ||
| 1487 | else | ||
| 1488 | intel_idle_init_cstates_acpi(drv); | ||
| 1489 | } | ||
| 1490 | |||
| 1491 | static void auto_demotion_disable(void) | ||
| 1492 | { | ||
| 1493 | unsigned long long msr_bits; | ||
| 1494 | |||
| 1495 | rdmsrl(MSR_PKG_CST_CONFIG_CONTROL, msr_bits); | ||
| 1496 | msr_bits &= ~(icpu->auto_demotion_disable_flags); | ||
| 1497 | wrmsrl(MSR_PKG_CST_CONFIG_CONTROL, msr_bits); | ||
| 1498 | } | ||
| 1499 | |||
| 1500 | static void c1e_promotion_disable(void) | ||
| 1501 | { | ||
| 1502 | unsigned long long msr_bits; | ||
| 1503 | |||
| 1504 | rdmsrl(MSR_IA32_POWER_CTL, msr_bits); | ||
| 1505 | msr_bits &= ~0x2; | ||
| 1506 | wrmsrl(MSR_IA32_POWER_CTL, msr_bits); | ||
| 1507 | } | ||
| 1382 | 1508 | ||
| 1383 | /* | 1509 | /* |
| 1384 | * intel_idle_cpu_init() | 1510 | * intel_idle_cpu_init() |
| @@ -1397,6 +1523,9 @@ static int intel_idle_cpu_init(unsigned int cpu) · 12 unmodified lines | |||
| 1397 | return -EIO; | 1523 | return -EIO; |
| 1398 | } | 1524 | } |
| 1399 | 1525 | ||
| 1526 | if (!icpu) | ||
| 1527 | return 0; | ||
| 1528 | |||
| 1400 | if (icpu->auto_demotion_disable_flags) | 1529 | if (icpu->auto_demotion_disable_flags) |
| 1401 | auto_demotion_disable(); | 1530 | auto_demotion_disable(); |
| 1402 | 1531 | ||
| @@ -1411,7 +1540,7 @@ static int intel_idle_cpu_online(unsigned int cpu) · 8 unmodified lines | |||
| 1411 | struct cpuidle_device *dev; | 1540 | struct cpuidle_device *dev; |
| 1412 | 1541 | ||
| 1413 | if (lapic_timer_reliable_states != LAPIC_TIMER_ALWAYS_RELIABLE) | 1542 | if (lapic_timer_reliable_states != LAPIC_TIMER_ALWAYS_RELIABLE) |
| 1414 | __setup_broadcast_timer(true); | 1543 | tick_broadcast_enable(); |
| 1415 | 1544 | ||
| 1416 | /* | 1545 | /* |
| 1417 | * Some systems can hotplug a cpu at runtime after | 1546 | * Some systems can hotplug a cpu at runtime after |
| @@ -1425,23 +1554,74 @@ static int intel_idle_cpu_online(unsigned int cpu) · 7 unmodified lines | |||
| 1425 | return 0; | 1554 | return 0; |
| 1426 | } | 1555 | } |
| 1427 | 1556 | ||
| 1557 | /** | ||
| 1558 | * intel_idle_cpuidle_devices_uninit - Unregister all cpuidle devices. | ||
| 1559 | */ | ||
| 1560 | static void __init intel_idle_cpuidle_devices_uninit(void) | ||
| 1561 | { | ||
| 1562 | int i; | ||
| 1563 | |||
| 1564 | for_each_online_cpu(i) | ||
| 1565 | cpuidle_unregister_device(per_cpu_ptr(intel_idle_cpuidle_devices, i)); | ||
| 1566 | } | ||
| 1567 | |||
| 1428 | static int __init intel_idle_init(void) | 1568 | static int __init intel_idle_init(void) |
| 1429 | { | 1569 | { |
| 1570 | const struct x86_cpu_id *id; | ||
| 1571 | unsigned int eax, ebx, ecx; | ||
| 1430 | int retval; | 1572 | int retval; |
| 1431 | 1573 | ||
| 1432 | /* Do not load intel_idle at all for now if idle= is passed */ | 1574 | /* Do not load intel_idle at all for now if idle= is passed */ |
| 1433 | if (boot_option_idle_override != IDLE_NO_OVERRIDE) | 1575 | if (boot_option_idle_override != IDLE_NO_OVERRIDE) |
| 1434 | return -ENODEV; | 1576 | return -ENODEV; |
| 1435 | 1577 | ||
| 1436 | retval = intel_idle_probe(); | 1578 | if (max_cstate == 0) { |
| 1437 | if (retval) | 1579 | pr_debug("disabled\n"); |
| 1438 | return retval; | 1580 | return -EPERM; |
| 1581 | } | ||
| 1582 | |||
| 1583 | id = x86_match_cpu(intel_idle_ids); | ||
| 1584 | if (id) { | ||
| 1585 | if (!boot_cpu_has(X86_FEATURE_MWAIT)) { | ||
| 1586 | pr_debug("Please enable MWAIT in BIOS SETUP\n"); | ||
| 1587 | return -ENODEV; | ||
| 1588 | } | ||
| 1589 | } else { | ||
| 1590 | id = x86_match_cpu(intel_mwait_ids); | ||
| 1591 | if (!id) | ||
| 1592 | return -ENODEV; | ||
| 1593 | } | ||
| 1594 | |||
| 1595 | if (boot_cpu_data.cpuid_level < CPUID_MWAIT_LEAF) | ||
| 1596 | return -ENODEV; | ||
| 1597 | |||
| 1598 | cpuid(CPUID_MWAIT_LEAF, &eax, &ebx, &ecx, &mwait_substates); | ||
| 1599 | |||
| 1600 | if (!(ecx & CPUID5_ECX_EXTENSIONS_SUPPORTED) || | ||
| 1601 | !(ecx & CPUID5_ECX_INTERRUPT_BREAK) || | ||
| 1602 | !mwait_substates) | ||
| 1603 | return -ENODEV; | ||
| 1604 | |||
| 1605 | pr_debug("MWAIT substates: 0x%x\n", mwait_substates); | ||
| 1606 | |||
| 1607 | icpu = (const struct idle_cpu *)id->driver_data; | ||
| 1608 | if (icpu) { | ||
| 1609 | cpuidle_state_table = icpu->state_table; | ||
| 1610 | if (icpu->use_acpi) | ||
| 1611 | intel_idle_acpi_cst_extract(); | ||
| 1612 | } else if (!intel_idle_acpi_cst_extract()) { | ||
| 1613 | return -ENODEV; | ||
| 1614 | } | ||
| 1615 | |||
| 1616 | pr_debug("v" INTEL_IDLE_VERSION " model 0x%X\n", | ||
| 1617 | boot_cpu_data.x86_model); | ||
| 1439 | 1618 | ||
| 1440 | intel_idle_cpuidle_devices = alloc_percpu(struct cpuidle_device); | 1619 | intel_idle_cpuidle_devices = alloc_percpu(struct cpuidle_device); |
| 1441 | if (intel_idle_cpuidle_devices == NULL) | 1620 | if (!intel_idle_cpuidle_devices) |
| 1442 | return -ENOMEM; | 1621 | return -ENOMEM; |
| 1443 | 1622 | ||
| 1444 | intel_idle_cpuidle_driver_init(); | 1623 | intel_idle_cpuidle_driver_init(&intel_idle_driver); |
| 1624 | |||
| 1445 | retval = cpuidle_register_driver(&intel_idle_driver); | 1625 | retval = cpuidle_register_driver(&intel_idle_driver); |
| 1446 | if (retval) { | 1626 | if (retval) { |
| 1447 | struct cpuidle_driver *drv = cpuidle_get_driver(); | 1627 | struct cpuidle_driver *drv = cpuidle_get_driver(); |
| @@ -279,6 +279,21 @@ static inline bool invalid_phys_cpuid(phys_cpuid_t phys_id) · 278 unmodified lines | |||
| 279 | 279 | ||
| 280 | /* Validate the processor object's proc_id */ | 280 | /* Validate the processor object's proc_id */ |
| 281 | bool acpi_duplicate_processor_id(int proc_id); | 281 | bool acpi_duplicate_processor_id(int proc_id); |
| 282 | /* Processor _CTS control */ | ||
| 283 | struct acpi_processor_power; | ||
| 284 | |||
| 285 | #ifdef CONFIG_ACPI_PROCESSOR_CSTATE | ||
| 286 | bool acpi_processor_claim_cst_control(void); | ||
| 287 | int acpi_processor_evaluate_cst(acpi_handle handle, u32 cpu, | ||
| 288 | struct acpi_processor_power *info); | ||
| 289 | #else | ||
| 290 | static inline bool acpi_processor_claim_cst_control(void) { return false; } | ||
| 291 | static inline int acpi_processor_evaluate_cst(acpi_handle handle, u32 cpu, | ||
| 292 | struct acpi_processor_power *info) | ||
| 293 | { | ||
| 294 | return -ENODEV; | ||
| 295 | } | ||
| 296 | #endif | ||
| 282 | 297 | ||
| 283 | #ifdef CONFIG_ACPI_HOTPLUG_CPU | 298 | #ifdef CONFIG_ACPI_HOTPLUG_CPU |
| 284 | /* Arch dependent functions for cpu hotplug support */ | 299 | /* Arch dependent functions for cpu hotplug support */ |
| @@ -77,6 +77,7 @@ struct cpuidle_state { · 76 unmodified lines | |||
| 77 | #define CPUIDLE_FLAG_COUPLED BIT(1) /* state applies to multiple cpus */ | 77 | #define CPUIDLE_FLAG_COUPLED BIT(1) /* state applies to multiple cpus */ |
| 78 | #define CPUIDLE_FLAG_TIMER_STOP BIT(2) /* timer is stopped on this state */ | 78 | #define CPUIDLE_FLAG_TIMER_STOP BIT(2) /* timer is stopped on this state */ |
| 79 | #define CPUIDLE_FLAG_UNUSABLE BIT(3) /* avoid using this state */ | 79 | #define CPUIDLE_FLAG_UNUSABLE BIT(3) /* avoid using this state */ |
| 80 | #define CPUIDLE_FLAG_OFF BIT(4) /* disable this state by default */ | ||
| 80 | 81 | ||
| 81 | struct cpuidle_device_kobj; | 82 | struct cpuidle_device_kobj; |
| 82 | struct cpuidle_state_kobj; | 83 | struct cpuidle_state_kobj; |
| @@ -115,7 +116,6 @@ DECLARE_PER_CPU(struct cpuidle_device, cpuidle_dev); · 32 unmodified lines | |||
| 115 | struct cpuidle_driver { | 116 | struct cpuidle_driver { |
| 116 | const char *name; | 117 | const char *name; |
| 117 | struct module *owner; | 118 | struct module *owner; |
| 118 | int refcnt; | ||
| 119 | 119 | ||
| 120 | /* used by the cpuidle framework to setup the broadcast timer */ | 120 | /* used by the cpuidle framework to setup the broadcast timer */ |
| 121 | unsigned int bctimer:1; | 121 | unsigned int bctimer:1; |
| @@ -147,8 +147,6 @@ extern u64 cpuidle_poll_time(struct cpuidle_driver *drv, · 25 unmodified lines | |||
| 147 | 147 | ||
| 148 | extern int cpuidle_register_driver(struct cpuidle_driver *drv); | 148 | extern int cpuidle_register_driver(struct cpuidle_driver *drv); |
| 149 | extern struct cpuidle_driver *cpuidle_get_driver(void); | 149 | extern struct cpuidle_driver *cpuidle_get_driver(void); |
| 150 | extern struct cpuidle_driver *cpuidle_driver_ref(void); | ||
| 151 | extern void cpuidle_driver_unref(void); | ||
| 152 | extern void cpuidle_driver_state_disabled(struct cpuidle_driver *drv, int idx, | 150 | extern void cpuidle_driver_state_disabled(struct cpuidle_driver *drv, int idx, |
| 153 | bool disable); | 151 | bool disable); |
| 154 | extern void cpuidle_unregister_driver(struct cpuidle_driver *drv); | 152 | extern void cpuidle_unregister_driver(struct cpuidle_driver *drv); |
| @@ -186,8 +184,6 @@ static inline u64 cpuidle_poll_time(struct cpuidle_driver *drv, · 31 unmodified lines | |||
| 186 | static inline int cpuidle_register_driver(struct cpuidle_driver *drv) | 184 | static inline int cpuidle_register_driver(struct cpuidle_driver *drv) |
| 187 | {return -ENODEV; } | 185 | {return -ENODEV; } |
| 188 | static inline struct cpuidle_driver *cpuidle_get_driver(void) {return NULL; } | 186 | static inline struct cpuidle_driver *cpuidle_get_driver(void) {return NULL; } |
| 189 | static inline struct cpuidle_driver *cpuidle_driver_ref(void) {return NULL; } | ||
| 190 | static inline void cpuidle_driver_unref(void) {} | ||
| 191 | static inline void cpuidle_driver_state_disabled(struct cpuidle_driver *drv, | 187 | static inline void cpuidle_driver_state_disabled(struct cpuidle_driver *drv, |
| 192 | int idx, bool disable) { } | 188 | int idx, bool disable) { } |
| 193 | static inline void cpuidle_unregister_driver(struct cpuidle_driver *drv) { } | 189 | static inline void cpuidle_unregister_driver(struct cpuidle_driver *drv) { } |