* pm-cpuidle: (27 commits) intel_idle: Clean up irtl_2_usec() intel_idle: Move 3 functions closer to their callers intel_idle: Annotate initialization code and data structures intel_idle: Move and clean up intel_idle_cpuidle_devices_uninit() intel_idle: Rearrange intel_idle_cpuidle_driver_init() intel_idle: Clean up NULL pointer check in intel_idle_init() intel_idle: Fold intel_idle_probe() into intel_idle_init() intel_idle: Eliminate __setup_broadcast_timer() cpuidle: fix cpuidle_find_deepest_state() kerneldoc warnings cpuidle: sysfs: fix warnings when compiling with W=1 cpuidle: coupled: fix warnings when compiling with W=1 Documentation: admin-guide: PM: Add intel_idle document cpuidle: arm: Enable compile testing for some of drivers cpuidle: Drop unused cpuidle_driver_ref/unref() functions intel_idle: Use ACPI _CST on server systems intel_idle: Add module parameter to prevent ACPI _CST from being used intel_idle: Allow ACPI _CST to be used for selected known processors cpuidle: Allow idle states to be disabled by default intel_idle: Use ACPI _CST for processor models without C-state tables intel_idle: Refactor intel_idle_cpuidle_driver_init() ...
Unified · Split
| @@ -196,6 +196,12 @@ Description: · 195 unmodified lines | ||
| 196 | 196 | does not reflect it. Likewise, if one enables a deep state but a |
| 197 | 197 | lighter state still is disabled, then this has no effect. |
| 198 | 198 | |
| 199 | +What: /sys/devices/system/cpu/cpuX/cpuidle/stateN/default_status | |
| 200 | +Date: December 2019 | |
| 201 | +KernelVersion: v5.6 | |
| 202 | +Contact: Linux power management list <linux-pm@vger.kernel.org> | |
| 203 | +Description: | |
| 204 | + (RO) The default status of this state, "enabled" or "disabled". | |
| 199 | 205 | |
| 200 | 206 | What: /sys/devices/system/cpu/cpuX/cpuidle/stateN/residency |
| 201 | 207 | Date: March 2014 |
| @@ -506,6 +506,9 @@ object corresponding to it, as follows: · 505 unmodified lines | ||
| 506 | 506 | ``disable`` |
| 507 | 507 | Whether or not this idle state is disabled. |
| 508 | 508 | |
| 509 | +``default_status`` | |
| 510 | + The default status of this state, "enabled" or "disabled". | |
| 511 | + | |
| 509 | 512 | ``latency`` |
| 510 | 513 | Exit latency of the idle state in microseconds. |
| 511 | 514 | |
| @@ -0,0 +1,246 @@ | ||
| 1 | +.. SPDX-License-Identifier: GPL-2.0 | |
| 2 | +.. include:: <isonum.txt> | |
| 3 | + | |
| 4 | +============================================== | |
| 5 | +``intel_idle`` CPU Idle Time Management Driver | |
| 6 | +============================================== | |
| 7 | + | |
| 8 | +:Copyright: |copy| 2020 Intel Corporation | |
| 9 | + | |
| 10 | +:Author: Rafael J. Wysocki <rafael.j.wysocki@intel.com> | |
| 11 | + | |
| 12 | + | |
| 13 | +General Information | |
| 14 | +=================== | |
| 15 | + | |
| 16 | +``intel_idle`` is a part of the | |
| 17 | +:doc:`CPU idle time management subsystem <cpuidle>` in the Linux kernel | |
| 18 | +(``CPUIdle``). It is the default CPU idle time management driver for the | |
| 19 | +Nehalem and later generations of Intel processors, but the level of support for | |
| 20 | +a particular processor model in it depends on whether or not it recognizes that | |
| 21 | +processor model and may also depend on information coming from the platform | |
| 22 | +firmware. [To understand ``intel_idle`` it is necessary to know how ``CPUIdle`` | |
| 23 | +works in general, so this is the time to get familiar with :doc:`cpuidle` if you | |
| 24 | +have not done that yet.] | |
| 25 | + | |
| 26 | +``intel_idle`` uses the ``MWAIT`` instruction to inform the processor that the | |
| 27 | +logical CPU executing it is idle and so it may be possible to put some of the | |
| 28 | +processor's functional blocks into low-power states. That instruction takes two | |
| 29 | +arguments (passed in the ``EAX`` and ``ECX`` registers of the target CPU), the | |
| 30 | +first of which, referred to as a *hint*, can be used by the processor to | |
| 31 | +determine what can be done (for details refer to Intel Software Developer’s | |
| 32 | +Manual [1]_). Accordingly, ``intel_idle`` refuses to work with processors in | |
| 33 | +which the support for the ``MWAIT`` instruction has been disabled (for example, | |
| 34 | +via the platform firmware configuration menu) or which do not support that | |
| 35 | +instruction at all. | |
| 36 | + | |
| 37 | +``intel_idle`` is not modular, so it cannot be unloaded, which means that the | |
| 38 | +only way to pass early-configuration-time parameters to it is via the kernel | |
| 39 | +command line. | |
| 40 | + | |
| 41 | + | |
| 42 | +.. _intel-idle-enumeration-of-states: | |
| 43 | + | |
| 44 | +Enumeration of Idle States | |
| 45 | +========================== | |
| 46 | + | |
| 47 | +Each ``MWAIT`` hint value is interpreted by the processor as a license to | |
| 48 | +reconfigure itself in a certain way in order to save energy. The processor | |
| 49 | +configurations (with reduced power draw) resulting from that are referred to | |
| 50 | +as C-states (in the ACPI terminology) or idle states. The list of meaningful | |
| 51 | +``MWAIT`` hint values and idle states (i.e. low-power configurations of the | |
| 52 | +processor) corresponding to them depends on the processor model and it may also | |
| 53 | +depend on the configuration of the platform. | |
| 54 | + | |
| 55 | +In order to create a list of available idle states required by the ``CPUIdle`` | |
| 56 | +subsystem (see :ref:`idle-states-representation` in :doc:`cpuidle`), | |
| 57 | +``intel_idle`` can use two sources of information: static tables of idle states | |
| 58 | +for different processor models included in the driver itself and the ACPI tables | |
| 59 | +of the system. The former are always used if the processor model at hand is | |
| 60 | +recognized by ``intel_idle`` and the latter are used if that is required for | |
| 61 | +the given processor model (which is the case for all server processor models | |
| 62 | +recognized by ``intel_idle``) or if the processor model is not recognized. | |
| 63 | + | |
| 64 | +If the ACPI tables are going to be used for building the list of available idle | |
| 65 | +states, ``intel_idle`` first looks for a ``_CST`` object under one of the ACPI | |
| 66 | +objects corresponding to the CPUs in the system (refer to the ACPI specification | |
| 67 | +[2]_ for the description of ``_CST`` and its output package). Because the | |
| 68 | +``CPUIdle`` subsystem expects that the list of idle states supplied by the | |
| 69 | +driver will be suitable for all of the CPUs handled by it and ``intel_idle`` is | |
| 70 | +registered as the ``CPUIdle`` driver for all of the CPUs in the system, the | |
| 71 | +driver looks for the first ``_CST`` object returning at least one valid idle | |
| 72 | +state description and such that all of the idle states included in its return | |
| 73 | +package are of the FFH (Functional Fixed Hardware) type, which means that the | |
| 74 | +``MWAIT`` instruction is expected to be used to tell the processor that it can | |
| 75 | +enter one of them. The return package of that ``_CST`` is then assumed to be | |
| 76 | +applicable to all of the other CPUs in the system and the idle state | |
| 77 | +descriptions extracted from it are stored in a preliminary list of idle states | |
| 78 | +coming from the ACPI tables. [This step is skipped if ``intel_idle`` is | |
| 79 | +configured to ignore the ACPI tables; see `below <intel-idle-parameters_>`_.] | |
| 80 | + | |
| 81 | +Next, the first (index 0) entry in the list of available idle states is | |
| 82 | +initialized to represent a "polling idle state" (a pseudo-idle state in which | |
| 83 | +the target CPU continuously fetches and executes instructions), and the | |
| 84 | +subsequent (real) idle state entries are populated as follows. | |
| 85 | + | |
| 86 | +If the processor model at hand is recognized by ``intel_idle``, there is a | |
| 87 | +(static) table of idle state descriptions for it in the driver. In that case, | |
| 88 | +the "internal" table is the primary source of information on idle states and the | |
| 89 | +information from it is copied to the final list of available idle states. If | |
| 90 | +using the ACPI tables for the enumeration of idle states is not required | |
| 91 | +(depending on the processor model), all of the listed idle state are enabled by | |
| 92 | +default (so all of them will be taken into consideration by ``CPUIdle`` | |
| 93 | +governors during CPU idle state selection). Otherwise, some of the listed idle | |
| 94 | +states may not be enabled by default if there are no matching entries in the | |
| 95 | +preliminary list of idle states coming from the ACPI tables. In that case user | |
| 96 | +space still can enable them later (on a per-CPU basis) with the help of | |
| 97 | +the ``disable`` idle state attribute in ``sysfs`` (see | |
| 98 | +:ref:`idle-states-representation` in :doc:`cpuidle`). This basically means that | |
| 99 | +the idle states "known" to the driver may not be enabled by default if they have | |
| 100 | +not been exposed by the platform firmware (through the ACPI tables). | |
| 101 | + | |
| 102 | +If the given processor model is not recognized by ``intel_idle``, but it | |
| 103 | +supports ``MWAIT``, the preliminary list of idle states coming from the ACPI | |
| 104 | +tables is used for building the final list that will be supplied to the | |
| 105 | +``CPUIdle`` core during driver registration. For each idle state in that list, | |
| 106 | +the description, ``MWAIT`` hint and exit latency are copied to the corresponding | |
| 107 | +entry in the final list of idle states. The name of the idle state represented | |
| 108 | +by it (to be returned by the ``name`` idle state attribute in ``sysfs``) is | |
| 109 | +"CX_ACPI", where X is the index of that idle state in the final list (note that | |
| 110 | +the minimum value of X is 1, because 0 is reserved for the "polling" state), and | |
| 111 | +its target residency is based on the exit latency value. Specifically, for | |
| 112 | +C1-type idle states the exit latency value is also used as the target residency | |
| 113 | +(for compatibility with the majority of the "internal" tables of idle states for | |
| 114 | +various processor models recognized by ``intel_idle``) and for the other idle | |
| 115 | +state types (C2 and C3) the target residency value is 3 times the exit latency | |
| 116 | +(again, that is because it reflects the target residency to exit latency ratio | |
| 117 | +in the majority of cases for the processor models recognized by ``intel_idle``). | |
| 118 | +All of the idle states in the final list are enabled by default in this case. | |
| 119 | + | |
| 120 | + | |
| 121 | +.. _intel-idle-initialization: | |
| 122 | + | |
| 123 | +Initialization | |
| 124 | +============== | |
| 125 | + | |
| 126 | +The initialization of ``intel_idle`` starts with checking if the kernel command | |
| 127 | +line options forbid the use of the ``MWAIT`` instruction. If that is the case, | |
| 128 | +an error code is returned right away. | |
| 129 | + | |
| 130 | +The next step is to check whether or not the processor model is known to the | |
| 131 | +driver, which determines the idle states enumeration method (see | |
| 132 | +`above <intel-idle-enumeration-of-states_>`_), and whether or not the processor | |
| 133 | +supports ``MWAIT`` (the initialization fails if that is not the case). Then, | |
| 134 | +the ``MWAIT`` support in the processor is enumerated through ``CPUID`` and the | |
| 135 | +driver initialization fails if the level of support is not as expected (for | |
| 136 | +example, if the total number of ``MWAIT`` substates returned is 0). | |
| 137 | + | |
| 138 | +Next, if the driver is not configured to ignore the ACPI tables (see | |
| 139 | +`below <intel-idle-parameters_>`_), the idle states information provided by the | |
| 140 | +platform firmware is extracted from them. | |
| 141 | + | |
| 142 | +Then, ``CPUIdle`` device objects are allocated for all CPUs and the list of | |
| 143 | +available idle states is created as explained | |
| 144 | +`above <intel-idle-enumeration-of-states_>`_. | |
| 145 | + | |
| 146 | +Finally, ``intel_idle`` is registered with the help of cpuidle_register_driver() | |
| 147 | +as the ``CPUIdle`` driver for all CPUs in the system and a CPU online callback | |
| 148 | +for configuring individual CPUs is registered via cpuhp_setup_state(), which | |
| 149 | +(among other things) causes the callback routine to be invoked for all of the | |
| 150 | +CPUs present in the system at that time (each CPU executes its own instance of | |
| 151 | +the callback routine). That routine registers a ``CPUIdle`` device for the CPU | |
| 152 | +running it (which enables the ``CPUIdle`` subsystem to operate that CPU) and | |
| 153 | +optionally performs some CPU-specific initialization actions that may be | |
| 154 | +required for the given processor model. | |
| 155 | + | |
| 156 | + | |
| 157 | +.. _intel-idle-parameters: | |
| 158 | + | |
| 159 | +Kernel Command Line Options and Module Parameters | |
| 160 | +================================================= | |
| 161 | + | |
| 162 | +The *x86* architecture support code recognizes three kernel command line | |
| 163 | +options related to CPU idle time management: ``idle=poll``, ``idle=halt``, | |
| 164 | +and ``idle=nomwait``. If any of them is present in the kernel command line, the | |
| 165 | +``MWAIT`` instruction is not allowed to be used, so the initialization of | |
| 166 | +``intel_idle`` will fail. | |
| 167 | + | |
| 168 | +Apart from that there are two module parameters recognized by ``intel_idle`` | |
| 169 | +itself that can be set via the kernel command line (they cannot be updated via | |
| 170 | +sysfs, so that is the only way to change their values). | |
| 171 | + | |
| 172 | +The ``max_cstate`` parameter value is the maximum idle state index in the list | |
| 173 | +of idle states supplied to the ``CPUIdle`` core during the registration of the | |
| 174 | +driver. It is also the maximum number of regular (non-polling) idle states that | |
| 175 | +can be used by ``intel_idle``, so the enumeration of idle states is terminated | |
| 176 | +after finding that number of usable idle states (the other idle states that | |
| 177 | +potentially might have been used if ``max_cstate`` had been greater are not | |
| 178 | +taken into consideration at all). Setting ``max_cstate`` can prevent | |
| 179 | +``intel_idle`` from exposing idle states that are regarded as "too deep" for | |
| 180 | +some reason to the ``CPUIdle`` core, but it does so by making them effectively | |
| 181 | +invisible until the system is shut down and started again which may not always | |
| 182 | +be desirable. In practice, it is only really necessary to do that if the idle | |
| 183 | +states in question cannot be enabled during system startup, because in the | |
| 184 | +working state of the system the CPU power management quality of service (PM | |
| 185 | +QoS) feature can be used to prevent ``CPUIdle`` from touching those idle states | |
| 186 | +even if they have been enumerated (see :ref:`cpu-pm-qos` in :doc:`cpuidle`). | |
| 187 | +Setting ``max_cstate`` to 0 causes the ``intel_idle`` initialization to fail. | |
| 188 | + | |
| 189 | +The ``noacpi`` module parameter (which is recognized by ``intel_idle`` if the | |
| 190 | +kernel has been configured with ACPI support), can be set to make the driver | |
| 191 | +ignore the system's ACPI tables entirely (it is unset by default). | |
| 192 | + | |
| 193 | + | |
| 194 | +.. _intel-idle-core-and-package-idle-states: | |
| 195 | + | |
| 196 | +Core and Package Levels of Idle States | |
| 197 | +====================================== | |
| 198 | + | |
| 199 | +Typically, in a processor supporting the ``MWAIT`` instruction there are (at | |
| 200 | +least) two levels of idle states (or C-states). One level, referred to as | |
| 201 | +"core C-states", covers individual cores in the processor, whereas the other | |
| 202 | +level, referred to as "package C-states", covers the entire processor package | |
| 203 | +and it may also involve other components of the system (GPUs, memory | |
| 204 | +controllers, I/O hubs etc.). | |
| 205 | + | |
| 206 | +Some of the ``MWAIT`` hint values allow the processor to use core C-states only | |
| 207 | +(most importantly, that is the case for the ``MWAIT`` hint value corresponding | |
| 208 | +to the ``C1`` idle state), but the majority of them give it a license to put | |
| 209 | +the target core (i.e. the core containing the logical CPU executing ``MWAIT`` | |
| 210 | +with the given hint value) into a specific core C-state and then (if possible) | |
| 211 | +to enter a specific package C-state at the deeper level. For example, the | |
| 212 | +``MWAIT`` hint value representing the ``C3`` idle state allows the processor to | |
| 213 | +put the target core into the low-power state referred to as "core ``C3``" (or | |
| 214 | +``CC3``), which happens if all of the logical CPUs (SMT siblings) in that core | |
| 215 | +have executed ``MWAIT`` with the ``C3`` hint value (or with a hint value | |
| 216 | +representing a deeper idle state), and in addition to that (in the majority of | |
| 217 | +cases) it gives the processor a license to put the entire package (possibly | |
| 218 | +including some non-CPU components such as a GPU or a memory controller) into the | |
| 219 | +low-power state referred to as "package ``C3``" (or ``PC3``), which happens if | |
| 220 | +all of the cores have gone into the ``CC3`` state and (possibly) some additional | |
| 221 | +conditions are satisfied (for instance, if the GPU is covered by ``PC3``, it may | |
| 222 | +be required to be in a certain GPU-specific low-power state for ``PC3`` to be | |
| 223 | +reachable). | |
| 224 | + | |
| 225 | +As a rule, there is no simple way to make the processor use core C-states only | |
| 226 | +if the conditions for entering the corresponding package C-states are met, so | |
| 227 | +the logical CPU executing ``MWAIT`` with a hint value that is not core-level | |
| 228 | +only (like for ``C1``) must always assume that this may cause the processor to | |
| 229 | +enter a package C-state. [That is why the exit latency and target residency | |
| 230 | +values corresponding to the majority of ``MWAIT`` hint values in the "internal" | |
| 231 | +tables of idle states in ``intel_idle`` reflect the properties of package | |
| 232 | +C-states.] If using package C-states is not desirable at all, either | |
| 233 | +:ref:`PM QoS <cpu-pm-qos>` or the ``max_cstate`` module parameter of | |
| 234 | +``intel_idle`` described `above <intel-idle-parameters_>`_ must be used to | |
| 235 | +restrict the range of permissible idle states to the ones with core-level only | |
| 236 | +``MWAIT`` hint values (like ``C1``). | |
| 237 | + | |
| 238 | + | |
| 239 | +References | |
| 240 | +========== | |
| 241 | + | |
| 242 | +.. [1] *Intel® 64 and IA-32 Architectures Software Developer’s Manual Volume 2B*, | |
| 243 | + https://www.intel.com/content/www/us/en/architecture-and-technology/64-ia-32-architectures-software-developer-vol-2b-manual.html | |
| 244 | + | |
| 245 | +.. [2] *Advanced Configuration and Power Interface (ACPI) Specification*, | |
| 246 | + https://uefi.org/specifications | |
| @@ -8,6 +8,7 @@ Working-State Power Management · 7 unmodified lines | ||
| 8 | 8 | :maxdepth: 2 |
| 9 | 9 | |
| 10 | 10 | cpuidle |
| 11 | + intel_idle | |
| 11 | 12 | cpufreq |
| 12 | 13 | intel_pstate |
| 13 | 14 | intel_epb |
| @@ -241,6 +241,7 @@ config ACPI_CPU_FREQ_PSS · 240 unmodified lines | ||
| 241 | 241 | |
| 242 | 242 | config ACPI_PROCESSOR_CSTATE |
| 243 | 243 | def_bool y |
| 244 | + depends on ACPI_PROCESSOR | |
| 244 | 245 | depends on IA64 || X86 |
| 245 | 246 | |
| 246 | 247 | config ACPI_PROCESSOR_IDLE |
| @@ -705,3 +705,185 @@ void __init acpi_processor_init(void) · 704 unmodified lines | ||
| 705 | 705 | acpi_scan_add_handler_with_hotplug(&processor_handler, "processor"); |
| 706 | 706 | acpi_scan_add_handler(&processor_container_handler); |
| 707 | 707 | } |
| 708 | + | |
| 709 | +#ifdef CONFIG_ACPI_PROCESSOR_CSTATE | |
| 710 | +/** | |
| 711 | + * acpi_processor_claim_cst_control - Request _CST control from the platform. | |
| 712 | + */ | |
| 713 | +bool acpi_processor_claim_cst_control(void) | |
| 714 | +{ | |
| 715 | + static bool cst_control_claimed; | |
| 716 | + acpi_status status; | |
| 717 | + | |
| 718 | + if (!acpi_gbl_FADT.cst_control || cst_control_claimed) | |
| 719 | + return true; | |
| 720 | + | |
| 721 | + status = acpi_os_write_port(acpi_gbl_FADT.smi_command, | |
| 722 | + acpi_gbl_FADT.cst_control, 8); | |
| 723 | + if (ACPI_FAILURE(status)) { | |
| 724 | + pr_warn("ACPI: Failed to claim processor _CST control\n"); | |
| 725 | + return false; | |
| 726 | + } | |
| 727 | + | |
| 728 | + cst_control_claimed = true; | |
| 729 | + return true; | |
| 730 | +} | |
| 731 | +EXPORT_SYMBOL_GPL(acpi_processor_claim_cst_control); | |
| 732 | + | |
| 733 | +/** | |
| 734 | + * acpi_processor_evaluate_cst - Evaluate the processor _CST control method. | |
| 735 | + * @handle: ACPI handle of the processor object containing the _CST. | |
| 736 | + * @cpu: The numeric ID of the target CPU. | |
| 737 | + * @info: Object write the C-states information into. | |
| 738 | + * | |
| 739 | + * Extract the C-state information for the given CPU from the output of the _CST | |
| 740 | + * control method under the corresponding ACPI processor object (or processor | |
| 741 | + * device object) and populate @info with it. | |
| 742 | + * | |
| 743 | + * If any ACPI_ADR_SPACE_FIXED_HARDWARE C-states are found, invoke | |
| 744 | + * acpi_processor_ffh_cstate_probe() to verify them and update the | |
| 745 | + * cpu_cstate_entry data for @cpu. | |
| 746 | + */ | |
| 747 | +int acpi_processor_evaluate_cst(acpi_handle handle, u32 cpu, | |
| 748 | + struct acpi_processor_power *info) | |
| 749 | +{ | |
| 750 | + struct acpi_buffer buffer = { ACPI_ALLOCATE_BUFFER, NULL }; | |
| 751 | + union acpi_object *cst; | |
| 752 | + acpi_status status; | |
| 753 | + u64 count; | |
| 754 | + int last_index = 0; | |
| 755 | + int i, ret = 0; | |
| 756 | + | |
| 757 | + status = acpi_evaluate_object(handle, "_CST", NULL, &buffer); | |
| 758 | + if (ACPI_FAILURE(status)) { | |
| 759 | + acpi_handle_debug(handle, "No _CST\n"); | |
| 760 | + return -ENODEV; | |
| 761 | + } | |
| 762 | + | |
| 763 | + cst = buffer.pointer; | |
| 764 | + | |
| 765 | + /* There must be at least 2 elements. */ | |
| 766 | + if (!cst || cst->type != ACPI_TYPE_PACKAGE || cst->package.count < 2) { | |
| 767 | + acpi_handle_warn(handle, "Invalid _CST output\n"); | |
| 768 | + ret = -EFAULT; | |
| 769 | + goto end; | |
| 770 | + } | |
| 771 | + | |
| 772 | + count = cst->package.elements[0].integer.value; | |
| 773 | + | |
| 774 | + /* Validate the number of C-states. */ | |
| 775 | + if (count < 1 || count != cst->package.count - 1) { | |
| 776 | + acpi_handle_warn(handle, "Inconsistent _CST data\n"); | |
| 777 | + ret = -EFAULT; | |
| 778 | + goto end; | |
| 779 | + } | |
| 780 | + | |
| 781 | + for (i = 1; i <= count; i++) { | |
| 782 | + union acpi_object *element; | |
| 783 | + union acpi_object *obj; | |
| 784 | + struct acpi_power_register *reg; | |
| 785 | + struct acpi_processor_cx cx; | |
| 786 | + | |
| 787 | + /* | |
| 788 | + * If there is not enough space for all C-states, skip the | |
| 789 | + * excess ones and log a warning. | |
| 790 | + */ | |
| 791 | + if (last_index >= ACPI_PROCESSOR_MAX_POWER - 1) { | |
| 792 | + acpi_handle_warn(handle, | |
| 793 | + "No room for more idle states (limit: %d)\n", | |
| 794 | + ACPI_PROCESSOR_MAX_POWER - 1); | |
| 795 | + break; | |
| 796 | + } | |
| 797 | + | |
| 798 | + memset(&cx, 0, sizeof(cx)); | |
| 799 | + | |
| 800 | + element = &cst->package.elements[i]; | |
| 801 | + if (element->type != ACPI_TYPE_PACKAGE) | |
| 802 | + continue; | |
| 803 | + | |
| 804 | + if (element->package.count != 4) | |
| 805 | + continue; | |
| 806 | + | |
| 807 | + obj = &element->package.elements[0]; | |
| 808 | + | |
| 809 | + if (obj->type != ACPI_TYPE_BUFFER) | |
| 810 | + continue; | |
| 811 | + | |
| 812 | + reg = (struct acpi_power_register *)obj->buffer.pointer; | |
| 813 | + | |
| 814 | + obj = &element->package.elements[1]; | |
| 815 | + if (obj->type != ACPI_TYPE_INTEGER) | |
| 816 | + continue; | |
| 817 | + | |
| 818 | + cx.type = obj->integer.value; | |
| 819 | + /* | |
| 820 | + * There are known cases in which the _CST output does not | |
| 821 | + * contain C1, so if the type of the first state found is not | |
| 822 | + * C1, leave an empty slot for C1 to be filled in later. | |
| 823 | + */ | |
| 824 | + if (i == 1 && cx.type != ACPI_STATE_C1) | |
| 825 | + last_index = 1; | |
| 826 | + | |
| 827 | + cx.address = reg->address; | |
| 828 | + cx.index = last_index + 1; | |
| 829 | + | |
| 830 | + if (reg->space_id == ACPI_ADR_SPACE_FIXED_HARDWARE) { | |
| 831 | + if (!acpi_processor_ffh_cstate_probe(cpu, &cx, reg)) { | |
| 832 | + /* | |
| 833 | + * In the majority of cases _CST describes C1 as | |
| 834 | + * a FIXED_HARDWARE C-state, but if the command | |
| 835 | + * line forbids using MWAIT, use CSTATE_HALT for | |
| 836 | + * C1 regardless. | |
| 837 | + */ | |
| 838 | + if (cx.type == ACPI_STATE_C1 && | |
| 839 | + boot_option_idle_override == IDLE_NOMWAIT) { | |
| 840 | + cx.entry_method = ACPI_CSTATE_HALT; | |
| 841 | + snprintf(cx.desc, ACPI_CX_DESC_LEN, "ACPI HLT"); | |
| 842 | + } else { | |
| 843 | + cx.entry_method = ACPI_CSTATE_FFH; | |
| 844 | + } | |
| 845 | + } else if (cx.type == ACPI_STATE_C1) { | |
| 846 | + /* | |
| 847 | + * In the special case of C1, FIXED_HARDWARE can | |
| 848 | + * be handled by executing the HLT instruction. | |
| 849 | + */ | |
| 850 | + cx.entry_method = ACPI_CSTATE_HALT; | |
| 851 | + snprintf(cx.desc, ACPI_CX_DESC_LEN, "ACPI HLT"); | |
| 852 | + } else { | |
| 853 | + continue; | |
| 854 | + } | |
| 855 | + } else if (reg->space_id == ACPI_ADR_SPACE_SYSTEM_IO) { | |
| 856 | + cx.entry_method = ACPI_CSTATE_SYSTEMIO; | |
| 857 | + snprintf(cx.desc, ACPI_CX_DESC_LEN, "ACPI IOPORT 0x%x", | |
| 858 | + cx.address); | |
| 859 | + } else { | |
| 860 | + continue; | |
| 861 | + } | |
| 862 | + | |
| 863 | + if (cx.type == ACPI_STATE_C1) | |
| 864 | + cx.valid = 1; | |
| 865 | + | |
| 866 | + obj = &element->package.elements[2]; | |
| 867 | + if (obj->type != ACPI_TYPE_INTEGER) | |
| 868 | + continue; | |
| 869 | + | |
| 870 | + cx.latency = obj->integer.value; | |
| 871 | + | |
| 872 | + obj = &element->package.elements[3]; | |
| 873 | + if (obj->type != ACPI_TYPE_INTEGER) | |
| 874 | + continue; | |
| 875 | + | |
| 876 | + memcpy(&info->states[++last_index], &cx, sizeof(cx)); | |
| 877 | + } | |
| 878 | + | |
| 879 | + acpi_handle_info(handle, "Found %d idle states\n", last_index); | |
| 880 | + | |
| 881 | + info->count = last_index; | |
| 882 | + | |
| 883 | + end: | |
| 884 | + kfree(buffer.pointer); | |
| 885 | + | |
| 886 | + return ret; | |
| 887 | +} | |
| 888 | +EXPORT_SYMBOL_GPL(acpi_processor_evaluate_cst); | |
| 889 | +#endif /* CONFIG_ACPI_PROCESSOR_CSTATE */ | |
| @@ -299,164 +299,24 @@ static int acpi_processor_get_power_info_default(struct acpi_processor *pr) · 298 unmodified lines | ||
| 299 | 299 | |
| 300 | 300 | static int acpi_processor_get_power_info_cst(struct acpi_processor *pr) |
| 301 | 301 | { |
| 302 | − acpi_status status; | |
| 303 | − u64 count; | |
| 304 | − int current_count; | |
| 305 | − int i, ret = 0; | |
| 306 | − struct acpi_buffer buffer = { ACPI_ALLOCATE_BUFFER, NULL }; | |
| 307 | − union acpi_object *cst; | |
| 302 | + int ret; | |
| 308 | 303 | |
| 309 | 304 | if (nocst) |
| 310 | 305 | return -ENODEV; |
| 311 | 306 | |
| 312 | − current_count = 0; | |
| 313 | − | |
| 314 | − status = acpi_evaluate_object(pr->handle, "_CST", NULL, &buffer); | |
| 315 | − if (ACPI_FAILURE(status)) { | |
| 316 | − ACPI_DEBUG_PRINT((ACPI_DB_INFO, "No _CST, giving up\n")); | |
| 317 | − return -ENODEV; | |
| 318 | − } | |
| 319 | − | |
| 320 | − cst = buffer.pointer; | |
| 321 | − | |
| 322 | − /* There must be at least 2 elements */ | |
| 323 | − if (!cst || (cst->type != ACPI_TYPE_PACKAGE) || cst->package.count < 2) { | |
| 324 | − pr_err("not enough elements in _CST\n"); | |
| 325 | − ret = -EFAULT; | |
| 326 | − goto end; | |
| 327 | − } | |
| 328 | − | |
| 329 | − count = cst->package.elements[0].integer.value; | |
| 307 | + ret = acpi_processor_evaluate_cst(pr->handle, pr->id, &pr->power); | |
| 308 | + if (ret) | |
| 309 | + return ret; | |
| 330 | 310 | |
| 331 | − /* Validate number of power states. */ | |
| 332 | − if (count < 1 || count != cst->package.count - 1) { | |
| 333 | − pr_err("count given by _CST is not valid\n"); | |
| 334 | − ret = -EFAULT; | |
| 335 | − goto end; | |
| 336 | − } | |
| 311 | + /* | |
| 312 | + * It is expected that there will be at least 2 states, C1 and | |
| 313 | + * something else (C2 or C3), so fail if that is not the case. | |
| 314 | + */ | |
| 315 | + if (pr->power.count < 2) | |
| 316 | + return -EFAULT; | |
| 337 | 317 | |
| 338 | − /* Tell driver that at least _CST is supported. */ | |
| 339 | 318 | pr->flags.has_cst = 1; |
| 340 | − | |
| 341 | − for (i = 1; i <= count; i++) { | |
| 342 | − union acpi_object *element; | |
| 343 | − union acpi_object *obj; | |
| 344 | − struct acpi_power_register *reg; | |
| 345 | − struct acpi_processor_cx cx; | |
| 346 | − | |
| 347 | − memset(&cx, 0, sizeof(cx)); | |
| 348 | − | |
| 349 | − element = &(cst->package.elements[i]); | |
| 350 | − if (element->type != ACPI_TYPE_PACKAGE) | |
| 351 | − continue; | |
| 352 | − | |
| 353 | − if (element->package.count != 4) | |
| 354 | − continue; | |
| 355 | − | |
| 356 | − obj = &(element->package.elements[0]); | |
| 357 | − | |
| 358 | − if (obj->type != ACPI_TYPE_BUFFER) | |
| 359 | − continue; | |
| 360 | − | |
| 361 | − reg = (struct acpi_power_register *)obj->buffer.pointer; | |
| 362 | − | |
| 363 | − if (reg->space_id != ACPI_ADR_SPACE_SYSTEM_IO && | |
| 364 | − (reg->space_id != ACPI_ADR_SPACE_FIXED_HARDWARE)) | |
| 365 | − continue; | |
| 366 | − | |
| 367 | − /* There should be an easy way to extract an integer... */ | |
| 368 | − obj = &(element->package.elements[1]); | |
| 369 | − if (obj->type != ACPI_TYPE_INTEGER) | |
| 370 | − continue; | |
| 371 | − | |
| 372 | − cx.type = obj->integer.value; | |
| 373 | − /* | |
| 374 | − * Some buggy BIOSes won't list C1 in _CST - | |
| 375 | − * Let acpi_processor_get_power_info_default() handle them later | |
| 376 | − */ | |
| 377 | − if (i == 1 && cx.type != ACPI_STATE_C1) | |
| 378 | − current_count++; | |
| 379 | − | |
| 380 | − cx.address = reg->address; | |
| 381 | − cx.index = current_count + 1; | |
| 382 | − | |
| 383 | − cx.entry_method = ACPI_CSTATE_SYSTEMIO; | |
| 384 | − if (reg->space_id == ACPI_ADR_SPACE_FIXED_HARDWARE) { | |
| 385 | − if (acpi_processor_ffh_cstate_probe | |
| 386 | − (pr->id, &cx, reg) == 0) { | |
| 387 | − cx.entry_method = ACPI_CSTATE_FFH; | |
| 388 | − } else if (cx.type == ACPI_STATE_C1) { | |
| 389 | − /* | |
| 390 | − * C1 is a special case where FIXED_HARDWARE | |
| 391 | − * can be handled in non-MWAIT way as well. | |
| 392 | − * In that case, save this _CST entry info. | |
| 393 | − * Otherwise, ignore this info and continue. | |
| 394 | − */ | |
| 395 | − cx.entry_method = ACPI_CSTATE_HALT; | |
| 396 | − snprintf(cx.desc, ACPI_CX_DESC_LEN, "ACPI HLT"); | |
| 397 | − } else { | |
| 398 | − continue; | |
| 399 | − } | |
| 400 | − if (cx.type == ACPI_STATE_C1 && | |
| 401 | − (boot_option_idle_override == IDLE_NOMWAIT)) { | |
| 402 | − /* | |
| 403 | − * In most cases the C1 space_id obtained from | |
| 404 | − * _CST object is FIXED_HARDWARE access mode. | |
| 405 | − * But when the option of idle=halt is added, | |
| 406 | − * the entry_method type should be changed from | |
| 407 | − * CSTATE_FFH to CSTATE_HALT. | |
| 408 | − * When the option of idle=nomwait is added, | |
| 409 | − * the C1 entry_method type should be | |
| 410 | − * CSTATE_HALT. | |
| 411 | − */ | |
| 412 | − cx.entry_method = ACPI_CSTATE_HALT; | |
| 413 | − snprintf(cx.desc, ACPI_CX_DESC_LEN, "ACPI HLT"); | |
| 414 | − } | |
| 415 | − } else { | |
| 416 | − snprintf(cx.desc, ACPI_CX_DESC_LEN, "ACPI IOPORT 0x%x", | |
| 417 | − cx.address); | |
| 418 | − } | |
| 419 | − | |
| 420 | − if (cx.type == ACPI_STATE_C1) { | |
| 421 | − cx.valid = 1; | |
| 422 | − } | |
| 423 | − | |
| 424 | − obj = &(element->package.elements[2]); | |
| 425 | − if (obj->type != ACPI_TYPE_INTEGER) | |
| 426 | − continue; | |
| 427 | − | |
| 428 | − cx.latency = obj->integer.value; | |
| 429 | − | |
| 430 | − obj = &(element->package.elements[3]); | |
| 431 | − if (obj->type != ACPI_TYPE_INTEGER) | |
| 432 | − continue; | |
| 433 | − | |
| 434 | − current_count++; | |
| 435 | − memcpy(&(pr->power.states[current_count]), &cx, sizeof(cx)); | |
| 436 | − | |
| 437 | − /* | |
| 438 | − * We support total ACPI_PROCESSOR_MAX_POWER - 1 | |
| 439 | − * (From 1 through ACPI_PROCESSOR_MAX_POWER - 1) | |
| 440 | − */ | |
| 441 | − if (current_count >= (ACPI_PROCESSOR_MAX_POWER - 1)) { | |
| 442 | − pr_warn("Limiting number of power states to max (%d)\n", | |
| 443 | − ACPI_PROCESSOR_MAX_POWER); | |
| 444 | − pr_warn("Please increase ACPI_PROCESSOR_MAX_POWER if needed.\n"); | |
| 445 | − break; | |
| 446 | − } | |
| 447 | − } | |
| 448 | − | |
| 449 | − ACPI_DEBUG_PRINT((ACPI_DB_INFO, "Found %d power states\n", | |
| 450 | − current_count)); | |
| 451 | − | |
| 452 | − /* Validate number of power states discovered */ | |
| 453 | − if (current_count < 2) | |
| 454 | − ret = -EFAULT; | |
| 455 | − | |
| 456 | − end: | |
| 457 | − kfree(buffer.pointer); | |
| 458 | − | |
| 459 | − return ret; | |
| 319 | + return 0; | |
| 460 | 320 | } |
| 461 | 321 | |
| 462 | 322 | static void acpi_processor_power_verify_c3(struct acpi_processor *pr, |
| @@ -909,7 +769,6 @@ static int acpi_processor_setup_cstates(struct acpi_processor *pr) · 446 unmodified lines | ||
| 909 | 769 | |
| 910 | 770 | static inline void acpi_processor_cstate_first_run_checks(void) |
| 911 | 771 | { |
| 912 | − acpi_status status; | |
| 913 | 772 | static int first_run; |
| 914 | 773 | |
| 915 | 774 | if (first_run) |
| @@ -921,13 +780,10 @@ static inline void acpi_processor_cstate_first_run_checks(void) · 5 unmodified lines | ||
| 921 | 780 | max_cstate); |
| 922 | 781 | first_run++; |
| 923 | 782 | |
| 924 | − if (acpi_gbl_FADT.cst_control && !nocst) { | |
| 925 | − status = acpi_os_write_port(acpi_gbl_FADT.smi_command, | |
| 926 | − acpi_gbl_FADT.cst_control, 8); | |
| 927 | − if (ACPI_FAILURE(status)) | |
| 928 | − ACPI_EXCEPTION((AE_INFO, status, | |
| 929 | − "Notifying BIOS of _CST ability failed")); | |
| 930 | − } | |
| 783 | + if (nocst) | |
| 784 | + return; | |
| 785 | + | |
| 786 | + acpi_processor_claim_cst_control(); | |
| 931 | 787 | } |
| 932 | 788 | #else |
| 933 | 789 | |
| @@ -25,7 +25,7 @@ config ARM_PSCI_CPUIDLE · 24 unmodified lines | ||
| 25 | 25 | |
| 26 | 26 | config ARM_BIG_LITTLE_CPUIDLE |
| 27 | 27 | bool "Support for ARM big.LITTLE processors" |
| 28 | − depends on ARCH_VEXPRESS_TC2_PM || ARCH_EXYNOS | |
| 28 | + depends on ARCH_VEXPRESS_TC2_PM || ARCH_EXYNOS || COMPILE_TEST | |
| 29 | 29 | depends on MCPM && !ARM64 |
| 30 | 30 | select ARM_CPU_SUSPEND |
| 31 | 31 | select CPU_IDLE_MULTIPLE_DRIVERS |
| @@ -51,13 +51,13 @@ config ARM_HIGHBANK_CPUIDLE · 19 unmodified lines | ||
| 51 | 51 | |
| 52 | 52 | config ARM_KIRKWOOD_CPUIDLE |
| 53 | 53 | bool "CPU Idle Driver for Marvell Kirkwood SoCs" |
| 54 | − depends on MACH_KIRKWOOD && !ARM64 | |
| 54 | + depends on (MACH_KIRKWOOD || COMPILE_TEST) && !ARM64 | |
| 55 | 55 | help |
| 56 | 56 | This adds the CPU Idle driver for Marvell Kirkwood SoCs. |
| 57 | 57 | |
| 58 | 58 | config ARM_ZYNQ_CPUIDLE |
| 59 | 59 | bool "CPU Idle Driver for Xilinx Zynq processors" |
| 60 | − depends on ARCH_ZYNQ && !ARM64 | |
| 60 | + depends on (ARCH_ZYNQ || COMPILE_TEST) && !ARM64 | |
| 61 | 61 | help |
| 62 | 62 | Select this to enable cpuidle on Xilinx Zynq processors. |
| 63 | 63 | |
| @@ -70,19 +70,19 @@ config ARM_U8500_CPUIDLE · 6 unmodified lines | ||
| 70 | 70 | config ARM_AT91_CPUIDLE |
| 71 | 71 | bool "Cpu Idle Driver for the AT91 processors" |
| 72 | 72 | default y |
| 73 | − depends on ARCH_AT91 && !ARM64 | |
| 73 | + depends on (ARCH_AT91 || COMPILE_TEST) && !ARM64 | |
| 74 | 74 | help |
| 75 | 75 | Select this to enable cpuidle for AT91 processors. |
| 76 | 76 | |
| 77 | 77 | config ARM_EXYNOS_CPUIDLE |
| 78 | 78 | bool "Cpu Idle Driver for the Exynos processors" |
| 79 | − depends on ARCH_EXYNOS && !ARM64 | |
| 79 | + depends on (ARCH_EXYNOS || COMPILE_TEST) && !ARM64 | |
| 80 | 80 | select ARCH_NEEDS_CPU_IDLE_COUPLED if SMP |
| 81 | 81 | help |
| 82 | 82 | Select this to enable cpuidle for Exynos processors. |
| 83 | 83 | |
| 84 | 84 | config ARM_MVEBU_V7_CPUIDLE |
| 85 | 85 | bool "CPU Idle Driver for mvebu v7 family processors" |
| 86 | − depends on ARCH_MVEBU && !ARM64 | |
| 86 | + depends on (ARCH_MVEBU || COMPILE_TEST) && !ARM64 | |
| 87 | 87 | help |
| 88 | 88 | Select this to enable cpuidle on Armada 370, 38x and XP processors. |
| @@ -89,6 +89,7 @@ · 88 unmodified lines | ||
| 89 | 89 | * @coupled_cpus: mask of cpus that are part of the coupled set |
| 90 | 90 | * @requested_state: array of requested states for cpus in the coupled set |
| 91 | 91 | * @ready_waiting_counts: combined count of cpus in ready or waiting loops |
| 92 | + * @abort_barrier: synchronisation point for abort cases | |
| 92 | 93 | * @online_count: count of cpus that are online |
| 93 | 94 | * @refcnt: reference count of cpuidle devices that are using this struct |
| 94 | 95 | * @prevent: flag to prevent coupled idle while a cpu is hotplugging |
| @@ -338,7 +339,7 @@ static void cpuidle_coupled_poke(int cpu) · 243 unmodified lines | ||
| 338 | 339 | |
| 339 | 340 | /** |
| 340 | 341 | * cpuidle_coupled_poke_others - wake up all other cpus that may be waiting |
| 341 | − * @dev: struct cpuidle_device for this cpu | |
| 342 | + * @this_cpu: target cpu | |
| 342 | 343 | * @coupled: the struct coupled that contains the current cpu |
| 343 | 344 | * |
| 344 | 345 | * Calls cpuidle_coupled_poke on all other online cpus. |
| @@ -355,7 +356,7 @@ static void cpuidle_coupled_poke_others(int this_cpu, · 10 unmodified lines | ||
| 355 | 356 | |
| 356 | 357 | /** |
| 357 | 358 | * cpuidle_coupled_set_waiting - mark this cpu as in the wait loop |
| 358 | − * @dev: struct cpuidle_device for this cpu | |
| 359 | + * @cpu: target cpu | |
| 359 | 360 | * @coupled: the struct coupled that contains the current cpu |
| 360 | 361 | * @next_state: the index in drv->states of the requested state for this cpu |
| 361 | 362 | * |
| @@ -376,7 +377,7 @@ static int cpuidle_coupled_set_waiting(int cpu, · 14 unmodified lines | ||
| 376 | 377 | |
| 377 | 378 | /** |
| 378 | 379 | * cpuidle_coupled_set_not_waiting - mark this cpu as leaving the wait loop |
| 379 | − * @dev: struct cpuidle_device for this cpu | |
| 380 | + * @cpu: target cpu | |
| 380 | 381 | * @coupled: the struct coupled that contains the current cpu |
| 381 | 382 | * |
| 382 | 383 | * Removes the requested idle state for the specified cpuidle device. |
| @@ -412,7 +413,7 @@ static void cpuidle_coupled_set_done(int cpu, struct cpuidle_coupled *coupled) · 29 unmodified lines | ||
| 412 | 413 | |
| 413 | 414 | /** |
| 414 | 415 | * cpuidle_coupled_clear_pokes - spin until the poke interrupt is processed |
| 415 | − * @cpu - this cpu | |
| 416 | + * @cpu: this cpu | |
| 416 | 417 | * |
| 417 | 418 | * Turns on interrupts and spins until any outstanding poke interrupts have |
| 418 | 419 | * been processed and the poke bit has been cleared. |
| @@ -37,10 +37,7 @@ static struct cpuidle_driver clps711x_idle_driver = { · 36 unmodified lines | ||
| 37 | 37 | |
| 38 | 38 | static int __init clps711x_cpuidle_probe(struct platform_device *pdev) |
| 39 | 39 | { |
| 40 | − struct resource *res; | |
| 41 | − | |
| 42 | − res = platform_get_resource(pdev, IORESOURCE_MEM, 0); | |
| 43 | − clps711x_halt = devm_ioremap_resource(&pdev->dev, res); | |
| 40 | + clps711x_halt = devm_platform_ioremap_resource(pdev, 0); | |
| 44 | 41 | if (IS_ERR(clps711x_halt)) |
| 45 | 42 | return PTR_ERR(clps711x_halt); |
| 46 | 43 | |
| @@ -55,10 +55,7 @@ static struct cpuidle_driver kirkwood_idle_driver = { · 54 unmodified lines | ||
| 55 | 55 | /* Initialize CPU idle by registering the idle states */ |
| 56 | 56 | static int kirkwood_cpuidle_probe(struct platform_device *pdev) |
| 57 | 57 | { |
| 58 | − struct resource *res; | |
| 59 | − | |
| 60 | − res = platform_get_resource(pdev, IORESOURCE_MEM, 0); | |
| 61 | − ddr_operation_base = devm_ioremap_resource(&pdev->dev, res); | |
| 58 | + ddr_operation_base = devm_platform_ioremap_resource(pdev, 0); | |
| 62 | 59 | if (IS_ERR(ddr_operation_base)) |
| 63 | 60 | return PTR_ERR(ddr_operation_base); |
| 64 | 61 | |
| @@ -121,6 +121,9 @@ void cpuidle_use_deepest_state(u64 latency_limit_ns) · 120 unmodified lines | ||
| 121 | 121 | * cpuidle_find_deepest_state - Find the deepest available idle state. |
| 122 | 122 | * @drv: cpuidle driver for the given CPU. |
| 123 | 123 | * @dev: cpuidle device for the given CPU. |
| 124 | + * @latency_limit_ns: Idle state exit latency limit | |
| 125 | + * | |
| 126 | + * Return: the index of the deepest available idle state. | |
| 124 | 127 | */ |
| 125 | 128 | int cpuidle_find_deepest_state(struct cpuidle_driver *drv, |
| 126 | 129 | struct cpuidle_device *dev, |
| @@ -572,10 +575,14 @@ static int __cpuidle_register_device(struct cpuidle_device *dev) · 445 unmodified lines | ||
| 572 | 575 | if (!try_module_get(drv->owner)) |
| 573 | 576 | return -EINVAL; |
| 574 | 577 | |
| 575 | − for (i = 0; i < drv->state_count; i++) | |
| 578 | + for (i = 0; i < drv->state_count; i++) { | |
| 576 | 579 | if (drv->states[i].flags & CPUIDLE_FLAG_UNUSABLE) |
| 577 | 580 | dev->states_usage[i].disable |= CPUIDLE_STATE_DISABLED_BY_DRIVER; |
| 578 | 581 | |
| 582 | + if (drv->states[i].flags & CPUIDLE_FLAG_OFF) | |
| 583 | + dev->states_usage[i].disable |= CPUIDLE_STATE_DISABLED_BY_USER; | |
| 584 | + } | |
| 585 | + | |
| 579 | 586 | per_cpu(cpuidle_devices, dev->cpu) = dev; |
| 580 | 587 | list_add(&dev->device_list, &cpuidle_detected_devices); |
| 581 | 588 | |
| @@ -155,8 +155,6 @@ static void __cpuidle_driver_init(struct cpuidle_driver *drv) · 154 unmodified lines | ||
| 155 | 155 | { |
| 156 | 156 | int i; |
| 157 | 157 | |
| 158 | − drv->refcnt = 0; | |
| 159 | − | |
| 160 | 158 | /* |
| 161 | 159 | * Use all possible CPUs as the default, because if the kernel boots |
| 162 | 160 | * with some CPUs offline and then we online one of them, the CPU |
| @@ -240,9 +238,6 @@ static int __cpuidle_register_driver(struct cpuidle_driver *drv) · 77 unmodified lines | ||
| 240 | 238 | */ |
| 241 | 239 | static void __cpuidle_unregister_driver(struct cpuidle_driver *drv) |
| 242 | 240 | { |
| 243 | − if (WARN_ON(drv->refcnt > 0)) | |
| 244 | − return; | |
| 245 | − | |
| 246 | 241 | if (drv->bctimer) { |
| 247 | 242 | drv->bctimer = 0; |
| 248 | 243 | on_each_cpu_mask(drv->cpumask, cpuidle_setup_broadcast_timer, |
| @@ -349,47 +344,6 @@ struct cpuidle_driver *cpuidle_get_cpu_driver(struct cpuidle_device *dev) · 100 unmodified lines | ||
| 349 | 344 | } |
| 350 | 345 | EXPORT_SYMBOL_GPL(cpuidle_get_cpu_driver); |
| 351 | 346 | |
| 352 | −/** | |
| 353 | − * cpuidle_driver_ref - get a reference to the driver. | |
| 354 | − * | |
| 355 | − * Increment the reference counter of the cpuidle driver associated with | |
| 356 | − * the current CPU. | |
| 357 | − * | |
| 358 | − * Returns a pointer to the driver, or NULL if the current CPU has no driver. | |
| 359 | − */ | |
| 360 | −struct cpuidle_driver *cpuidle_driver_ref(void) | |
| 361 | −{ | |
| 362 | − struct cpuidle_driver *drv; | |
| 363 | − | |
| 364 | − spin_lock(&cpuidle_driver_lock); | |
| 365 | − | |
| 366 | − drv = cpuidle_get_driver(); | |
| 367 | − if (drv) | |
| 368 | − drv->refcnt++; | |
| 369 | − | |
| 370 | − spin_unlock(&cpuidle_driver_lock); | |
| 371 | − return drv; | |
| 372 | −} | |
| 373 | − | |
| 374 | −/** | |
| 375 | − * cpuidle_driver_unref - puts down the refcount for the driver | |
| 376 | − * | |
| 377 | − * Decrement the reference counter of the cpuidle driver associated with | |
| 378 | − * the current CPU. | |
| 379 | − */ | |
| 380 | −void cpuidle_driver_unref(void) | |
| 381 | −{ | |
| 382 | − struct cpuidle_driver *drv; | |
| 383 | − | |
| 384 | − spin_lock(&cpuidle_driver_lock); | |
| 385 | − | |
| 386 | − drv = cpuidle_get_driver(); | |
| 387 | − if (drv && !WARN_ON(drv->refcnt <= 0)) | |
| 388 | − drv->refcnt--; | |
| 389 | − | |
| 390 | − spin_unlock(&cpuidle_driver_lock); | |
| 391 | −} | |
| 392 | − | |
| 393 | 347 | /** |
| 394 | 348 | * cpuidle_driver_state_disabled - Disable or enable an idle state |
| 395 | 349 | * @drv: cpuidle driver owning the state |
| @@ -142,6 +142,7 @@ static struct attribute_group cpuidle_attr_group = { · 141 unmodified lines | ||
| 142 | 142 | |
| 143 | 143 | /** |
| 144 | 144 | * cpuidle_add_interface - add CPU global sysfs attributes |
| 145 | + * @dev: the target device | |
| 145 | 146 | */ |
| 146 | 147 | int cpuidle_add_interface(struct device *dev) |
| 147 | 148 | { |
| @@ -153,6 +154,7 @@ int cpuidle_add_interface(struct device *dev) · 5 unmodified lines | ||
| 153 | 154 | |
| 154 | 155 | /** |
| 155 | 156 | * cpuidle_remove_interface - remove CPU global sysfs attributes |
| 157 | + * @dev: the target device | |
| 156 | 158 | */ |
| 157 | 159 | void cpuidle_remove_interface(struct device *dev) |
| 158 | 160 | { |
| @@ -327,6 +329,14 @@ static ssize_t store_state_disable(struct cpuidle_state *state, · 168 unmodified lines | ||
| 327 | 329 | return size; |
| 328 | 330 | } |
| 329 | 331 | |
| 332 | +static ssize_t show_state_default_status(struct cpuidle_state *state, | |
| 333 | + struct cpuidle_state_usage *state_usage, | |
| 334 | + char *buf) | |
| 335 | +{ | |
| 336 | + return sprintf(buf, "%s\n", | |
| 337 | + state->flags & CPUIDLE_FLAG_OFF ? "disabled" : "enabled"); | |
| 338 | +} | |
| 339 | + | |
| 330 | 340 | define_one_state_ro(name, show_state_name); |
| 331 | 341 | define_one_state_ro(desc, show_state_desc); |
| 332 | 342 | define_one_state_ro(latency, show_state_exit_latency); |
| @@ -337,6 +347,7 @@ define_one_state_ro(time, show_state_time); · 4 unmodified lines | ||
| 337 | 347 | define_one_state_rw(disable, show_state_disable, store_state_disable); |
| 338 | 348 | define_one_state_ro(above, show_state_above); |
| 339 | 349 | define_one_state_ro(below, show_state_below); |
| 350 | +define_one_state_ro(default_status, show_state_default_status); | |
| 340 | 351 | |
| 341 | 352 | static struct attribute *cpuidle_state_default_attrs[] = { |
| 342 | 353 | &attr_name.attr, |
| @@ -349,6 +360,7 @@ static struct attribute *cpuidle_state_default_attrs[] = { · 6 unmodified lines | ||
| 349 | 360 | &attr_disable.attr, |
| 350 | 361 | &attr_above.attr, |
| 351 | 362 | &attr_below.attr, |
| 363 | + &attr_default_status.attr, | |
| 352 | 364 | NULL |
| 353 | 365 | }; |
| 354 | 366 | |
| @@ -615,7 +627,7 @@ static struct kobj_type ktype_driver_cpuidle = { · 260 unmodified lines | ||
| 615 | 627 | |
| 616 | 628 | /** |
| 617 | 629 | * cpuidle_add_driver_sysfs - adds the driver name sysfs attribute |
| 618 | − * @device: the target device | |
| 630 | + * @dev: the target device | |
| 619 | 631 | */ |
| 620 | 632 | static int cpuidle_add_driver_sysfs(struct cpuidle_device *dev) |
| 621 | 633 | { |
| @@ -646,7 +658,7 @@ static int cpuidle_add_driver_sysfs(struct cpuidle_device *dev) · 24 unmodified lines | ||
| 646 | 658 | |
| 647 | 659 | /** |
| 648 | 660 | * cpuidle_remove_driver_sysfs - removes the driver name sysfs attribute |
| 649 | − * @device: the target device | |
| 661 | + * @dev: the target device | |
| 650 | 662 | */ |
| 651 | 663 | static void cpuidle_remove_driver_sysfs(struct cpuidle_device *dev) |
| 652 | 664 | { |
| @@ -41,6 +41,7 @@ · 40 unmodified lines | ||
| 41 | 41 | |
| 42 | 42 | #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt |
| 43 | 43 | |
| 44 | +#include <linux/acpi.h> | |
| 44 | 45 | #include <linux/kernel.h> |
| 45 | 46 | #include <linux/cpuidle.h> |
| 46 | 47 | #include <linux/tick.h> |
| @@ -79,6 +80,7 @@ struct idle_cpu { · 32 unmodified lines | ||
| 79 | 80 | unsigned long auto_demotion_disable_flags; |
| 80 | 81 | bool byt_auto_demotion_disable_flag; |
| 81 | 82 | bool disable_promotion_to_c1e; |
| 83 | + bool use_acpi; | |
| 82 | 84 | }; |
| 83 | 85 | |
| 84 | 86 | static const struct idle_cpu *icpu; |
| @@ -89,6 +91,11 @@ static void intel_idle_s2idle(struct cpuidle_device *dev, · 4 unmodified lines | ||
| 89 | 91 | struct cpuidle_driver *drv, int index); |
| 90 | 92 | static struct cpuidle_state *cpuidle_state_table; |
| 91 | 93 | |
| 94 | +/* | |
| 95 | + * Enable this state by default even if the ACPI _CST does not list it. | |
| 96 | + */ | |
| 97 | +#define CPUIDLE_FLAG_ALWAYS_ENABLE BIT(15) | |
| 98 | + | |
| 92 | 99 | /* |
| 93 | 100 | * Set this flag for states where the HW flushes the TLB for us |
| 94 | 101 | * and so we don't need cross-calls to keep it consistent. |
| @@ -124,7 +131,7 @@ static struct cpuidle_state nehalem_cstates[] = { · 29 unmodified lines | ||
| 124 | 131 | { |
| 125 | 132 | .name = "C1E", |
| 126 | 133 | .desc = "MWAIT 0x01", |
| 127 | − .flags = MWAIT2flg(0x01), | |
| 134 | + .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE, | |
| 128 | 135 | .exit_latency = 10, |
| 129 | 136 | .target_residency = 20, |
| 130 | 137 | .enter = &intel_idle, |
| @@ -161,7 +168,7 @@ static struct cpuidle_state snb_cstates[] = { · 30 unmodified lines | ||
| 161 | 168 | { |
| 162 | 169 | .name = "C1E", |
| 163 | 170 | .desc = "MWAIT 0x01", |
| 164 | − .flags = MWAIT2flg(0x01), | |
| 171 | + .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE, | |
| 165 | 172 | .exit_latency = 10, |
| 166 | 173 | .target_residency = 20, |
| 167 | 174 | .enter = &intel_idle, |
| @@ -296,7 +303,7 @@ static struct cpuidle_state ivb_cstates[] = { · 128 unmodified lines | ||
| 296 | 303 | { |
| 297 | 304 | .name = "C1E", |
| 298 | 305 | .desc = "MWAIT 0x01", |
| 299 | − .flags = MWAIT2flg(0x01), | |
| 306 | + .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE, | |
| 300 | 307 | .exit_latency = 10, |
| 301 | 308 | .target_residency = 20, |
| 302 | 309 | .enter = &intel_idle, |
| @@ -341,7 +348,7 @@ static struct cpuidle_state ivt_cstates[] = { · 38 unmodified lines | ||
| 341 | 348 | { |
| 342 | 349 | .name = "C1E", |
| 343 | 350 | .desc = "MWAIT 0x01", |
| 344 | − .flags = MWAIT2flg(0x01), | |
| 351 | + .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE, | |
| 345 | 352 | .exit_latency = 10, |
| 346 | 353 | .target_residency = 80, |
| 347 | 354 | .enter = &intel_idle, |
| @@ -378,7 +385,7 @@ static struct cpuidle_state ivt_cstates_4s[] = { · 30 unmodified lines | ||
| 378 | 385 | { |
| 379 | 386 | .name = "C1E", |
| 380 | 387 | .desc = "MWAIT 0x01", |
| 381 | − .flags = MWAIT2flg(0x01), | |
| 388 | + .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE, | |
| 382 | 389 | .exit_latency = 10, |
| 383 | 390 | .target_residency = 250, |
| 384 | 391 | .enter = &intel_idle, |
| @@ -415,7 +422,7 @@ static struct cpuidle_state ivt_cstates_8s[] = { · 30 unmodified lines | ||
| 415 | 422 | { |
| 416 | 423 | .name = "C1E", |
| 417 | 424 | .desc = "MWAIT 0x01", |
| 418 | − .flags = MWAIT2flg(0x01), | |
| 425 | + .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE, | |
| 419 | 426 | .exit_latency = 10, |
| 420 | 427 | .target_residency = 500, |
| 421 | 428 | .enter = &intel_idle, |
| @@ -452,7 +459,7 @@ static struct cpuidle_state hsw_cstates[] = { · 30 unmodified lines | ||
| 452 | 459 | { |
| 453 | 460 | .name = "C1E", |
| 454 | 461 | .desc = "MWAIT 0x01", |
| 455 | − .flags = MWAIT2flg(0x01), | |
| 462 | + .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE, | |
| 456 | 463 | .exit_latency = 10, |
| 457 | 464 | .target_residency = 20, |
| 458 | 465 | .enter = &intel_idle, |
| @@ -520,7 +527,7 @@ static struct cpuidle_state bdw_cstates[] = { · 61 unmodified lines | ||
| 520 | 527 | { |
| 521 | 528 | .name = "C1E", |
| 522 | 529 | .desc = "MWAIT 0x01", |
| 523 | − .flags = MWAIT2flg(0x01), | |
| 530 | + .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE, | |
| 524 | 531 | .exit_latency = 10, |
| 525 | 532 | .target_residency = 20, |
| 526 | 533 | .enter = &intel_idle, |
| @@ -589,7 +596,7 @@ static struct cpuidle_state skl_cstates[] = { · 62 unmodified lines | ||
| 589 | 596 | { |
| 590 | 597 | .name = "C1E", |
| 591 | 598 | .desc = "MWAIT 0x01", |
| 592 | − .flags = MWAIT2flg(0x01), | |
| 599 | + .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE, | |
| 593 | 600 | .exit_latency = 10, |
| 594 | 601 | .target_residency = 20, |
| 595 | 602 | .enter = &intel_idle, |
| @@ -658,7 +665,7 @@ static struct cpuidle_state skx_cstates[] = { · 62 unmodified lines | ||
| 658 | 665 | { |
| 659 | 666 | .name = "C1E", |
| 660 | 667 | .desc = "MWAIT 0x01", |
| 661 | − .flags = MWAIT2flg(0x01), | |
| 668 | + .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE, | |
| 662 | 669 | .exit_latency = 10, |
| 663 | 670 | .target_residency = 20, |
| 664 | 671 | .enter = &intel_idle, |
| @@ -808,7 +815,7 @@ static struct cpuidle_state bxt_cstates[] = { · 143 unmodified lines | ||
| 808 | 815 | { |
| 809 | 816 | .name = "C1E", |
| 810 | 817 | .desc = "MWAIT 0x01", |
| 811 | − .flags = MWAIT2flg(0x01), | |
| 818 | + .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE, | |
| 812 | 819 | .exit_latency = 10, |
| 813 | 820 | .target_residency = 20, |
| 814 | 821 | .enter = &intel_idle, |
| @@ -869,7 +876,7 @@ static struct cpuidle_state dnv_cstates[] = { · 54 unmodified lines | ||
| 869 | 876 | { |
| 870 | 877 | .name = "C1E", |
| 871 | 878 | .desc = "MWAIT 0x01", |
| 872 | − .flags = MWAIT2flg(0x01), | |
| 879 | + .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE, | |
| 873 | 880 | .exit_latency = 10, |
| 874 | 881 | .target_residency = 20, |
| 875 | 882 | .enter = &intel_idle, |
| @@ -944,37 +951,19 @@ static void intel_idle_s2idle(struct cpuidle_device *dev, · 68 unmodified lines | ||
| 944 | 951 | mwait_idle_with_hints(eax, ecx); |
| 945 | 952 | } |
| 946 | 953 | |
| 947 | −static void __setup_broadcast_timer(bool on) | |
| 948 | −{ | |
| 949 | − if (on) | |
| 950 | − tick_broadcast_enable(); | |
| 951 | − else | |
| 952 | − tick_broadcast_disable(); | |
| 953 | −} | |
| 954 | − | |
| 955 | −static void auto_demotion_disable(void) | |
| 956 | −{ | |
| 957 | − unsigned long long msr_bits; | |
| 958 | − | |
| 959 | − rdmsrl(MSR_PKG_CST_CONFIG_CONTROL, msr_bits); | |
| 960 | − msr_bits &= ~(icpu->auto_demotion_disable_flags); | |
| 961 | − wrmsrl(MSR_PKG_CST_CONFIG_CONTROL, msr_bits); | |
| 962 | −} | |
| 963 | −static void c1e_promotion_disable(void) | |
| 964 | −{ | |
| 965 | − unsigned long long msr_bits; | |
| 966 | − | |
| 967 | − rdmsrl(MSR_IA32_POWER_CTL, msr_bits); | |
| 968 | − msr_bits &= ~0x2; | |
| 969 | − wrmsrl(MSR_IA32_POWER_CTL, msr_bits); | |
| 970 | −} | |
| 971 | − | |
| 972 | 954 | static const struct idle_cpu idle_cpu_nehalem = { |
| 973 | 955 | .state_table = nehalem_cstates, |
| 974 | 956 | .auto_demotion_disable_flags = NHM_C1_AUTO_DEMOTE | NHM_C3_AUTO_DEMOTE, |
| 975 | 957 | .disable_promotion_to_c1e = true, |
| 976 | 958 | }; |
| 977 | 959 | |
| 960 | +static const struct idle_cpu idle_cpu_nhx = { | |
| 961 | + .state_table = nehalem_cstates, | |
| 962 | + .auto_demotion_disable_flags = NHM_C1_AUTO_DEMOTE | NHM_C3_AUTO_DEMOTE, | |
| 963 | + .disable_promotion_to_c1e = true, | |
| 964 | + .use_acpi = true, | |
| 965 | +}; | |
| 966 | + | |
| 978 | 967 | static const struct idle_cpu idle_cpu_atom = { |
| 979 | 968 | .state_table = atom_cstates, |
| 980 | 969 | }; |
| @@ -993,6 +982,12 @@ static const struct idle_cpu idle_cpu_snb = { · 12 unmodified lines | ||
| 993 | 982 | .disable_promotion_to_c1e = true, |
| 994 | 983 | }; |
| 995 | 984 | |
| 985 | +static const struct idle_cpu idle_cpu_snx = { | |
| 986 | + .state_table = snb_cstates, | |
| 987 | + .disable_promotion_to_c1e = true, | |
| 988 | + .use_acpi = true, | |
| 989 | +}; | |
| 990 | + | |
| 996 | 991 | static const struct idle_cpu idle_cpu_byt = { |
| 997 | 992 | .state_table = byt_cstates, |
| 998 | 993 | .disable_promotion_to_c1e = true, |
| @@ -1013,6 +1008,7 @@ static const struct idle_cpu idle_cpu_ivb = { · 14 unmodified lines | ||
| 1013 | 1008 | static const struct idle_cpu idle_cpu_ivt = { |
| 1014 | 1009 | .state_table = ivt_cstates, |
| 1015 | 1010 | .disable_promotion_to_c1e = true, |
| 1011 | + .use_acpi = true, | |
| 1016 | 1012 | }; |
| 1017 | 1013 | |
| 1018 | 1014 | static const struct idle_cpu idle_cpu_hsw = { |
| @@ -1020,11 +1016,23 @@ static const struct idle_cpu idle_cpu_hsw = { · 1 unmodified line | ||
| 1020 | 1016 | .disable_promotion_to_c1e = true, |
| 1021 | 1017 | }; |
| 1022 | 1018 | |
| 1019 | +static const struct idle_cpu idle_cpu_hsx = { | |
| 1020 | + .state_table = hsw_cstates, | |
| 1021 | + .disable_promotion_to_c1e = true, | |
| 1022 | + .use_acpi = true, | |
| 1023 | +}; | |
| 1024 | + | |
| 1023 | 1025 | static const struct idle_cpu idle_cpu_bdw = { |
| 1024 | 1026 | .state_table = bdw_cstates, |
| 1025 | 1027 | .disable_promotion_to_c1e = true, |
| 1026 | 1028 | }; |
| 1027 | 1029 | |
| 1030 | +static const struct idle_cpu idle_cpu_bdx = { | |
| 1031 | + .state_table = bdw_cstates, | |
| 1032 | + .disable_promotion_to_c1e = true, | |
| 1033 | + .use_acpi = true, | |
| 1034 | +}; | |
| 1035 | + | |
| 1028 | 1036 | static const struct idle_cpu idle_cpu_skl = { |
| 1029 | 1037 | .state_table = skl_cstates, |
| 1030 | 1038 | .disable_promotion_to_c1e = true, |
| @@ -1033,15 +1041,18 @@ static const struct idle_cpu idle_cpu_skl = { · 2 unmodified lines | ||
| 1033 | 1041 | static const struct idle_cpu idle_cpu_skx = { |
| 1034 | 1042 | .state_table = skx_cstates, |
| 1035 | 1043 | .disable_promotion_to_c1e = true, |
| 1044 | + .use_acpi = true, | |
| 1036 | 1045 | }; |
| 1037 | 1046 | |
| 1038 | 1047 | static const struct idle_cpu idle_cpu_avn = { |
| 1039 | 1048 | .state_table = avn_cstates, |
| 1040 | 1049 | .disable_promotion_to_c1e = true, |
| 1050 | + .use_acpi = true, | |
| 1041 | 1051 | }; |
| 1042 | 1052 | |
| 1043 | 1053 | static const struct idle_cpu idle_cpu_knl = { |
| 1044 | 1054 | .state_table = knl_cstates, |
| 1055 | + .use_acpi = true, | |
| 1045 | 1056 | }; |
| 1046 | 1057 | |
| 1047 | 1058 | static const struct idle_cpu idle_cpu_bxt = { |
| @@ -1052,20 +1063,21 @@ static const struct idle_cpu idle_cpu_bxt = { · 4 unmodified lines | ||
| 1052 | 1063 | static const struct idle_cpu idle_cpu_dnv = { |
| 1053 | 1064 | .state_table = dnv_cstates, |
| 1054 | 1065 | .disable_promotion_to_c1e = true, |
| 1066 | + .use_acpi = true, | |
| 1055 | 1067 | }; |
| 1056 | 1068 | |
| 1057 | 1069 | static const struct x86_cpu_id intel_idle_ids[] __initconst = { |
| 1058 | − INTEL_CPU_FAM6(NEHALEM_EP, idle_cpu_nehalem), | |
| 1070 | + INTEL_CPU_FAM6(NEHALEM_EP, idle_cpu_nhx), | |
| 1059 | 1071 | INTEL_CPU_FAM6(NEHALEM, idle_cpu_nehalem), |
| 1060 | 1072 | INTEL_CPU_FAM6(NEHALEM_G, idle_cpu_nehalem), |
| 1061 | 1073 | INTEL_CPU_FAM6(WESTMERE, idle_cpu_nehalem), |
| 1062 | − INTEL_CPU_FAM6(WESTMERE_EP, idle_cpu_nehalem), | |
| 1063 | − INTEL_CPU_FAM6(NEHALEM_EX, idle_cpu_nehalem), | |
| 1074 | + INTEL_CPU_FAM6(WESTMERE_EP, idle_cpu_nhx), | |
| 1075 | + INTEL_CPU_FAM6(NEHALEM_EX, idle_cpu_nhx), | |
| 1064 | 1076 | INTEL_CPU_FAM6(ATOM_BONNELL, idle_cpu_atom), |
| 1065 | 1077 | INTEL_CPU_FAM6(ATOM_BONNELL_MID, idle_cpu_lincroft), |
| 1066 | − INTEL_CPU_FAM6(WESTMERE_EX, idle_cpu_nehalem), | |
| 1078 | + INTEL_CPU_FAM6(WESTMERE_EX, idle_cpu_nhx), | |
| 1067 | 1079 | INTEL_CPU_FAM6(SANDYBRIDGE, idle_cpu_snb), |
| 1068 | − INTEL_CPU_FAM6(SANDYBRIDGE_X, idle_cpu_snb), | |
| 1080 | + INTEL_CPU_FAM6(SANDYBRIDGE_X, idle_cpu_snx), | |
| 1069 | 1081 | INTEL_CPU_FAM6(ATOM_SALTWELL, idle_cpu_atom), |
| 1070 | 1082 | INTEL_CPU_FAM6(ATOM_SILVERMONT, idle_cpu_byt), |
| 1071 | 1083 | INTEL_CPU_FAM6(ATOM_SILVERMONT_MID, idle_cpu_tangier), |
| @@ -1073,14 +1085,14 @@ static const struct x86_cpu_id intel_idle_ids[] __initconst = { · 1 unmodified line | ||
| 1073 | 1085 | INTEL_CPU_FAM6(IVYBRIDGE, idle_cpu_ivb), |
| 1074 | 1086 | INTEL_CPU_FAM6(IVYBRIDGE_X, idle_cpu_ivt), |
| 1075 | 1087 | INTEL_CPU_FAM6(HASWELL, idle_cpu_hsw), |
| 1076 | − INTEL_CPU_FAM6(HASWELL_X, idle_cpu_hsw), | |
| 1088 | + INTEL_CPU_FAM6(HASWELL_X, idle_cpu_hsx), | |
| 1077 | 1089 | INTEL_CPU_FAM6(HASWELL_L, idle_cpu_hsw), |
| 1078 | 1090 | INTEL_CPU_FAM6(HASWELL_G, idle_cpu_hsw), |
| 1079 | 1091 | INTEL_CPU_FAM6(ATOM_SILVERMONT_D, idle_cpu_avn), |
| 1080 | 1092 | INTEL_CPU_FAM6(BROADWELL, idle_cpu_bdw), |
| 1081 | 1093 | INTEL_CPU_FAM6(BROADWELL_G, idle_cpu_bdw), |
| 1082 | − INTEL_CPU_FAM6(BROADWELL_X, idle_cpu_bdw), | |
| 1083 | − INTEL_CPU_FAM6(BROADWELL_D, idle_cpu_bdw), | |
| 1094 | + INTEL_CPU_FAM6(BROADWELL_X, idle_cpu_bdx), | |
| 1095 | + INTEL_CPU_FAM6(BROADWELL_D, idle_cpu_bdx), | |
| 1084 | 1096 | INTEL_CPU_FAM6(SKYLAKE_L, idle_cpu_skl), |
| 1085 | 1097 | INTEL_CPU_FAM6(SKYLAKE, idle_cpu_skl), |
| 1086 | 1098 | INTEL_CPU_FAM6(KABYLAKE_L, idle_cpu_skl), |
| @@ -1095,76 +1107,169 @@ static const struct x86_cpu_id intel_idle_ids[] __initconst = { · 8 unmodified lines | ||
| 1095 | 1107 | {} |
| 1096 | 1108 | }; |
| 1097 | 1109 | |
| 1098 | −/* | |
| 1099 | − * intel_idle_probe() | |
| 1110 | +#define INTEL_CPU_FAM6_MWAIT \ | |
| 1111 | + { X86_VENDOR_INTEL, 6, X86_MODEL_ANY, X86_FEATURE_MWAIT, 0 } | |
| 1112 | + | |
| 1113 | +static const struct x86_cpu_id intel_mwait_ids[] __initconst = { | |
| 1114 | + INTEL_CPU_FAM6_MWAIT, | |
| 1115 | + {} | |
| 1116 | +}; | |
| 1117 | + | |
| 1118 | +static bool __init intel_idle_max_cstate_reached(int cstate) | |
| 1119 | +{ | |
| 1120 | + if (cstate + 1 > max_cstate) { | |
| 1121 | + pr_info("max_cstate %d reached\n", max_cstate); | |
| 1122 | + return true; | |
| 1123 | + } | |
| 1124 | + return false; | |
| 1125 | +} | |
| 1126 | + | |
| 1127 | +#ifdef CONFIG_ACPI_PROCESSOR_CSTATE | |
| 1128 | +#include <acpi/processor.h> | |
| 1129 | + | |
| 1130 | +static bool no_acpi __read_mostly; | |
| 1131 | +module_param(no_acpi, bool, 0444); | |
| 1132 | +MODULE_PARM_DESC(no_acpi, "Do not use ACPI _CST for building the idle states list"); | |
| 1133 | + | |
| 1134 | +static struct acpi_processor_power acpi_state_table __initdata; | |
| 1135 | + | |
| 1136 | +/** | |
| 1137 | + * intel_idle_cst_usable - Check if the _CST information can be used. | |
| 1138 | + * | |
| 1139 | + * Check if all of the C-states listed by _CST in the max_cstate range are | |
| 1140 | + * ACPI_CSTATE_FFH, which means that they should be entered via MWAIT. | |
| 1100 | 1141 | */ |
| 1101 | −static int __init intel_idle_probe(void) | |
| 1142 | +static bool __init intel_idle_cst_usable(void) | |
| 1102 | 1143 | { |
| 1103 | − unsigned int eax, ebx, ecx; | |
| 1104 | − const struct x86_cpu_id *id; | |
| 1144 | + int cstate, limit; | |
| 1105 | 1145 | |
| 1106 | − if (max_cstate == 0) { | |
| 1107 | − pr_debug("disabled\n"); | |
| 1108 | − return -EPERM; | |
| 1109 | − } | |
| 1146 | + limit = min_t(int, min_t(int, CPUIDLE_STATE_MAX, max_cstate + 1), | |
| 1147 | + acpi_state_table.count); | |
| 1110 | 1148 | |
| 1111 | − id = x86_match_cpu(intel_idle_ids); | |
| 1112 | − if (!id) { | |
| 1113 | − if (boot_cpu_data.x86_vendor == X86_VENDOR_INTEL && | |
| 1114 | − boot_cpu_data.x86 == 6) | |
| 1115 | − pr_debug("does not run on family %d model %d\n", | |
| 1116 | − boot_cpu_data.x86, boot_cpu_data.x86_model); | |
| 1117 | − return -ENODEV; | |
| 1149 | + for (cstate = 1; cstate < limit; cstate++) { | |
| 1150 | + struct acpi_processor_cx *cx = &acpi_state_table.states[cstate]; | |
| 1151 | + | |
| 1152 | + if (cx->entry_method != ACPI_CSTATE_FFH) | |
| 1153 | + return false; | |
| 1118 | 1154 | } |
| 1119 | 1155 | |
| 1120 | − if (!boot_cpu_has(X86_FEATURE_MWAIT)) { | |
| 1121 | − pr_debug("Please enable MWAIT in BIOS SETUP\n"); | |
| 1122 | − return -ENODEV; | |
| 1156 | + return true; | |
| 1157 | +} | |
| 1158 | + | |
| 1159 | +static bool __init intel_idle_acpi_cst_extract(void) | |
| 1160 | +{ | |
| 1161 | + unsigned int cpu; | |
| 1162 | + | |
| 1163 | + if (no_acpi) { | |
| 1164 | + pr_debug("Not allowed to use ACPI _CST\n"); | |
| 1165 | + return false; | |
| 1123 | 1166 | } |
| 1124 | 1167 | |
| 1125 | − if (boot_cpu_data.cpuid_level < CPUID_MWAIT_LEAF) | |
| 1126 | − return -ENODEV; | |
| 1168 | + for_each_possible_cpu(cpu) { | |
| 1169 | + struct acpi_processor *pr = per_cpu(processors, cpu); | |
| 1127 | 1170 | |
| 1128 | − cpuid(CPUID_MWAIT_LEAF, &eax, &ebx, &ecx, &mwait_substates); | |
| 1171 | + if (!pr) | |
| 1172 | + continue; | |
| 1129 | 1173 | |
| 1130 | − if (!(ecx & CPUID5_ECX_EXTENSIONS_SUPPORTED) || | |
| 1131 | − !(ecx & CPUID5_ECX_INTERRUPT_BREAK) || | |
| 1132 | − !mwait_substates) | |
| 1133 | − return -ENODEV; | |
| 1174 | + if (acpi_processor_evaluate_cst(pr->handle, cpu, &acpi_state_table)) | |
| 1175 | + continue; | |
| 1134 | 1176 | |
| 1135 | − pr_debug("MWAIT substates: 0x%x\n", mwait_substates); | |
| 1177 | + acpi_state_table.count++; | |
| 1136 | 1178 | |
| 1137 | − icpu = (const struct idle_cpu *)id->driver_data; | |
| 1138 | − cpuidle_state_table = icpu->state_table; | |
| 1179 | + if (!intel_idle_cst_usable()) | |
| 1180 | + continue; | |
| 1139 | 1181 | |
| 1140 | − pr_debug("v" INTEL_IDLE_VERSION " model 0x%X\n", | |
| 1141 | − boot_cpu_data.x86_model); | |
| 1182 | + if (!acpi_processor_claim_cst_control()) { | |
| 1183 | + acpi_state_table.count = 0; | |
| 1184 | + return false; | |
| 1185 | + } | |
| 1142 | 1186 | |
| 1143 | − return 0; | |
| 1187 | + return true; | |
| 1188 | + } | |
| 1189 | + | |
| 1190 | + pr_debug("ACPI _CST not found or not usable\n"); | |
| 1191 | + return false; | |
| 1144 | 1192 | } |
| 1145 | 1193 | |
| 1146 | −/* | |
| 1147 | − * intel_idle_cpuidle_devices_uninit() | |
| 1148 | − * Unregisters the cpuidle devices. | |
| 1149 | − */ | |
| 1150 | −static void intel_idle_cpuidle_devices_uninit(void) | |
| 1194 | +static void __init intel_idle_init_cstates_acpi(struct cpuidle_driver *drv) | |
| 1151 | 1195 | { |
| 1152 | − int i; | |
| 1153 | − struct cpuidle_device *dev; | |
| 1196 | + int cstate, limit = min_t(int, CPUIDLE_STATE_MAX, acpi_state_table.count); | |
| 1197 | + | |
| 1198 | + /* | |
| 1199 | + * If limit > 0, intel_idle_cst_usable() has returned 'true', so all of | |
| 1200 | + * the interesting states are ACPI_CSTATE_FFH. | |
| 1201 | + */ | |
| 1202 | + for (cstate = 1; cstate < limit; cstate++) { | |
| 1203 | + struct acpi_processor_cx *cx; | |
| 1204 | + struct cpuidle_state *state; | |
| 1154 | 1205 | |
| 1155 | − for_each_online_cpu(i) { | |
| 1156 | − dev = per_cpu_ptr(intel_idle_cpuidle_devices, i); | |
| 1157 | − cpuidle_unregister_device(dev); | |
| 1206 | + if (intel_idle_max_cstate_reached(cstate)) | |
| 1207 | + break; | |
| 1208 | + | |
| 1209 | + cx = &acpi_state_table.states[cstate]; | |
| 1210 | + | |
| 1211 | + state = &drv->states[drv->state_count++]; | |
| 1212 | + | |
| 1213 | + snprintf(state->name, CPUIDLE_NAME_LEN, "C%d_ACPI", cstate); | |
| 1214 | + strlcpy(state->desc, cx->desc, CPUIDLE_DESC_LEN); | |
| 1215 | + state->exit_latency = cx->latency; | |
| 1216 | + /* | |
| 1217 | + * For C1-type C-states use the same number for both the exit | |
| 1218 | + * latency and target residency, because that is the case for | |
| 1219 | + * C1 in the majority of the static C-states tables above. | |
| 1220 | + * For the other types of C-states, however, set the target | |
| 1221 | + * residency to 3 times the exit latency which should lead to | |
| 1222 | + * a reasonable balance between energy-efficiency and | |
| 1223 | + * performance in the majority of interesting cases. | |
| 1224 | + */ | |
| 1225 | + state->target_residency = cx->latency; | |
| 1226 | + if (cx->type > ACPI_STATE_C1) | |
| 1227 | + state->target_residency *= 3; | |
| 1228 | + | |
| 1229 | + state->flags = MWAIT2flg(cx->address); | |
| 1230 | + if (cx->type > ACPI_STATE_C2) | |
| 1231 | + state->flags |= CPUIDLE_FLAG_TLB_FLUSHED; | |
| 1232 | + | |
| 1233 | + state->enter = intel_idle; | |
| 1234 | + state->enter_s2idle = intel_idle_s2idle; | |
| 1158 | 1235 | } |
| 1159 | 1236 | } |
| 1160 | 1237 | |
| 1238 | +static bool __init intel_idle_off_by_default(u32 mwait_hint) | |
| 1239 | +{ | |
| 1240 | + int cstate, limit; | |
| 1241 | + | |
| 1242 | + /* | |
| 1243 | + * If there are no _CST C-states, do not disable any C-states by | |
| 1244 | + * default. | |
| 1245 | + */ | |
| 1246 | + if (!acpi_state_table.count) | |
| 1247 | + return false; | |
| 1248 | + | |
| 1249 | + limit = min_t(int, CPUIDLE_STATE_MAX, acpi_state_table.count); | |
| 1250 | + /* | |
| 1251 | + * If limit > 0, intel_idle_cst_usable() has returned 'true', so all of | |
| 1252 | + * the interesting states are ACPI_CSTATE_FFH. | |
| 1253 | + */ | |
| 1254 | + for (cstate = 1; cstate < limit; cstate++) { | |
| 1255 | + if (acpi_state_table.states[cstate].address == mwait_hint) | |
| 1256 | + return false; | |
| 1257 | + } | |
| 1258 | + return true; | |
| 1259 | +} | |
| 1260 | +#else /* !CONFIG_ACPI_PROCESSOR_CSTATE */ | |
| 1261 | +static inline bool intel_idle_acpi_cst_extract(void) { return false; } | |
| 1262 | +static inline void intel_idle_init_cstates_acpi(struct cpuidle_driver *drv) { } | |
| 1263 | +static inline bool intel_idle_off_by_default(u32 mwait_hint) { return false; } | |
| 1264 | +#endif /* !CONFIG_ACPI_PROCESSOR_CSTATE */ | |
| 1265 | + | |
| 1161 | 1266 | /* |
| 1162 | 1267 | * ivt_idle_state_table_update(void) |
| 1163 | 1268 | * |
| 1164 | 1269 | * Tune IVT multi-socket targets |
| 1165 | 1270 | * Assumption: num_sockets == (max_package_num + 1) |
| 1166 | 1271 | */ |
| 1167 | −static void ivt_idle_state_table_update(void) | |
| 1272 | +static void __init ivt_idle_state_table_update(void) | |
| 1168 | 1273 | { |
| 1169 | 1274 | /* IVT uses a different table for 1-2, 3-4, and > 4 sockets */ |
| 1170 | 1275 | int cpu, package_num, num_sockets = 1; |
| @@ -1187,15 +1292,17 @@ static void ivt_idle_state_table_update(void) · 16 unmodified lines | ||
| 1187 | 1292 | /* else, 1 and 2 socket systems use default ivt_cstates */ |
| 1188 | 1293 | } |
| 1189 | 1294 | |
| 1190 | −/* | |
| 1191 | − * Translate IRTL (Interrupt Response Time Limit) MSR to usec | |
| 1295 | +/** | |
| 1296 | + * irtl_2_usec - IRTL to microseconds conversion. | |
| 1297 | + * @irtl: IRTL MSR value. | |
| 1298 | + * | |
| 1299 | + * Translate the IRTL (Interrupt Response Time Limit) MSR value to microseconds. | |
| 1192 | 1300 | */ |
| 1193 | − | |
| 1194 | −static unsigned int irtl_ns_units[] = { | |
| 1195 | − 1, 32, 1024, 32768, 1048576, 33554432, 0, 0 }; | |
| 1196 | − | |
| 1197 | −static unsigned long long irtl_2_usec(unsigned long long irtl) | |
| 1301 | +static unsigned long long __init irtl_2_usec(unsigned long long irtl) | |
| 1198 | 1302 | { |
| 1303 | + static const unsigned int irtl_ns_units[] __initconst = { | |
| 1304 | + 1, 32, 1024, 32768, 1048576, 33554432, 0, 0 | |
| 1305 | + }; | |
| 1199 | 1306 | unsigned long long ns; |
| 1200 | 1307 | |
| 1201 | 1308 | if (!irtl) |
| @@ -1203,15 +1310,16 @@ static unsigned long long irtl_2_usec(unsigned long long irtl) · 1 unmodified line | ||
| 1203 | 1310 | |
| 1204 | 1311 | ns = irtl_ns_units[(irtl >> 10) & 0x7]; |
| 1205 | 1312 | |
| 1206 | − return div64_u64((irtl & 0x3FF) * ns, 1000); | |
| 1313 | + return div_u64((irtl & 0x3FF) * ns, NSEC_PER_USEC); | |
| 1207 | 1314 | } |
| 1315 | + | |
| 1208 | 1316 | /* |
| 1209 | 1317 | * bxt_idle_state_table_update(void) |
| 1210 | 1318 | * |
| 1211 | 1319 | * On BXT, we trust the IRTL to show the definitive maximum latency |
| 1212 | 1320 | * We use the same value for target_residency. |
| 1213 | 1321 | */ |
| 1214 | −static void bxt_idle_state_table_update(void) | |
| 1322 | +static void __init bxt_idle_state_table_update(void) | |
| 1215 | 1323 | { |
| 1216 | 1324 | unsigned long long msr; |
| 1217 | 1325 | unsigned int usec; |
| @@ -1258,7 +1366,7 @@ static void bxt_idle_state_table_update(void) · 40 unmodified lines | ||
| 1258 | 1366 | * On SKL-H (model 0x5e) disable C8 and C9 if: |
| 1259 | 1367 | * C10 is enabled and SGX disabled |
| 1260 | 1368 | */ |
| 1261 | −static void sklh_idle_state_table_update(void) | |
| 1369 | +static void __init sklh_idle_state_table_update(void) | |
| 1262 | 1370 | { |
| 1263 | 1371 | unsigned long long msr; |
| 1264 | 1372 | unsigned int eax, ebx, ecx, edx; |
| @@ -1294,16 +1402,28 @@ static void sklh_idle_state_table_update(void) · 29 unmodified lines | ||
| 1294 | 1402 | skl_cstates[5].flags |= CPUIDLE_FLAG_UNUSABLE; /* C8-SKL */ |
| 1295 | 1403 | skl_cstates[6].flags |= CPUIDLE_FLAG_UNUSABLE; /* C9-SKL */ |
| 1296 | 1404 | } |
| 1297 | −/* | |
| 1298 | − * intel_idle_state_table_update() | |
| 1299 | − * | |
| 1300 | − * Update the default state_table for this CPU-id | |
| 1301 | − */ | |
| 1302 | 1405 | |
| 1303 | −static void intel_idle_state_table_update(void) | |
| 1406 | +static bool __init intel_idle_verify_cstate(unsigned int mwait_hint) | |
| 1304 | 1407 | { |
| 1305 | − switch (boot_cpu_data.x86_model) { | |
| 1408 | + unsigned int mwait_cstate = MWAIT_HINT2CSTATE(mwait_hint) + 1; | |
| 1409 | + unsigned int num_substates = (mwait_substates >> mwait_cstate * 4) & | |
| 1410 | + MWAIT_SUBSTATE_MASK; | |
| 1411 | + | |
| 1412 | + /* Ignore the C-state if there are NO sub-states in CPUID for it. */ | |
| 1413 | + if (num_substates == 0) | |
| 1414 | + return false; | |
| 1415 | + | |
| 1416 | + if (mwait_cstate > 2 && !boot_cpu_has(X86_FEATURE_NONSTOP_TSC)) | |
| 1417 | + mark_tsc_unstable("TSC halts in idle states deeper than C2"); | |
| 1306 | 1418 | |
| 1419 | + return true; | |
| 1420 | +} | |
| 1421 | + | |
| 1422 | +static void __init intel_idle_init_cstates_icpu(struct cpuidle_driver *drv) | |
| 1423 | +{ | |
| 1424 | + int cstate; | |
| 1425 | + | |
| 1426 | + switch (boot_cpu_data.x86_model) { | |
| 1307 | 1427 | case INTEL_FAM6_IVYBRIDGE_X: |
| 1308 | 1428 | ivt_idle_state_table_update(); |
| 1309 | 1429 | break; |
| @@ -1315,62 +1435,36 @@ static void intel_idle_state_table_update(void) · 5 unmodified lines | ||
| 1315 | 1435 | sklh_idle_state_table_update(); |
| 1316 | 1436 | break; |
| 1317 | 1437 | } |
| 1318 | −} | |
| 1319 | − | |
| 1320 | −/* | |
| 1321 | − * intel_idle_cpuidle_driver_init() | |
| 1322 | − * allocate, initialize cpuidle_states | |
| 1323 | − */ | |
| 1324 | −static void __init intel_idle_cpuidle_driver_init(void) | |
| 1325 | −{ | |
| 1326 | − int cstate; | |
| 1327 | − struct cpuidle_driver *drv = &intel_idle_driver; | |
| 1328 | − | |
| 1329 | − intel_idle_state_table_update(); | |
| 1330 | − | |
| 1331 | − cpuidle_poll_state_init(drv); | |
| 1332 | − drv->state_count = 1; | |
| 1333 | 1438 | |
| 1334 | 1439 | for (cstate = 0; cstate < CPUIDLE_STATE_MAX; ++cstate) { |
| 1335 | − int num_substates, mwait_hint, mwait_cstate; | |
| 1440 | + unsigned int mwait_hint; | |
| 1336 | 1441 | |
| 1337 | − if ((cpuidle_state_table[cstate].enter == NULL) && | |
| 1338 | − (cpuidle_state_table[cstate].enter_s2idle == NULL)) | |
| 1442 | + if (intel_idle_max_cstate_reached(cstate)) | |
| 1339 | 1443 | break; |
| 1340 | 1444 | |
| 1341 | − if (cstate + 1 > max_cstate) { | |
| 1342 | − pr_info("max_cstate %d reached\n", max_cstate); | |
| 1445 | + if (!cpuidle_state_table[cstate].enter && | |
| 1446 | + !cpuidle_state_table[cstate].enter_s2idle) | |
| 1343 | 1447 | break; |
| 1344 | − } | |
| 1345 | − | |
| 1346 | − mwait_hint = flg2MWAIT(cpuidle_state_table[cstate].flags); | |
| 1347 | − mwait_cstate = MWAIT_HINT2CSTATE(mwait_hint); | |
| 1348 | − | |
| 1349 | − /* number of sub-states for this state in CPUID.MWAIT */ | |
| 1350 | − num_substates = (mwait_substates >> ((mwait_cstate + 1) * 4)) | |
| 1351 | − & MWAIT_SUBSTATE_MASK; | |
| 1352 | − | |
| 1353 | − /* if NO sub-states for this state in CPUID, skip it */ | |
| 1354 | − if (num_substates == 0) | |
| 1355 | − continue; | |
| 1356 | 1448 | |
| 1357 | − /* if state marked as disabled, skip it */ | |
| 1449 | + /* If marked as unusable, skip this state. */ | |
| 1358 | 1450 | if (cpuidle_state_table[cstate].flags & CPUIDLE_FLAG_UNUSABLE) { |
| 1359 | 1451 | pr_debug("state %s is disabled\n", |
| 1360 | 1452 | cpuidle_state_table[cstate].name); |
| 1361 | 1453 | continue; |
| 1362 | 1454 | } |
| 1363 | 1455 | |
| 1456 | + mwait_hint = flg2MWAIT(cpuidle_state_table[cstate].flags); | |
| 1457 | + if (!intel_idle_verify_cstate(mwait_hint)) | |
| 1458 | + continue; | |
| 1364 | 1459 | |
| 1365 | − if (((mwait_cstate + 1) > 2) && | |
| 1366 | − !boot_cpu_has(X86_FEATURE_NONSTOP_TSC)) | |
| 1367 | − mark_tsc_unstable("TSC halts in idle" | |
| 1368 | − " states deeper than C2"); | |
| 1460 | + /* Structure copy. */ | |
| 1461 | + drv->states[drv->state_count] = cpuidle_state_table[cstate]; | |
| 1369 | 1462 | |
| 1370 | − drv->states[drv->state_count] = /* structure copy */ | |
| 1371 | − cpuidle_state_table[cstate]; | |
| 1463 | + if (icpu->use_acpi && intel_idle_off_by_default(mwait_hint) && | |
| 1464 | + !(cpuidle_state_table[cstate].flags & CPUIDLE_FLAG_ALWAYS_ENABLE)) | |
| 1465 | + drv->states[drv->state_count].flags |= CPUIDLE_FLAG_OFF; | |
| 1372 | 1466 | |
| 1373 | − drv->state_count += 1; | |
| 1467 | + drv->state_count++; | |
| 1374 | 1468 | } |
| 1375 | 1469 | |
| 1376 | 1470 | if (icpu->byt_auto_demotion_disable_flag) { |
| @@ -1379,6 +1473,38 @@ static void __init intel_idle_cpuidle_driver_init(void) · 2 unmodified lines | ||
| 1379 | 1473 | } |
| 1380 | 1474 | } |
| 1381 | 1475 | |
| 1476 | +/* | |
| 1477 | + * intel_idle_cpuidle_driver_init() | |
| 1478 | + * allocate, initialize cpuidle_states | |
| 1479 | + */ | |
| 1480 | +static void __init intel_idle_cpuidle_driver_init(struct cpuidle_driver *drv) | |
| 1481 | +{ | |
| 1482 | + cpuidle_poll_state_init(drv); | |
| 1483 | + drv->state_count = 1; | |
| 1484 | + | |
| 1485 | + if (icpu) | |
| 1486 | + intel_idle_init_cstates_icpu(drv); | |
| 1487 | + else | |
| 1488 | + intel_idle_init_cstates_acpi(drv); | |
| 1489 | +} | |
| 1490 | + | |
| 1491 | +static void auto_demotion_disable(void) | |
| 1492 | +{ | |
| 1493 | + unsigned long long msr_bits; | |
| 1494 | + | |
| 1495 | + rdmsrl(MSR_PKG_CST_CONFIG_CONTROL, msr_bits); | |
| 1496 | + msr_bits &= ~(icpu->auto_demotion_disable_flags); | |
| 1497 | + wrmsrl(MSR_PKG_CST_CONFIG_CONTROL, msr_bits); | |
| 1498 | +} | |
| 1499 | + | |
| 1500 | +static void c1e_promotion_disable(void) | |
| 1501 | +{ | |
| 1502 | + unsigned long long msr_bits; | |
| 1503 | + | |
| 1504 | + rdmsrl(MSR_IA32_POWER_CTL, msr_bits); | |
| 1505 | + msr_bits &= ~0x2; | |
| 1506 | + wrmsrl(MSR_IA32_POWER_CTL, msr_bits); | |
| 1507 | +} | |
| 1382 | 1508 | |
| 1383 | 1509 | /* |
| 1384 | 1510 | * intel_idle_cpu_init() |
| @@ -1397,6 +1523,9 @@ static int intel_idle_cpu_init(unsigned int cpu) · 12 unmodified lines | ||
| 1397 | 1523 | return -EIO; |
| 1398 | 1524 | } |
| 1399 | 1525 | |
| 1526 | + if (!icpu) | |
| 1527 | + return 0; | |
| 1528 | + | |
| 1400 | 1529 | if (icpu->auto_demotion_disable_flags) |
| 1401 | 1530 | auto_demotion_disable(); |
| 1402 | 1531 | |
| @@ -1411,7 +1540,7 @@ static int intel_idle_cpu_online(unsigned int cpu) · 8 unmodified lines | ||
| 1411 | 1540 | struct cpuidle_device *dev; |
| 1412 | 1541 | |
| 1413 | 1542 | if (lapic_timer_reliable_states != LAPIC_TIMER_ALWAYS_RELIABLE) |
| 1414 | − __setup_broadcast_timer(true); | |
| 1543 | + tick_broadcast_enable(); | |
| 1415 | 1544 | |
| 1416 | 1545 | /* |
| 1417 | 1546 | * Some systems can hotplug a cpu at runtime after |
| @@ -1425,23 +1554,74 @@ static int intel_idle_cpu_online(unsigned int cpu) · 7 unmodified lines | ||
| 1425 | 1554 | return 0; |
| 1426 | 1555 | } |
| 1427 | 1556 | |
| 1557 | +/** | |
| 1558 | + * intel_idle_cpuidle_devices_uninit - Unregister all cpuidle devices. | |
| 1559 | + */ | |
| 1560 | +static void __init intel_idle_cpuidle_devices_uninit(void) | |
| 1561 | +{ | |
| 1562 | + int i; | |
| 1563 | + | |
| 1564 | + for_each_online_cpu(i) | |
| 1565 | + cpuidle_unregister_device(per_cpu_ptr(intel_idle_cpuidle_devices, i)); | |
| 1566 | +} | |
| 1567 | + | |
| 1428 | 1568 | static int __init intel_idle_init(void) |
| 1429 | 1569 | { |
| 1570 | + const struct x86_cpu_id *id; | |
| 1571 | + unsigned int eax, ebx, ecx; | |
| 1430 | 1572 | int retval; |
| 1431 | 1573 | |
| 1432 | 1574 | /* Do not load intel_idle at all for now if idle= is passed */ |
| 1433 | 1575 | if (boot_option_idle_override != IDLE_NO_OVERRIDE) |
| 1434 | 1576 | return -ENODEV; |
| 1435 | 1577 | |
| 1436 | − retval = intel_idle_probe(); | |
| 1437 | − if (retval) | |
| 1438 | − return retval; | |
| 1578 | + if (max_cstate == 0) { | |
| 1579 | + pr_debug("disabled\n"); | |
| 1580 | + return -EPERM; | |
| 1581 | + } | |
| 1582 | + | |
| 1583 | + id = x86_match_cpu(intel_idle_ids); | |
| 1584 | + if (id) { | |
| 1585 | + if (!boot_cpu_has(X86_FEATURE_MWAIT)) { | |
| 1586 | + pr_debug("Please enable MWAIT in BIOS SETUP\n"); | |
| 1587 | + return -ENODEV; | |
| 1588 | + } | |
| 1589 | + } else { | |
| 1590 | + id = x86_match_cpu(intel_mwait_ids); | |
| 1591 | + if (!id) | |
| 1592 | + return -ENODEV; | |
| 1593 | + } | |
| 1594 | + | |
| 1595 | + if (boot_cpu_data.cpuid_level < CPUID_MWAIT_LEAF) | |
| 1596 | + return -ENODEV; | |
| 1597 | + | |
| 1598 | + cpuid(CPUID_MWAIT_LEAF, &eax, &ebx, &ecx, &mwait_substates); | |
| 1599 | + | |
| 1600 | + if (!(ecx & CPUID5_ECX_EXTENSIONS_SUPPORTED) || | |
| 1601 | + !(ecx & CPUID5_ECX_INTERRUPT_BREAK) || | |
| 1602 | + !mwait_substates) | |
| 1603 | + return -ENODEV; | |
| 1604 | + | |
| 1605 | + pr_debug("MWAIT substates: 0x%x\n", mwait_substates); | |
| 1606 | + | |
| 1607 | + icpu = (const struct idle_cpu *)id->driver_data; | |
| 1608 | + if (icpu) { | |
| 1609 | + cpuidle_state_table = icpu->state_table; | |
| 1610 | + if (icpu->use_acpi) | |
| 1611 | + intel_idle_acpi_cst_extract(); | |
| 1612 | + } else if (!intel_idle_acpi_cst_extract()) { | |
| 1613 | + return -ENODEV; | |
| 1614 | + } | |
| 1615 | + | |
| 1616 | + pr_debug("v" INTEL_IDLE_VERSION " model 0x%X\n", | |
| 1617 | + boot_cpu_data.x86_model); | |
| 1439 | 1618 | |
| 1440 | 1619 | intel_idle_cpuidle_devices = alloc_percpu(struct cpuidle_device); |
| 1441 | − if (intel_idle_cpuidle_devices == NULL) | |
| 1620 | + if (!intel_idle_cpuidle_devices) | |
| 1442 | 1621 | return -ENOMEM; |
| 1443 | 1622 | |
| 1444 | − intel_idle_cpuidle_driver_init(); | |
| 1623 | + intel_idle_cpuidle_driver_init(&intel_idle_driver); | |
| 1624 | + | |
| 1445 | 1625 | retval = cpuidle_register_driver(&intel_idle_driver); |
| 1446 | 1626 | if (retval) { |
| 1447 | 1627 | struct cpuidle_driver *drv = cpuidle_get_driver(); |
| @@ -279,6 +279,21 @@ static inline bool invalid_phys_cpuid(phys_cpuid_t phys_id) · 278 unmodified lines | ||
| 279 | 279 | |
| 280 | 280 | /* Validate the processor object's proc_id */ |
| 281 | 281 | bool acpi_duplicate_processor_id(int proc_id); |
| 282 | +/* Processor _CTS control */ | |
| 283 | +struct acpi_processor_power; | |
| 284 | + | |
| 285 | +#ifdef CONFIG_ACPI_PROCESSOR_CSTATE | |
| 286 | +bool acpi_processor_claim_cst_control(void); | |
| 287 | +int acpi_processor_evaluate_cst(acpi_handle handle, u32 cpu, | |
| 288 | + struct acpi_processor_power *info); | |
| 289 | +#else | |
| 290 | +static inline bool acpi_processor_claim_cst_control(void) { return false; } | |
| 291 | +static inline int acpi_processor_evaluate_cst(acpi_handle handle, u32 cpu, | |
| 292 | + struct acpi_processor_power *info) | |
| 293 | +{ | |
| 294 | + return -ENODEV; | |
| 295 | +} | |
| 296 | +#endif | |
| 282 | 297 | |
| 283 | 298 | #ifdef CONFIG_ACPI_HOTPLUG_CPU |
| 284 | 299 | /* Arch dependent functions for cpu hotplug support */ |
| @@ -77,6 +77,7 @@ struct cpuidle_state { · 76 unmodified lines | ||
| 77 | 77 | #define CPUIDLE_FLAG_COUPLED BIT(1) /* state applies to multiple cpus */ |
| 78 | 78 | #define CPUIDLE_FLAG_TIMER_STOP BIT(2) /* timer is stopped on this state */ |
| 79 | 79 | #define CPUIDLE_FLAG_UNUSABLE BIT(3) /* avoid using this state */ |
| 80 | +#define CPUIDLE_FLAG_OFF BIT(4) /* disable this state by default */ | |
| 80 | 81 | |
| 81 | 82 | struct cpuidle_device_kobj; |
| 82 | 83 | struct cpuidle_state_kobj; |
| @@ -115,7 +116,6 @@ DECLARE_PER_CPU(struct cpuidle_device, cpuidle_dev); · 32 unmodified lines | ||
| 115 | 116 | struct cpuidle_driver { |
| 116 | 117 | const char *name; |
| 117 | 118 | struct module *owner; |
| 118 | − int refcnt; | |
| 119 | 119 | |
| 120 | 120 | /* used by the cpuidle framework to setup the broadcast timer */ |
| 121 | 121 | unsigned int bctimer:1; |
| @@ -147,8 +147,6 @@ extern u64 cpuidle_poll_time(struct cpuidle_driver *drv, · 25 unmodified lines | ||
| 147 | 147 | |
| 148 | 148 | extern int cpuidle_register_driver(struct cpuidle_driver *drv); |
| 149 | 149 | extern struct cpuidle_driver *cpuidle_get_driver(void); |
| 150 | −extern struct cpuidle_driver *cpuidle_driver_ref(void); | |
| 151 | −extern void cpuidle_driver_unref(void); | |
| 152 | 150 | extern void cpuidle_driver_state_disabled(struct cpuidle_driver *drv, int idx, |
| 153 | 151 | bool disable); |
| 154 | 152 | extern void cpuidle_unregister_driver(struct cpuidle_driver *drv); |
| @@ -186,8 +184,6 @@ static inline u64 cpuidle_poll_time(struct cpuidle_driver *drv, · 31 unmodified lines | ||
| 186 | 184 | static inline int cpuidle_register_driver(struct cpuidle_driver *drv) |
| 187 | 185 | {return -ENODEV; } |
| 188 | 186 | static inline struct cpuidle_driver *cpuidle_get_driver(void) {return NULL; } |
| 189 | −static inline struct cpuidle_driver *cpuidle_driver_ref(void) {return NULL; } | |
| 190 | −static inline void cpuidle_driver_unref(void) {} | |
| 191 | 187 | static inline void cpuidle_driver_state_disabled(struct cpuidle_driver *drv, |
| 192 | 188 | int idx, bool disable) { } |
| 193 | 189 | static inline void cpuidle_unregister_driver(struct cpuidle_driver *drv) { } |