History

Merge branch 'pm-cpuidle'

* pm-cpuidle: (27 commits) intel_idle: Clean up irtl_2_usec() intel_idle: Move 3 functions closer to their callers intel_idle: Annotate initialization code and data structures intel_idle: Move and clean up intel_idle_cpuidle_devices_uninit() intel_idle: Rearrange intel_idle_cpuidle_driver_init() intel_idle: Clean up NULL pointer check in intel_idle_init() intel_idle: Fold intel_idle_probe() into intel_idle_init() intel_idle: Eliminate __setup_broadcast_timer() cpuidle: fix cpuidle_find_deepest_state() kerneldoc warnings cpuidle: sysfs: fix warnings when compiling with W=1 cpuidle: coupled: fix warnings when compiling with W=1 Documentation: admin-guide: PM: Add intel_idle document cpuidle: arm: Enable compile testing for some of drivers cpuidle: Drop unused cpuidle_driver_ref/unref() functions intel_idle: Use ACPI _CST on server systems intel_idle: Add module parameter to prevent ACPI _CST from being used intel_idle: Allow ACPI _CST to be used for selected known processors cpuidle: Allow idle states to be disabled by default intel_idle: Use ACPI _CST for processor models without C-state tables intel_idle: Refactor intel_idle_cpuidle_driver_init() ...

Rafael J. Wysocki committed 6 years ago

Commit 5b82ba72 parents 3a83c8c 86e946617 files · +836 −382Browse files

Unified · Split

17 files
  1. Documentation/ABI/testing/sysfs-devices-system-cpu +6 −0
  2. Documentation/admin-guide/pm/cpuidle.rst +3 −0
  3. Documentation/admin-guide/pm/intel_idle.rst +246 −0
  4. Documentation/admin-guide/pm/working-state.rst +1 −0
  5. drivers/acpi/Kconfig +1 −0
  6. drivers/acpi/acpi_processor.c +182 −0
  7. drivers/acpi/processor_idle.c +15 −159
  8. drivers/cpuidle/Kconfig.arm +6 −6
  9. drivers/cpuidle/coupled.c +5 −4
  10. drivers/cpuidle/cpuidle-clps711x.c +1 −4
  11. drivers/cpuidle/cpuidle-kirkwood.c +1 −4
  12. drivers/cpuidle/cpuidle.c +8 −1
  13. drivers/cpuidle/driver.c +0 −46
  14. drivers/cpuidle/sysfs.c +14 −2
  15. drivers/idle/intel_idle.c +331 −151
  16. include/linux/acpi.h +15 −0
  17. include/linux/cpuidle.h +1 −5
Documentation/ABI/testing/sysfs-devices-system-cpu+6−0
@@ -196,6 +196,12 @@ Description: · 195 unmodified lines
196196 does not reflect it. Likewise, if one enables a deep state but a
197197 lighter state still is disabled, then this has no effect.
198198
199+What: /sys/devices/system/cpu/cpuX/cpuidle/stateN/default_status
200+Date: December 2019
201+KernelVersion: v5.6
202+Contact: Linux power management list <linux-pm@vger.kernel.org>
203+Description:
204+ (RO) The default status of this state, "enabled" or "disabled".
199205
200206 What: /sys/devices/system/cpu/cpuX/cpuidle/stateN/residency
201207 Date: March 2014
Documentation/admin-guide/pm/cpuidle.rst+3−0
@@ -506,6 +506,9 @@ object corresponding to it, as follows: · 505 unmodified lines
506506 ``disable``
507507 Whether or not this idle state is disabled.
508508
509+``default_status``
510+ The default status of this state, "enabled" or "disabled".
511+
509512 ``latency``
510513 Exit latency of the idle state in microseconds.
511514
Documentation/admin-guide/pm/intel_idle.rst+246−0
@@ -0,0 +1,246 @@
1+.. SPDX-License-Identifier: GPL-2.0
2+.. include:: <isonum.txt>
3+
4+==============================================
5+``intel_idle`` CPU Idle Time Management Driver
6+==============================================
7+
8+:Copyright: |copy| 2020 Intel Corporation
9+
10+:Author: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
11+
12+
13+General Information
14+===================
15+
16+``intel_idle`` is a part of the
17+:doc:`CPU idle time management subsystem <cpuidle>` in the Linux kernel
18+(``CPUIdle``). It is the default CPU idle time management driver for the
19+Nehalem and later generations of Intel processors, but the level of support for
20+a particular processor model in it depends on whether or not it recognizes that
21+processor model and may also depend on information coming from the platform
22+firmware. [To understand ``intel_idle`` it is necessary to know how ``CPUIdle``
23+works in general, so this is the time to get familiar with :doc:`cpuidle` if you
24+have not done that yet.]
25+
26+``intel_idle`` uses the ``MWAIT`` instruction to inform the processor that the
27+logical CPU executing it is idle and so it may be possible to put some of the
28+processor's functional blocks into low-power states. That instruction takes two
29+arguments (passed in the ``EAX`` and ``ECX`` registers of the target CPU), the
30+first of which, referred to as a *hint*, can be used by the processor to
31+determine what can be done (for details refer to Intel Software Developer’s
32+Manual [1]_). Accordingly, ``intel_idle`` refuses to work with processors in
33+which the support for the ``MWAIT`` instruction has been disabled (for example,
34+via the platform firmware configuration menu) or which do not support that
35+instruction at all.
36+
37+``intel_idle`` is not modular, so it cannot be unloaded, which means that the
38+only way to pass early-configuration-time parameters to it is via the kernel
39+command line.
40+
41+
42+.. _intel-idle-enumeration-of-states:
43+
44+Enumeration of Idle States
45+==========================
46+
47+Each ``MWAIT`` hint value is interpreted by the processor as a license to
48+reconfigure itself in a certain way in order to save energy. The processor
49+configurations (with reduced power draw) resulting from that are referred to
50+as C-states (in the ACPI terminology) or idle states. The list of meaningful
51+``MWAIT`` hint values and idle states (i.e. low-power configurations of the
52+processor) corresponding to them depends on the processor model and it may also
53+depend on the configuration of the platform.
54+
55+In order to create a list of available idle states required by the ``CPUIdle``
56+subsystem (see :ref:`idle-states-representation` in :doc:`cpuidle`),
57+``intel_idle`` can use two sources of information: static tables of idle states
58+for different processor models included in the driver itself and the ACPI tables
59+of the system. The former are always used if the processor model at hand is
60+recognized by ``intel_idle`` and the latter are used if that is required for
61+the given processor model (which is the case for all server processor models
62+recognized by ``intel_idle``) or if the processor model is not recognized.
63+
64+If the ACPI tables are going to be used for building the list of available idle
65+states, ``intel_idle`` first looks for a ``_CST`` object under one of the ACPI
66+objects corresponding to the CPUs in the system (refer to the ACPI specification
67+[2]_ for the description of ``_CST`` and its output package). Because the
68+``CPUIdle`` subsystem expects that the list of idle states supplied by the
69+driver will be suitable for all of the CPUs handled by it and ``intel_idle`` is
70+registered as the ``CPUIdle`` driver for all of the CPUs in the system, the
71+driver looks for the first ``_CST`` object returning at least one valid idle
72+state description and such that all of the idle states included in its return
73+package are of the FFH (Functional Fixed Hardware) type, which means that the
74+``MWAIT`` instruction is expected to be used to tell the processor that it can
75+enter one of them. The return package of that ``_CST`` is then assumed to be
76+applicable to all of the other CPUs in the system and the idle state
77+descriptions extracted from it are stored in a preliminary list of idle states
78+coming from the ACPI tables. [This step is skipped if ``intel_idle`` is
79+configured to ignore the ACPI tables; see `below <intel-idle-parameters_>`_.]
80+
81+Next, the first (index 0) entry in the list of available idle states is
82+initialized to represent a "polling idle state" (a pseudo-idle state in which
83+the target CPU continuously fetches and executes instructions), and the
84+subsequent (real) idle state entries are populated as follows.
85+
86+If the processor model at hand is recognized by ``intel_idle``, there is a
87+(static) table of idle state descriptions for it in the driver. In that case,
88+the "internal" table is the primary source of information on idle states and the
89+information from it is copied to the final list of available idle states. If
90+using the ACPI tables for the enumeration of idle states is not required
91+(depending on the processor model), all of the listed idle state are enabled by
92+default (so all of them will be taken into consideration by ``CPUIdle``
93+governors during CPU idle state selection). Otherwise, some of the listed idle
94+states may not be enabled by default if there are no matching entries in the
95+preliminary list of idle states coming from the ACPI tables. In that case user
96+space still can enable them later (on a per-CPU basis) with the help of
97+the ``disable`` idle state attribute in ``sysfs`` (see
98+:ref:`idle-states-representation` in :doc:`cpuidle`). This basically means that
99+the idle states "known" to the driver may not be enabled by default if they have
100+not been exposed by the platform firmware (through the ACPI tables).
101+
102+If the given processor model is not recognized by ``intel_idle``, but it
103+supports ``MWAIT``, the preliminary list of idle states coming from the ACPI
104+tables is used for building the final list that will be supplied to the
105+``CPUIdle`` core during driver registration. For each idle state in that list,
106+the description, ``MWAIT`` hint and exit latency are copied to the corresponding
107+entry in the final list of idle states. The name of the idle state represented
108+by it (to be returned by the ``name`` idle state attribute in ``sysfs``) is
109+"CX_ACPI", where X is the index of that idle state in the final list (note that
110+the minimum value of X is 1, because 0 is reserved for the "polling" state), and
111+its target residency is based on the exit latency value. Specifically, for
112+C1-type idle states the exit latency value is also used as the target residency
113+(for compatibility with the majority of the "internal" tables of idle states for
114+various processor models recognized by ``intel_idle``) and for the other idle
115+state types (C2 and C3) the target residency value is 3 times the exit latency
116+(again, that is because it reflects the target residency to exit latency ratio
117+in the majority of cases for the processor models recognized by ``intel_idle``).
118+All of the idle states in the final list are enabled by default in this case.
119+
120+
121+.. _intel-idle-initialization:
122+
123+Initialization
124+==============
125+
126+The initialization of ``intel_idle`` starts with checking if the kernel command
127+line options forbid the use of the ``MWAIT`` instruction. If that is the case,
128+an error code is returned right away.
129+
130+The next step is to check whether or not the processor model is known to the
131+driver, which determines the idle states enumeration method (see
132+`above <intel-idle-enumeration-of-states_>`_), and whether or not the processor
133+supports ``MWAIT`` (the initialization fails if that is not the case). Then,
134+the ``MWAIT`` support in the processor is enumerated through ``CPUID`` and the
135+driver initialization fails if the level of support is not as expected (for
136+example, if the total number of ``MWAIT`` substates returned is 0).
137+
138+Next, if the driver is not configured to ignore the ACPI tables (see
139+`below <intel-idle-parameters_>`_), the idle states information provided by the
140+platform firmware is extracted from them.
141+
142+Then, ``CPUIdle`` device objects are allocated for all CPUs and the list of
143+available idle states is created as explained
144+`above <intel-idle-enumeration-of-states_>`_.
145+
146+Finally, ``intel_idle`` is registered with the help of cpuidle_register_driver()
147+as the ``CPUIdle`` driver for all CPUs in the system and a CPU online callback
148+for configuring individual CPUs is registered via cpuhp_setup_state(), which
149+(among other things) causes the callback routine to be invoked for all of the
150+CPUs present in the system at that time (each CPU executes its own instance of
151+the callback routine). That routine registers a ``CPUIdle`` device for the CPU
152+running it (which enables the ``CPUIdle`` subsystem to operate that CPU) and
153+optionally performs some CPU-specific initialization actions that may be
154+required for the given processor model.
155+
156+
157+.. _intel-idle-parameters:
158+
159+Kernel Command Line Options and Module Parameters
160+=================================================
161+
162+The *x86* architecture support code recognizes three kernel command line
163+options related to CPU idle time management: ``idle=poll``, ``idle=halt``,
164+and ``idle=nomwait``. If any of them is present in the kernel command line, the
165+``MWAIT`` instruction is not allowed to be used, so the initialization of
166+``intel_idle`` will fail.
167+
168+Apart from that there are two module parameters recognized by ``intel_idle``
169+itself that can be set via the kernel command line (they cannot be updated via
170+sysfs, so that is the only way to change their values).
171+
172+The ``max_cstate`` parameter value is the maximum idle state index in the list
173+of idle states supplied to the ``CPUIdle`` core during the registration of the
174+driver. It is also the maximum number of regular (non-polling) idle states that
175+can be used by ``intel_idle``, so the enumeration of idle states is terminated
176+after finding that number of usable idle states (the other idle states that
177+potentially might have been used if ``max_cstate`` had been greater are not
178+taken into consideration at all). Setting ``max_cstate`` can prevent
179+``intel_idle`` from exposing idle states that are regarded as "too deep" for
180+some reason to the ``CPUIdle`` core, but it does so by making them effectively
181+invisible until the system is shut down and started again which may not always
182+be desirable. In practice, it is only really necessary to do that if the idle
183+states in question cannot be enabled during system startup, because in the
184+working state of the system the CPU power management quality of service (PM
185+QoS) feature can be used to prevent ``CPUIdle`` from touching those idle states
186+even if they have been enumerated (see :ref:`cpu-pm-qos` in :doc:`cpuidle`).
187+Setting ``max_cstate`` to 0 causes the ``intel_idle`` initialization to fail.
188+
189+The ``noacpi`` module parameter (which is recognized by ``intel_idle`` if the
190+kernel has been configured with ACPI support), can be set to make the driver
191+ignore the system's ACPI tables entirely (it is unset by default).
192+
193+
194+.. _intel-idle-core-and-package-idle-states:
195+
196+Core and Package Levels of Idle States
197+======================================
198+
199+Typically, in a processor supporting the ``MWAIT`` instruction there are (at
200+least) two levels of idle states (or C-states). One level, referred to as
201+"core C-states", covers individual cores in the processor, whereas the other
202+level, referred to as "package C-states", covers the entire processor package
203+and it may also involve other components of the system (GPUs, memory
204+controllers, I/O hubs etc.).
205+
206+Some of the ``MWAIT`` hint values allow the processor to use core C-states only
207+(most importantly, that is the case for the ``MWAIT`` hint value corresponding
208+to the ``C1`` idle state), but the majority of them give it a license to put
209+the target core (i.e. the core containing the logical CPU executing ``MWAIT``
210+with the given hint value) into a specific core C-state and then (if possible)
211+to enter a specific package C-state at the deeper level. For example, the
212+``MWAIT`` hint value representing the ``C3`` idle state allows the processor to
213+put the target core into the low-power state referred to as "core ``C3``" (or
214+``CC3``), which happens if all of the logical CPUs (SMT siblings) in that core
215+have executed ``MWAIT`` with the ``C3`` hint value (or with a hint value
216+representing a deeper idle state), and in addition to that (in the majority of
217+cases) it gives the processor a license to put the entire package (possibly
218+including some non-CPU components such as a GPU or a memory controller) into the
219+low-power state referred to as "package ``C3``" (or ``PC3``), which happens if
220+all of the cores have gone into the ``CC3`` state and (possibly) some additional
221+conditions are satisfied (for instance, if the GPU is covered by ``PC3``, it may
222+be required to be in a certain GPU-specific low-power state for ``PC3`` to be
223+reachable).
224+
225+As a rule, there is no simple way to make the processor use core C-states only
226+if the conditions for entering the corresponding package C-states are met, so
227+the logical CPU executing ``MWAIT`` with a hint value that is not core-level
228+only (like for ``C1``) must always assume that this may cause the processor to
229+enter a package C-state. [That is why the exit latency and target residency
230+values corresponding to the majority of ``MWAIT`` hint values in the "internal"
231+tables of idle states in ``intel_idle`` reflect the properties of package
232+C-states.] If using package C-states is not desirable at all, either
233+:ref:`PM QoS <cpu-pm-qos>` or the ``max_cstate`` module parameter of
234+``intel_idle`` described `above <intel-idle-parameters_>`_ must be used to
235+restrict the range of permissible idle states to the ones with core-level only
236+``MWAIT`` hint values (like ``C1``).
237+
238+
239+References
240+==========
241+
242+.. [1] *Intel® 64 and IA-32 Architectures Software Developer’s Manual Volume 2B*,
243+ https://www.intel.com/content/www/us/en/architecture-and-technology/64-ia-32-architectures-software-developer-vol-2b-manual.html
244+
245+.. [2] *Advanced Configuration and Power Interface (ACPI) Specification*,
246+ https://uefi.org/specifications
Documentation/admin-guide/pm/working-state.rst+1−0
@@ -8,6 +8,7 @@ Working-State Power Management · 7 unmodified lines
88 :maxdepth: 2
99
1010 cpuidle
11+ intel_idle
1112 cpufreq
1213 intel_pstate
1314 intel_epb
drivers/acpi/Kconfig+1−0
@@ -241,6 +241,7 @@ config ACPI_CPU_FREQ_PSS · 240 unmodified lines
241241
242242 config ACPI_PROCESSOR_CSTATE
243243 def_bool y
244+ depends on ACPI_PROCESSOR
244245 depends on IA64 || X86
245246
246247 config ACPI_PROCESSOR_IDLE
drivers/acpi/acpi_processor.c+182−0
@@ -705,3 +705,185 @@ void __init acpi_processor_init(void) · 704 unmodified lines
705705 acpi_scan_add_handler_with_hotplug(&processor_handler, "processor");
706706 acpi_scan_add_handler(&processor_container_handler);
707707 }
708+
709+#ifdef CONFIG_ACPI_PROCESSOR_CSTATE
710+/**
711+ * acpi_processor_claim_cst_control - Request _CST control from the platform.
712+ */
713+bool acpi_processor_claim_cst_control(void)
714+{
715+ static bool cst_control_claimed;
716+ acpi_status status;
717+
718+ if (!acpi_gbl_FADT.cst_control || cst_control_claimed)
719+ return true;
720+
721+ status = acpi_os_write_port(acpi_gbl_FADT.smi_command,
722+ acpi_gbl_FADT.cst_control, 8);
723+ if (ACPI_FAILURE(status)) {
724+ pr_warn("ACPI: Failed to claim processor _CST control\n");
725+ return false;
726+ }
727+
728+ cst_control_claimed = true;
729+ return true;
730+}
731+EXPORT_SYMBOL_GPL(acpi_processor_claim_cst_control);
732+
733+/**
734+ * acpi_processor_evaluate_cst - Evaluate the processor _CST control method.
735+ * @handle: ACPI handle of the processor object containing the _CST.
736+ * @cpu: The numeric ID of the target CPU.
737+ * @info: Object write the C-states information into.
738+ *
739+ * Extract the C-state information for the given CPU from the output of the _CST
740+ * control method under the corresponding ACPI processor object (or processor
741+ * device object) and populate @info with it.
742+ *
743+ * If any ACPI_ADR_SPACE_FIXED_HARDWARE C-states are found, invoke
744+ * acpi_processor_ffh_cstate_probe() to verify them and update the
745+ * cpu_cstate_entry data for @cpu.
746+ */
747+int acpi_processor_evaluate_cst(acpi_handle handle, u32 cpu,
748+ struct acpi_processor_power *info)
749+{
750+ struct acpi_buffer buffer = { ACPI_ALLOCATE_BUFFER, NULL };
751+ union acpi_object *cst;
752+ acpi_status status;
753+ u64 count;
754+ int last_index = 0;
755+ int i, ret = 0;
756+
757+ status = acpi_evaluate_object(handle, "_CST", NULL, &buffer);
758+ if (ACPI_FAILURE(status)) {
759+ acpi_handle_debug(handle, "No _CST\n");
760+ return -ENODEV;
761+ }
762+
763+ cst = buffer.pointer;
764+
765+ /* There must be at least 2 elements. */
766+ if (!cst || cst->type != ACPI_TYPE_PACKAGE || cst->package.count < 2) {
767+ acpi_handle_warn(handle, "Invalid _CST output\n");
768+ ret = -EFAULT;
769+ goto end;
770+ }
771+
772+ count = cst->package.elements[0].integer.value;
773+
774+ /* Validate the number of C-states. */
775+ if (count < 1 || count != cst->package.count - 1) {
776+ acpi_handle_warn(handle, "Inconsistent _CST data\n");
777+ ret = -EFAULT;
778+ goto end;
779+ }
780+
781+ for (i = 1; i <= count; i++) {
782+ union acpi_object *element;
783+ union acpi_object *obj;
784+ struct acpi_power_register *reg;
785+ struct acpi_processor_cx cx;
786+
787+ /*
788+ * If there is not enough space for all C-states, skip the
789+ * excess ones and log a warning.
790+ */
791+ if (last_index >= ACPI_PROCESSOR_MAX_POWER - 1) {
792+ acpi_handle_warn(handle,
793+ "No room for more idle states (limit: %d)\n",
794+ ACPI_PROCESSOR_MAX_POWER - 1);
795+ break;
796+ }
797+
798+ memset(&cx, 0, sizeof(cx));
799+
800+ element = &cst->package.elements[i];
801+ if (element->type != ACPI_TYPE_PACKAGE)
802+ continue;
803+
804+ if (element->package.count != 4)
805+ continue;
806+
807+ obj = &element->package.elements[0];
808+
809+ if (obj->type != ACPI_TYPE_BUFFER)
810+ continue;
811+
812+ reg = (struct acpi_power_register *)obj->buffer.pointer;
813+
814+ obj = &element->package.elements[1];
815+ if (obj->type != ACPI_TYPE_INTEGER)
816+ continue;
817+
818+ cx.type = obj->integer.value;
819+ /*
820+ * There are known cases in which the _CST output does not
821+ * contain C1, so if the type of the first state found is not
822+ * C1, leave an empty slot for C1 to be filled in later.
823+ */
824+ if (i == 1 && cx.type != ACPI_STATE_C1)
825+ last_index = 1;
826+
827+ cx.address = reg->address;
828+ cx.index = last_index + 1;
829+
830+ if (reg->space_id == ACPI_ADR_SPACE_FIXED_HARDWARE) {
831+ if (!acpi_processor_ffh_cstate_probe(cpu, &cx, reg)) {
832+ /*
833+ * In the majority of cases _CST describes C1 as
834+ * a FIXED_HARDWARE C-state, but if the command
835+ * line forbids using MWAIT, use CSTATE_HALT for
836+ * C1 regardless.
837+ */
838+ if (cx.type == ACPI_STATE_C1 &&
839+ boot_option_idle_override == IDLE_NOMWAIT) {
840+ cx.entry_method = ACPI_CSTATE_HALT;
841+ snprintf(cx.desc, ACPI_CX_DESC_LEN, "ACPI HLT");
842+ } else {
843+ cx.entry_method = ACPI_CSTATE_FFH;
844+ }
845+ } else if (cx.type == ACPI_STATE_C1) {
846+ /*
847+ * In the special case of C1, FIXED_HARDWARE can
848+ * be handled by executing the HLT instruction.
849+ */
850+ cx.entry_method = ACPI_CSTATE_HALT;
851+ snprintf(cx.desc, ACPI_CX_DESC_LEN, "ACPI HLT");
852+ } else {
853+ continue;
854+ }
855+ } else if (reg->space_id == ACPI_ADR_SPACE_SYSTEM_IO) {
856+ cx.entry_method = ACPI_CSTATE_SYSTEMIO;
857+ snprintf(cx.desc, ACPI_CX_DESC_LEN, "ACPI IOPORT 0x%x",
858+ cx.address);
859+ } else {
860+ continue;
861+ }
862+
863+ if (cx.type == ACPI_STATE_C1)
864+ cx.valid = 1;
865+
866+ obj = &element->package.elements[2];
867+ if (obj->type != ACPI_TYPE_INTEGER)
868+ continue;
869+
870+ cx.latency = obj->integer.value;
871+
872+ obj = &element->package.elements[3];
873+ if (obj->type != ACPI_TYPE_INTEGER)
874+ continue;
875+
876+ memcpy(&info->states[++last_index], &cx, sizeof(cx));
877+ }
878+
879+ acpi_handle_info(handle, "Found %d idle states\n", last_index);
880+
881+ info->count = last_index;
882+
883+ end:
884+ kfree(buffer.pointer);
885+
886+ return ret;
887+}
888+EXPORT_SYMBOL_GPL(acpi_processor_evaluate_cst);
889+#endif /* CONFIG_ACPI_PROCESSOR_CSTATE */
drivers/acpi/processor_idle.c+15−159
@@ -299,164 +299,24 @@ static int acpi_processor_get_power_info_default(struct acpi_processor *pr) · 298 unmodified lines
299299
300300 static int acpi_processor_get_power_info_cst(struct acpi_processor *pr)
301301 {
302− acpi_status status;
303− u64 count;
304− int current_count;
305− int i, ret = 0;
306− struct acpi_buffer buffer = { ACPI_ALLOCATE_BUFFER, NULL };
307− union acpi_object *cst;
302+ int ret;
308303
309304 if (nocst)
310305 return -ENODEV;
311306
312− current_count = 0;
313−
314− status = acpi_evaluate_object(pr->handle, "_CST", NULL, &buffer);
315− if (ACPI_FAILURE(status)) {
316− ACPI_DEBUG_PRINT((ACPI_DB_INFO, "No _CST, giving up\n"));
317− return -ENODEV;
318− }
319−
320− cst = buffer.pointer;
321−
322− /* There must be at least 2 elements */
323− if (!cst || (cst->type != ACPI_TYPE_PACKAGE) || cst->package.count < 2) {
324− pr_err("not enough elements in _CST\n");
325− ret = -EFAULT;
326− goto end;
327− }
328−
329− count = cst->package.elements[0].integer.value;
307+ ret = acpi_processor_evaluate_cst(pr->handle, pr->id, &pr->power);
308+ if (ret)
309+ return ret;
330310
331− /* Validate number of power states. */
332− if (count < 1 || count != cst->package.count - 1) {
333− pr_err("count given by _CST is not valid\n");
334− ret = -EFAULT;
335− goto end;
336− }
311+ /*
312+ * It is expected that there will be at least 2 states, C1 and
313+ * something else (C2 or C3), so fail if that is not the case.
314+ */
315+ if (pr->power.count < 2)
316+ return -EFAULT;
337317
338− /* Tell driver that at least _CST is supported. */
339318 pr->flags.has_cst = 1;
340−
341− for (i = 1; i <= count; i++) {
342− union acpi_object *element;
343− union acpi_object *obj;
344− struct acpi_power_register *reg;
345− struct acpi_processor_cx cx;
346−
347− memset(&cx, 0, sizeof(cx));
348−
349− element = &(cst->package.elements[i]);
350− if (element->type != ACPI_TYPE_PACKAGE)
351− continue;
352−
353− if (element->package.count != 4)
354− continue;
355−
356− obj = &(element->package.elements[0]);
357−
358− if (obj->type != ACPI_TYPE_BUFFER)
359− continue;
360−
361− reg = (struct acpi_power_register *)obj->buffer.pointer;
362−
363− if (reg->space_id != ACPI_ADR_SPACE_SYSTEM_IO &&
364− (reg->space_id != ACPI_ADR_SPACE_FIXED_HARDWARE))
365− continue;
366−
367− /* There should be an easy way to extract an integer... */
368− obj = &(element->package.elements[1]);
369− if (obj->type != ACPI_TYPE_INTEGER)
370− continue;
371−
372− cx.type = obj->integer.value;
373− /*
374− * Some buggy BIOSes won't list C1 in _CST -
375− * Let acpi_processor_get_power_info_default() handle them later
376− */
377− if (i == 1 && cx.type != ACPI_STATE_C1)
378− current_count++;
379−
380− cx.address = reg->address;
381− cx.index = current_count + 1;
382−
383− cx.entry_method = ACPI_CSTATE_SYSTEMIO;
384− if (reg->space_id == ACPI_ADR_SPACE_FIXED_HARDWARE) {
385− if (acpi_processor_ffh_cstate_probe
386− (pr->id, &cx, reg) == 0) {
387− cx.entry_method = ACPI_CSTATE_FFH;
388− } else if (cx.type == ACPI_STATE_C1) {
389− /*
390− * C1 is a special case where FIXED_HARDWARE
391− * can be handled in non-MWAIT way as well.
392− * In that case, save this _CST entry info.
393− * Otherwise, ignore this info and continue.
394− */
395− cx.entry_method = ACPI_CSTATE_HALT;
396− snprintf(cx.desc, ACPI_CX_DESC_LEN, "ACPI HLT");
397− } else {
398− continue;
399− }
400− if (cx.type == ACPI_STATE_C1 &&
401− (boot_option_idle_override == IDLE_NOMWAIT)) {
402− /*
403− * In most cases the C1 space_id obtained from
404− * _CST object is FIXED_HARDWARE access mode.
405− * But when the option of idle=halt is added,
406− * the entry_method type should be changed from
407− * CSTATE_FFH to CSTATE_HALT.
408− * When the option of idle=nomwait is added,
409− * the C1 entry_method type should be
410− * CSTATE_HALT.
411− */
412− cx.entry_method = ACPI_CSTATE_HALT;
413− snprintf(cx.desc, ACPI_CX_DESC_LEN, "ACPI HLT");
414− }
415− } else {
416− snprintf(cx.desc, ACPI_CX_DESC_LEN, "ACPI IOPORT 0x%x",
417− cx.address);
418− }
419−
420− if (cx.type == ACPI_STATE_C1) {
421− cx.valid = 1;
422− }
423−
424− obj = &(element->package.elements[2]);
425− if (obj->type != ACPI_TYPE_INTEGER)
426− continue;
427−
428− cx.latency = obj->integer.value;
429−
430− obj = &(element->package.elements[3]);
431− if (obj->type != ACPI_TYPE_INTEGER)
432− continue;
433−
434− current_count++;
435− memcpy(&(pr->power.states[current_count]), &cx, sizeof(cx));
436−
437− /*
438− * We support total ACPI_PROCESSOR_MAX_POWER - 1
439− * (From 1 through ACPI_PROCESSOR_MAX_POWER - 1)
440− */
441− if (current_count >= (ACPI_PROCESSOR_MAX_POWER - 1)) {
442− pr_warn("Limiting number of power states to max (%d)\n",
443− ACPI_PROCESSOR_MAX_POWER);
444− pr_warn("Please increase ACPI_PROCESSOR_MAX_POWER if needed.\n");
445− break;
446− }
447− }
448−
449− ACPI_DEBUG_PRINT((ACPI_DB_INFO, "Found %d power states\n",
450− current_count));
451−
452− /* Validate number of power states discovered */
453− if (current_count < 2)
454− ret = -EFAULT;
455−
456− end:
457− kfree(buffer.pointer);
458−
459− return ret;
319+ return 0;
460320 }
461321
462322 static void acpi_processor_power_verify_c3(struct acpi_processor *pr,
@@ -909,7 +769,6 @@ static int acpi_processor_setup_cstates(struct acpi_processor *pr) · 446 unmodified lines
909769
910770 static inline void acpi_processor_cstate_first_run_checks(void)
911771 {
912− acpi_status status;
913772 static int first_run;
914773
915774 if (first_run)
@@ -921,13 +780,10 @@ static inline void acpi_processor_cstate_first_run_checks(void) · 5 unmodified lines
921780 max_cstate);
922781 first_run++;
923782
924− if (acpi_gbl_FADT.cst_control && !nocst) {
925− status = acpi_os_write_port(acpi_gbl_FADT.smi_command,
926− acpi_gbl_FADT.cst_control, 8);
927− if (ACPI_FAILURE(status))
928− ACPI_EXCEPTION((AE_INFO, status,
929− "Notifying BIOS of _CST ability failed"));
930− }
783+ if (nocst)
784+ return;
785+
786+ acpi_processor_claim_cst_control();
931787 }
932788 #else
933789
drivers/cpuidle/Kconfig.arm+6−6
@@ -25,7 +25,7 @@ config ARM_PSCI_CPUIDLE · 24 unmodified lines
2525
2626 config ARM_BIG_LITTLE_CPUIDLE
2727 bool "Support for ARM big.LITTLE processors"
28− depends on ARCH_VEXPRESS_TC2_PM || ARCH_EXYNOS
28+ depends on ARCH_VEXPRESS_TC2_PM || ARCH_EXYNOS || COMPILE_TEST
2929 depends on MCPM && !ARM64
3030 select ARM_CPU_SUSPEND
3131 select CPU_IDLE_MULTIPLE_DRIVERS
@@ -51,13 +51,13 @@ config ARM_HIGHBANK_CPUIDLE · 19 unmodified lines
5151
5252 config ARM_KIRKWOOD_CPUIDLE
5353 bool "CPU Idle Driver for Marvell Kirkwood SoCs"
54− depends on MACH_KIRKWOOD && !ARM64
54+ depends on (MACH_KIRKWOOD || COMPILE_TEST) && !ARM64
5555 help
5656 This adds the CPU Idle driver for Marvell Kirkwood SoCs.
5757
5858 config ARM_ZYNQ_CPUIDLE
5959 bool "CPU Idle Driver for Xilinx Zynq processors"
60− depends on ARCH_ZYNQ && !ARM64
60+ depends on (ARCH_ZYNQ || COMPILE_TEST) && !ARM64
6161 help
6262 Select this to enable cpuidle on Xilinx Zynq processors.
6363
@@ -70,19 +70,19 @@ config ARM_U8500_CPUIDLE · 6 unmodified lines
7070 config ARM_AT91_CPUIDLE
7171 bool "Cpu Idle Driver for the AT91 processors"
7272 default y
73− depends on ARCH_AT91 && !ARM64
73+ depends on (ARCH_AT91 || COMPILE_TEST) && !ARM64
7474 help
7575 Select this to enable cpuidle for AT91 processors.
7676
7777 config ARM_EXYNOS_CPUIDLE
7878 bool "Cpu Idle Driver for the Exynos processors"
79− depends on ARCH_EXYNOS && !ARM64
79+ depends on (ARCH_EXYNOS || COMPILE_TEST) && !ARM64
8080 select ARCH_NEEDS_CPU_IDLE_COUPLED if SMP
8181 help
8282 Select this to enable cpuidle for Exynos processors.
8383
8484 config ARM_MVEBU_V7_CPUIDLE
8585 bool "CPU Idle Driver for mvebu v7 family processors"
86− depends on ARCH_MVEBU && !ARM64
86+ depends on (ARCH_MVEBU || COMPILE_TEST) && !ARM64
8787 help
8888 Select this to enable cpuidle on Armada 370, 38x and XP processors.
drivers/cpuidle/coupled.c+5−4
@@ -89,6 +89,7 @@ · 88 unmodified lines
8989 * @coupled_cpus: mask of cpus that are part of the coupled set
9090 * @requested_state: array of requested states for cpus in the coupled set
9191 * @ready_waiting_counts: combined count of cpus in ready or waiting loops
92+ * @abort_barrier: synchronisation point for abort cases
9293 * @online_count: count of cpus that are online
9394 * @refcnt: reference count of cpuidle devices that are using this struct
9495 * @prevent: flag to prevent coupled idle while a cpu is hotplugging
@@ -338,7 +339,7 @@ static void cpuidle_coupled_poke(int cpu) · 243 unmodified lines
338339
339340 /**
340341 * cpuidle_coupled_poke_others - wake up all other cpus that may be waiting
341− * @dev: struct cpuidle_device for this cpu
342+ * @this_cpu: target cpu
342343 * @coupled: the struct coupled that contains the current cpu
343344 *
344345 * Calls cpuidle_coupled_poke on all other online cpus.
@@ -355,7 +356,7 @@ static void cpuidle_coupled_poke_others(int this_cpu, · 10 unmodified lines
355356
356357 /**
357358 * cpuidle_coupled_set_waiting - mark this cpu as in the wait loop
358− * @dev: struct cpuidle_device for this cpu
359+ * @cpu: target cpu
359360 * @coupled: the struct coupled that contains the current cpu
360361 * @next_state: the index in drv->states of the requested state for this cpu
361362 *
@@ -376,7 +377,7 @@ static int cpuidle_coupled_set_waiting(int cpu, · 14 unmodified lines
376377
377378 /**
378379 * cpuidle_coupled_set_not_waiting - mark this cpu as leaving the wait loop
379− * @dev: struct cpuidle_device for this cpu
380+ * @cpu: target cpu
380381 * @coupled: the struct coupled that contains the current cpu
381382 *
382383 * Removes the requested idle state for the specified cpuidle device.
@@ -412,7 +413,7 @@ static void cpuidle_coupled_set_done(int cpu, struct cpuidle_coupled *coupled) · 29 unmodified lines
412413
413414 /**
414415 * cpuidle_coupled_clear_pokes - spin until the poke interrupt is processed
415− * @cpu - this cpu
416+ * @cpu: this cpu
416417 *
417418 * Turns on interrupts and spins until any outstanding poke interrupts have
418419 * been processed and the poke bit has been cleared.
drivers/cpuidle/cpuidle-clps711x.c+1−4
@@ -37,10 +37,7 @@ static struct cpuidle_driver clps711x_idle_driver = { · 36 unmodified lines
3737
3838 static int __init clps711x_cpuidle_probe(struct platform_device *pdev)
3939 {
40− struct resource *res;
41−
42− res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
43− clps711x_halt = devm_ioremap_resource(&pdev->dev, res);
40+ clps711x_halt = devm_platform_ioremap_resource(pdev, 0);
4441 if (IS_ERR(clps711x_halt))
4542 return PTR_ERR(clps711x_halt);
4643
drivers/cpuidle/cpuidle-kirkwood.c+1−4
@@ -55,10 +55,7 @@ static struct cpuidle_driver kirkwood_idle_driver = { · 54 unmodified lines
5555 /* Initialize CPU idle by registering the idle states */
5656 static int kirkwood_cpuidle_probe(struct platform_device *pdev)
5757 {
58− struct resource *res;
59−
60− res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
61− ddr_operation_base = devm_ioremap_resource(&pdev->dev, res);
58+ ddr_operation_base = devm_platform_ioremap_resource(pdev, 0);
6259 if (IS_ERR(ddr_operation_base))
6360 return PTR_ERR(ddr_operation_base);
6461
drivers/cpuidle/cpuidle.c+8−1
@@ -121,6 +121,9 @@ void cpuidle_use_deepest_state(u64 latency_limit_ns) · 120 unmodified lines
121121 * cpuidle_find_deepest_state - Find the deepest available idle state.
122122 * @drv: cpuidle driver for the given CPU.
123123 * @dev: cpuidle device for the given CPU.
124+ * @latency_limit_ns: Idle state exit latency limit
125+ *
126+ * Return: the index of the deepest available idle state.
124127 */
125128 int cpuidle_find_deepest_state(struct cpuidle_driver *drv,
126129 struct cpuidle_device *dev,
@@ -572,10 +575,14 @@ static int __cpuidle_register_device(struct cpuidle_device *dev) · 445 unmodified lines
572575 if (!try_module_get(drv->owner))
573576 return -EINVAL;
574577
575− for (i = 0; i < drv->state_count; i++)
578+ for (i = 0; i < drv->state_count; i++) {
576579 if (drv->states[i].flags & CPUIDLE_FLAG_UNUSABLE)
577580 dev->states_usage[i].disable |= CPUIDLE_STATE_DISABLED_BY_DRIVER;
578581
582+ if (drv->states[i].flags & CPUIDLE_FLAG_OFF)
583+ dev->states_usage[i].disable |= CPUIDLE_STATE_DISABLED_BY_USER;
584+ }
585+
579586 per_cpu(cpuidle_devices, dev->cpu) = dev;
580587 list_add(&dev->device_list, &cpuidle_detected_devices);
581588
drivers/cpuidle/driver.c+0−46
@@ -155,8 +155,6 @@ static void __cpuidle_driver_init(struct cpuidle_driver *drv) · 154 unmodified lines
155155 {
156156 int i;
157157
158− drv->refcnt = 0;
159−
160158 /*
161159 * Use all possible CPUs as the default, because if the kernel boots
162160 * with some CPUs offline and then we online one of them, the CPU
@@ -240,9 +238,6 @@ static int __cpuidle_register_driver(struct cpuidle_driver *drv) · 77 unmodified lines
240238 */
241239 static void __cpuidle_unregister_driver(struct cpuidle_driver *drv)
242240 {
243− if (WARN_ON(drv->refcnt > 0))
244− return;
245−
246241 if (drv->bctimer) {
247242 drv->bctimer = 0;
248243 on_each_cpu_mask(drv->cpumask, cpuidle_setup_broadcast_timer,
@@ -349,47 +344,6 @@ struct cpuidle_driver *cpuidle_get_cpu_driver(struct cpuidle_device *dev) · 100 unmodified lines
349344 }
350345 EXPORT_SYMBOL_GPL(cpuidle_get_cpu_driver);
351346
352−/**
353− * cpuidle_driver_ref - get a reference to the driver.
354− *
355− * Increment the reference counter of the cpuidle driver associated with
356− * the current CPU.
357− *
358− * Returns a pointer to the driver, or NULL if the current CPU has no driver.
359− */
360−struct cpuidle_driver *cpuidle_driver_ref(void)
361−{
362− struct cpuidle_driver *drv;
363−
364− spin_lock(&cpuidle_driver_lock);
365−
366− drv = cpuidle_get_driver();
367− if (drv)
368− drv->refcnt++;
369−
370− spin_unlock(&cpuidle_driver_lock);
371− return drv;
372−}
373−
374−/**
375− * cpuidle_driver_unref - puts down the refcount for the driver
376− *
377− * Decrement the reference counter of the cpuidle driver associated with
378− * the current CPU.
379− */
380−void cpuidle_driver_unref(void)
381−{
382− struct cpuidle_driver *drv;
383−
384− spin_lock(&cpuidle_driver_lock);
385−
386− drv = cpuidle_get_driver();
387− if (drv && !WARN_ON(drv->refcnt <= 0))
388− drv->refcnt--;
389−
390− spin_unlock(&cpuidle_driver_lock);
391−}
392−
393347 /**
394348 * cpuidle_driver_state_disabled - Disable or enable an idle state
395349 * @drv: cpuidle driver owning the state
drivers/cpuidle/sysfs.c+14−2
@@ -142,6 +142,7 @@ static struct attribute_group cpuidle_attr_group = { · 141 unmodified lines
142142
143143 /**
144144 * cpuidle_add_interface - add CPU global sysfs attributes
145+ * @dev: the target device
145146 */
146147 int cpuidle_add_interface(struct device *dev)
147148 {
@@ -153,6 +154,7 @@ int cpuidle_add_interface(struct device *dev) · 5 unmodified lines
153154
154155 /**
155156 * cpuidle_remove_interface - remove CPU global sysfs attributes
157+ * @dev: the target device
156158 */
157159 void cpuidle_remove_interface(struct device *dev)
158160 {
@@ -327,6 +329,14 @@ static ssize_t store_state_disable(struct cpuidle_state *state, · 168 unmodified lines
327329 return size;
328330 }
329331
332+static ssize_t show_state_default_status(struct cpuidle_state *state,
333+ struct cpuidle_state_usage *state_usage,
334+ char *buf)
335+{
336+ return sprintf(buf, "%s\n",
337+ state->flags & CPUIDLE_FLAG_OFF ? "disabled" : "enabled");
338+}
339+
330340 define_one_state_ro(name, show_state_name);
331341 define_one_state_ro(desc, show_state_desc);
332342 define_one_state_ro(latency, show_state_exit_latency);
@@ -337,6 +347,7 @@ define_one_state_ro(time, show_state_time); · 4 unmodified lines
337347 define_one_state_rw(disable, show_state_disable, store_state_disable);
338348 define_one_state_ro(above, show_state_above);
339349 define_one_state_ro(below, show_state_below);
350+define_one_state_ro(default_status, show_state_default_status);
340351
341352 static struct attribute *cpuidle_state_default_attrs[] = {
342353 &attr_name.attr,
@@ -349,6 +360,7 @@ static struct attribute *cpuidle_state_default_attrs[] = { · 6 unmodified lines
349360 &attr_disable.attr,
350361 &attr_above.attr,
351362 &attr_below.attr,
363+ &attr_default_status.attr,
352364 NULL
353365 };
354366
@@ -615,7 +627,7 @@ static struct kobj_type ktype_driver_cpuidle = { · 260 unmodified lines
615627
616628 /**
617629 * cpuidle_add_driver_sysfs - adds the driver name sysfs attribute
618− * @device: the target device
630+ * @dev: the target device
619631 */
620632 static int cpuidle_add_driver_sysfs(struct cpuidle_device *dev)
621633 {
@@ -646,7 +658,7 @@ static int cpuidle_add_driver_sysfs(struct cpuidle_device *dev) · 24 unmodified lines
646658
647659 /**
648660 * cpuidle_remove_driver_sysfs - removes the driver name sysfs attribute
649− * @device: the target device
661+ * @dev: the target device
650662 */
651663 static void cpuidle_remove_driver_sysfs(struct cpuidle_device *dev)
652664 {
drivers/idle/intel_idle.c+331−151
@@ -41,6 +41,7 @@ · 40 unmodified lines
4141
4242 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
4343
44+#include <linux/acpi.h>
4445 #include <linux/kernel.h>
4546 #include <linux/cpuidle.h>
4647 #include <linux/tick.h>
@@ -79,6 +80,7 @@ struct idle_cpu { · 32 unmodified lines
7980 unsigned long auto_demotion_disable_flags;
8081 bool byt_auto_demotion_disable_flag;
8182 bool disable_promotion_to_c1e;
83+ bool use_acpi;
8284 };
8385
8486 static const struct idle_cpu *icpu;
@@ -89,6 +91,11 @@ static void intel_idle_s2idle(struct cpuidle_device *dev, · 4 unmodified lines
8991 struct cpuidle_driver *drv, int index);
9092 static struct cpuidle_state *cpuidle_state_table;
9193
94+/*
95+ * Enable this state by default even if the ACPI _CST does not list it.
96+ */
97+#define CPUIDLE_FLAG_ALWAYS_ENABLE BIT(15)
98+
9299 /*
93100 * Set this flag for states where the HW flushes the TLB for us
94101 * and so we don't need cross-calls to keep it consistent.
@@ -124,7 +131,7 @@ static struct cpuidle_state nehalem_cstates[] = { · 29 unmodified lines
124131 {
125132 .name = "C1E",
126133 .desc = "MWAIT 0x01",
127− .flags = MWAIT2flg(0x01),
134+ .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE,
128135 .exit_latency = 10,
129136 .target_residency = 20,
130137 .enter = &intel_idle,
@@ -161,7 +168,7 @@ static struct cpuidle_state snb_cstates[] = { · 30 unmodified lines
161168 {
162169 .name = "C1E",
163170 .desc = "MWAIT 0x01",
164− .flags = MWAIT2flg(0x01),
171+ .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE,
165172 .exit_latency = 10,
166173 .target_residency = 20,
167174 .enter = &intel_idle,
@@ -296,7 +303,7 @@ static struct cpuidle_state ivb_cstates[] = { · 128 unmodified lines
296303 {
297304 .name = "C1E",
298305 .desc = "MWAIT 0x01",
299− .flags = MWAIT2flg(0x01),
306+ .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE,
300307 .exit_latency = 10,
301308 .target_residency = 20,
302309 .enter = &intel_idle,
@@ -341,7 +348,7 @@ static struct cpuidle_state ivt_cstates[] = { · 38 unmodified lines
341348 {
342349 .name = "C1E",
343350 .desc = "MWAIT 0x01",
344− .flags = MWAIT2flg(0x01),
351+ .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE,
345352 .exit_latency = 10,
346353 .target_residency = 80,
347354 .enter = &intel_idle,
@@ -378,7 +385,7 @@ static struct cpuidle_state ivt_cstates_4s[] = { · 30 unmodified lines
378385 {
379386 .name = "C1E",
380387 .desc = "MWAIT 0x01",
381− .flags = MWAIT2flg(0x01),
388+ .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE,
382389 .exit_latency = 10,
383390 .target_residency = 250,
384391 .enter = &intel_idle,
@@ -415,7 +422,7 @@ static struct cpuidle_state ivt_cstates_8s[] = { · 30 unmodified lines
415422 {
416423 .name = "C1E",
417424 .desc = "MWAIT 0x01",
418− .flags = MWAIT2flg(0x01),
425+ .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE,
419426 .exit_latency = 10,
420427 .target_residency = 500,
421428 .enter = &intel_idle,
@@ -452,7 +459,7 @@ static struct cpuidle_state hsw_cstates[] = { · 30 unmodified lines
452459 {
453460 .name = "C1E",
454461 .desc = "MWAIT 0x01",
455− .flags = MWAIT2flg(0x01),
462+ .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE,
456463 .exit_latency = 10,
457464 .target_residency = 20,
458465 .enter = &intel_idle,
@@ -520,7 +527,7 @@ static struct cpuidle_state bdw_cstates[] = { · 61 unmodified lines
520527 {
521528 .name = "C1E",
522529 .desc = "MWAIT 0x01",
523− .flags = MWAIT2flg(0x01),
530+ .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE,
524531 .exit_latency = 10,
525532 .target_residency = 20,
526533 .enter = &intel_idle,
@@ -589,7 +596,7 @@ static struct cpuidle_state skl_cstates[] = { · 62 unmodified lines
589596 {
590597 .name = "C1E",
591598 .desc = "MWAIT 0x01",
592− .flags = MWAIT2flg(0x01),
599+ .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE,
593600 .exit_latency = 10,
594601 .target_residency = 20,
595602 .enter = &intel_idle,
@@ -658,7 +665,7 @@ static struct cpuidle_state skx_cstates[] = { · 62 unmodified lines
658665 {
659666 .name = "C1E",
660667 .desc = "MWAIT 0x01",
661− .flags = MWAIT2flg(0x01),
668+ .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE,
662669 .exit_latency = 10,
663670 .target_residency = 20,
664671 .enter = &intel_idle,
@@ -808,7 +815,7 @@ static struct cpuidle_state bxt_cstates[] = { · 143 unmodified lines
808815 {
809816 .name = "C1E",
810817 .desc = "MWAIT 0x01",
811− .flags = MWAIT2flg(0x01),
818+ .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE,
812819 .exit_latency = 10,
813820 .target_residency = 20,
814821 .enter = &intel_idle,
@@ -869,7 +876,7 @@ static struct cpuidle_state dnv_cstates[] = { · 54 unmodified lines
869876 {
870877 .name = "C1E",
871878 .desc = "MWAIT 0x01",
872− .flags = MWAIT2flg(0x01),
879+ .flags = MWAIT2flg(0x01) | CPUIDLE_FLAG_ALWAYS_ENABLE,
873880 .exit_latency = 10,
874881 .target_residency = 20,
875882 .enter = &intel_idle,
@@ -944,37 +951,19 @@ static void intel_idle_s2idle(struct cpuidle_device *dev, · 68 unmodified lines
944951 mwait_idle_with_hints(eax, ecx);
945952 }
946953
947−static void __setup_broadcast_timer(bool on)
948−{
949− if (on)
950− tick_broadcast_enable();
951− else
952− tick_broadcast_disable();
953−}
954−
955−static void auto_demotion_disable(void)
956−{
957− unsigned long long msr_bits;
958−
959− rdmsrl(MSR_PKG_CST_CONFIG_CONTROL, msr_bits);
960− msr_bits &= ~(icpu->auto_demotion_disable_flags);
961− wrmsrl(MSR_PKG_CST_CONFIG_CONTROL, msr_bits);
962−}
963−static void c1e_promotion_disable(void)
964−{
965− unsigned long long msr_bits;
966−
967− rdmsrl(MSR_IA32_POWER_CTL, msr_bits);
968− msr_bits &= ~0x2;
969− wrmsrl(MSR_IA32_POWER_CTL, msr_bits);
970−}
971−
972954 static const struct idle_cpu idle_cpu_nehalem = {
973955 .state_table = nehalem_cstates,
974956 .auto_demotion_disable_flags = NHM_C1_AUTO_DEMOTE | NHM_C3_AUTO_DEMOTE,
975957 .disable_promotion_to_c1e = true,
976958 };
977959
960+static const struct idle_cpu idle_cpu_nhx = {
961+ .state_table = nehalem_cstates,
962+ .auto_demotion_disable_flags = NHM_C1_AUTO_DEMOTE | NHM_C3_AUTO_DEMOTE,
963+ .disable_promotion_to_c1e = true,
964+ .use_acpi = true,
965+};
966+
978967 static const struct idle_cpu idle_cpu_atom = {
979968 .state_table = atom_cstates,
980969 };
@@ -993,6 +982,12 @@ static const struct idle_cpu idle_cpu_snb = { · 12 unmodified lines
993982 .disable_promotion_to_c1e = true,
994983 };
995984
985+static const struct idle_cpu idle_cpu_snx = {
986+ .state_table = snb_cstates,
987+ .disable_promotion_to_c1e = true,
988+ .use_acpi = true,
989+};
990+
996991 static const struct idle_cpu idle_cpu_byt = {
997992 .state_table = byt_cstates,
998993 .disable_promotion_to_c1e = true,
@@ -1013,6 +1008,7 @@ static const struct idle_cpu idle_cpu_ivb = { · 14 unmodified lines
10131008 static const struct idle_cpu idle_cpu_ivt = {
10141009 .state_table = ivt_cstates,
10151010 .disable_promotion_to_c1e = true,
1011+ .use_acpi = true,
10161012 };
10171013
10181014 static const struct idle_cpu idle_cpu_hsw = {
@@ -1020,11 +1016,23 @@ static const struct idle_cpu idle_cpu_hsw = { · 1 unmodified line
10201016 .disable_promotion_to_c1e = true,
10211017 };
10221018
1019+static const struct idle_cpu idle_cpu_hsx = {
1020+ .state_table = hsw_cstates,
1021+ .disable_promotion_to_c1e = true,
1022+ .use_acpi = true,
1023+};
1024+
10231025 static const struct idle_cpu idle_cpu_bdw = {
10241026 .state_table = bdw_cstates,
10251027 .disable_promotion_to_c1e = true,
10261028 };
10271029
1030+static const struct idle_cpu idle_cpu_bdx = {
1031+ .state_table = bdw_cstates,
1032+ .disable_promotion_to_c1e = true,
1033+ .use_acpi = true,
1034+};
1035+
10281036 static const struct idle_cpu idle_cpu_skl = {
10291037 .state_table = skl_cstates,
10301038 .disable_promotion_to_c1e = true,
@@ -1033,15 +1041,18 @@ static const struct idle_cpu idle_cpu_skl = { · 2 unmodified lines
10331041 static const struct idle_cpu idle_cpu_skx = {
10341042 .state_table = skx_cstates,
10351043 .disable_promotion_to_c1e = true,
1044+ .use_acpi = true,
10361045 };
10371046
10381047 static const struct idle_cpu idle_cpu_avn = {
10391048 .state_table = avn_cstates,
10401049 .disable_promotion_to_c1e = true,
1050+ .use_acpi = true,
10411051 };
10421052
10431053 static const struct idle_cpu idle_cpu_knl = {
10441054 .state_table = knl_cstates,
1055+ .use_acpi = true,
10451056 };
10461057
10471058 static const struct idle_cpu idle_cpu_bxt = {
@@ -1052,20 +1063,21 @@ static const struct idle_cpu idle_cpu_bxt = { · 4 unmodified lines
10521063 static const struct idle_cpu idle_cpu_dnv = {
10531064 .state_table = dnv_cstates,
10541065 .disable_promotion_to_c1e = true,
1066+ .use_acpi = true,
10551067 };
10561068
10571069 static const struct x86_cpu_id intel_idle_ids[] __initconst = {
1058− INTEL_CPU_FAM6(NEHALEM_EP, idle_cpu_nehalem),
1070+ INTEL_CPU_FAM6(NEHALEM_EP, idle_cpu_nhx),
10591071 INTEL_CPU_FAM6(NEHALEM, idle_cpu_nehalem),
10601072 INTEL_CPU_FAM6(NEHALEM_G, idle_cpu_nehalem),
10611073 INTEL_CPU_FAM6(WESTMERE, idle_cpu_nehalem),
1062− INTEL_CPU_FAM6(WESTMERE_EP, idle_cpu_nehalem),
1063− INTEL_CPU_FAM6(NEHALEM_EX, idle_cpu_nehalem),
1074+ INTEL_CPU_FAM6(WESTMERE_EP, idle_cpu_nhx),
1075+ INTEL_CPU_FAM6(NEHALEM_EX, idle_cpu_nhx),
10641076 INTEL_CPU_FAM6(ATOM_BONNELL, idle_cpu_atom),
10651077 INTEL_CPU_FAM6(ATOM_BONNELL_MID, idle_cpu_lincroft),
1066− INTEL_CPU_FAM6(WESTMERE_EX, idle_cpu_nehalem),
1078+ INTEL_CPU_FAM6(WESTMERE_EX, idle_cpu_nhx),
10671079 INTEL_CPU_FAM6(SANDYBRIDGE, idle_cpu_snb),
1068− INTEL_CPU_FAM6(SANDYBRIDGE_X, idle_cpu_snb),
1080+ INTEL_CPU_FAM6(SANDYBRIDGE_X, idle_cpu_snx),
10691081 INTEL_CPU_FAM6(ATOM_SALTWELL, idle_cpu_atom),
10701082 INTEL_CPU_FAM6(ATOM_SILVERMONT, idle_cpu_byt),
10711083 INTEL_CPU_FAM6(ATOM_SILVERMONT_MID, idle_cpu_tangier),
@@ -1073,14 +1085,14 @@ static const struct x86_cpu_id intel_idle_ids[] __initconst = { · 1 unmodified line
10731085 INTEL_CPU_FAM6(IVYBRIDGE, idle_cpu_ivb),
10741086 INTEL_CPU_FAM6(IVYBRIDGE_X, idle_cpu_ivt),
10751087 INTEL_CPU_FAM6(HASWELL, idle_cpu_hsw),
1076− INTEL_CPU_FAM6(HASWELL_X, idle_cpu_hsw),
1088+ INTEL_CPU_FAM6(HASWELL_X, idle_cpu_hsx),
10771089 INTEL_CPU_FAM6(HASWELL_L, idle_cpu_hsw),
10781090 INTEL_CPU_FAM6(HASWELL_G, idle_cpu_hsw),
10791091 INTEL_CPU_FAM6(ATOM_SILVERMONT_D, idle_cpu_avn),
10801092 INTEL_CPU_FAM6(BROADWELL, idle_cpu_bdw),
10811093 INTEL_CPU_FAM6(BROADWELL_G, idle_cpu_bdw),
1082− INTEL_CPU_FAM6(BROADWELL_X, idle_cpu_bdw),
1083− INTEL_CPU_FAM6(BROADWELL_D, idle_cpu_bdw),
1094+ INTEL_CPU_FAM6(BROADWELL_X, idle_cpu_bdx),
1095+ INTEL_CPU_FAM6(BROADWELL_D, idle_cpu_bdx),
10841096 INTEL_CPU_FAM6(SKYLAKE_L, idle_cpu_skl),
10851097 INTEL_CPU_FAM6(SKYLAKE, idle_cpu_skl),
10861098 INTEL_CPU_FAM6(KABYLAKE_L, idle_cpu_skl),
@@ -1095,76 +1107,169 @@ static const struct x86_cpu_id intel_idle_ids[] __initconst = { · 8 unmodified lines
10951107 {}
10961108 };
10971109
1098−/*
1099− * intel_idle_probe()
1110+#define INTEL_CPU_FAM6_MWAIT \
1111+ { X86_VENDOR_INTEL, 6, X86_MODEL_ANY, X86_FEATURE_MWAIT, 0 }
1112+
1113+static const struct x86_cpu_id intel_mwait_ids[] __initconst = {
1114+ INTEL_CPU_FAM6_MWAIT,
1115+ {}
1116+};
1117+
1118+static bool __init intel_idle_max_cstate_reached(int cstate)
1119+{
1120+ if (cstate + 1 > max_cstate) {
1121+ pr_info("max_cstate %d reached\n", max_cstate);
1122+ return true;
1123+ }
1124+ return false;
1125+}
1126+
1127+#ifdef CONFIG_ACPI_PROCESSOR_CSTATE
1128+#include <acpi/processor.h>
1129+
1130+static bool no_acpi __read_mostly;
1131+module_param(no_acpi, bool, 0444);
1132+MODULE_PARM_DESC(no_acpi, "Do not use ACPI _CST for building the idle states list");
1133+
1134+static struct acpi_processor_power acpi_state_table __initdata;
1135+
1136+/**
1137+ * intel_idle_cst_usable - Check if the _CST information can be used.
1138+ *
1139+ * Check if all of the C-states listed by _CST in the max_cstate range are
1140+ * ACPI_CSTATE_FFH, which means that they should be entered via MWAIT.
11001141 */
1101−static int __init intel_idle_probe(void)
1142+static bool __init intel_idle_cst_usable(void)
11021143 {
1103− unsigned int eax, ebx, ecx;
1104− const struct x86_cpu_id *id;
1144+ int cstate, limit;
11051145
1106− if (max_cstate == 0) {
1107− pr_debug("disabled\n");
1108− return -EPERM;
1109− }
1146+ limit = min_t(int, min_t(int, CPUIDLE_STATE_MAX, max_cstate + 1),
1147+ acpi_state_table.count);
11101148
1111− id = x86_match_cpu(intel_idle_ids);
1112− if (!id) {
1113− if (boot_cpu_data.x86_vendor == X86_VENDOR_INTEL &&
1114− boot_cpu_data.x86 == 6)
1115− pr_debug("does not run on family %d model %d\n",
1116− boot_cpu_data.x86, boot_cpu_data.x86_model);
1117− return -ENODEV;
1149+ for (cstate = 1; cstate < limit; cstate++) {
1150+ struct acpi_processor_cx *cx = &acpi_state_table.states[cstate];
1151+
1152+ if (cx->entry_method != ACPI_CSTATE_FFH)
1153+ return false;
11181154 }
11191155
1120− if (!boot_cpu_has(X86_FEATURE_MWAIT)) {
1121− pr_debug("Please enable MWAIT in BIOS SETUP\n");
1122− return -ENODEV;
1156+ return true;
1157+}
1158+
1159+static bool __init intel_idle_acpi_cst_extract(void)
1160+{
1161+ unsigned int cpu;
1162+
1163+ if (no_acpi) {
1164+ pr_debug("Not allowed to use ACPI _CST\n");
1165+ return false;
11231166 }
11241167
1125− if (boot_cpu_data.cpuid_level < CPUID_MWAIT_LEAF)
1126− return -ENODEV;
1168+ for_each_possible_cpu(cpu) {
1169+ struct acpi_processor *pr = per_cpu(processors, cpu);
11271170
1128− cpuid(CPUID_MWAIT_LEAF, &eax, &ebx, &ecx, &mwait_substates);
1171+ if (!pr)
1172+ continue;
11291173
1130− if (!(ecx & CPUID5_ECX_EXTENSIONS_SUPPORTED) ||
1131− !(ecx & CPUID5_ECX_INTERRUPT_BREAK) ||
1132− !mwait_substates)
1133− return -ENODEV;
1174+ if (acpi_processor_evaluate_cst(pr->handle, cpu, &acpi_state_table))
1175+ continue;
11341176
1135− pr_debug("MWAIT substates: 0x%x\n", mwait_substates);
1177+ acpi_state_table.count++;
11361178
1137− icpu = (const struct idle_cpu *)id->driver_data;
1138− cpuidle_state_table = icpu->state_table;
1179+ if (!intel_idle_cst_usable())
1180+ continue;
11391181
1140− pr_debug("v" INTEL_IDLE_VERSION " model 0x%X\n",
1141− boot_cpu_data.x86_model);
1182+ if (!acpi_processor_claim_cst_control()) {
1183+ acpi_state_table.count = 0;
1184+ return false;
1185+ }
11421186
1143− return 0;
1187+ return true;
1188+ }
1189+
1190+ pr_debug("ACPI _CST not found or not usable\n");
1191+ return false;
11441192 }
11451193
1146−/*
1147− * intel_idle_cpuidle_devices_uninit()
1148− * Unregisters the cpuidle devices.
1149− */
1150−static void intel_idle_cpuidle_devices_uninit(void)
1194+static void __init intel_idle_init_cstates_acpi(struct cpuidle_driver *drv)
11511195 {
1152− int i;
1153− struct cpuidle_device *dev;
1196+ int cstate, limit = min_t(int, CPUIDLE_STATE_MAX, acpi_state_table.count);
1197+
1198+ /*
1199+ * If limit > 0, intel_idle_cst_usable() has returned 'true', so all of
1200+ * the interesting states are ACPI_CSTATE_FFH.
1201+ */
1202+ for (cstate = 1; cstate < limit; cstate++) {
1203+ struct acpi_processor_cx *cx;
1204+ struct cpuidle_state *state;
11541205
1155− for_each_online_cpu(i) {
1156− dev = per_cpu_ptr(intel_idle_cpuidle_devices, i);
1157− cpuidle_unregister_device(dev);
1206+ if (intel_idle_max_cstate_reached(cstate))
1207+ break;
1208+
1209+ cx = &acpi_state_table.states[cstate];
1210+
1211+ state = &drv->states[drv->state_count++];
1212+
1213+ snprintf(state->name, CPUIDLE_NAME_LEN, "C%d_ACPI", cstate);
1214+ strlcpy(state->desc, cx->desc, CPUIDLE_DESC_LEN);
1215+ state->exit_latency = cx->latency;
1216+ /*
1217+ * For C1-type C-states use the same number for both the exit
1218+ * latency and target residency, because that is the case for
1219+ * C1 in the majority of the static C-states tables above.
1220+ * For the other types of C-states, however, set the target
1221+ * residency to 3 times the exit latency which should lead to
1222+ * a reasonable balance between energy-efficiency and
1223+ * performance in the majority of interesting cases.
1224+ */
1225+ state->target_residency = cx->latency;
1226+ if (cx->type > ACPI_STATE_C1)
1227+ state->target_residency *= 3;
1228+
1229+ state->flags = MWAIT2flg(cx->address);
1230+ if (cx->type > ACPI_STATE_C2)
1231+ state->flags |= CPUIDLE_FLAG_TLB_FLUSHED;
1232+
1233+ state->enter = intel_idle;
1234+ state->enter_s2idle = intel_idle_s2idle;
11581235 }
11591236 }
11601237
1238+static bool __init intel_idle_off_by_default(u32 mwait_hint)
1239+{
1240+ int cstate, limit;
1241+
1242+ /*
1243+ * If there are no _CST C-states, do not disable any C-states by
1244+ * default.
1245+ */
1246+ if (!acpi_state_table.count)
1247+ return false;
1248+
1249+ limit = min_t(int, CPUIDLE_STATE_MAX, acpi_state_table.count);
1250+ /*
1251+ * If limit > 0, intel_idle_cst_usable() has returned 'true', so all of
1252+ * the interesting states are ACPI_CSTATE_FFH.
1253+ */
1254+ for (cstate = 1; cstate < limit; cstate++) {
1255+ if (acpi_state_table.states[cstate].address == mwait_hint)
1256+ return false;
1257+ }
1258+ return true;
1259+}
1260+#else /* !CONFIG_ACPI_PROCESSOR_CSTATE */
1261+static inline bool intel_idle_acpi_cst_extract(void) { return false; }
1262+static inline void intel_idle_init_cstates_acpi(struct cpuidle_driver *drv) { }
1263+static inline bool intel_idle_off_by_default(u32 mwait_hint) { return false; }
1264+#endif /* !CONFIG_ACPI_PROCESSOR_CSTATE */
1265+
11611266 /*
11621267 * ivt_idle_state_table_update(void)
11631268 *
11641269 * Tune IVT multi-socket targets
11651270 * Assumption: num_sockets == (max_package_num + 1)
11661271 */
1167−static void ivt_idle_state_table_update(void)
1272+static void __init ivt_idle_state_table_update(void)
11681273 {
11691274 /* IVT uses a different table for 1-2, 3-4, and > 4 sockets */
11701275 int cpu, package_num, num_sockets = 1;
@@ -1187,15 +1292,17 @@ static void ivt_idle_state_table_update(void) · 16 unmodified lines
11871292 /* else, 1 and 2 socket systems use default ivt_cstates */
11881293 }
11891294
1190−/*
1191− * Translate IRTL (Interrupt Response Time Limit) MSR to usec
1295+/**
1296+ * irtl_2_usec - IRTL to microseconds conversion.
1297+ * @irtl: IRTL MSR value.
1298+ *
1299+ * Translate the IRTL (Interrupt Response Time Limit) MSR value to microseconds.
11921300 */
1193−
1194−static unsigned int irtl_ns_units[] = {
1195− 1, 32, 1024, 32768, 1048576, 33554432, 0, 0 };
1196−
1197−static unsigned long long irtl_2_usec(unsigned long long irtl)
1301+static unsigned long long __init irtl_2_usec(unsigned long long irtl)
11981302 {
1303+ static const unsigned int irtl_ns_units[] __initconst = {
1304+ 1, 32, 1024, 32768, 1048576, 33554432, 0, 0
1305+ };
11991306 unsigned long long ns;
12001307
12011308 if (!irtl)
@@ -1203,15 +1310,16 @@ static unsigned long long irtl_2_usec(unsigned long long irtl) · 1 unmodified line
12031310
12041311 ns = irtl_ns_units[(irtl >> 10) & 0x7];
12051312
1206− return div64_u64((irtl & 0x3FF) * ns, 1000);
1313+ return div_u64((irtl & 0x3FF) * ns, NSEC_PER_USEC);
12071314 }
1315+
12081316 /*
12091317 * bxt_idle_state_table_update(void)
12101318 *
12111319 * On BXT, we trust the IRTL to show the definitive maximum latency
12121320 * We use the same value for target_residency.
12131321 */
1214−static void bxt_idle_state_table_update(void)
1322+static void __init bxt_idle_state_table_update(void)
12151323 {
12161324 unsigned long long msr;
12171325 unsigned int usec;
@@ -1258,7 +1366,7 @@ static void bxt_idle_state_table_update(void) · 40 unmodified lines
12581366 * On SKL-H (model 0x5e) disable C8 and C9 if:
12591367 * C10 is enabled and SGX disabled
12601368 */
1261−static void sklh_idle_state_table_update(void)
1369+static void __init sklh_idle_state_table_update(void)
12621370 {
12631371 unsigned long long msr;
12641372 unsigned int eax, ebx, ecx, edx;
@@ -1294,16 +1402,28 @@ static void sklh_idle_state_table_update(void) · 29 unmodified lines
12941402 skl_cstates[5].flags |= CPUIDLE_FLAG_UNUSABLE; /* C8-SKL */
12951403 skl_cstates[6].flags |= CPUIDLE_FLAG_UNUSABLE; /* C9-SKL */
12961404 }
1297−/*
1298− * intel_idle_state_table_update()
1299− *
1300− * Update the default state_table for this CPU-id
1301− */
13021405
1303−static void intel_idle_state_table_update(void)
1406+static bool __init intel_idle_verify_cstate(unsigned int mwait_hint)
13041407 {
1305− switch (boot_cpu_data.x86_model) {
1408+ unsigned int mwait_cstate = MWAIT_HINT2CSTATE(mwait_hint) + 1;
1409+ unsigned int num_substates = (mwait_substates >> mwait_cstate * 4) &
1410+ MWAIT_SUBSTATE_MASK;
1411+
1412+ /* Ignore the C-state if there are NO sub-states in CPUID for it. */
1413+ if (num_substates == 0)
1414+ return false;
1415+
1416+ if (mwait_cstate > 2 && !boot_cpu_has(X86_FEATURE_NONSTOP_TSC))
1417+ mark_tsc_unstable("TSC halts in idle states deeper than C2");
13061418
1419+ return true;
1420+}
1421+
1422+static void __init intel_idle_init_cstates_icpu(struct cpuidle_driver *drv)
1423+{
1424+ int cstate;
1425+
1426+ switch (boot_cpu_data.x86_model) {
13071427 case INTEL_FAM6_IVYBRIDGE_X:
13081428 ivt_idle_state_table_update();
13091429 break;
@@ -1315,62 +1435,36 @@ static void intel_idle_state_table_update(void) · 5 unmodified lines
13151435 sklh_idle_state_table_update();
13161436 break;
13171437 }
1318−}
1319−
1320−/*
1321− * intel_idle_cpuidle_driver_init()
1322− * allocate, initialize cpuidle_states
1323− */
1324−static void __init intel_idle_cpuidle_driver_init(void)
1325−{
1326− int cstate;
1327− struct cpuidle_driver *drv = &intel_idle_driver;
1328−
1329− intel_idle_state_table_update();
1330−
1331− cpuidle_poll_state_init(drv);
1332− drv->state_count = 1;
13331438
13341439 for (cstate = 0; cstate < CPUIDLE_STATE_MAX; ++cstate) {
1335− int num_substates, mwait_hint, mwait_cstate;
1440+ unsigned int mwait_hint;
13361441
1337− if ((cpuidle_state_table[cstate].enter == NULL) &&
1338− (cpuidle_state_table[cstate].enter_s2idle == NULL))
1442+ if (intel_idle_max_cstate_reached(cstate))
13391443 break;
13401444
1341− if (cstate + 1 > max_cstate) {
1342− pr_info("max_cstate %d reached\n", max_cstate);
1445+ if (!cpuidle_state_table[cstate].enter &&
1446+ !cpuidle_state_table[cstate].enter_s2idle)
13431447 break;
1344− }
1345−
1346− mwait_hint = flg2MWAIT(cpuidle_state_table[cstate].flags);
1347− mwait_cstate = MWAIT_HINT2CSTATE(mwait_hint);
1348−
1349− /* number of sub-states for this state in CPUID.MWAIT */
1350− num_substates = (mwait_substates >> ((mwait_cstate + 1) * 4))
1351− & MWAIT_SUBSTATE_MASK;
1352−
1353− /* if NO sub-states for this state in CPUID, skip it */
1354− if (num_substates == 0)
1355− continue;
13561448
1357− /* if state marked as disabled, skip it */
1449+ /* If marked as unusable, skip this state. */
13581450 if (cpuidle_state_table[cstate].flags & CPUIDLE_FLAG_UNUSABLE) {
13591451 pr_debug("state %s is disabled\n",
13601452 cpuidle_state_table[cstate].name);
13611453 continue;
13621454 }
13631455
1456+ mwait_hint = flg2MWAIT(cpuidle_state_table[cstate].flags);
1457+ if (!intel_idle_verify_cstate(mwait_hint))
1458+ continue;
13641459
1365− if (((mwait_cstate + 1) > 2) &&
1366− !boot_cpu_has(X86_FEATURE_NONSTOP_TSC))
1367− mark_tsc_unstable("TSC halts in idle"
1368− " states deeper than C2");
1460+ /* Structure copy. */
1461+ drv->states[drv->state_count] = cpuidle_state_table[cstate];
13691462
1370− drv->states[drv->state_count] = /* structure copy */
1371− cpuidle_state_table[cstate];
1463+ if (icpu->use_acpi && intel_idle_off_by_default(mwait_hint) &&
1464+ !(cpuidle_state_table[cstate].flags & CPUIDLE_FLAG_ALWAYS_ENABLE))
1465+ drv->states[drv->state_count].flags |= CPUIDLE_FLAG_OFF;
13721466
1373− drv->state_count += 1;
1467+ drv->state_count++;
13741468 }
13751469
13761470 if (icpu->byt_auto_demotion_disable_flag) {
@@ -1379,6 +1473,38 @@ static void __init intel_idle_cpuidle_driver_init(void) · 2 unmodified lines
13791473 }
13801474 }
13811475
1476+/*
1477+ * intel_idle_cpuidle_driver_init()
1478+ * allocate, initialize cpuidle_states
1479+ */
1480+static void __init intel_idle_cpuidle_driver_init(struct cpuidle_driver *drv)
1481+{
1482+ cpuidle_poll_state_init(drv);
1483+ drv->state_count = 1;
1484+
1485+ if (icpu)
1486+ intel_idle_init_cstates_icpu(drv);
1487+ else
1488+ intel_idle_init_cstates_acpi(drv);
1489+}
1490+
1491+static void auto_demotion_disable(void)
1492+{
1493+ unsigned long long msr_bits;
1494+
1495+ rdmsrl(MSR_PKG_CST_CONFIG_CONTROL, msr_bits);
1496+ msr_bits &= ~(icpu->auto_demotion_disable_flags);
1497+ wrmsrl(MSR_PKG_CST_CONFIG_CONTROL, msr_bits);
1498+}
1499+
1500+static void c1e_promotion_disable(void)
1501+{
1502+ unsigned long long msr_bits;
1503+
1504+ rdmsrl(MSR_IA32_POWER_CTL, msr_bits);
1505+ msr_bits &= ~0x2;
1506+ wrmsrl(MSR_IA32_POWER_CTL, msr_bits);
1507+}
13821508
13831509 /*
13841510 * intel_idle_cpu_init()
@@ -1397,6 +1523,9 @@ static int intel_idle_cpu_init(unsigned int cpu) · 12 unmodified lines
13971523 return -EIO;
13981524 }
13991525
1526+ if (!icpu)
1527+ return 0;
1528+
14001529 if (icpu->auto_demotion_disable_flags)
14011530 auto_demotion_disable();
14021531
@@ -1411,7 +1540,7 @@ static int intel_idle_cpu_online(unsigned int cpu) · 8 unmodified lines
14111540 struct cpuidle_device *dev;
14121541
14131542 if (lapic_timer_reliable_states != LAPIC_TIMER_ALWAYS_RELIABLE)
1414− __setup_broadcast_timer(true);
1543+ tick_broadcast_enable();
14151544
14161545 /*
14171546 * Some systems can hotplug a cpu at runtime after
@@ -1425,23 +1554,74 @@ static int intel_idle_cpu_online(unsigned int cpu) · 7 unmodified lines
14251554 return 0;
14261555 }
14271556
1557+/**
1558+ * intel_idle_cpuidle_devices_uninit - Unregister all cpuidle devices.
1559+ */
1560+static void __init intel_idle_cpuidle_devices_uninit(void)
1561+{
1562+ int i;
1563+
1564+ for_each_online_cpu(i)
1565+ cpuidle_unregister_device(per_cpu_ptr(intel_idle_cpuidle_devices, i));
1566+}
1567+
14281568 static int __init intel_idle_init(void)
14291569 {
1570+ const struct x86_cpu_id *id;
1571+ unsigned int eax, ebx, ecx;
14301572 int retval;
14311573
14321574 /* Do not load intel_idle at all for now if idle= is passed */
14331575 if (boot_option_idle_override != IDLE_NO_OVERRIDE)
14341576 return -ENODEV;
14351577
1436− retval = intel_idle_probe();
1437− if (retval)
1438− return retval;
1578+ if (max_cstate == 0) {
1579+ pr_debug("disabled\n");
1580+ return -EPERM;
1581+ }
1582+
1583+ id = x86_match_cpu(intel_idle_ids);
1584+ if (id) {
1585+ if (!boot_cpu_has(X86_FEATURE_MWAIT)) {
1586+ pr_debug("Please enable MWAIT in BIOS SETUP\n");
1587+ return -ENODEV;
1588+ }
1589+ } else {
1590+ id = x86_match_cpu(intel_mwait_ids);
1591+ if (!id)
1592+ return -ENODEV;
1593+ }
1594+
1595+ if (boot_cpu_data.cpuid_level < CPUID_MWAIT_LEAF)
1596+ return -ENODEV;
1597+
1598+ cpuid(CPUID_MWAIT_LEAF, &eax, &ebx, &ecx, &mwait_substates);
1599+
1600+ if (!(ecx & CPUID5_ECX_EXTENSIONS_SUPPORTED) ||
1601+ !(ecx & CPUID5_ECX_INTERRUPT_BREAK) ||
1602+ !mwait_substates)
1603+ return -ENODEV;
1604+
1605+ pr_debug("MWAIT substates: 0x%x\n", mwait_substates);
1606+
1607+ icpu = (const struct idle_cpu *)id->driver_data;
1608+ if (icpu) {
1609+ cpuidle_state_table = icpu->state_table;
1610+ if (icpu->use_acpi)
1611+ intel_idle_acpi_cst_extract();
1612+ } else if (!intel_idle_acpi_cst_extract()) {
1613+ return -ENODEV;
1614+ }
1615+
1616+ pr_debug("v" INTEL_IDLE_VERSION " model 0x%X\n",
1617+ boot_cpu_data.x86_model);
14391618
14401619 intel_idle_cpuidle_devices = alloc_percpu(struct cpuidle_device);
1441− if (intel_idle_cpuidle_devices == NULL)
1620+ if (!intel_idle_cpuidle_devices)
14421621 return -ENOMEM;
14431622
1444− intel_idle_cpuidle_driver_init();
1623+ intel_idle_cpuidle_driver_init(&intel_idle_driver);
1624+
14451625 retval = cpuidle_register_driver(&intel_idle_driver);
14461626 if (retval) {
14471627 struct cpuidle_driver *drv = cpuidle_get_driver();
include/linux/acpi.h+15−0
@@ -279,6 +279,21 @@ static inline bool invalid_phys_cpuid(phys_cpuid_t phys_id) · 278 unmodified lines
279279
280280 /* Validate the processor object's proc_id */
281281 bool acpi_duplicate_processor_id(int proc_id);
282+/* Processor _CTS control */
283+struct acpi_processor_power;
284+
285+#ifdef CONFIG_ACPI_PROCESSOR_CSTATE
286+bool acpi_processor_claim_cst_control(void);
287+int acpi_processor_evaluate_cst(acpi_handle handle, u32 cpu,
288+ struct acpi_processor_power *info);
289+#else
290+static inline bool acpi_processor_claim_cst_control(void) { return false; }
291+static inline int acpi_processor_evaluate_cst(acpi_handle handle, u32 cpu,
292+ struct acpi_processor_power *info)
293+{
294+ return -ENODEV;
295+}
296+#endif
282297
283298 #ifdef CONFIG_ACPI_HOTPLUG_CPU
284299 /* Arch dependent functions for cpu hotplug support */
include/linux/cpuidle.h+1−5
@@ -77,6 +77,7 @@ struct cpuidle_state { · 76 unmodified lines
7777 #define CPUIDLE_FLAG_COUPLED BIT(1) /* state applies to multiple cpus */
7878 #define CPUIDLE_FLAG_TIMER_STOP BIT(2) /* timer is stopped on this state */
7979 #define CPUIDLE_FLAG_UNUSABLE BIT(3) /* avoid using this state */
80+#define CPUIDLE_FLAG_OFF BIT(4) /* disable this state by default */
8081
8182 struct cpuidle_device_kobj;
8283 struct cpuidle_state_kobj;
@@ -115,7 +116,6 @@ DECLARE_PER_CPU(struct cpuidle_device, cpuidle_dev); · 32 unmodified lines
115116 struct cpuidle_driver {
116117 const char *name;
117118 struct module *owner;
118− int refcnt;
119119
120120 /* used by the cpuidle framework to setup the broadcast timer */
121121 unsigned int bctimer:1;
@@ -147,8 +147,6 @@ extern u64 cpuidle_poll_time(struct cpuidle_driver *drv, · 25 unmodified lines
147147
148148 extern int cpuidle_register_driver(struct cpuidle_driver *drv);
149149 extern struct cpuidle_driver *cpuidle_get_driver(void);
150−extern struct cpuidle_driver *cpuidle_driver_ref(void);
151−extern void cpuidle_driver_unref(void);
152150 extern void cpuidle_driver_state_disabled(struct cpuidle_driver *drv, int idx,
153151 bool disable);
154152 extern void cpuidle_unregister_driver(struct cpuidle_driver *drv);
@@ -186,8 +184,6 @@ static inline u64 cpuidle_poll_time(struct cpuidle_driver *drv, · 31 unmodified lines
186184 static inline int cpuidle_register_driver(struct cpuidle_driver *drv)
187185 {return -ENODEV; }
188186 static inline struct cpuidle_driver *cpuidle_get_driver(void) {return NULL; }
189−static inline struct cpuidle_driver *cpuidle_driver_ref(void) {return NULL; }
190−static inline void cpuidle_driver_unref(void) {}
191187 static inline void cpuidle_driver_state_disabled(struct cpuidle_driver *drv,
192188 int idx, bool disable) { }
193189 static inline void cpuidle_unregister_driver(struct cpuidle_driver *drv) { }

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t