History

Merge branch 'pm-cpufreq'

* pm-cpufreq: cpufreq: intel_pstate: Implement passive mode with HWP enabled

Rafael J. Wysocki committed 6 years ago

Commit f3db6de2 parents f6235eb f6ebbcf4 files · +229 −113Browse files

Unified · Split

4 files
  1. Documentation/admin-guide/pm/intel_pstate.rst +43 −46
  2. drivers/cpufreq/cpufreq.c +2 −4
  3. drivers/cpufreq/intel_pstate.c +182 −63
  4. include/linux/cpufreq.h +2 −0
Documentation/admin-guide/pm/intel_pstate.rst+43−46
@@ -54,10 +54,13 @@ registered (see `below <status_attr_>`_). · 53 unmodified lines
5454 Operation Modes
5555 ===============
5656
57−``intel_pstate`` can operate in three different modes: in the active mode with
58−or without hardware-managed P-states support and in the passive mode. Which of
59−them will be in effect depends on what kernel command line options are used and
60−on the capabilities of the processor.
57+``intel_pstate`` can operate in two different modes, active or passive. In the
58+active mode, it uses its own internal performance scaling governor algorithm or
59+allows the hardware to do preformance scaling by itself, while in the passive
60+mode it responds to requests made by a generic ``CPUFreq`` governor implementing
61+a certain performance scaling algorithm. Which of them will be in effect
62+depends on what kernel command line options are used and on the capabilities of
63+the processor.
6164
6265 Active Mode
6366 -----------
@@ -194,10 +197,11 @@ This is the default operation mode of ``intel_pstate`` for processors without · 130 unmodified lines
194197 hardware-managed P-states (HWP) support. It is always used if the
195198 ``intel_pstate=passive`` argument is passed to the kernel in the command line
196199 regardless of whether or not the given processor supports HWP. [Note that the
197−``intel_pstate=no_hwp`` setting implies ``intel_pstate=passive`` if it is used
198−without ``intel_pstate=active``.] Like in the active mode without HWP support,
199−in this mode ``intel_pstate`` may refuse to work with processors that are not
200−recognized by it.
200+``intel_pstate=no_hwp`` setting causes the driver to start in the passive mode
201+if it is not combined with ``intel_pstate=active``.] Like in the active mode
202+without HWP support, in this mode ``intel_pstate`` may refuse to work with
203+processors that are not recognized by it if HWP is prevented from being enabled
204+through the kernel command line.
201205
202206 If the driver works in this mode, the ``scaling_driver`` policy attribute in
203207 ``sysfs`` for all ``CPUFreq`` policies contains the string "intel_cpufreq".
@@ -318,10 +322,9 @@ manuals need to be consulted to get to it too. · 114 unmodified lines
318322
319323 For this reason, there is a list of supported processors in ``intel_pstate`` and
320324 the driver initialization will fail if the detected processor is not in that
321−list, unless it supports the `HWP feature <Active Mode_>`_. [The interface to
322−obtain all of the information listed above is the same for all of the processors
323−supporting the HWP feature, which is why they all are supported by
324−``intel_pstate``.]
325+list, unless it supports the HWP feature. [The interface to obtain all of the
326+information listed above is the same for all of the processors supporting the
327+HWP feature, which is why ``intel_pstate`` works with all of them.]
325328
326329
327330 User Space Interface in ``sysfs``
@@ -425,22 +428,16 @@ argument is passed to the kernel in the command line. · 97 unmodified lines
425428 as well as the per-policy ones) are then reset to their default
426429 values, possibly depending on the target operation mode.]
427430
428− That only is supported in some configurations, though (for example, if
429− the `HWP feature is enabled in the processor <Active Mode With HWP_>`_,
430− the operation mode of the driver cannot be changed), and if it is not
431− supported in the current configuration, writes to this attribute will
432− fail with an appropriate error.
433−
434431 ``energy_efficiency``
435− This attribute is only present on platforms, which have CPUs matching
436− Kaby Lake or Coffee Lake desktop CPU model. By default
437− energy efficiency optimizations are disabled on these CPU models in HWP
438− mode by this driver. Enabling energy efficiency may limit maximum
439− operating frequency in both HWP and non HWP mode. In non HWP mode,
440− optimizations are done only in the turbo frequency range. In HWP mode,
441− optimizations are done in the entire frequency range. Setting this
442− attribute to "1" enables energy efficiency optimizations and setting
443− to "0" disables energy efficiency optimizations.
432+ This attribute is only present on platforms with CPUs matching the Kaby
433+ Lake or Coffee Lake desktop CPU model. By default, energy-efficiency
434+ optimizations are disabled on these CPU models if HWP is enabled.
435+ Enabling energy-efficiency optimizations may limit maximum operating
436+ frequency with or without the HWP feature. With HWP enabled, the
437+ optimizations are done only in the turbo frequency range. Without it,
438+ they are done in the entire available frequency range. Setting this
439+ attribute to "1" enables the energy-efficiency optimizations and setting
440+ to "0" disables them.
444441
445442 Interpretation of Policy Attributes
446443 -----------------------------------
@@ -484,8 +481,8 @@ Next, the following policy attributes have special meaning if · 37 unmodified lines
484481 policy for the time interval between the last two invocations of the
485482 driver's utilization update callback by the CPU scheduler for that CPU.
486483
487−One more policy attribute is present if the `HWP feature is enabled in the
488−processor <Active Mode With HWP_>`_:
484+One more policy attribute is present if the HWP feature is enabled in the
485+processor:
489486
490487 ``base_frequency``
491488 Shows the base frequency of the CPU. Any frequency above this will be
@@ -526,11 +523,11 @@ on the following rules, regardless of the current operation mode of the driver: · 34 unmodified lines
526523
527524 3. The global and per-policy limits can be set independently.
528525
529−If the `HWP feature is enabled in the processor <Active Mode With HWP_>`_, the
530−resulting effective values are written into its registers whenever the limits
531−change in order to request its internal P-state selection logic to always set
532−P-states within these limits. Otherwise, the limits are taken into account by
533−scaling governors (in the `passive mode <Passive Mode_>`_) and by the driver
526+In the `active mode with the HWP feature enabled <Active Mode With HWP_>`_, the
527+resulting effective values are written into hardware registers whenever the
528+limits change in order to request its internal P-state selection logic to always
529+set P-states within these limits. Otherwise, the limits are taken into account
530+by scaling governors (in the `passive mode <Passive Mode_>`_) and by the driver
534531 every time before setting a new P-state for a CPU.
535532
536533 Additionally, if the ``intel_pstate=per_cpu_perf_limits`` command line argument
@@ -541,12 +538,11 @@ at all and the only way to set the limits is by using the policy attributes. · 4 unmodified lines
541538 Energy vs Performance Hints
542539 ---------------------------
543540
544−If ``intel_pstate`` works in the `active mode with the HWP feature enabled
545−<Active Mode With HWP_>`_ in the processor, additional attributes are present
546−in every ``CPUFreq`` policy directory in ``sysfs``. They are intended to allow
547−user space to help ``intel_pstate`` to adjust the processor's internal P-state
548−selection logic by focusing it on performance or on energy-efficiency, or
549−somewhere between the two extremes:
541+If the hardware-managed P-states (HWP) is enabled in the processor, additional
542+attributes, intended to allow user space to help ``intel_pstate`` to adjust the
543+processor's internal P-state selection logic by focusing it on performance or on
544+energy-efficiency, or somewhere between the two extremes, are present in every
545+``CPUFreq`` policy directory in ``sysfs``. They are :
550546
551547 ``energy_performance_preference``
552548 Current value of the energy vs performance hint for the given policy
@@ -650,12 +646,14 @@ of them have to be prepended with the ``intel_pstate=`` prefix. · 97 unmodified lines
650646 Do not register ``intel_pstate`` as the scaling driver even if the
651647 processor is supported by it.
652648
649+``active``
650+ Register ``intel_pstate`` in the `active mode <Active Mode_>`_ to start
651+ with.
652+
653653 ``passive``
654654 Register ``intel_pstate`` in the `passive mode <Passive Mode_>`_ to
655655 start with.
656656
657− This option implies the ``no_hwp`` one described below.
658−
659657 ``force``
660658 Register ``intel_pstate`` as the scaling driver instead of
661659 ``acpi-cpufreq`` even if the latter is preferred on the given system.
@@ -670,13 +668,12 @@ of them have to be prepended with the ``intel_pstate=`` prefix. · 8 unmodified lines
670668 driver is used instead of ``acpi-cpufreq``.
671669
672670 ``no_hwp``
673− Do not enable the `hardware-managed P-states (HWP) feature
674− <Active Mode With HWP_>`_ even if it is supported by the processor.
671+ Do not enable the hardware-managed P-states (HWP) feature even if it is
672+ supported by the processor.
675673
676674 ``hwp_only``
677675 Register ``intel_pstate`` as the scaling driver only if the
678− `hardware-managed P-states (HWP) feature <Active Mode With HWP_>`_ is
679− supported by the processor.
676+ hardware-managed P-states (HWP) feature is supported by the processor.
680677
681678 ``support_acpi_ppc``
682679 Take ACPI ``_PPC`` performance limits into account.
drivers/cpufreq/cpufreq.c+2−4
@@ -73,8 +73,6 @@ static inline bool has_target(void) · 72 unmodified lines
7373 static unsigned int __cpufreq_get(struct cpufreq_policy *policy);
7474 static int cpufreq_init_governor(struct cpufreq_policy *policy);
7575 static void cpufreq_exit_governor(struct cpufreq_policy *policy);
76−static int cpufreq_start_governor(struct cpufreq_policy *policy);
77−static void cpufreq_stop_governor(struct cpufreq_policy *policy);
7876 static void cpufreq_governor_limits(struct cpufreq_policy *policy);
7977 static int cpufreq_set_policy(struct cpufreq_policy *policy,
8078 struct cpufreq_governor *new_gov,
@@ -2266,7 +2264,7 @@ static void cpufreq_exit_governor(struct cpufreq_policy *policy) · 2185 unmodified lines
22662264 module_put(policy->governor->owner);
22672265 }
22682266
2269−static int cpufreq_start_governor(struct cpufreq_policy *policy)
2267+int cpufreq_start_governor(struct cpufreq_policy *policy)
22702268 {
22712269 int ret;
22722270
@@ -2293,7 +2291,7 @@ static int cpufreq_start_governor(struct cpufreq_policy *policy) · 20 unmodified lines
22932291 return 0;
22942292 }
22952293
2296−static void cpufreq_stop_governor(struct cpufreq_policy *policy)
2294+void cpufreq_stop_governor(struct cpufreq_policy *policy)
22972295 {
22982296 if (cpufreq_suspended || !policy->governor)
22992297 return;
drivers/cpufreq/intel_pstate.c+182−63
@@ -36,6 +36,7 @@ · 35 unmodified lines
3636 #define INTEL_PSTATE_SAMPLING_INTERVAL (10 * NSEC_PER_MSEC)
3737
3838 #define INTEL_CPUFREQ_TRANSITION_LATENCY 20000
39+#define INTEL_CPUFREQ_TRANSITION_DELAY_HWP 5000
3940 #define INTEL_CPUFREQ_TRANSITION_DELAY 500
4041
4142 #ifdef CONFIG_ACPI
@@ -220,6 +221,7 @@ struct global_params { · 178 unmodified lines
220221 * preference/bias
221222 * @epp_saved: Saved EPP/EPB during system suspend or CPU offline
222223 * operation
224+ * @epp_cached Cached HWP energy-performance preference value
223225 * @hwp_req_cached: Cached value of the last HWP Request MSR
224226 * @hwp_cap_cached: Cached value of the last HWP Capabilities MSR
225227 * @last_io_update: Last time when IO wake flag was set
@@ -257,6 +259,7 @@ struct cpudata { · 31 unmodified lines
257259 s16 epp_policy;
258260 s16 epp_default;
259261 s16 epp_saved;
262+ s16 epp_cached;
260263 u64 hwp_req_cached;
261264 u64 hwp_cap_cached;
262265 u64 last_io_update;
@@ -639,6 +642,26 @@ static int intel_pstate_get_energy_pref_index(struct cpudata *cpu_data, int *raw · 376 unmodified lines
639642 return index;
640643 }
641644
645+static int intel_pstate_set_epp(struct cpudata *cpu, u32 epp)
646+{
647+ /*
648+ * Use the cached HWP Request MSR value, because in the active mode the
649+ * register itself may be updated by intel_pstate_hwp_boost_up() or
650+ * intel_pstate_hwp_boost_down() at any time.
651+ */
652+ u64 value = READ_ONCE(cpu->hwp_req_cached);
653+
654+ value &= ~GENMASK_ULL(31, 24);
655+ value |= (u64)epp << 24;
656+ /*
657+ * The only other updater of hwp_req_cached in the active mode,
658+ * intel_pstate_hwp_set(), is called under the same lock as this
659+ * function, so it cannot run in parallel with the update below.
660+ */
661+ WRITE_ONCE(cpu->hwp_req_cached, value);
662+ return wrmsrl_on_cpu(cpu->cpu, MSR_HWP_REQUEST, value);
663+}
664+
642665 static int intel_pstate_set_energy_pref_index(struct cpudata *cpu_data,
643666 int pref_index, bool use_raw,
644667 u32 raw_epp)
@@ -650,28 +673,12 @@ static int intel_pstate_set_energy_pref_index(struct cpudata *cpu_data, · 5 unmodified lines
650673 epp = cpu_data->epp_default;
651674
652675 if (boot_cpu_has(X86_FEATURE_HWP_EPP)) {
653− /*
654− * Use the cached HWP Request MSR value, because the register
655− * itself may be updated by intel_pstate_hwp_boost_up() or
656− * intel_pstate_hwp_boost_down() at any time.
657− */
658− u64 value = READ_ONCE(cpu_data->hwp_req_cached);
659−
660− value &= ~GENMASK_ULL(31, 24);
661−
662676 if (use_raw)
663677 epp = raw_epp;
664678 else if (epp == -EINVAL)
665679 epp = epp_values[pref_index - 1];
666680
667− value |= (u64)epp << 24;
668− /*
669− * The only other updater of hwp_req_cached in the active mode,
670− * intel_pstate_hwp_set(), is called under the same lock as this
671− * function, so it cannot run in parallel with the update below.
672− */
673− WRITE_ONCE(cpu_data->hwp_req_cached, value);
674− ret = wrmsrl_on_cpu(cpu_data->cpu, MSR_HWP_REQUEST, value);
681+ ret = intel_pstate_set_epp(cpu_data, epp);
675682 } else {
676683 if (epp == -EINVAL)
677684 epp = (pref_index - 1) << 2;
@@ -697,10 +704,12 @@ static ssize_t show_energy_performance_available_preferences( · 19 unmodified lines
697704
698705 cpufreq_freq_attr_ro(energy_performance_available_preferences);
699706
707+static struct cpufreq_driver intel_pstate;
708+
700709 static ssize_t store_energy_performance_preference(
701710 struct cpufreq_policy *policy, const char *buf, size_t count)
702711 {
703− struct cpudata *cpu_data = all_cpu_data[policy->cpu];
712+ struct cpudata *cpu = all_cpu_data[policy->cpu];
704713 char str_preference[21];
705714 bool raw = false;
706715 ssize_t ret;
@@ -725,15 +734,44 @@ static ssize_t store_energy_performance_preference( · 18 unmodified lines
725734 raw = true;
726735 }
727736
737+ /*
738+ * This function runs with the policy R/W semaphore held, which
739+ * guarantees that the driver pointer will not change while it is
740+ * running.
741+ */
742+ if (!intel_pstate_driver)
743+ return -EAGAIN;
744+
728745 mutex_lock(&intel_pstate_limits_lock);
729746
730− ret = intel_pstate_set_energy_pref_index(cpu_data, ret, raw, epp);
731− if (!ret)
732− ret = count;
747+ if (intel_pstate_driver == &intel_pstate) {
748+ ret = intel_pstate_set_energy_pref_index(cpu, ret, raw, epp);
749+ } else {
750+ /*
751+ * In the passive mode the governor needs to be stopped on the
752+ * target CPU before the EPP update and restarted after it,
753+ * which is super-heavy-weight, so make sure it is worth doing
754+ * upfront.
755+ */
756+ if (!raw)
757+ epp = ret ? epp_values[ret - 1] : cpu->epp_default;
758+
759+ if (cpu->epp_cached != epp) {
760+ int err;
761+
762+ cpufreq_stop_governor(policy);
763+ ret = intel_pstate_set_epp(cpu, epp);
764+ err = cpufreq_start_governor(policy);
765+ if (!ret) {
766+ cpu->epp_cached = epp;
767+ ret = err;
768+ }
769+ }
770+ }
733771
734772 mutex_unlock(&intel_pstate_limits_lock);
735773
736− return ret;
774+ return ret ?: count;
737775 }
738776
739777 static ssize_t show_energy_performance_preference(
@@ -1145,8 +1183,6 @@ static ssize_t store_no_turbo(struct kobject *a, struct kobj_attribute *b, · 405 unmodified lines
11451183 return count;
11461184 }
11471185
1148−static struct cpufreq_driver intel_pstate;
1149−
11501186 static void update_qos_request(enum freq_qos_req_type type)
11511187 {
11521188 int max_state, turbo_max, freq, i, perf_pct;
@@ -1330,9 +1366,10 @@ static const struct attribute_group intel_pstate_attr_group = { · 177 unmodified lines
13301366
13311367 static const struct x86_cpu_id intel_pstate_cpu_ee_disable_ids[];
13321368
1369+static struct kobject *intel_pstate_kobject;
1370+
13331371 static void __init intel_pstate_sysfs_expose_params(void)
13341372 {
1335− struct kobject *intel_pstate_kobject;
13361373 int rc;
13371374
13381375 intel_pstate_kobject = kobject_create_and_add("intel_pstate",
@@ -1357,17 +1394,31 @@ static void __init intel_pstate_sysfs_expose_params(void) · 18 unmodified lines
13571394 rc = sysfs_create_file(intel_pstate_kobject, &min_perf_pct.attr);
13581395 WARN_ON(rc);
13591396
1360− if (hwp_active) {
1361− rc = sysfs_create_file(intel_pstate_kobject,
1362− &hwp_dynamic_boost.attr);
1363− WARN_ON(rc);
1364− }
1365−
13661397 if (x86_match_cpu(intel_pstate_cpu_ee_disable_ids)) {
13671398 rc = sysfs_create_file(intel_pstate_kobject, &energy_efficiency.attr);
13681399 WARN_ON(rc);
13691400 }
13701401 }
1402+
1403+static void intel_pstate_sysfs_expose_hwp_dynamic_boost(void)
1404+{
1405+ int rc;
1406+
1407+ if (!hwp_active)
1408+ return;
1409+
1410+ rc = sysfs_create_file(intel_pstate_kobject, &hwp_dynamic_boost.attr);
1411+ WARN_ON_ONCE(rc);
1412+}
1413+
1414+static void intel_pstate_sysfs_hide_hwp_dynamic_boost(void)
1415+{
1416+ if (!hwp_active)
1417+ return;
1418+
1419+ sysfs_remove_file(intel_pstate_kobject, &hwp_dynamic_boost.attr);
1420+}
1421+
13711422 /************************** sysfs end ************************/
13721423
13731424 static void intel_pstate_hwp_enable(struct cpudata *cpudata)
@@ -2247,7 +2298,10 @@ static int intel_pstate_verify_policy(struct cpufreq_policy_data *policy) · 873 unmodified lines
22472298
22482299 static void intel_cpufreq_stop_cpu(struct cpufreq_policy *policy)
22492300 {
2250− intel_pstate_set_min_pstate(all_cpu_data[policy->cpu]);
2301+ if (hwp_active)
2302+ intel_pstate_hwp_force_min_perf(policy->cpu);
2303+ else
2304+ intel_pstate_set_min_pstate(all_cpu_data[policy->cpu]);
22512305 }
22522306
22532307 static void intel_pstate_stop_cpu(struct cpufreq_policy *policy)
@@ -2255,12 +2309,10 @@ static void intel_pstate_stop_cpu(struct cpufreq_policy *policy) · 1 unmodified line
22552309 pr_debug("CPU %d exiting\n", policy->cpu);
22562310
22572311 intel_pstate_clear_update_util_hook(policy->cpu);
2258− if (hwp_active) {
2312+ if (hwp_active)
22592313 intel_pstate_hwp_save_state(policy);
2260− intel_pstate_hwp_force_min_perf(policy->cpu);
2261− } else {
2262− intel_cpufreq_stop_cpu(policy);
2263− }
2314+
2315+ intel_cpufreq_stop_cpu(policy);
22642316 }
22652317
22662318 static int intel_pstate_cpu_exit(struct cpufreq_policy *policy)
@@ -2390,13 +2442,71 @@ static void intel_cpufreq_trace(struct cpudata *cpu, unsigned int trace_type, in · 123 unmodified lines
23902442 fp_toint(cpu->iowait_boost * 100));
23912443 }
23922444
2445+static void intel_cpufreq_adjust_hwp(struct cpudata *cpu, u32 target_pstate,
2446+ bool fast_switch)
2447+{
2448+ u64 prev = READ_ONCE(cpu->hwp_req_cached), value = prev;
2449+
2450+ value &= ~HWP_MIN_PERF(~0L);
2451+ value |= HWP_MIN_PERF(target_pstate);
2452+
2453+ /*
2454+ * The entire MSR needs to be updated in order to update the HWP min
2455+ * field in it, so opportunistically update the max too if needed.
2456+ */
2457+ value &= ~HWP_MAX_PERF(~0L);
2458+ value |= HWP_MAX_PERF(cpu->max_perf_ratio);
2459+
2460+ if (value == prev)
2461+ return;
2462+
2463+ WRITE_ONCE(cpu->hwp_req_cached, value);
2464+ if (fast_switch)
2465+ wrmsrl(MSR_HWP_REQUEST, value);
2466+ else
2467+ wrmsrl_on_cpu(cpu->cpu, MSR_HWP_REQUEST, value);
2468+}
2469+
2470+static void intel_cpufreq_adjust_perf_ctl(struct cpudata *cpu,
2471+ u32 target_pstate, bool fast_switch)
2472+{
2473+ if (fast_switch)
2474+ wrmsrl(MSR_IA32_PERF_CTL,
2475+ pstate_funcs.get_val(cpu, target_pstate));
2476+ else
2477+ wrmsrl_on_cpu(cpu->cpu, MSR_IA32_PERF_CTL,
2478+ pstate_funcs.get_val(cpu, target_pstate));
2479+}
2480+
2481+static int intel_cpufreq_update_pstate(struct cpudata *cpu, int target_pstate,
2482+ bool fast_switch)
2483+{
2484+ int old_pstate = cpu->pstate.current_pstate;
2485+
2486+ target_pstate = intel_pstate_prepare_request(cpu, target_pstate);
2487+ if (target_pstate != old_pstate) {
2488+ cpu->pstate.current_pstate = target_pstate;
2489+ if (hwp_active)
2490+ intel_cpufreq_adjust_hwp(cpu, target_pstate,
2491+ fast_switch);
2492+ else
2493+ intel_cpufreq_adjust_perf_ctl(cpu, target_pstate,
2494+ fast_switch);
2495+ }
2496+
2497+ intel_cpufreq_trace(cpu, fast_switch ? INTEL_PSTATE_TRACE_FAST_SWITCH :
2498+ INTEL_PSTATE_TRACE_TARGET, old_pstate);
2499+
2500+ return target_pstate;
2501+}
2502+
23932503 static int intel_cpufreq_target(struct cpufreq_policy *policy,
23942504 unsigned int target_freq,
23952505 unsigned int relation)
23962506 {
23972507 struct cpudata *cpu = all_cpu_data[policy->cpu];
23982508 struct cpufreq_freqs freqs;
2399− int target_pstate, old_pstate;
2509+ int target_pstate;
24002510
24012511 update_turbo_state();
24022512
@@ -2404,6 +2514,7 @@ static int intel_cpufreq_target(struct cpufreq_policy *policy, · 1 unmodified line
24042514 freqs.new = target_freq;
24052515
24062516 cpufreq_freq_transition_begin(policy, &freqs);
2517+
24072518 switch (relation) {
24082519 case CPUFREQ_RELATION_L:
24092520 target_pstate = DIV_ROUND_UP(freqs.new, cpu->pstate.scaling);
@@ -2415,15 +2526,11 @@ static int intel_cpufreq_target(struct cpufreq_policy *policy, · 5 unmodified lines
24152526 target_pstate = DIV_ROUND_CLOSEST(freqs.new, cpu->pstate.scaling);
24162527 break;
24172528 }
2418− target_pstate = intel_pstate_prepare_request(cpu, target_pstate);
2419− old_pstate = cpu->pstate.current_pstate;
2420− if (target_pstate != cpu->pstate.current_pstate) {
2421− cpu->pstate.current_pstate = target_pstate;
2422− wrmsrl_on_cpu(policy->cpu, MSR_IA32_PERF_CTL,
2423− pstate_funcs.get_val(cpu, target_pstate));
2424− }
2529+
2530+ target_pstate = intel_cpufreq_update_pstate(cpu, target_pstate, false);
2531+
24252532 freqs.new = target_pstate * cpu->pstate.scaling;
2426− intel_cpufreq_trace(cpu, INTEL_PSTATE_TRACE_TARGET, old_pstate);
2533+
24272534 cpufreq_freq_transition_end(policy, &freqs, false);
24282535
24292536 return 0;
@@ -2433,15 +2540,14 @@ static unsigned int intel_cpufreq_fast_switch(struct cpufreq_policy *policy, · 3 unmodified lines
24332540 unsigned int target_freq)
24342541 {
24352542 struct cpudata *cpu = all_cpu_data[policy->cpu];
2436− int target_pstate, old_pstate;
2543+ int target_pstate;
24372544
24382545 update_turbo_state();
24392546
24402547 target_pstate = DIV_ROUND_UP(target_freq, cpu->pstate.scaling);
2441− target_pstate = intel_pstate_prepare_request(cpu, target_pstate);
2442− old_pstate = cpu->pstate.current_pstate;
2443− intel_pstate_update_pstate(cpu, target_pstate);
2444− intel_cpufreq_trace(cpu, INTEL_PSTATE_TRACE_FAST_SWITCH, old_pstate);
2548+
2549+ target_pstate = intel_cpufreq_update_pstate(cpu, target_pstate, true);
2550+
24452551 return target_pstate * cpu->pstate.scaling;
24462552 }
24472553
@@ -2461,7 +2567,6 @@ static int intel_cpufreq_cpu_init(struct cpufreq_policy *policy) · 13 unmodified lines
24612567 return ret;
24622568
24632569 policy->cpuinfo.transition_latency = INTEL_CPUFREQ_TRANSITION_LATENCY;
2464− policy->transition_delay_us = INTEL_CPUFREQ_TRANSITION_DELAY;
24652570 /* This reflects the intel_pstate_get_cpu_pstates() setting. */
24662571 policy->cur = policy->cpuinfo.min_freq;
24672572
@@ -2473,10 +2578,18 @@ static int intel_cpufreq_cpu_init(struct cpufreq_policy *policy) · 5 unmodified lines
24732578
24742579 cpu = all_cpu_data[policy->cpu];
24752580
2476− if (hwp_active)
2581+ if (hwp_active) {
2582+ u64 value;
2583+
24772584 intel_pstate_get_hwp_max(policy->cpu, &turbo_max, &max_state);
2478− else
2585+ policy->transition_delay_us = INTEL_CPUFREQ_TRANSITION_DELAY_HWP;
2586+ rdmsrl_on_cpu(cpu->cpu, MSR_HWP_REQUEST, &value);
2587+ WRITE_ONCE(cpu->hwp_req_cached, value);
2588+ cpu->epp_cached = (value & GENMASK_ULL(31, 24)) >> 24;
2589+ } else {
24792590 turbo_max = cpu->pstate.turbo_pstate;
2591+ policy->transition_delay_us = INTEL_CPUFREQ_TRANSITION_DELAY;
2592+ }
24802593
24812594 min_freq = DIV_ROUND_UP(turbo_max * global.min_perf_pct, 100);
24822595 min_freq *= cpu->pstate.scaling;
@@ -2553,6 +2666,10 @@ static void intel_pstate_driver_cleanup(void) · 70 unmodified lines
25532666 }
25542667 }
25552668 put_online_cpus();
2669+
2670+ if (intel_pstate_driver == &intel_pstate)
2671+ intel_pstate_sysfs_hide_hwp_dynamic_boost();
2672+
25562673 intel_pstate_driver = NULL;
25572674 }
25582675
@@ -2560,6 +2677,9 @@ static int intel_pstate_register_driver(struct cpufreq_driver *driver) · 1 unmodified line
25602677 {
25612678 int ret;
25622679
2680+ if (driver == &intel_pstate)
2681+ intel_pstate_sysfs_expose_hwp_dynamic_boost();
2682+
25632683 memset(&global, 0, sizeof(global));
25642684 global.max_perf_pct = 100;
25652685
@@ -2577,9 +2697,6 @@ static int intel_pstate_register_driver(struct cpufreq_driver *driver) · 11 unmodified lines
25772697
25782698 static int intel_pstate_unregister_driver(void)
25792699 {
2580− if (hwp_active)
2581− return -EBUSY;
2582−
25832700 cpufreq_unregister_driver(intel_pstate_driver);
25842701 intel_pstate_driver_cleanup();
25852702
@@ -2835,7 +2952,10 @@ static int __init intel_pstate_init(void) · 249 unmodified lines
28352952 hwp_active++;
28362953 hwp_mode_bdw = id->driver_data;
28372954 intel_pstate.attr = hwp_cpufreq_attrs;
2838− default_driver = &intel_pstate;
2955+ intel_cpufreq.attr = hwp_cpufreq_attrs;
2956+ if (!default_driver)
2957+ default_driver = &intel_pstate;
2958+
28392959 goto hwp_cpu_matched;
28402960 }
28412961 } else {
@@ -2906,14 +3026,13 @@ static int __init intel_pstate_setup(char *str) · 64 unmodified lines
29063026 if (!str)
29073027 return -EINVAL;
29083028
2909− if (!strcmp(str, "disable")) {
3029+ if (!strcmp(str, "disable"))
29103030 no_load = 1;
2911− } else if (!strcmp(str, "active")) {
3031+ else if (!strcmp(str, "active"))
29123032 default_driver = &intel_pstate;
2913− } else if (!strcmp(str, "passive")) {
3033+ else if (!strcmp(str, "passive"))
29143034 default_driver = &intel_cpufreq;
2915− no_hwp = 1;
2916− }
3035+
29173036 if (!strcmp(str, "no_hwp")) {
29183037 pr_info("HWP disabled\n");
29193038 no_hwp = 1;
include/linux/cpufreq.h+2−0
@@ -576,6 +576,8 @@ unsigned int cpufreq_driver_resolve_freq(struct cpufreq_policy *policy, · 575 unmodified lines
576576 unsigned int cpufreq_policy_transition_delay_us(struct cpufreq_policy *policy);
577577 int cpufreq_register_governor(struct cpufreq_governor *governor);
578578 void cpufreq_unregister_governor(struct cpufreq_governor *governor);
579+int cpufreq_start_governor(struct cpufreq_policy *policy);
580+void cpufreq_stop_governor(struct cpufreq_policy *policy);
579581
580582 #define cpufreq_governor_init(__governor) \
581583 static int __init __governor##_init(void) \

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t