History

mm, compaction: capture a page under direct compaction

Compaction is inherently race-prone as a suitable page freed during compaction can be allocated by any parallel task. This patch uses a capture_control structure to isolate a page immediately when it is freed by a direct compactor in the slow path of the page allocator. The intent is to avoid redundant scanning. 5.0.0-rc1 5.0.0-rc1 selective-v3r17 capture-v3r19 Amean fault-both-1 0.00 ( 0.00%) 0.00 * 0.00%* Amean fault-both-3 2582.11 ( 0.00%) 2563.68 ( 0.71%) Amean fault-both-5 4500.26 ( 0.00%) 4233.52 ( 5.93%) Amean fault-both-7 5819.53 ( 0.00%) 6333.65 ( -8.83%) Amean fault-both-12 9321.18 ( 0.00%) 9759.38 ( -4.70%) Amean fault-both-18 9782.76 ( 0.00%) 10338.76 ( -5.68%) Amean fault-both-24 15272.81 ( 0.00%) 13379.55 * 12.40%* Amean fault-both-30 15121.34 ( 0.00%) 16158.25 ( -6.86%) Amean fault-both-32 18466.67 ( 0.00%) 18971.21 ( -2.73%) Latency is only moderately affected but the devil is in the details. A closer examination indicates that base page fault latency is reduced but latency of huge pages is increased as it takes creater care to succeed. Part of the "problem" is that allocation success rates are close to 100% even when under pressure and compaction gets harder 5.0.0-rc1 5.0.0-rc1 selective-v3r17 capture-v3r19 Percentage huge-3 96.70 ( 0.00%) 98.23 ( 1.58%) Percentage huge-5 96.99 ( 0.00%) 95.30 ( -1.75%) Percentage huge-7 94.19 ( 0.00%) 97.24 ( 3.24%) Percentage huge-12 94.95 ( 0.00%) 97.35 ( 2.53%) Percentage huge-18 96.74 ( 0.00%) 97.30 ( 0.58%) Percentage huge-24 97.07 ( 0.00%) 97.55 ( 0.50%) Percentage huge-30 95.69 ( 0.00%) 98.50 ( 2.95%) Percentage huge-32 96.70 ( 0.00%) 99.27 ( 2.65%) And scan rates are reduced as expected by 6% for the migration scanner and 29% for the free scanner indicating that there is less redundant work. Compaction migrate scanned 20815362 19573286 Compaction free scanned 16352612 11510663 [mgorman@techsingularity.net: remove redundant check] Link: http://lkml.kernel.org/r/20190201143853.GH9565@techsingularity.net Link: http://lkml.kernel.org/r/20190118175136.31341-23-mgorman@techsingularity.net Signed-off-by: Mel Gorman <mgorman@techsingularity.net> Acked-by: Vlastimil Babka <vbabka@suse.cz> Cc: Andrea Arcangeli <aarcange@redhat.com> Cc: Dan Carpenter <dan.carpenter@oracle.com> Cc: David Rientjes <rientjes@google.com> Cc: YueHaibing <yuehaibing@huawei.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

Mel Gorman committed 7 years ago

Commit 5e1f0f01 parent e332f746 files · +111 −12Browse files

Showing the changes to include/linux (2 of 6 files). Show all

Unified · Split

2 files
  1. include/linux/compaction.h +2 −1
  2. include/linux/sched.h +4 −0
include/linux/compaction.h+2−1
@@ -93,7 +93,8 @@ extern int sysctl_compact_unevictable_allowed; · 92 unmodified lines
93extern int fragmentation_index(struct zone *zone, unsigned int order);93extern int fragmentation_index(struct zone *zone, unsigned int order);
94extern enum compact_result try_to_compact_pages(gfp_t gfp_mask,94extern enum compact_result try_to_compact_pages(gfp_t gfp_mask,
95 unsigned int order, unsigned int alloc_flags,95 unsigned int order, unsigned int alloc_flags,
96 const struct alloc_context *ac, enum compact_priority prio);96 const struct alloc_context *ac, enum compact_priority prio,
97 struct page **page);
97extern void reset_isolation_suitable(pg_data_t *pgdat);98extern void reset_isolation_suitable(pg_data_t *pgdat);
98extern enum compact_result compaction_suitable(struct zone *zone, int order,99extern enum compact_result compaction_suitable(struct zone *zone, int order,
99 unsigned int alloc_flags, int classzone_idx);100 unsigned int alloc_flags, int classzone_idx);
include/linux/sched.h+4−0
@@ -47,6 +47,7 @@ struct pid_namespace; · 46 unmodified lines
47struct pipe_inode_info;47struct pipe_inode_info;
48struct rcu_node;48struct rcu_node;
49struct reclaim_state;49struct reclaim_state;
50struct capture_control;
50struct robust_list_head;51struct robust_list_head;
51struct sched_attr;52struct sched_attr;
52struct sched_param;53struct sched_param;
@@ -958,6 +959,9 @@ struct task_struct { · 905 unmodified lines
958 959
959 struct io_context *io_context;960 struct io_context *io_context;
960 961
962#ifdef CONFIG_COMPACTION
963 struct capture_control *capture_control;
964#endif
961 /* Ptrace state: */965 /* Ptrace state: */
962 unsigned long ptrace_message;966 unsigned long ptrace_message;
963 kernel_siginfo_t *last_siginfo;967 kernel_siginfo_t *last_siginfo;

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t