History

mm, compaction: capture a page under direct compaction

Compaction is inherently race-prone as a suitable page freed during compaction can be allocated by any parallel task. This patch uses a capture_control structure to isolate a page immediately when it is freed by a direct compactor in the slow path of the page allocator. The intent is to avoid redundant scanning. 5.0.0-rc1 5.0.0-rc1 selective-v3r17 capture-v3r19 Amean fault-both-1 0.00 ( 0.00%) 0.00 * 0.00%* Amean fault-both-3 2582.11 ( 0.00%) 2563.68 ( 0.71%) Amean fault-both-5 4500.26 ( 0.00%) 4233.52 ( 5.93%) Amean fault-both-7 5819.53 ( 0.00%) 6333.65 ( -8.83%) Amean fault-both-12 9321.18 ( 0.00%) 9759.38 ( -4.70%) Amean fault-both-18 9782.76 ( 0.00%) 10338.76 ( -5.68%) Amean fault-both-24 15272.81 ( 0.00%) 13379.55 * 12.40%* Amean fault-both-30 15121.34 ( 0.00%) 16158.25 ( -6.86%) Amean fault-both-32 18466.67 ( 0.00%) 18971.21 ( -2.73%) Latency is only moderately affected but the devil is in the details. A closer examination indicates that base page fault latency is reduced but latency of huge pages is increased as it takes creater care to succeed. Part of the "problem" is that allocation success rates are close to 100% even when under pressure and compaction gets harder 5.0.0-rc1 5.0.0-rc1 selective-v3r17 capture-v3r19 Percentage huge-3 96.70 ( 0.00%) 98.23 ( 1.58%) Percentage huge-5 96.99 ( 0.00%) 95.30 ( -1.75%) Percentage huge-7 94.19 ( 0.00%) 97.24 ( 3.24%) Percentage huge-12 94.95 ( 0.00%) 97.35 ( 2.53%) Percentage huge-18 96.74 ( 0.00%) 97.30 ( 0.58%) Percentage huge-24 97.07 ( 0.00%) 97.55 ( 0.50%) Percentage huge-30 95.69 ( 0.00%) 98.50 ( 2.95%) Percentage huge-32 96.70 ( 0.00%) 99.27 ( 2.65%) And scan rates are reduced as expected by 6% for the migration scanner and 29% for the free scanner indicating that there is less redundant work. Compaction migrate scanned 20815362 19573286 Compaction free scanned 16352612 11510663 [mgorman@techsingularity.net: remove redundant check] Link: http://lkml.kernel.org/r/20190201143853.GH9565@techsingularity.net Link: http://lkml.kernel.org/r/20190118175136.31341-23-mgorman@techsingularity.net Signed-off-by: Mel Gorman <mgorman@techsingularity.net> Acked-by: Vlastimil Babka <vbabka@suse.cz> Cc: Andrea Arcangeli <aarcange@redhat.com> Cc: Dan Carpenter <dan.carpenter@oracle.com> Cc: David Rientjes <rientjes@google.com> Cc: YueHaibing <yuehaibing@huawei.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

Mel Gorman committed 7 years ago

Commit 5e1f0f01 parent e332f746 files · +111 −12Browse files

Showing the changes to include/linux (2 of 6 files). Show all

Unified · Split

2 files
  1. include/linux/compaction.h +2 −1
  2. include/linux/sched.h +4 −0
include/linux/compaction.h+2−1
@@ -93,7 +93,8 @@ extern int sysctl_compact_unevictable_allowed; · 92 unmodified lines
9393 extern int fragmentation_index(struct zone *zone, unsigned int order);
9494 extern enum compact_result try_to_compact_pages(gfp_t gfp_mask,
9595 unsigned int order, unsigned int alloc_flags,
96− const struct alloc_context *ac, enum compact_priority prio);
96+ const struct alloc_context *ac, enum compact_priority prio,
97+ struct page **page);
9798 extern void reset_isolation_suitable(pg_data_t *pgdat);
9899 extern enum compact_result compaction_suitable(struct zone *zone, int order,
99100 unsigned int alloc_flags, int classzone_idx);
include/linux/sched.h+4−0
@@ -47,6 +47,7 @@ struct pid_namespace; · 46 unmodified lines
4747 struct pipe_inode_info;
4848 struct rcu_node;
4949 struct reclaim_state;
50+struct capture_control;
5051 struct robust_list_head;
5152 struct sched_attr;
5253 struct sched_param;
@@ -958,6 +959,9 @@ struct task_struct { · 905 unmodified lines
958959
959960 struct io_context *io_context;
960961
962+#ifdef CONFIG_COMPACTION
963+ struct capture_control *capture_control;
964+#endif
961965 /* Ptrace state: */
962966 unsigned long ptrace_message;
963967 kernel_siginfo_t *last_siginfo;

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t