History

mm, thp: do not allow thp faults to avoid cpuset restrictions

The page allocator relies on __GFP_WAIT to determine if ALLOC_CPUSET should be set in allocflags. ALLOC_CPUSET controls if a page allocation should be restricted only to the set of allowed cpuset mems. Transparent hugepages clears __GFP_WAIT when defrag is disabled to prevent the fault path from using memory compaction or direct reclaim. Thus, it is unfairly able to allocate outside of its cpuset mems restriction as a side-effect. This patch ensures that ALLOC_CPUSET is only cleared when the gfp mask is truly GFP_ATOMIC by verifying it is also not a thp allocation.

David Rientjes committed 12 years ago

Commit b104a351 parent f6789591 file · +8 −8Browse files

Unified · Split

mm/page_alloc.c+8−8
@@ -2447,7 +2447,7 @@ static inline int · 2446 unmodified lines
24472447 gfp_to_alloc_flags(gfp_t gfp_mask)
24482448 {
24492449 int alloc_flags = ALLOC_WMARK_MIN | ALLOC_CPUSET;
2450− const gfp_t wait = gfp_mask & __GFP_WAIT;
2450+ const bool atomic = !(gfp_mask & (__GFP_WAIT | __GFP_NO_KSWAPD));
24512451
24522452 /* __GFP_HIGH is assumed to be the same as ALLOC_HIGH to save a branch. */
24532453 BUILD_BUG_ON(__GFP_HIGH != (__force gfp_t) ALLOC_HIGH);
@@ -2456,20 +2456,20 @@ gfp_to_alloc_flags(gfp_t gfp_mask) · 2 unmodified lines
24562456 * The caller may dip into page reserves a bit more if the caller
24572457 * cannot run direct reclaim, or if the caller has realtime scheduling
24582458 * policy or is asking for __GFP_HIGH memory. GFP_ATOMIC requests will
2459− * set both ALLOC_HARDER (!wait) and ALLOC_HIGH (__GFP_HIGH).
2459+ * set both ALLOC_HARDER (atomic == true) and ALLOC_HIGH (__GFP_HIGH).
24602460 */
24612461 alloc_flags |= (__force int) (gfp_mask & __GFP_HIGH);
24622462
2463− if (!wait) {
2463+ if (atomic) {
24642464 /*
2465− * Not worth trying to allocate harder for
2466− * __GFP_NOMEMALLOC even if it can't schedule.
2465+ * Not worth trying to allocate harder for __GFP_NOMEMALLOC even
2466+ * if it can't schedule.
24672467 */
2468− if (!(gfp_mask & __GFP_NOMEMALLOC))
2468+ if (!(gfp_mask & __GFP_NOMEMALLOC))
24692469 alloc_flags |= ALLOC_HARDER;
24702470 /*
2471− * Ignore cpuset if GFP_ATOMIC (!wait) rather than fail alloc.
2472− * See also cpuset_zone_allowed() comment in kernel/cpuset.c.
2471+ * Ignore cpuset mems for GFP_ATOMIC rather than fail, see the
2472+ * comment for __cpuset_node_allowed_softwall().
24732473 */
24742474 alloc_flags &= ~ALLOC_CPUSET;
24752475 } else if (unlikely(rt_task(current)) && !in_interrupt())

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t